#597 "bös".chars.reverse is "s̈ob" for decomposed string - Ruby on Rails

Type	To find
responsible:me	tickets assigned to you
tagged:"@high"	tickets tagged @high
milestone:next	tickets in the upcoming milestone
state:invalid	tickets with the state invalid
created:"last week"	tickets created last week
sort:number, importance, updated	tickets sorted by #, importance or updated
Combine keywords for powerful searching.
Use advanced searching »

This project is archived and is in readonly mode.

#597 ✓committed

"bös".chars.reverse is "s̈ob" for decomposed string

Reported by Henrik Nyh | July 10th, 2008 @ 06:12 PM | in 2.3.6

Failing test attached.

I would make an attempt at patching this myself, but I'm not sure what would be preferred behavior.

Should chars normalize everything to NFKC? Should chars.to_s (assuming chars has not been modified) always return the original string, un-normalized?

chars_reverse_decomposed.diff 1.3 KB

Comments and changes to this ticket

You flagged this item as spam.
Henrik Nyh July 10th, 2008 @ 06:13 PM
The failing test isn't likely something that would be used as-is if this issue is fixed, but is rather meant to illustrate the issue.
josh October 28th, 2008 @ 04:24 PM
- State changed from “new” to “stale”
Staling out, please reopen if this is still a problem.
You flagged this item as spam.
Henrik Nyh October 28th, 2008 @ 04:30 PM
Works fine in 2.2, so problem solved.
You flagged this item as spam.
Henrik Nyh October 28th, 2008 @ 04:32 PM
Oops, no. I forgot that the ticket was about decomposed strings, specifically, and they still fail:
```
>> dc = "bös".mb_chars.normalize(:kd).reverse.to_s
=> "s̈ob"
```
I'm not empowered to reopen tickets, I think, but someone else please do.
CancelProfileIsBroken August 3rd, 2009 @ 07:34 PM
- Tag changed from “activesupport, bug, chars, multibyte, tested” to “activesupport, bug, bugmash, chars, multibyte, tested”
- Milestone cleared.
You flagged this item as spam.
Dana Jones August 9th, 2009 @ 11:48 PM
+1 - This is still a problem in 3.0. If you create an @string variable that equals "bös", then @string.chars.reverse has the desired result ("söb"), but @string.mb_chars.normalize(:kd).reverse.to_s outputs "s̈ob".
Pratik August 9th, 2009 @ 11:49 PM
- Assigned user set to “Jeremy Kemper”
You flagged this item as spam.
Elad Meidar August 10th, 2009 @ 12:05 AM
+1 indeed verified on master, unsuccessful to reproduce with other UTF8 strings (hebrew)
Jeremy Kemper August 10th, 2009 @ 05:51 AM
- State changed from “stale” to “open”
- Milestone set to “2.3.4”
You flagged this item as spam.
Derander August 10th, 2009 @ 07:18 AM
+1 verified on master, this also happens w/ the string "bés"
Jeremy Kemper August 10th, 2009 @ 07:36 AM
- State changed from “open” to “incomplete”
Needs a fix.
Jeremy Kemper September 11th, 2009 @ 11:04 PM
- Milestone changed from “2.3.4” to “2.3.6”
[milestone:id#50064 bulk edit command]
You flagged this item as spam.
Matías Flores September 27th, 2009 @ 01:43 AM
I've attached a patch that fixes this issue. It applies cleanly to both master and 2-3-stable.
- fix_chars_reverse_decomposed.diff 2.2 KB
You flagged this item as spam.
John Pignata September 27th, 2009 @ 05:14 AM
+1 - applies cleanly to master and 2-3-stable and tests pass
You flagged this item as spam.
sr.iniv.t September 27th, 2009 @ 05:38 AM
+1 verified.

The patch applies and all tests pass.
You flagged this item as spam.
Rishav Rastogi September 27th, 2009 @ 07:20 AM
+1 verified.

Patch applies to master.
CancelProfileIsBroken September 27th, 2009 @ 11:32 AM
- State changed from “incomplete” to “open”
- Tag changed from “activesupport, bug, bugmash, chars, multibyte, tested” to “activesupport, bug, bugmash, bugmash-review, chars, multibyte, tested”
CancelProfileIsBroken September 27th, 2009 @ 12:31 PM
- Tag changed from “activesupport, bug, bugmash, bugmash-review, chars, multibyte, tested” to “activesupport, bug, bugmash-review, chars, multibyte, tested”
Manfred Stienstra November 1st, 2009 @ 03:46 PM
- Tag changed from “activesupport, bug, bugmash-review, chars, multibyte, tested” to “activesupport, bug, bugmash-review, chars, multibyte, tested”
Matías' fix looks good, it has my blessing (;

In a pure theoretical sense this is the correct way of fixing the problem. This makes sure that elaborate Hangul strings get reversed properly as well. It is a bit slow though. I don't know of many real world cases where you would reverse a string, so I guess if you're doing it you probably have a great incentive to do it properly and you wouldn't mind the speed penalty.
Repository November 3rd, 2009 @ 02:00 AM
- State changed from “open” to “committed”
(from [4ae03b2d5a7b81a8b8d330c12de40f970525a67a]) Fix chars.reverse for multibyte decomposed strings

[#597 state:committed]

Signed-off-by: Jeremy Kemper jeremy@bitsweat.net
http://github.com/rails/rails/commit/4ae03b2d5a7b81a8b8d330c12de40f...
Repository November 3rd, 2009 @ 02:00 AM
(from [23780850237876cf81038534d8f59fa307af0b31]) Fix chars.reverse for multibyte decomposed strings

[#597 state:committed]

Signed-off-by: Jeremy Kemper jeremy@bitsweat.net
http://github.com/rails/rails/commit/23780850237876cf81038534d8f59f...
Rizwan Reza May 15th, 2010 @ 06:42 PM
- Tag changed from “activesupport, bug, bugmash-review, chars, multibyte, tested” to “activesupport, bug, chars, multibyte, tested”
[bulk edit]