This project is archived and is in readonly mode.
Parameterize should accept malformated utf8 strings
Reported by Kristopher Murata | April 4th, 2010 @ 07:02 AM | in 2.3.6
When using parameterize I ended up with the following errors:
NoMethodError: undefined method normalize' for
"Fort\351":String
This occurred because user data (user name, city name...)
sometimes were inserted with malformated characters (something like
"Now \251") and parameterize was not able to transform that string
into a slug. I assume that parameterize should transform
any string into a nice slug, even if it ends up removing
characters or generating a slightly different slug from the
original string.
So what I did is first verify if the string is a valid utf8, if not
it should force the convert to utf8 (with losses if necessary) and
then translate to ascii equivalents using
Inflector:transliterate.
This should solve this bug too #3504.
Comments and changes to this ticket
-
Kristopher Murata April 4th, 2010 @ 07:05 AM
- Tag set to “patch”
-
Yaroslav Markin April 4th, 2010 @ 09:31 AM
- Tag changed from “patch” to “activesupport, patch, utf8”
- Assigned user set to “José Valim”
Works for me; Need this as well.
-
Jeremy Kemper April 4th, 2010 @ 03:10 PM
- State changed from “new” to “incomplete”
- Milestone set to “2.3.6”
NoMethodError: undefined method `toutf8' for "Test with malformed utf8 \xA9":String
-
Kristopher Murata April 4th, 2010 @ 03:42 PM
Hi Jeremy,
I think that this problem is due to the missing Kconv module, we can call Kconv directly since the toutf8 method on String is just an alias to Kconv.toutf8().
Also, I see that you marked for the next 2.3.6 release, should I branch from stable 2.3.5 (instead of master) and redo the patch?
-
Jeremy Kemper April 4th, 2010 @ 03:49 PM
Ah ok - be sure to require kconv.
We always work from master, then
git cherry-pickback to 2-3-stable if appropriate. I marked this as 2.3.6 since it'd be a welcome update for 2.3.6 users also. -
-
Repository April 4th, 2010 @ 07:13 PM
- State changed from “incomplete” to “resolved”
(from [ef84e691ffe7b43ce64e859f43b17128a405785a]) Parameterize should accept malformed utf8 characters [#4323 state:resolved]
Signed-off-by: Jeremy Kemper jeremy@bitsweat.net
http://github.com/rails/rails/commit/ef84e691ffe7b43ce64e859f43b171... -
Jeremy Kemper April 4th, 2010 @ 07:15 PM
- State changed from “resolved” to “incomplete”
Kristopher, could you backport to 2-3-stable?
-
Kristopher Murata April 4th, 2010 @ 07:39 PM
Yep, I'm installing ruby 1.9 env to test a particular case on 2-3-stable branch.
-
-
Repository April 4th, 2010 @ 07:59 PM
- State changed from “incomplete” to “resolved”
(from [642d5d297e9569a76047c958b8e4aa23522f300d]) Parameterize should accept malformed utf8 characters [#4323 state:resolved]
Signed-off-by: Jeremy Kemper jeremy@bitsweat.net
http://github.com/rails/rails/commit/642d5d297e9569a76047c958b8e4aa... -
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile »
<h2 style="font-size: 14px">Tickets have moved to Github</h2>
The new ticket tracker is available at <a href="https://github.com/rails/rails/issues">https://github.com/rails/rails/issues</a>
People watching this ticket
Jeremy Kemper
José Valim
Kristopher Murata
Yaroslav Markin
Attachments
Tags
Referenced by
-
4323
Parameterize should accept malformated utf8 strings
(from [ef84e691ffe7b43ce64e859f43b17128a405785a])
Paramet...
-
4323
Parameterize should accept malformated utf8 strings
(from [642d5d297e9569a76047c958b8e4aa23522f300d])
Paramet...