Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Parameterize should accept malformated utf8 strings

#4323

When using parameterize I ended up with the following errors:

NoMethodError: undefined method normalize' for "Fort\351":String

This occurred because user data (user name, city name...) sometimes were inserted with malformated characters (something like "Now \251") and parameterize was not able to transform that string into a slug. I assume that parameterize should transform any string into a nice slug, even if it ends up removing characters or generating a slightly different slug from the original string.
So what I did is first verify if the string is a valid utf8, if not it should force the convert to utf8 (with losses if necessary) and then translate to ascii equivalents using Inflector:transliterate.

This should solve this bug too #3504 String parameterize / normalize bug.

Reported by Kristopher Murata · April 4th, 2010 @ 07:02 AM

State: resolved
Milestone: 2.3.6
Assigned to: José Valim José Valim
Importance: none

Activity

  1. Kristopher Murata
  2. Yaroslav Markin
    Yaroslav Markin
    • Tag changed from patch to activesupport, patch, utf8
    • Assigned user set to José Valim

    Works for me; Need this as well.

    April 4th, 2010 @ 09:31 AM

  3. Jeremy Kemper
    Jeremy Kemper
    • State changed from new to incomplete
    • Milestone set to 2.3.6
    NoMethodError: undefined method `toutf8' for "Test with malformed utf8 \xA9":String
    

    April 4th, 2010 @ 03:10 PM

  4. Kristopher Murata
    Kristopher Murata

    Hi Jeremy,

    I think that this problem is due to the missing Kconv module, we can call Kconv directly since the toutf8 method on String is just an alias to Kconv.toutf8().

    Also, I see that you marked for the next 2.3.6 release, should I branch from stable 2.3.5 (instead of master) and redo the patch?

    April 4th, 2010 @ 03:42 PM

  5. Jeremy Kemper
    Jeremy Kemper

    Ah ok - be sure to require kconv.

    We always work from master, then git cherry-pick back to 2-3-stable if appropriate. I marked this as 2.3.6 since it'd be a welcome update for 2.3.6 users also.

    April 4th, 2010 @ 03:49 PM

  6. Kristopher Murata
  7. Repository
    Repository
    • State changed from incomplete to resolved

    April 4th, 2010 @ 07:13 PM

  8. Jeremy Kemper
    Jeremy Kemper
    • State changed from resolved to incomplete

    Kristopher, could you backport to 2-3-stable?

    April 4th, 2010 @ 07:15 PM

  9. Kristopher Murata
    Kristopher Murata

    Yep, I'm installing ruby 1.9 env to test a particular case on 2-3-stable branch.

    April 4th, 2010 @ 07:39 PM

  10. Kristopher Murata
  11. Repository
    Repository
    • State changed from incomplete to resolved

    April 4th, 2010 @ 07:59 PM

  12. bingbing