This project is archived and is in readonly mode.
Ruby1.9: submitted string form parameters with non-ASCII characters cause encoding errors
-
Jeremy Kemper
- State changed from new to open
- Assigned user set to Jeremy Kemper
- Milestone cleared.
We need to solve this at the Rack level by having a middleware force_encoding on incoming params and headers.
-
Szymon Nowak
In our projects (Rails 2.3.5) we're currently using something like this to solve the problem with params:
before_filter :force_utf8_params def force_utf8_params traverse = lambda do |object, block| if object.kind_of?(Hash) object.each_value { |o| traverse.call(o, block) } elsif object.kind_of?(Array) object.each { |o| traverse.call(o, block) } else block.call(object) end object end force_encoding = lambda do |o| o.force_encoding(Encoding::UTF_8) if o.respond_to?(:force_encoding) end traverse.call(params, force_encoding) endbut we want to move it into a middleware as well.
-
Marcin Kulik
Jeremy, I think this middleware should not blindly do force_encoding("utf8") because browser could send data in other encoding without telling which. It could be really in ISO-8859-1 and forcing UTF8 will damage the string. Maybe the middleware should use some heuristics to guess encoding if HTTP encoding header wasn't set. Wycats wrote in one if his recent posts that there are some ideas for such a middleware. Do you know if anyone tried to do sth with it?
I'm willing to help as this is real PITA for me at the moment. -
Dan Pickett
- Tag changed from encoding, rails3, ruby19, utf8 to bugmash, encoding, rails3, ruby19, utf8
-
Sasha Koss
@Szymon Nowak
Thanks! It works fine for me!
-
Marcin Kulik
@Szymon
Thanks! It works pretty well so far.
-
David Morton
My first attempt at creating a rails 3 project hit this one. @Szymon solution works, if a bit hackish. :)
-
Rohit Arondekar
- State changed from open to duplicate
- Importance changed from to Low
I think this issue was resolved in this ticket https://rails.lighthouseapp.com/projects/8994/tickets/4807
Can you please confirm? If there are still issues that remain then can you continue the discussion on that ticket?