Rails Ruby on Rails

Comments and changes to this ticket

• You flagged this item as spam.
  

  Yaroslav Markin April 10th, 2010 @ 08:11 PM

  • Tag changed from “normalize, parameterize, string” to “bug, normalize, parameterize, string”

  Can't reproduce on Ruby 1.8.7 and Rails 2.3.5

  >> str = "test ščž"
  => "test šč\305\276"
  >> str.parameterize
  => #
  >> str.parameterize.to_s
  => "test-scz"
  

  Can you please be more specific with Ruby and Rails versions?

• You flagged this item as spam.
  

  Nathaniel Bibler April 11th, 2010 @ 03:36 AM

  • Tag changed from “bug, normalize, parameterize, string” to “bug, invalid, normalize, parameterize, string”

  I could not reproduce with Rails 2.3.5 and Ruby 1.8.7, either. HOWEVER! Attempting to recreate with 1.8.7p248 and Rails 3, I got:

  /gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/inflector/transliterate.rb:26:in `transliterate': undefined method `normalize' for "test \305\241\304\215\305\276":String (NoMethodError)
      from /gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/inflector/transliterate.rb:48:in `parameterize'
      from /gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/core_ext/string/inflections.rb:102:in `parameterize'
      from /gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/multibyte/chars.rb:102:in `__send__'
      from /gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/multibyte/chars.rb:102:in `method_missing'
      from test.rb:12
  

  To fix it, I had to define the $KCODE (encoding) for the document:

  require 'rubygems'
  require 'active_support/all'
  
  $KCODE = 'UTF8'
  
  str = "test ščž"
  puts str.parameterize.inspect
  puts str.parameterize.to_s
  
  => #<ActiveSupport::Multibyte::Chars:0x101876110 @wrapped_string="test-scz">
  => test-scz
  

  The author should update their config/environment.rb or create an "encoding" initializer properly setting their $KCODE for their application. This is a known requirement of Ruby 1.8 when working with multibyte strings.

  Tagging with invalid.

• 

  Ryan Bigg April 11th, 2010 @ 02:16 PM

  • Tag cleared.

  Shouldn't the $KCODE be set by Rails to something that supports this, such as UTF8, by default? I do not think this ticket is invalid.

• You flagged this item as spam.
  

  Nathaniel Bibler April 11th, 2010 @ 05:45 PM

  Perhaps it should.. does anyone know of any downside of setting the $KCODE to UTF8 all the time? Perhaps a time penalty when working with string comparisons and whatnot?

• You flagged this item as spam.
  

  Yaroslav Markin April 11th, 2010 @ 05:52 PM

  • Tag set to “activesupport, utf8”
  • Assigned user set to “Ryan Bigg”

  Rails 3:

  railties/lib/rails.rb:# For Ruby 1.8, this initialization sets $KCODE to 'u' to enable the
  railties/lib/rails.rb:  $KCODE='u'
  

  Rails 2.3:

  railties/lib/initializer.rb:    # For Ruby 1.8, this initialization sets $KCODE to 'u' to enable the
  railties/lib/initializer.rb:      $KCODE='u' if RUBY_VERSION < '1.9'
  

  And it was there for quite a long time. Unicode KCODE is default in Rails 2.3+ at least.

• 

  Ryan Bigg April 11th, 2010 @ 09:24 PM

  • State changed from “new” to “open”

  Nathaniel Bibler, you seem to be knowledgeable in this area, could you submit a test case and if you could, a fix for it? Thank you!

• You flagged this item as spam.
  

  releod April 11th, 2010 @ 10:32 PM

  If it is relevant - use of $KCODE in 1.9 will produce a warning...

  warning: variable $KCODE is no longer effective; ignored

• You flagged this item as spam.
  

  Yaroslav Markin April 12th, 2010 @ 12:59 AM

  You can ensure $KCODE defaults to "UTF8" in Rails apps if you open up script/console in any of your applications and inspect $KCODE.

  @releod -- you are right, $KCODE is for Ruby 1.8 and less only.

• You flagged this item as spam.
  

  Nathaniel Bibler April 12th, 2010 @ 08:13 PM

  Part of the problem is that the original author didn't post the Ruby/Rails versions they were using to cause the problem. So, they may actually be using an older version of Rails (<2.3). For the sake of argument, I assumed Rails 2.3.5 / Rails 3 for testing.

  You can see in my sample script, I didn't fully load Rails 3; instead loading only the ActiveSupport module, which didn't execute the Rails $KCODE setter that @Yaroslav mentioned.

  So, if this ticket isn't invalid or wontfix I would think it should at least be needs-more-info.

• 

  Ryan Bigg April 12th, 2010 @ 09:52 PM

  If it's working on 2.3.5 then that's fine.

  If it is broken in 3.0.0, then please write a parch.

• You flagged this item as spam.
  

  Norman Clarke April 13th, 2010 @ 01:24 PM

  • Tag changed from “activesupport, utf8” to “activesupport, invalid, utf8”

  Jeremy committed a patch of mine today that modifies the way Inflector.parameterize and Inflector.transliterate work. The #transliterate method invoked by parameterize now always operates on an instance of ActiveSupport::Multibyte::Chars rather than a proxy class, so it should not be possible to reach this error by invoking #parameterize. Marking the ticket invalid.

• 

  Ryan Bigg April 13th, 2010 @ 09:41 PM

  • Tag changed from “activesupport, invalid, utf8” to “activesupport, utf8”
  • State changed from “open” to “invalid”

  confirmed.