String parameterize / normalize bug

Can't reproduce on Ruby 1.8.7 and Rails 2.3.5

>> str = "test ščž"
=> "test šč\305\276"
>> str.parameterize
=> #
>> str.parameterize.to_s
=> "test-scz"

Can you please be more specific with Ruby and Rails versions?

I could not reproduce with Rails 2.3.5 and Ruby 1.8.7, either. HOWEVER! Attempting to recreate with 1.8.7p248 and Rails 3, I got:

/gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/inflector/transliterate.rb:26:in `transliterate': undefined method `normalize' for "test \305\241\304\215\305\276":String (NoMethodError)
    from /gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/inflector/transliterate.rb:48:in `parameterize'
    from /gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/core_ext/string/inflections.rb:102:in `parameterize'
    from /gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/multibyte/chars.rb:102:in `__send__'
    from /gems/ruby-1.8.7-p248/gems/activesupport-3.0.0.beta/lib/active_support/multibyte/chars.rb:102:in `method_missing'
    from test.rb:12

To fix it, I had to define the $KCODE (encoding) for the document:

require 'rubygems'
require 'active_support/all'

$KCODE = 'UTF8'

str = "test ščž"
puts str.parameterize.inspect
puts str.parameterize.to_s

=> #<ActiveSupport::Multibyte::Chars:0x101876110 @wrapped_string="test-scz">
=> test-scz

The author should update their config/environment.rb or create an "encoding" initializer properly setting their $KCODE for their application. This is a known requirement of Ruby 1.8 when working with multibyte strings.

Tagging with invalid.

Shouldn't the $KCODE be set by Rails to something that supports this, such as UTF8, by default? I do not think this ticket is invalid.

Rails 3:

railties/lib/rails.rb:# For Ruby 1.8, this initialization sets $KCODE to 'u' to enable the
railties/lib/rails.rb:  $KCODE='u'

Rails 2.3:

railties/lib/initializer.rb:    # For Ruby 1.8, this initialization sets $KCODE to 'u' to enable the
railties/lib/initializer.rb:      $KCODE='u' if RUBY_VERSION < '1.9'

And it was there for quite a long time. Unicode KCODE is default in Rails 2.3+ at least.

Nathaniel Bibler, you seem to be knowledgeable in this area, could you submit a test case and if you could, a fix for it? Thank you!

Jeremy committed a patch of mine today that modifies the way Inflector.parameterize and Inflector.transliterate work. The #transliterate method invoked by parameterize now always operates on an instance of ActiveSupport::Multibyte::Chars rather than a proxy class, so it should not be possible to reach this error by invoking #parameterize. Marking the ticket invalid.

confirmed.