Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Response should be encoded in charset

#4256

I have an app which is UTF-8, but needs to output ISO-8859-1 in response to a particular request. I set the appropriate headers in the controller method

response.headers['Content-Type'] = 'text/html; charset=ISO-8859-1'

and my template file contains output like

<%= some_string.encode("iso-8859-1") %>

However, Rails renders the file into an ASCII-8BIT string, which falls over when we try to create an eTag out of it...

if etag.blank?
  headers.delete('ETag')
else
  headers['ETag'] = %("#{Digest::MD5.hexdigest(ActiveSupport::Cache.expand_cache_key(etag))}")
end

...as the call to blank? assumes the string is UTF-8 and fails with an ArgumentError: (invalid byte sequence in UTF-8). See #2628 Ruby 1.9 and ActiveSupport for reference.

In my opinion, the response.body should be a string in the same encoding as the charset?

In the meantime, the attached patch is working for me.

Reported by donncha · March 23rd, 2010 @ 04:14 AM

State: invalid
Milestone: none
Assigned to: Jeremy Kemper Jeremy Kemper
Importance: none

Activity

  1. donncha
    donncha

    Err, allowing for no charset... whoops! :)

    March 23rd, 2010 @ 05:22 AM

  2. Jeremy Kemper
    Jeremy Kemper
    • State changed from new to invalid

    Your template needs to have the correct encoding. Set Encoding.default_external = "iso-8859-1".

    March 23rd, 2010 @ 02:03 PM

  3. donncha
    donncha

    Sorry, doesn't work.

    If I add

    Encoding.default_external = 'iso-8859-1'
    
    to the controller's method I still get the "invalid sequence in UTF-8" error. Debugging into ActionController#Base.render shows that External.default_external is in fact ISO-8859-1, but the response.body string still thinks it's UTF-8 (although it contains ISO-8859-1 characters), so it fails at the blank? call.
    [188, 197] in /Users/donch/Development/Exact/Steam/3583/vendor/rails/actionpack/lib/action_controller/response.rb
       188        def handle_conditional_get!
       189          if etag? || last_modified?
       190            set_conditional_cache_control!
       191          elsif nonempty_ok_response?
       192            debugger
    => 193            self.etag = body
       194  
       195            if request && request.etag_matches?(etag)
       196              self.status = '304 Not Modified'
       197              self.body = ''
    /Users/donch/Development/Exact/Steam/3583/vendor/rails/actionpack/lib/action_controller/response.rb:193
    self.etag = body
    (rdb:1) Encoding.default_external
    #<Encoding:ISO-8859-1>
    (rdb:1) body.encoding
    #<Encoding:UTF-8>
    (rdb:1)
    

    Furthermore, my encoding is now also set to ISO-8859-1 for all future requests, which is not what I want. I want to be able to output ISO-8859-1 for a single particular request, but have the rest of my app remain in UTF-8.

    If I set the charset for the response to ISO-8859-1, why is the response encoding in UTF-8?

    March 24th, 2010 @ 12:04 AM