Ruby 1.9 and ActiveSupport

Hi Dimitrij,

This issue just bit me on Ruby 1.9.1 as well. For me, it occurred on an action involving a CSV data export.

For the time being, I just wrapped the method body in a begin / rescue block and called headers.delete('ETag') if the test failed. As the Etag is not critical to the functionality of the application, it seemed a decent (if hasty) response.

Please keep me posted if you've found a better workaround, or if this issue has been resolved.

Thanks,

C. Scott Andreas
Developer, Sunago.org

Hello,

I've taken some time to better understand this behavior, patch the underlying issue, and prepare a test demonstrating the bug and verifying the resolution.

In my case, the application was also failing in ActionController::Response. The data rendered in my action contained a non-UTF8 character that, in the rendering process, is coerced to UTF-8 (likely from ASCII_8BIT) - this behavior is tested in part in actionpack/test/new_base/render_test.rb:

    def test_render_utf8_template_with_magic_comment
      with_external_encoding Encoding::ASCII_8BIT do
        result = @view.render(:file => "test/utf8_magic.html.erb", :layouts => "layouts/yield")
        assert_equal "Русский текст\nUTF-8\nUTF-8\nUTF-8\n", result
        assert_equal Encoding::UTF_8, result.encoding
      end
    end

However, in the event that non-UTF8 data is coerced into UTF-8 and String.blank? is called on it, an ArgumentError is raised because the current implementation (a regular expression) throws an exception when it receives a string with invalid encoding. So when the rendered response is evaluated to prepare etags in ActionController::Response, the application breaks by attempting to test for blank-ness on the document body.

Here are steps to reproduce the bug, both in a Rails 2.3.2 app on Ruby 1.9.1 in a console, and in IRB by requiring ActiveSupport: http://gist.github.com/146722

See the backtrace here: http://u.phoreo.com/lu.html

While it is not surprising that an error could result at some level when working with data of multiple and perhaps invalid character encodings, the point at which affected applications fail (in ActionController::Response) is a problem. Should an error occur due to character encoding, the request should at least not die deep inside ActionController.

More specifically, the current behavior of String.blank? does not conform to the statement atop active_support/core_ext/object/blank.rb, which states: An object is blank if it's false, empty, or a whitespace string. This method should not throw an exception when called upon a String object, regardless of encoding. The modified test included in this patch demonstrates the failure by supplying a string, converting it to UTF-8, and attempting to call .blank?.

The behavior of the patched version in Ruby 1.9 is as follows: if a string of invalid encoding is supplied and a regex test would result in fireworks, the method falls back to String.strip.size to test for blankness.

This seems to be a faithful implementation of the method that is compatible with both 1.8.x and 1.9.x while remaining minimally-invasive.

Apologies in advance for writing so much about such a simple bug and a one-line patch! Its obscurity seemed to warrant a little explanation.

I'd love to hear your feedback, and of any plans to include this or a similar patch in a future release.

Regards,

Scott Andreas (@cscotta)

Got an S3 error when attempting to download the patch attached to this ticket.

I've posted it here in case this doesn't fix itself: http://u.phoreo.com/01.diff

+1 to this patch.

An alternative might be to output a warning to the log file if a template contains invalid UTF-8 bytes, then use iconv to strip the offending bytes out. That way the template will always be UTF-8 and will hopefully avoid other similar issues like this blank?() one.