Rails Ruby on Rails → 2.x

Comments and changes to this ticket

• You flagged this item as spam.
  

  Scott July 14th, 2009 @ 03:03 AM

  Hi Dimitrij,

  This issue just bit me on Ruby 1.9.1 as well. For me, it occurred on an action involving a CSV data export.

  For the time being, I just wrapped the method body in a begin / rescue block and called headers.delete('ETag') if the test failed. As the Etag is not critical to the functionality of the application, it seemed a decent (if hasty) response.

  Please keep me posted if you've found a better workaround, or if this issue has been resolved.

  Thanks,

  C. Scott Andreas
  Developer, Sunago.org

• You flagged this item as spam.
  

  Scott July 14th, 2009 @ 08:01 AM

  Hello,

  I've taken some time to better understand this behavior, patch the underlying issue, and prepare a test demonstrating the bug and verifying the resolution.

  In my case, the application was also failing in ActionController::Response. The data rendered in my action contained a non-UTF8 character that, in the rendering process, is coerced to UTF-8 (likely from ASCII_8BIT) - this behavior is tested in part in actionpack/test/new_base/render_test.rb:

      def test_render_utf8_template_with_magic_comment
        with_external_encoding Encoding::ASCII_8BIT do
          result = @view.render(:file => "test/utf8_magic.html.erb", :layouts => "layouts/yield")
          assert_equal "Русский текст\nUTF-8\nUTF-8\nUTF-8\n", result
          assert_equal Encoding::UTF_8, result.encoding
        end
      end
  

  However, in the event that non-UTF8 data is coerced into UTF-8 and String.blank? is called on it, an ArgumentError is raised because the current implementation (a regular expression) throws an exception when it receives a string with invalid encoding. So when the rendered response is evaluated to prepare etags in ActionController::Response, the application breaks by attempting to test for blank-ness on the document body.
  

  Here are steps to reproduce the bug, both in a Rails 2.3.2 app on Ruby 1.9.1 in a console, and in IRB by requiring ActiveSupport: http://gist.github.com/146722

  See the backtrace here: http://u.phoreo.com/lu.html

  While it is not surprising that an error could result at some level when working with data of multiple and perhaps invalid character encodings, the point at which affected applications fail (in ActionController::Response) is a problem. Should an error occur due to character encoding, the request should at least not die deep inside ActionController.

  More specifically, the current behavior of String.blank? does not conform to the statement atop active_support/core_ext/object/blank.rb, which states: An object is blank if it's false, empty, or a whitespace string. This method should not throw an exception when called upon a String object, regardless of encoding. The modified test included in this patch demonstrates the failure by supplying a string, converting it to UTF-8, and attempting to call .blank?.

  The behavior of the patched version in Ruby 1.9 is as follows: if a string of invalid encoding is supplied and a regex test would result in fireworks, the method falls back to String.strip.size to test for blankness.

  This seems to be a faithful implementation of the method that is compatible with both 1.8.x and 1.9.x while remaining minimally-invasive.

  Apologies in advance for writing so much about such a simple bug and a one-line patch! Its obscurity seemed to warrant a little explanation.

  I'd love to hear your feedback, and of any plans to include this or a similar patch in a future release.

  Regards,

  Scott Andreas (@cscotta)

  • string_blank_patch_for_19.diff 1.7 KB

• You flagged this item as spam.
  

  Scott July 14th, 2009 @ 08:05 AM

  Got an S3 error when attempting to download the patch attached to this ticket.

  I've posted it here in case this doesn't fix itself: http://u.phoreo.com/01.diff

• 

  Michael Koziarski July 15th, 2009 @ 12:48 AM

  • Assigned user set to “Jeremy Kemper”

  Jeremy's the 1.9 guy

• You flagged this item as spam.
  

  James Healy July 19th, 2009 @ 01:22 PM

  +1 to this patch.

  An alternative might be to output a warning to the log file if a template contains invalid UTF-8 bytes, then use iconv to strip the offending bytes out. That way the template will always be UTF-8 and will hopefully avoid other similar issues like this blank?() one.

• 

  Jeremy Kemper August 2nd, 2009 @ 04:32 AM

  • State changed from “new” to “wontfix”

  Changing blank? masks the underlying error instead of fixing the root cause. Why do you have a string with invalid encoding?

• You flagged this item as spam.
  

  pederbl (at jobstar) January 11th, 2011 @ 01:20 PM

  • Tag changed from “2-3-stable, activesupport, ruby1.9, ruby19” to “2-3-stable, activesupport, rails3, ruby1.9, ruby19”
  • Importance changed from “” to “”

  Strings with invalid encoding is supported by Ruby. So it seems natural that calling blank? on strings with invalid encoding should be supported by Rails.

  There is another way to figure out if strings are invalid, e.g. valid_encoding?

  What is the benefit of making blank? fail on strings with invalid encoding?

• You flagged this item as spam.
  

  csnk May 18th, 2011 @ 08:27 AM

  We are the professional underwear manufacturer, underwear supplier, underwear factory, custom underwear.