Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Bad Content Type Error in Rack 1.0 with first CGI Mongrel Request

#2844

This issue is similar to Tickets #2788 Bad Content Type Error in Rack 1.0 and #1875 Posting a multipart form with file data can break params assignment (in 2.3) with the same backtrace. I have been able to reproduce this reliably by sending an initial multipart request to mongrel upon a mongrel_rails (or cluster) startup.

Ruby => 1.8.6
OS => OS X Leopard (also a Linux Distro)
Rails => 2.3.2
Mongrel => 1.1.5

This issue has its origins in this change in Rails 2.3 in action_controller.rb from:

require 'action_controller/cgi_process'

to

# DEPRECATE: Remove CGI support
autoload :CgiRequest, 'action_controller/cgi_process'
autoload :CGIHandler, 'action_controller/cgi_process'

This causes this patch in /action_controller/cgi_ext/query_extension.rb to be lazy loaded:

# Neuter CGI parameter parsing.
def initialize_query
  # Fix some strange request environments.
  env_table['REQUEST_METHOD'] ||= 'GET'

  # POST assumes missing Content-Type is application/x-www-form-urlencoded.
  if env_table['CONTENT_TYPE'].blank? && env_table['REQUEST_METHOD'] == 'POST'
    env_table['CONTENT_TYPE'] = 'application/x-www-form-urlencoded'
  end

  @cookies = CGI::Cookie::parse(env_table['HTTP_COOKIE'] || env_table['COOKIE'])
  @params = {}
end

This has the implication of the original initialize_query running for the first request if it's a multipart request. The original ruby cgi method calls read on stdinput moving the offset to the end which causes the method parse_multipart to fail as it doesn't rewind the data before trying to parse it. After the first request, the Rails cgi extension has a chance to load making all subsequent multipart request work correctly. This also explains why passenger seems to be immune to this issue, as is starting the app via script/server as it does not process the request via cgi (mongrel_rails, though it does use mongrel).

This bug, although an isolated case, can be annoying for a production app of several mongrel instances that just listen for multipart submissions when the first one for each instance fails (Especially when you are watching the logs like a hawk :) ). I reviewed the commit 25fde77 which updates the rack version but does not address this issue.

The resolution to this may very well be that cgi support is deprecated so move on, but as this issue cost me many hours in rdebug I wanted it recorded for reference, and I would bet there are still a few mongrel_clusters still out there in the wild.

Reported by Brad Sumersford · June 27th, 2009 @ 04:00 AM

State: resolved
Milestone: 3.x
Assigned to: nobody
Importance: none

Activity

  1. Brad Sumersford
    Brad Sumersford

    It appears as http://github.com/rack/rack/commit/95f068e84b90fdfd6440314b0f86862a... takes care of the issue by calling .rewind on the input before trying to parse it. Packaged gem for ActionPack 2.3.2 (w/ Rack 1.0.0) as of June 26th 2009 didn't appear to have this fix, today it does.

    June 29th, 2009 @ 03:08 PM

  2. Brad Sumersford
    Brad Sumersford

    It appears as though I was incorrect, the current version on rubyforge does not include the rewind fix. I had to perform wizardry beyond specifying remote and clearing my cache to prevent gems from installing the version I built from source (which does work).

    June 29th, 2009 @ 04:46 PM

  3. Yehuda Katz (wycats)
    Yehuda Katz (wycats)

    Does this still happen on 2-3-stable? It appears that Josh vendered the latest rack with the following commit: http://github.com/rails/rails/commit/25fde77674cd6f9906b7a8e7eef046...

    July 2nd, 2009 @ 06:34 PM

  4. Yehuda Katz (wycats)
    Yehuda Katz (wycats)
    • State changed from new to open

    July 2nd, 2009 @ 06:34 PM

  5. Brad Sumersford
    Brad Sumersford

    After checking out 2-3-stable is does look like the vendor/ directory includes the rack-1.0.x version which does include the rewind addition that addresses the above issue. The packaged gem still does not include this however (last time I checked), in the mean time I was able to successfully solve the issue by installing the rack gem (1.0.0) which also includes the fix.

    Though this issue has been fixed (in 2-3-stable) the behaviour of the first request routing through the original CGI code and all subsequent requests routing through the Rails CGI code could still result in all sorts of weird and wonderful behaviours.

    July 3rd, 2009 @ 01:30 AM

  6. scott stewart
    scott stewart

    I'm seeing a painful variation on this with Rails right through 2.3.3. When mongrel is started via mongrel_cluster, and the first request to a newly started mongrel is a "POST", no request parameters or session is set inside the controller.

    Once the restarted mongrel had handled its first request, all is well.

    I've repeated this with a freshly created Rails app, adding a single controller and form. In my case this is just a regular Form (application/x-www-form-urlencoded ) and not a multi-part.

    To repeat:
    Start up mongrel_cluster of at least 1 mongrel.
    Serve up a page with a form.
    Restart mongrel_cluster.
    Post form.

    Notice no session data or form parameters in controller.

    July 27th, 2009 @ 06:12 PM

  7. Michael Koziarski
    Michael Koziarski
    • Tag set to bugmash

    August 3rd, 2009 @ 06:01 AM

  8. Ryan Carmelo Briones
    Ryan Carmelo Briones

    The scenario outlined by Scott Stewart happened to my coworkers and I today on the first deployment of our app after an upgrade from vendored 2.0.2 to vendored 2.3.4. FYI. Will see if I can come up with some kind of patch...

    September 16th, 2009 @ 10:26 PM

  9. Ryan Carmelo Briones
    Ryan Carmelo Briones

    http://gist.github.com/195369 is the interim fix that I've come up with to handle my issue as it relates to this ticket. It basically adds back the code that caused this scenario to break. The other possible fix that has worked for me was to add "cgi.stdinput.rewind" after L25 here (http://github.com/rails/rails/blob/2-3-stable/actionpack/lib/action...) but I don't fully understand the implications, and I'm not sure where to start adding a test that is related to the dispatcher but confirms that params were parsed in the proper way. Of course, I assume this will all be moot since .dispatch and .dispatch_cgi will probably go away in Rails 3.0.

    September 28th, 2009 @ 02:54 PM

  10. Frederick Cheung
    Frederick Cheung

    November 30th, 2009 @ 02:12 PM

  11. Rizwan Reza
    Rizwan Reza
    • Tag cleared.

    February 12th, 2010 @ 12:46 PM

  12. Jeremy Kemper
    Jeremy Kemper
    • Milestone changed from 2.x to 3.x

    May 4th, 2010 @ 06:48 PM

  13. Rohit Arondekar
    Rohit Arondekar
    • State changed from open to resolved
    • Importance changed from to

    October 9th, 2010 @ 03:59 AM