This project is archived and is in readonly mode.

#2652 ✓resolved
Markus Seeger

multipart form upload: mongrel memory consumption

Reported by Markus Seeger | May 15th, 2009 @ 04:53 PM | in 2.x

When uploading big files with a multipart form, the mongrel process will eat up lots of memory and not release it anytime, thus permanently stalling the whole machine after some uploads.
Occured on Debian Linux and Windows XP Pro machines, also with a mongrel cluster balanced by apache.
This didn't happen when switching back to rails 2.2.2.

Reproduction instructions:

rails: 2.3.2
mongrel: 1.1.5

$> rails upload_test
$> cd upload_test
$> script/generate controller upload index

Modify app/views/upload/index.html.erb to:

<% form_tag({}, { :multipart => true }) do %>
  <label for="file">File to Upload</label>
  <%= file_field_tag "file" %>
  <%= submit_tag %>
<% end %>
then start the server, create a huge file
$> mongrel_rails start
$> dd if=/dev/random of=big_file.txt bs=1000000 count=300

and upload using the form:


Comments and changes to this ticket

  • Daniel Guettler

    Daniel Guettler May 16th, 2009 @ 01:57 AM

    I tracked this down to this commit:


    1adc1496f9152c893e1f08abcb1e5e7272829899 is first bad commit
    commit 1adc1496f9152c893e1f08abcb1e5e7272829899
    Author: Joshua Peek
    Date: Tue Jan 13 16:09:51 2009 -0600

    Add RewindableInput wrapper to fix issues with middleware that impolitely eat up non-rewindable input

    :040000 040000 dea255e41b37ac558b28750a1086ace2c9c35f46 8c6e1795511808984f105270a9ce5949bf128cd4 M actionpack


    Which adds a RewindableInput to the standard Rack stack used internally in Rails. Now when you upload a file this input is not rewindable by itself. So it is read into a StringIO, which means your large file is read into memory.
    The rest is pretty much depends on the GC. After ~700 request to /upload all memory got freed again under mongrel. With passenger memory did not get freed again (tried 4000 additional requests without change in memory consumption).

    Not sure if it's an option for you but the RewindableInput can be removed from the Rack stack which brings back memory to "normal". Not sure about the consequences of removing it. So do it on your own risk... would be nice to get a comment from somebody who knows more about why it was added and which middleware impolitely eats up non-rewindable inputs.

  • Daniel Guettler

    Daniel Guettler May 16th, 2009 @ 02:01 AM

    Ah forgot to tell you how to remove it. Under config/initializers/ add a file containing (or use an existing one):


    This will remove remove RewindableInput from the Rack stack

  • Daniel Guettler

    Daniel Guettler May 26th, 2009 @ 04:23 AM

    This ticket should be closed! RewindableInput has been removed in this commit: which solves the problem described above.

  • Markus Seeger

    Markus Seeger May 26th, 2009 @ 01:52 PM

    I can confirm the fix (removing the RewindableInput).
    Thank you very much!

  • Steve St. Martin

    Steve St. Martin April 16th, 2010 @ 01:27 AM

    • Assigned user set to “Ryan Bigg”

    OP confirms fix, can be marked as resolved.

  • Ryan Bigg

    Ryan Bigg April 16th, 2010 @ 01:28 AM

    • State changed from “new” to “resolved”
  • Ryan Bigg

    Ryan Bigg October 9th, 2010 @ 09:55 PM

    • Tag cleared.

    Automatic cleanup of spam.

  • Jeff Kreeftmeijer

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2 style="font-size: 14px">Tickets have moved to Github</h2>

The new ticket tracker is available at <a href=""></a>