This project is archived and is in readonly mode.

#3926 ✓stale

Scalability: Lock contention on ActiveRecord::ConnectionAdapters::ConnectionPool.@connection_mutex

Reported by xbaldauf | February 10th, 2010 @ 11:44 PM

In a heavily used multi-threaded rails application (on JRuby), it appears that when CPU usage goes to about 350% (on an 8 core machine), almost all threads (e.g. more than 100 threads) wait on the lock "@connection_mutex". This means that incoming requests cannot be served fast enough anymore.

Nick Sieger provided an ad-hoc work-around in by switching off all connection pooling, but this work-around means a considerable increase of latency, as for each new request a new connection would be needed to be established.

A partial solution would be to reduce the work during the time the lock is held.

A complete solution would be to have multiple connection pools, e.g. as many pools as the system has CPU cores. When a connection is needed, any random pool is chosen and only this pool's lock is acquired. This way, it becomes unlikely that there is lock contention. If the chosen pool happens to be empty, the other pools are checked first before attempting to create a new connection.

An even better solution would be to not use a random pool, but to use the pool whose number is the number of the CPU core the current thread is running on. This way, each pool tends to stay stable in only one CPUs cache, presumably making things very fast.

Comments and changes to this ticket

  • Santiago Pastorino

    Santiago Pastorino February 2nd, 2011 @ 04:44 PM

    • State changed from “new” to “open”

    This issue has been automatically marked as stale because it has not been commented on for at least three months.

    The resources of the Rails core team are limited, and so we are asking for your help. If you can still reproduce this error on the 3-0-stable branch or on master, please reply with all of the information you have about it and add "[state:open]" to your comment. This will reopen the ticket for review. Likewise, if you feel that this is a very important feature for Rails to include, please reply with your explanation so we can consider it.

    Thank you for all your contributions, and we hope you will understand this step to focus our efforts where they are most helpful.

  • Santiago Pastorino

    Santiago Pastorino February 2nd, 2011 @ 04:44 PM

    • State changed from “open” to “stale”

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2 style="font-size: 14px">Tickets have moved to Github</h2>

The new ticket tracker is available at <a href=""></a>

People watching this ticket