Rails Ruby on Rails

In a heavily used multi-threaded rails application (on JRuby), it appears that when CPU usage goes to about 350% (on an 8 core machine), almost all threads (e.g. more than 100 threads) wait on the lock "@connection_mutex". This means that incoming requests cannot be served fast enough anymore.

Nick Sieger provided an ad-hoc work-around in http://gist.github.com/300782 by switching off all connection pooling, but this work-around means a considerable increase of latency, as for each new request a new connection would be needed to be established.

A partial solution would be to reduce the work during the time the lock is held.

A complete solution would be to have multiple connection pools, e.g. as many pools as the system has CPU cores. When a connection is needed, any random pool is chosen and only this pool's lock is acquired. This way, it becomes unlikely that there is lock contention. If the chosen pool happens to be empty, the other pools are checked first before attempting to create a new connection.

An even better solution would be to not use a random pool, but to use the pool whose number is the number of the CPU core the current thread is running on. This way, each pool tends to stay stable in only one CPUs cache, presumably making things very fast.