Ruby Rack servers benchmark

Facing the question which Ruby Rack server perform best behind Nginx front-end and failing to google out any exact comparison, I decided to do a quick test myself.

The servers:

Later I tried to test UWSGI server too as it now boasts built-in RACK module, but dropped it for two reasons: (1) it required tweaking OS to raise kern.ipc.somaxconn above 128 (which none other server needed) and later Nginx’s worker_connections above 1024 too and (2) it still lagged far behind at ~ 130 req/s, so after successful concurrency of 1000 requests, I got tired of waiting for the tests to complete and gave up seeking it’s break point. Still, UWSGI is very interesting project that I will keep my eye on, mostly because of it’s Emperor and Zerg modes and ease of deployment for dynamic mass-hosting Rack apps.

As UWSGI was originally developed for Python, I wasted a bit of time trying to get it working with some simple Python framework for comparison, but probably lack of knowledge on my part was the failure of it.

Testing

The test platform consisted of:

To set up a basic testcase, I wrote a simple Rack app that responds every request with the request IP address. I dediced to output IP because this involves some Ruby code in the app, but should be rather simple still.

ip = lambda do |env|
  [200, {"Content-Type" => "text/plain"}, [env["REMOTE_ADDR"]]]
end
run ip

Tweaking the concurrency number N (see below) with resolution of 100, I found out the break point of each of the servers (when they started giving errors) and recorded the previous throughput (the one that didn’t give any errors).

Results

The results are as follows:

  1. Unicorn – 2451 req/s @ 1500 concurrent request
  2. Thin – 2102 req/s @ 900 concurrent requests
  3. Passenger – 1549 req/s @ 400 concurrent requests

The following are screenshots from JMeter results:

Unicorn @1500 concurrent request
Thin @900 concurrent requests
Passenger @400 concurrent requests

None of these throughputs are bad, but still Unicorn and Thin beat the crap out of Passenger.

Details

The JMeter testcase

  1. ramp up to N requests concurrently
  2. send request to the server
  3. assert that response contains IP address
  4. loop all of this 10 times

Nginx configuration:

    # Passenger
    server {
      listen 8080;
      server_name localhost;
      root /Users/laas/proged/rack_test/public;
      passenger_enabled on;
      rack_env production;
      passenger_min_instances 4;
    }

    # Unicorn
    upstream unicorn_server {
      server unix:/Users/laas/proged/rack_test/tmp/unicorn.sock fail_timeout=0;
    }

    server {
      listen 8081;
      server_name localhost;
      root /Users/laas/proged/rack_test/public;

      location / {
        proxy_pass http://unicorn_server;
      }
    }

    # Thin
    upstream thin_server{
      server unix:/Users/laas/proged/rack_test/tmp/thin.0.sock fail_timeout=0;
      server unix:/Users/laas/proged/rack_test/tmp/thin.1.sock fail_timeout=0;
      server unix:/Users/laas/proged/rack_test/tmp/thin.2.sock fail_timeout=0;
      server unix:/Users/laas/proged/rack_test/tmp/thin.3.sock fail_timeout=0;
    }

    server {
      listen 8082;
      server_name localhost;
      root /Users/laas/proged/rack_test/public;

      location / {
        proxy_pass http://thin_server;
      }
    }

As is only logical, having processes match the number of cores (dual HT = 4 cores) gave best results for both Thin and Unicorn (thouch the variations were small).

Unicorn configuration

Passenger requires no additional configuration and Thin was configured from command line to use 4 servers and Unix sockets, but Unicorn required a separate file (I modified Unicorn example config for my purpose):

worker_processes 4
working_directory "/Users/laas/proged/rack_test/"
listen '/Users/laas/proged/rack_test/tmp/unicorn.sock', :backlog => 512
timeout 120
pid "/Users/laas/proged/rack_test/tmp/pids/unicorn.pid"

preload_app true
  if GC.respond_to?(:copy_on_write_friendly=)
  GC.copy_on_write_friendly = true
end

Disclaimer

I admit that this is extremely basic test and with better configuration much can be squeezed out from all of these servers, but this simple test surved my purpose and hopefully is of help to others too.

2 thoughts on “Ruby Rack servers benchmark

  1. Please ask for help/info/support in uWSGI ml/irc channel. (without rubysts support the uWSGI rack plugin will always stay behind the others)

    You will be surprised by the values you will get tuning it for ‘hello world’ (read the rack/rails doc on uWSGI site about how the GC is configured by default). About somaxconn it is needed by unicorn and thin too as this is a system-dependent value. You did not noted this as they do not spend 90% of the time running GC as uWSGI by default. Ruby is a memory devourer, and as stated in the uWSGI wiki, being aggressive about this is a design choice. You will discover how for non-hello world app this is a very good choice 🙂 Even the values you get from passenger are lowered by this, as they care memory usage a lot.

    Like

    • Thank you. I had already suspected that the results might vary if I did a “real” test with some complex application. I will give some thought to the app selection and try to get around to testing them again.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s