Tag: ruby

RubyMine high CPU, no background tasks

LaasLeave a comment

RubyMine, being an IDE and not light editor, is expected to hog some CPU. But after each RSpec run, the CPU usage jumps to 300 % for several minutes. If I do fast iteration on specs, then this high CPU usage is constant and heats up my Mac, causing fan noise and of course drains battery.

Screen Shot 2017-08-23 at 17.13.19.png

Googling around for solution did not find anything, especially as there are not background processes running. I even tried profiling the app to see if it gives any  ideas (hint: it did not).

One support ticket for WebStorm though tipped me off and turned out that the culprit are Inspections – those little yellow or red markers around right gutter that tell you when your code smells or is outright broken.

 

Handpick the Inspections

Screen Shot 2017-08-23 at 17.11.58.png

Navigate to Preferences > Editor > Inspections and go through the long list. Unfortunately RubyMine does not tell you which Inspection it currently runs, so it is trial and error to find out what causes the slowness.

In my case there were a lot of “Double quotes” as well as whitespace whitespace warnings, but growing tired of not finding any low hanging fruits, I decided to just disable almost all warnings and left only error-level Inspections.

Now, after running specs, RubyMine CPU usage jumps to around 100 % for only a little while and then returns to normal.

 

A note on tabs

RubyMine has default setting of allowing only a handful of tabs open at the same time. Increasing this limit affects, among other things, the Inspections performance too as these are run for all tabs after every spec run. (Don’t ask me why JetBrains thinks this is necessary.)

Faster RubyMine RSpec with Vagrant

LaasLeave a comment

I use RubyMine and Vagrant for my Rails development and run RSpec specs from RubyMine for convenience. The problem is that every time I start rspec, it takes a few seconds to connect to the box, before actually doing anything. This is especially annoying when running single spec which should be fast.

vagrant ssh is slow to connect

% time vagrant ssh -c whoami
vagrant
Connection to 10.211.55.45 closed.
2,57s user 0,73s system 82% cpu 3,984 total

That’s probably because vagrant ssh-config takes most of that time to sort things out:

% time vagrant ssh-config > /dev/null
2,53s user 0,79s system 77% cpu 4,269 total

When researching this issue, some people mention DNS or other Vagrant provider specific issues, but they complain of an order of magnitude slower (30-40 sec) connection. Might just be that I have everything set up correctly and it just takes that much time to get the configuration.

But what this means is that if you run RSpecs from RubyMine, they are slow to start, because RubyMine executes vagrant ssh-config every time (RUBY-16186).

Direct SSH to the rescue

Fortunately, direct SSH with keypair authentication is blazing fast:

time ssh vagrant@10.211.55.45 -i ~/.ssh/parallels_key whoami
vagrant
0,01s user 0,01s system 21% cpu 0,114 total

So, to take advantage of that you need to:

  1. set up SSH configuration
  2. configure RubyMine remote ruby over SSH, not Vagrant
  3. Add RubyMine helpers to RUBYLIB to be able to start persistent spring instance.

Set up SSH configuration

Vagrant might be using their vagrant_insecure_private_key for the SSH or, in my case, Parallels custom keypair, as I don’t use VirtualBox. So run this command to find out which key is in use:

% vagrant ssh-config
Host default
  HostName 10.211.55.45
  User vagrant
  Port 22
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /Users/laas/project/.vagrant/machines/default/parallels/private_key
  IdentitiesOnly yes
  LogLevel FATAL

Now we know the IP address of the box and what keyfile is used. For easier access, I symlinked the key file to my .ssh folder:

ln -s IdentityFile /Users/laas/project/.vagrant/machines/default/parallels/private_key ~/.ssh/parallels_key

You can try out SSH connection:

ssh vagrant@10.211.55.45 -i ~/.ssh/parallels_key whoami
vagrant

Configure RubyMine to use remote ruby over SSH

Next step is to configure new Ruby SDK for the project. Open Preferences and navigate Language & Frameworks > Ruby SDK and Gems, click + button and pick New remote…. Then fill in the data as shown in the screenshot and when closing dialog, ensure that the newly added ruby is selected as active.

Configure Remote Ruby Interpreter

Now specs should still run (but take several seconds every time to boot up Rails, because spring is shut down after every connection).

Note: I actually use landrush to manage development boxes names and so I can configure remote ruby with a domain name and not worry about IP address changes when recreating the devbox.

Add RubyMine helpers to RUBYLIB

The problem above is that spring closes itself down every time SSH connection is closed. One could start a long-running spring from terminal SSH, but that would result in RubyMine complaining that it can not load teamcity formatter:

cannot load such file -- teamcity/spec/runner/formatter/teamcity/formatter (LoadError)

This is because RubyMine injects a special formatter into rspec so that it can parse the output better. Thanks to Oleg at Jetbrains (RUBY-16324) I discovered that the required formatters are present in the vagrant box and all I need is to add them to ruby load path, before starting spring.

So, I added following lines to my /home/vagrant/.bashrc:

# RubyMine-RUBYLIB
if [[ -d "$HOME/.rubymine_helpers" ]]
then
  export RUBYLIB=$HOME/.rubymine_helpers/rb/testing/patch/common/:$HOME/.rubymine_helpers/rb/testing/patch/bdd/:$RUBYLIB
fi

This sets up RUBYLIB variable with RubyMine helpers. I skipped over testunit folder, as I do not use that, but feel free to add that also if you need it.

Let the specs fly

Now, just SSH into your Vagrant box, start Spring, e.g. by running:

bin/rspec --help

And then RubyMine can re-use the already running Spring server to speed itself up.

Rails redirect_back_or_default

LaasLeave a comment

In a recent project I found myself writing recirect_to :back alot, and then found myself worrying that what if for some reason there is no :back.

Drawing inspiration from this blog, I wrote two helpers in my application_controller.rb.

The store_location stores current URI (or referer URI in case of non-GET request) into session[:return_to] for later usage:

#!ruby
def store_location
  session[:return_to] = if request.get?
    request.request_uri
  else
    request.referer
  end
end

And redirect_back_or_default tries its best to redirect the user to somewhere, in the following order:

  1. previously stored session[:return_to]
  2. Referer URI
  3. Given default URI
  4. or root_url if all else fails

The code itself

#!ruby
def redirect_back_or_default(default = root_url, options)
  redirect_to(session.delete(:return_to) || request.referer || default, options)
end

I’ve found that when rewriting redirect_to :back, notice: 'something' into redirect_back_or_default-call, adding this alias helps:

#!ruby
alias_method :redirect_to_back_or_default, :redirect_back_or_default

But of course, if you are testing your code (and you should be), it’s better to stick to one variant of above and use tests to catch all erroneous incarnations.

Rails 3: Merge scopes

LaasLeave a comment

I run into a case where I had User.search method and I wanted the GroupMember model be searchable by the user’s attributes. The most DRY way to accomplish this in Rails 3 is to merge scopes. In the User model:

#!ruby
# user.rb
class User < ActiveRecord::Base
  has_many :memberships, :class_name => "GroupMember", :foreign_key => "user_id"

  def self.search(search)
    if search.present?
      query = []
      params = []
      %w(uid email name).each do |field|
        # The field name must be fully qualified to merge scopes
        query << "#{self.table_name}.#{field} LIKE ?"
        params << "%#{search}%"
      end
      query = query.join(" OR ")
      where(query, *params)
    else
      scoped
    end
  end
end

NB! It’s important to have the User’s field names fully qualified so that they won’t be applied to the GroupMember table. And in the GroupMember model:

#!ruby
# group_member.rb
class GroupMember < ActiveRecord::Base
  belongs_to :user
  belongs_to :group

  def self.search(search)
    if search.present?
      # We search GroupMembers by the user attributes
      scoped.joins(:user).merge(User.search(search))
    else
      scoped
    end
  end
end

Now it’s possible to search for GroupMembers by the User attributes:

#!ruby
group = Group.find 1
group.group_members.search('david')

This results in SQL query:

#!sql
SELECT "group_members".* FROM "group_members" INNER JOIN "users"
ON "users"."id" = "group_members"."user_id" WHERE "group_members"."group_id" = 1
AND (users.uid LIKE '%david%' OR users.email LIKE '%david%'
OR users.name LIKE '%david%')

uses of Ruby’s Object#tap

LaasLeave a comment

Ruby 1.9’s Object#tap method has always seemed useful to me, but until now I hadn’t met the chance to use it. Every other time it seemed like abusing it in some way.

Now I came to an old code. Consider this:

cgi.text_field( "name" => "myfield",
                "value" => value,
                "size" => 20,
                "maxlength" => maxlength
              )

I needed to turn that maxlength into a conditional attribute (omiting it if it’s nil).
One way would have been extracting the attributes into a separate variable:

attrs = {
          "name" => "myfield",
          "value" => value,
          "size" => 20
        }
attrs['maxlength'] = maxlength if maxlength
cgi.text_field( attrs )

But this separates visually the cgi.text_field() call from it’s arguments, which I don’t like. Tap to the resque:

cgi.text_field({  "name" => "myfield",
                  "value" => value,
                  "size" => 20
                }.tap{|attrs| 
                  attrs['maxlength'] = maxlength if maxlength
                })

Now, isn’t that nice! (OK, maybe it isn’t, but at least it is encompassed in the method call and makes it easy to spot all attributes).

Ruby (on Rails) toolchest for Windows users

LaasLeave a comment

Setting up solid Ruby on Rails developer box based on Windows can be tedious task. More so than on other platforms, because vanilla Windows is meant for end user and lacks proper development tools that exist on other platforms. But fear not, there are many good people out there that have jumped through multitude of hoops to get different parts of the ecosystem working. All that remains is to build a solid foundation for developmer from them.Read More »

Ruby Rack servers benchmark

Laas2 Comments

Facing the question which Ruby Rack server perform best behind Nginx front-end and failing to google out any exact comparison, I decided to do a quick test myself.

The servers:

Later I tried to test UWSGI server too as it now boasts built-in RACK module, but dropped it for two reasons: (1) it required tweaking OS to raise kern.ipc.somaxconn above 128 (which none other server needed) and later Nginx’s worker_connections above 1024 too and (2) it still lagged far behind at ~ 130 req/s, so after successful concurrency of 1000 requests, I got tired of waiting for the tests to complete and gave up seeking it’s break point. Still, UWSGI is very interesting project that I will keep my eye on, mostly because of it’s Emperor and Zerg modes and ease of deployment for dynamic mass-hosting Rack apps.

As UWSGI was originally developed for Python, I wasted a bit of time trying to get it working with some simple Python framework for comparison, but probably lack of knowledge on my part was the failure of it.

Testing

The test platform consisted of:

To set up a basic testcase, I wrote a simple Rack app that responds every request with the request IP address. I dediced to output IP because this involves some Ruby code in the app, but should be rather simple still.

ip = lambda do |env|
  [200, {"Content-Type" => "text/plain"}, [env["REMOTE_ADDR"]]]
end
run ip

Tweaking the concurrency number N (see below) with resolution of 100, I found out the break point of each of the servers (when they started giving errors) and recorded the previous throughput (the one that didn’t give any errors).

Results

The results are as follows:

  1. Unicorn – 2451 req/s @ 1500 concurrent request
  2. Thin – 2102 req/s @ 900 concurrent requests
  3. Passenger – 1549 req/s @ 400 concurrent requests

The following are screenshots from JMeter results:

Unicorn @1500 concurrent request
Thin @900 concurrent requests
Passenger @400 concurrent requests

None of these throughputs are bad, but still Unicorn and Thin beat the crap out of Passenger.

Details

The JMeter testcase

  1. ramp up to N requests concurrently
  2. send request to the server
  3. assert that response contains IP address
  4. loop all of this 10 times

Nginx configuration:

    # Passenger
    server {
      listen 8080;
      server_name localhost;
      root /Users/laas/proged/rack_test/public;
      passenger_enabled on;
      rack_env production;
      passenger_min_instances 4;
    }

    # Unicorn
    upstream unicorn_server {
      server unix:/Users/laas/proged/rack_test/tmp/unicorn.sock fail_timeout=0;
    }

    server {
      listen 8081;
      server_name localhost;
      root /Users/laas/proged/rack_test/public;

      location / {
        proxy_pass http://unicorn_server;
      }
    }

    # Thin
    upstream thin_server{
      server unix:/Users/laas/proged/rack_test/tmp/thin.0.sock fail_timeout=0;
      server unix:/Users/laas/proged/rack_test/tmp/thin.1.sock fail_timeout=0;
      server unix:/Users/laas/proged/rack_test/tmp/thin.2.sock fail_timeout=0;
      server unix:/Users/laas/proged/rack_test/tmp/thin.3.sock fail_timeout=0;
    }

    server {
      listen 8082;
      server_name localhost;
      root /Users/laas/proged/rack_test/public;

      location / {
        proxy_pass http://thin_server;
      }
    }

As is only logical, having processes match the number of cores (dual HT = 4 cores) gave best results for both Thin and Unicorn (thouch the variations were small).

Unicorn configuration

Passenger requires no additional configuration and Thin was configured from command line to use 4 servers and Unix sockets, but Unicorn required a separate file (I modified Unicorn example config for my purpose):

worker_processes 4
working_directory "/Users/laas/proged/rack_test/"
listen '/Users/laas/proged/rack_test/tmp/unicorn.sock', :backlog => 512
timeout 120
pid "/Users/laas/proged/rack_test/tmp/pids/unicorn.pid"

preload_app true
  if GC.respond_to?(:copy_on_write_friendly=)
  GC.copy_on_write_friendly = true
end

Disclaimer

I admit that this is extremely basic test and with better configuration much can be squeezed out from all of these servers, but this simple test surved my purpose and hopefully is of help to others too.

Inkscape CSV merge

LaasLeave a comment

I just uploaded inkscape_merge gem v0.1.0.

This is a script to merge SVG files with CSV data-files using Inkscape, to produce one outputfile (e.g. PDF) per data-row.

Script inspired by and based on Aurélio A. Heckert excellent InkscapeGenerator (wiki.colivre.net/Aurium/InkscapeGenerator)

Heckert’s original script unfortunately broke for me several times and I took the opportunity to rewrite it and make it more extendable for future.

 

USAGE

Install the gem

gem install inkscape_merge

Create files

Create CSV data file with first row as a header. The values from this row are used as keys in the SVG file substitution.

Create SVG file that contains some variables in the form:

%VAR_name%

Where `name` is the name of a column in the CSV file created previously. These variables can be anywhere inside the SVG, from plain text nodes to color values. This script just brute-forcedly `gsubs` these values as text w/o any thought.

Run the script

The script requires at least three arguments:

  • the input SVG file
  • the input CSV file
  • and the output file `pattern`

Note: output pattern undergoes the same substitutions as the SVG file, so to create easily unique file names. Additionally the output pattern can contain `%d` which is replaced with current row number.

Example:

inkscape_merge -f postcard.svg -d names.csv -o postcards/card_%d.pdf

This produces files like:

  • postcards/
    • card_1.pdf
    • card_2.pdf