Archive for category Rails

(Do not) Use Rails cache as a distributed lock system

You should strive to design your application in a way that eliminates the need of critical sections as possible.

It is not robust, it adds layers of complexity and is VERY difficult to test.

You can avoid such burdens by using alternative threading constrains such as actors

Having said that there are still cases in a large distributed environment that you need to use Distributed lock manager in order to wrap a critical section of your flow.

It is preferable to use event based distributed locking – can be achieved with Zookeeper (based on Google’s Chubby)

I will show here a very simple locking mechanism based on Rails cache store.

You can set a shared cache between all your rails processes by setting config/environments/#{env}.rb

In the example I use Dalli (excellent memcached client by Mike Perham) but any store that is shared between the machines will work.

You can scroll to the bottom to see the CacheMutex spec and code

Usage is simple:

As you can see, the main disadvantage here is using a sleep-lock instead of event to notify the waiting thread.

CacheMutex source code:

<Edit>

Due to potential race conditions scenario identified by http://en.gravatar.com/cryo28 I switched to provided lock abilities of Redis-Objects https://github.com/nateware/redis-objects#lock

</Edit>

, , , , , ,

2 Comments

ActiveRecord default_scope

ActiveRecord, starting at v2.3.2, provides with a convenient way to perform default scoping on a model. This feature must be handled with extra thought before applying. Let’s look at the following class:

class Post < ActiveRecord::Base
default_scope where(:deleted_at => nil)
end

We’re going to use a nice trick here in order to display the actual query rails generates for us:
ActiveRecord::Base.logger = Logger.new(STDOUT)

Now let’s start playing with our Post model a little bit:

:046 >; Post.all
SELECT `posts`.* FROM `posts` WHERE `posts`.`deleted_at` IS NULL
=>; []

As expected, activerecord chained the default scope into our initial query.

:047 >; Post.unscoped
SELECT `posts`.* FROM `posts`
=>; []

The unscoped method rips the model of any chained scopes it has been applied with.

To make things a little bit more interesting I’m adding another scope to the Post model:

class Post
scope :id_is_one, lambda { where(:id =>; 1) }
end

and a post(:id =>; 1, :deleted_at =>; Time.now) to the database.

Back to rails console:

:060 >; Post.id_is_one
SELECT `posts`.* FROM `posts`
WHERE `posts`.`deleted_at` IS NULL AND `posts`.`id` = 1
=>; []

Returns an empty result since our post does have a delete_at field which is different than nil.

:061 >; Post.unscoped.id_is_one
SELECT `posts`.* FROM `posts`
WHERE `posts`.`deleted_at` IS NULL AND `posts`.`id` = 1
=>; []

What just happened here? didn’t we unscope our Post model before querying for a post with id 1? We did, but it looks like rails just ignored it and chained the default_scope again.

More interestingly:

:062 >; Post.unscoped.where(:id =>; 1)
SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 1
=>; [#;]

Replacing the named scope with a simple where clause of ActiveRecord does the job.
So, what do we have so far?
1. Chaining an unscoped with named scopes does not supply with expected results.
2. Chaining unscoped with ActiveRecord query methods does supply with expected results.

This raises the obvious question: Can’t we use named scopes ignoring the default scope of a model? And the answer is: yes, we can! (sounds familiar?!) How? It is very simple:

:063 >; Post.unscoped { Post.id_is_one }
SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 1
=>; [#;]

Passing the desired query inside a block does the job as expected.

To conclude: Dealing with default_scope may look appealing at first, but may also give you some hard times when you want to ignore it. It’s efficiency decreases the more times you find yourself trying to ignore it, so make a serious consideration before going this slippery slope.

, , ,

Leave a comment

git, migrations and everything in between

Simultaneously working on multiple branches with git could be quite confusing at times. Very confusing.

Recently I found myself in a very strange situation: I changed some old migration file to suit a change of requirements in the application. After changing it and in order to apply the changes to the DB I ran

rake db:migrate:redo

Which should have performed rollback and migrate on the last migration.

What did I get in response you ask? A new, fresh shell prompt line and an unchanged DB. Not very informative of rake isn’t it?

Next step in investigating the mystery I’ve decided to let rake tell me what hurts him:

rake --trace db:rollback

This time the response was better, still not really indicating the problem:

** Invoke db:rollback (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:rollback
** Invoke db:schema:dump (first_time)
** Invoke environment
** Execute db:schema:dump

If that’s the same response you’re getting, most chances that you have migrated your DB on another git branch using migration X, and then switched back to your current branch, in which the migration file X does not exist.

What you should look for is the schema_migrations table on your DB, and compare it against your db/migrate/* files. The table lists the various versions of your DB and it should match the timestamps on your migration files. A mismatch between the two sets of data will reproduce the responses I got from rake.

The cleanest solution for this is to switch back to the old branch and rollback your DB to a common version with your current branch (you can perform multiple rollbacks with rake db:rollback STEP=n, where n is the number of consecutive rollbacks you’d like to perform),  then switch back to the branch you’re working on and viola! rake responds as expected again and you are free to migrate your DB up and down as you wish.

, , ,

Leave a comment

Our version of mysql backup using rake

https://gist.github.com/1258918

Leave a comment

Delayed Jobs troubleshooting – when all delayed jobs processes are dead

– ssh to the server that suppose to run the jobs,
1. Check that there is free disk space
2. Go to application root
3. Try running delayed job with

RAILS_ENV=production script/delayed_job start -n 8

4. Get a clue regarding the failure reason from

tail log/delayed_job

— Probably there is some data corruption in the delayed jobs table in database – cleaning it will probably solve the problem

, ,

Leave a comment

Short testing feedback in rails 3

Good TDD requires Fast Feedback

When practicing TDD, testing feedback time should be super fast. You cannot sit and wait for your tests to run minutes before getting your red / green notification.

Tests should not be IO bound (IO == slow), you should use in memory db / preloaded data.

Another factor is the environment start-up time.
In rails 3 environment, sometimes for applications with heavy gem dependencies, loading time can be very long. Even up to 1 minute in some cases. We cannot accept that when it comes to test that depends on rails environment to be up. What can we do?

Spork & Guard to the rescue

Spork is using fork of rails environment in the background. You can configure your RSpec / Test::Unit to use the preloaded environment using drb (ruby rpc).

But what if you change initialization files? -changes will only affect next rails start-up

For this task guard, which uses fs events to monitor changes in files can be configured to inflict spork reload of rails environment.

The following post describes well how to install and configure Spork & Guard with RSpec

Next step: autotest

Now that the environment is preloaded we are nearly done.

But… why should we run rspec manually each time we wish to run test? – if we are practicing TDD well we probably wish to run the tests VERY often.

‘autotest’ uses fs-event to listen to file modification within our rails directory tree and to run the tests each time we SAVE a pre-configured file (e.g. spec files)

growl is a nice tool which shows a visual notification of the testing status green / red with number of failures and total number of test run.

We find this setting to work fine:
In your Gemfile (test group)

gem 'webrat'
gem 'ZenTest'
gem 'autotest-rails'
gem 'autotest-growl'
gem 'autotest-fsevent'

and in your .autotest file (rails root):

require "ZenTest"
require "autotest/rails"
require "autotest/growl"
require "autotest/fsevent"
Autotest.add_hook :initialize do |autotest|
  %w{.git .svn .hg .DS_Store ._* vendor tmp log doc config .rvmrc Gemfile .autotest README Rakefile}.each do |exception|
    autotest.add_exception(exception)
  end
end

Side note: Continuous testing vs. Compilation

When you are getting use to test drive you code, it starts feeling similar to compiling your code (if you are using a language that require compilation ;-))

Auto-testing inline notification is pretty much like modern IDEs notifying syntax errors and compilation errors before you run your build.
The major difference is that compilation only tells you the code can run, whereas testing provides evidence that it actually work!!

, ,

Leave a comment

Oh my god! where all my digits went?

Case 1: decimals
You are troubled to find out that some of your precision is lost…

1. First place to look is whether you are using the correct number class.

2. next place to look is db column definition
decimal 10,4 for instance, will support 6 digits number with 4 digits after the decimal point.
Don’t get too depressed, you can easily migrate your column to support more digit numbers:


class ExpandDecimals < ActiveRecord::Migration
def self.up
change_column :[table], :[column], :decimal, :precision => 12, :scale => 2
end

def self.down
change_column :[table], :[column], :decimal, :precision => 10, :scale => 4
end
end

Case 2: blob
You are using a table column to store data blob and it’s too short to contain all data?

Migration can be something like:

class ExpandLongtext < ActiveRecord::Migration
def self.up
change_column :[table], :[column], :text, :limit => 16777215
end

def self.down
change_column :[table], :[column], :string, :limit => 255
end
end

 

Migration is very easy and makes your life easy.

Use it.

, ,

Leave a comment