ruby-ffi, Tokyo Cabinet

I know, we have a Tyrant but I wanted to focus just on the Cabinet, staying on my side of the network interface.

Tokyo Cabinet is a great [C] library for managing DBM hashes, it’s very efficient, so efficient it’s one of the workhorses behind Mixi, the top japanese social network (damn me, I used the word ‘social’…)

I really wanted to use Tokyo Cabinet from Ruby, I could only find the a datamapper related gem, no ‘sudo gem install’ 1-stop-only option for me. Hirabayashi-san, the original author of Tokyo Cabinet wrote a Ruby lib, but it’s not packaged as a gem, and then I wanted something that looked like a ruby hash, enumerable included.

ruby-ffi is a ruby extension for binding ruby to dynamic libraries, it originated in the Rubinius project and is now available for MRI Ruby (the ‘ruby-ffi’ gem) and JRuby as well.

So, if you have the Tokyo Cabinet library installed on your system, you can venture into :

sudo gem install rufus-tokyo

(it should install its dependency ruby-ffi in the same run)

and then

  require 'rubygems'
  require 'rufus/tokyo'

  db = Rufus::Tokyo::Cabinet.new('data.tch')

  db['nada'] = 'surf'

  p db['nada'] # => 'surf'
  p db['lost'] # => nil

  5000.times { |i| db[i.to_s] = "x" }

  p db.inject { |r, (k, v)| k } # => '4999'

  db.close

will create/open the hash database in “data.tch” and store all its stuff in there. It does it via the dynamic library for Tokyo Cabinet that it looks up in either /opt/local/lib or /usr/local/lib (or another path specified by the environment variable TOKYO_CABINET_LIB).

OK, nothing exciting, just that it’s fast, handy, works on the major Ruby platforms just a ‘sudo gem install’ away.

It’s rather rudimentary for now, but with a bit time…

source : http://github.com/jmettraux/rufus-tokyo
rdoc : http://rufus.rubyforge.org/rufus-tokyo
rest of rufus : http://rufus.rubyforge.org/
mailing list : http://groups.google.com/group/rufus-ruby

Many thanks to the authors of Tokyo Cabinet and ruby-ffi !