processi

about processes and engines

rufus-tokyo 0.1.9 (Rufus::Edo)

picture-11I have just released rufus-tokyo 0.1.9 (sudo gem install rufus-tokyo)

Rufus-tokyo provides rubyist friendly classes for accessing Tokyo Cabinet and Tokyo Tyrant structures. It does so by binding Tokyo C libraries via FFI and …

Hirabayashi-san, the author of Tokyo Cabinet and Tyrant provides a ‘native’ Ruby library for Tokyo Cabinet and a pure Ruby library for Tokyo Tyrant. My initial motivation for writing rufus-tokyo was that those classes are very C oriented and I wanted to interact in an elegant way with the Tokyo ‘products’.

Another issue was that the author was not providing his libraries via gem install. But this has changed, Dane Jensen is now mirroring the libraries (cabinet, tyrant) at github and releasing them as gems (sudo gem install careo-tokyocabinet careo-tokyotyrant –source http://gems.github.com)

As Ilya Grigorik pointed out FFI is slower than extconf.rb classical Ruby to C bindings. I wanted to benefit from the speed of the ‘native’ bindings.

I have added to rufus-tokyo a Rufus::Edo namespace with 2 classes (cabinet, table) wrapping the ‘native’ ruby gem and 2 classes wrapping the ‘pure’ Ruby classes (cabinet, table) for Tyrant. The ‘pure’ ruby classes for Tyrant, despite being a bit slow are interesting because they don’t require the installation of the Cabinet + Tyrant libraries.

There is a bit of documentation at :

http://github.com/jmettraux/rufus-tokyo/tree/master/lib/rufus/edo

The Rufus::Tokyo classes and the Rufus::Edo classes have the same methods, making it easy to plug one for the other. For instance, in my workflow engine (ruote), I’ve made sure the native bindings are used if present. At instantiation/connection time, the right class (native vs FFI, Edo vs Tokyo) is chosen, the rest of the code is the same.

Here is a tiny decision table for choosing which ruby lib to use :

decision table

If only need a btree or a hash (no tables) served by a Tyrant, a native libmemcached wrapper might be a good option if speed is a must (yes, Tokyo Tyrant speaks ‘memcached’).

I have some very naive benchmarks at http://gist.github.com/71285 (Tokyo Cabinet 1.4.8 and Tyrant 1.1.16). Unfortunately the memcached part is not done with a native libmemcached wrapper.

rufus-tokyo : http://github.com/jmettraux/rufus-tokyo/

Written by John Mettraux

February 27, 2009 at 6:38 am

8 Responses

Subscribe to comments with RSS.

  1. Great stuff John, looking forward to playing with the updated gem on the weekend!

    Ilya Grigorik

    February 27, 2009 at 1:52 pm

  2. Thanks Ilya, suggestions are welcome !

    John Mettraux

    February 27, 2009 at 2:22 pm

  3. Just came across this gem – really wonderful – thanks so much for your work on it, it’ll be a real life-saver as I start diving into the Tokyo Cabinet world!

    Matt Jaynes

    March 19, 2009 at 7:30 am

  4. Thanks Matt !

    John Mettraux

    March 19, 2009 at 7:31 am

  5. Hi John,
    Do you have the source code of your benchmark to share?

    Marko Anastasov

    April 18, 2009 at 2:48 pm

  6. Hi Marko,

    it’s mostly bm0.rb under http://github.com/jmettraux/rufus-tokyo/tree/master/test

    Best regards,

    John Mettraux

    April 18, 2009 at 2:50 pm

  7. […] portable across all Ruby implementations. John, author of rufus-tokyo, made some benchmarks and wrote a more detailed explanation on his blog. I have used rufus-tokyo’s wrapper to the native bindings for file access, and […]

  8. […] Rufus 0.1.9 Release Notes – https://jmettraux.wordpress.com/2009/02/27/rufus-tokyo-018-rufusedo/ […]


Comments are closed.

%d bloggers like this: