about processes and engines

Archive for the ‘jruby’ Category

rufus-tokyo 1.0.0

climaxJust released rufus-tokyo 1.0.0

This is mostly a “cleanup” release (spec reorg, to_s enforcement on key and values, and dropped backward compatibility with older TC/TT releases). Hence the 1.0.0.

There are new features though :

* Matthew King helped me add process method for table queries.

* Jeremy Hinegardner did an initial contribution on the front FFI / Tokyo Dystopia.

* Adam Keys contributed the ext method for the table structures (sorry, I had forgotten).

* full text search operators for table queries (:ftsphrase, :ftsec, :ftsor: ftsand) and the accompanying :token, :qgram and :opt inverted index types. I haven’t played too much with them. Feedback is welcome.

For those of you interested in extending Tokyo Tyrant via Lua functions, I warmly recommend Ilya Grigork’s post about it.

Many thanks to all the people who helped.

There will be a next release soon, as I’m busy trying to catch up with Hirabayashi-san.

rufus-tokyo :

(picture graciously from @michelledasilva)


Written by John Mettraux

July 23, 2009 at 1:24 am

rufus-tokyo 0.1.13

shinjukuWith the initial releases of rufus-tokyo, I happily cut corners and went with C strings (ending with NUL).

This isn’t optimal, often you need to store binary data as the value, the resulting ‘string’ contains NUL characters and values get truncated at restitution. I was working around that with Base64 encoding. Fine, the perf cost isn’t too heavy. But then, binary data was working fine on the Rufus::Edo side (wrapping the original tokyo{cabinet|tyrant}-ruby), why should Rufus::Tokyo be an exception ?

rufus-tokyo 0.1.13 which just got released fixes that issue.

Thanks to Ben Mabey for pointing out the issue.

Kamal Fariz Mahyuddin was kind enough to fork rufus-tokyo and add the #putkeep(k, v) method which only put if there was no previous value associated with the key. The collaboration with Kamal also prompted me to provide #add_int and #add_double for int/double counters in cabinet/tyrant.


Written by John Mettraux

June 2, 2009 at 12:47 am

ruby to lua

luaWith the help from Alain I just released rufus-lua 0.1.0.

This ruby gem talks to Lua via Ruby FFI. One could say it’s “embedding Lua”, as Lua is described as “Lua is a powerful, fast, lightweight, embeddable scripting language.”

I feel tempted by Lua, there’s the speed, the austerity and that javascript feeling.


Rufus-lua is not the first bridge between Lua and Ruby, there are also :


They are worth a look, but I was a bit disappointed, they are not available via ‘sudo gem install …’.

Rufus-lua is available via Rubygems, but since it’s a FFI thing, it needs to talk to the dynamic library packaging of Lua. Well, it’s not available by default, there’s some work involved (but not too much).

Something great about FFI is that it works for Ruby 1.8.6, Ruby 1.9.1 and JRuby (1.1.6) (and will work soon for Rubinius, its original platform).


Once liblua.dylib is on your system, along with rufus-lua (sudo gem install rufus-lua), you can play with code like :

require 'rubygems'
require 'rufus/lua'

s =

t = s.eval(%{
  return { message = { 'hello', 'from', 'Lua' },
    funk = function (x) return 2^x end }

p t['message'].to_a # => ["hello", "from", "lua"]
p t['funk'].call(8) # => 256.0


Well, it’s only a 0.1.0 release…


As rufus-lua is using FFI it’s slower at the “frontier”, here is a naive benchmark for computing a Fibonacci suite, with Fiber (Ruby 1.9) and Coroutines on the Lua side :

require 'benchmark'
require 'rubygems'
require 'rufus/lua'

  n1 = n2 = 1
  loop do
    Fiber.yield n1
    n1, n2 = n2, n1+n2

s = %{
  co = coroutine.create(function ()
    local n1 = 1; local n2 = 1
    while true do
      n1, n2 = n2, n1+n2
  return co

LUAFIBS = LUA.eval(s)

N = 10_000
Benchmark.benchmark(' ' * 31 + Benchmark::Tms::CAPTION, 31) do |b|'ruby') do
    N.times { RUBYFIBS.resume }
  end'lua via ruby') do
    N.times { LUAFIBS.resume }
  end'lua') do
    LUA.eval("for i = 0, #{N} do coroutine.resume(co) end")


For the original. Running it on my board yields something along the lines of :

$ ruby19 test/bm_fiber.rb
                  user     system      total        real
ruby          0.050000   0.010000   0.060000 (  0.052032)
lua via ruby  0.180000   0.000000   0.180000 (  0.195411)
lua           0.010000   0.000000   0.010000 (  0.006394)

Well, nothing spectacular, that just states that it’s better to not cross the border too often : the “lua via ruby” bench has Ruby resuming the Lua coroutine 10_000 times and it’s costly.


on github :
rdoc :
the rest of rufus :

lua :
ffi :


Written by John Mettraux

March 16, 2009 at 9:39 am

Posted in ffi, jruby, lua, ruby, rufus

ruby-ffi, Tokyo Cabinet

tcabinetI know, we have a Tyrant but I wanted to focus just on the Cabinet, staying on my side of the network interface.

Tokyo Cabinet is a great [C] library for managing DBM hashes, it’s very efficient, so efficient it’s one of the workhorses behind Mixi, the top japanese social network (damn me, I used the word ‘social’…)

I really wanted to use Tokyo Cabinet from Ruby, I could only find the a datamapper related gem, no ‘sudo gem install’ 1-stop-only option for me. Hirabayashi-san, the original author of Tokyo Cabinet wrote a Ruby lib, but it’s not packaged as a gem, and then I wanted something that looked like a ruby hash, enumerable included.

ruby-ffi is a ruby extension for binding ruby to dynamic libraries, it originated in the Rubinius project and is now available for MRI Ruby (the ‘ruby-ffi’ gem) and JRuby as well.

So, if you have the Tokyo Cabinet library installed on your system, you can venture into :

sudo gem install rufus-tokyo

(it should install its dependency ruby-ffi in the same run)

and then

  require 'rubygems'
  require 'rufus/tokyo'

  db ='data.tch')

  db['nada'] = 'surf'

  p db['nada'] # => 'surf'
  p db['lost'] # => nil

  5000.times { |i| db[i.to_s] = "x" }

  p db.inject { |r, (k, v)| k } # => '4999'


will create/open the hash database in “data.tch” and store all its stuff in there. It does it via the dynamic library for Tokyo Cabinet that it looks up in either /opt/local/lib or /usr/local/lib (or another path specified by the environment variable TOKYO_CABINET_LIB).

OK, nothing exciting, just that it’s fast, handy, works on the major Ruby platforms just a ‘sudo gem install’ away.

It’s rather rudimentary for now, but with a bit time…

source :
rdoc :
rest of rufus :
mailing list :

Many thanks to the authors of Tokyo Cabinet and ruby-ffi !

Written by John Mettraux

January 23, 2009 at 8:13 am

Posted in jruby, ruby, rufus, tokyocabinet

Fujikawa-san on JRuby at Ninjava

I attended a talk by Kouichi Fujikawa at the June Ninjava meeting.

Was an excellent warm-up before the RubyKaigi 2008.

Sorry for the hashed post, “live blogging” before the post-talk beers.

– Fujikawa-san is working for a company developing finance derivatives trading systems
– had the chance to work with ThoughtWorks China
– went to JavaOne 2008
– met Thomas E Enebo, Charles Oliver Nutter, Nick Sieger, Ola Bini

– JRuby supported by Sun Microsystems (especially Tim Bray) and ThoughtWorks
– JRuby is ‘performant’
– good way to get Ruby into the enterprise

– small demo… (classical JFrame demo)

– leveraging JDBC for database support (many drivers, drivers with connection pooling)
– integration of Java libraries (new systems and legacy services)
– at JavaOne08, Ola Bini presented a SpellCheck java library used in a JRuby on Rails application
– Warbler : turns a Rails app into a Java .war
– JRuby Rack (included in Warbler) integration of the Rack Ruby (low level) web framework

– next some slides courtesy of Nick Sieger
— may JRuby interpreters in 1 JVM
— Glassfish [web] app server
— servlet context initialization creates JRuby runtimes and pool them (acquire/dispatch/release)

– more information :
— book by Ola Bini “Practical JRuby on Rails (japanese, about JRuby and Haskell)

– then questions from the audience :
— Zev asking if threads do really magically make things better (more scalable), answer by Lars was mainly about JRuby being able to leverage ‘native’ threads thus multicore
— questions about JRuby and ObjectSpace (various tries between jirb and irb do follow)

The talk by Fujikawa-san was followed by a presentation of iKnow! by Zev Blut :

– “developing and scaling iKnow!”
– (stopping here as he will present this talk on Sunday at the RubyKaigi)

Written by John Mettraux

June 19, 2008 at 12:00 pm

Posted in java, jruby, ninjava, opensource, ruby

webdavhandler.rb on jruby 1.1[.1]

There is this hidden gem in Ruby land, the webrick webdav handler.

I’ve been using it to power some webdav accesses to an application here. When the time arrived to upgrade from JRuby 1.1 “pre from last fall” to JRuby 1.1[.1], this webdav handler broke.

jmettraux@sonora:~/tmp$ ~/jruby-1.1.1/bin/jruby dav.rb
[2008-04-24 10:09:24] INFO  WEBrick 1.3.1
[2008-04-24 10:09:24] INFO  ruby 1.8.6 (2008-04-22) [java]
[2008-04-24 10:09:24] INFO  WEBrick::HTTPServer#start: pid=3998 port=10080
[2008-04-24 10:09:37] ERROR ArgumentError: wrong number of arguments (4 for 3)
        /home/jmettraux/tmp/./webdavhandler.rb:135:in `chk_utf8'
        /home/jmettraux/tmp/./webdavhandler.rb:135:in `detect'

Two small changes to webdavhandler.rb fixed this issue and a subsequent issue.

The updated webdavhandler.rb is available.

Written by John Mettraux

April 24, 2008 at 1:46 am

Posted in jruby, ruby

on the JRuby front

jruby.pngNicolas has been tracking JRuby and testing OpenWFEru on this platform. He had submitted a few patches that turned “OpenWFEru on JRuby” from a big NO to a firm Yes. A business process engine is quite a demanding beast.

This week, JRuby seems to have made big progresses (we test with JRuby trunk, JRuby 1.0.1 is a “no go”), OpenWFEru almost runs perfectly on it (except for a few baroque test cases). The performance are good (slightly lesser).

See Nicolas’ post for the full details.

Time to think about giving true flexibility to some business processes, currently hard-coded in Java here at work.

Written by John Mettraux

September 21, 2007 at 4:43 am

Posted in java, jruby, openwferu