moving to lambda.io/jmettraux/

This blog is moving to https://jmettraux.skepti.ch

Meet you there.

rufus-scheduler 2.0.12 released

rufus-scheduler is a thread-based scheduler written in Ruby. It lets you write code like:

require 'rufus-scheduler'

s = Rufus::Scheduler.start_new

s.every '10m' do
  puts 'open the window for the cat'
end

s.at '2012-01-01 12:00' do
  puts 'reminder: wife's birthday'
end

s.cron '0 22 * * 1-5' do
  puts 'activate the security system'
end

s.join # in case of stand-alone script...

The main addition brought by this release is the :mutex attribute when scheduling blocks of code. I was seeing people misusing :blocking => true to exclude block execution overlapping. It works but the scheduler is blocked as well, and crons might get skipped:

s.every '10m', :blocking => true do
  puts 'doing this...'
  sleep 60 * 60 # 1 hour
  puts 'done.'
end

# if the scheduler is in the blocking task above, crons will get skipped...
s.at '2012-01-01 12:00' do
  puts 'do that.'
end

My advice was to use mutexes instead:

$m = Mutex.new

s.every '10m' do
  $m.synchronize do
    puts 'doing this...'
    sleep 60 * 60 # 1 hour
    puts 'done.'
  end
end

# if the scheduler is in the blocking task above, crons will get skipped...
s.at '2012-01-01 12:00' do
  $m.synchronize do
    puts 'do that.'
  end
end

For those of you who use such mutexes and are OK with them wrapping the whole block, rufus-scheduler 2.0.12 introduces the :mutex attribute:

s.every '10m', :mutex => 'my_mutex_name' do
  puts 'doing this...'
  sleep 60 * 60 # 1 hour
  puts 'done.'
end

# if the scheduler is in the blocking task above, crons will get skipped...
s.at '2012-01-01 12:00', :mutex => 'my_mutex_name'  do
  puts 'do that.'
end

Where rufus-scheduler receives a mutex name and manages it for you.

When one wants more control over the granularity, it’s OK to do:

$m = Mutex.new

s.every '10m', :mutex => $m do
  puts 'doing this...'
  sleep 60 * 60 # 1 hour
  puts 'done.'
end

# if the scheduler is in the blocking task above, crons will get skipped...
s.at '2012-01-01 12:00' do
  puts 'do that'
  $m.synchronize do
    puts 'and that.'
  end
end

 

Remember that rufus-scheduler is not a cron replacement. Many thanks to all the people who complained or helped in the development of this piece of software over the years.

 

source: https://github.com/jmettraux/rufus-scheduler
issues: https://github.com/jmettraux/rufus-scheduler/issues
mailing list: http://groups.google.com/group/rufus-ruby
irc: freenode #ruote

 

Parslet and JSON

Parslet is a small Ruby library for constructing parsers based on Parsing Expression Grammars (PEG). It’s written by Kaspar Schiess and various contributors.

This blog post introduces Parslet with a parser example. Since JSON has very easy to grasp railroad diagrams for its syntax, it might make for a good example.

Please note that the JSON parser here won’t compete for speed with available libraries. No benchmarks here.

Our goal is to take as input JSON strings and output the resulting value.

For the impatient, the end result is at https://gist.github.com/966020

How is an array encoded in JSON ?

How would that look in our parser ?

  class Parser < Parslet::Parser

    rule(:spaces) { match('\s').repeat(1) }
      # at least 1 space character (space, tab, new line, carriage return)

    rule(:spaces?) { spaces.maybe }
      # a bunch of spaces or not

    rule(:comma) { spaces? >> str(',') >> spaces? }
      # a comma surrounded by optional spaces

    rule(:array) {
      str('[') >> spaces? >>
      (value >> (comma >> value).repeat).maybe.as(:array) >>
      spaces? >> str(']')
    }
  end

What is this value thing ?

string or number or object or …

    rule(:value) {
      string | number |
      object | array |
      str('true').as(:true) | str('false').as(:false) |
      str('null').as(:null)
    }

All is good, a few parsing rules laters, we have a complete JSON parser, but wait, what does it output ?

p MyJson::Parser.new.parse(%{
  [ 1, 2, 3, null,
    "asdfasdf asdfds", { "a": -1.2 }, { "b": true, "c": false },
    0.1e24, true, false, [ 1 ] ]
})
# => {:array=>[{:number=>"1"@5}, {:number=>"2"@8}, {:number=>"3"@11}, {:null=>"null"@14}, {:string=>"asdfasdf asdfds"@25}, {:object=>{:entry=>{:val=>{:number=>"-1.2"@50}, :key=>{:string=>"a"@46}}}}, {:object=>[{:entry=>{:val=>{:true=>"true"@65}, :key=>{:string=>"b"@61}}}, {:entry=>{:val=>{:false=>"false"@76}, :key=>{:string=>"c"@72}}}]}, {:number=>"0.1e24"@89}, {:true=>"true"@97}, {:false=>"false"@103}, {:array=>{:number=>"1"@112}}]}

Oh well, that is not exactly what we want as final result. Parslet calls the output of its parser a “intermediate tree”. It separates parsing from transformation.

We need a transformer and it looks like :

  class Transformer < Parslet::Transform

    class Entry < Struct.new(:key, :val); end

    rule(:array => subtree(:ar)) {
      ar.is_a?(Array) ? ar : [ ar ]
    }
    rule(:object => subtree(:ob)) {
      (ob.is_a?(Array) ? ob : [ ob ]).inject({}) { |h, e| h[e.key] = e.val; h }
    }

    rule(:entry => { :key => simple(:ke), :val => simple(:va) }) {
      Entry.new(ke, va)
    }

    rule(:string => simple(:st)) {
      st.to_s
    }
    rule(:number => simple(:nb)) {
      nb.match(/[eE\.]/) ? Float(nb) : Integer(nb)
    }

    rule(:null => simple(:nu)) { nil }
    rule(:true => simple(:tr)) { true }
    rule(:false => simple(:fa)) { false }
  end

Patterns in the intermediate tree are indentified and replaced, producing a final output (or yet another intermediate result, it’s up to you).

The complete parser (and transformer and small test) is at https://gist.github.com/966020

There isn’t much more I could say. Ah yes, about testing. Kaspar explains it in the tricks, you can directly test parsing rules individually :

class MyJsonTest < Test::Unit::TestCase
  def parser
    MyJson::Parser.new
  end
  def test_parser_number_integer
    assert_equal 1, parser.number("1")
  end
  def test_parser_number_float
    assert_equal 1.0, parser.number("1.0")
  end
  def test_parser_number_not_a_number
    assert_raise Parslet::ParseFailed do
      parser.number("whatever")
    end
  end
end

Happy parsing (and transforming) !

 

the json parser : https://gist.github.com/966020

documentation : http://kschiess.github.com/parslet/
source code : https://github.com/kschiess/parslet
mailing list : ruby.parslet@librelist.com
irc : freenode.net #parslet

No animals got benchmarked during this blog post.

 

ruote 2.2.0 released

Just released version 2.2.0 of ruote, a Ruby workflow engine. It interprets workflow definitions, routing tasks / work among participants.

Ruote.process_definition do
  alice :task => 'prepare offer'
  bob :task => 'revise offer'
  concurrence do
    david :task => 'revise offer'
    fred :task => 'revise offer', :if => '${offer.total} > 1000'
    elie :notification => 'offer for ${customer.name} (${customer.city}) out'
  end
  charly :task => 'submit offer'
  accounting :task => 'emit invoice'
end

 

the changelog

2.2.x

Why a 2.2.x ? It’s not that ruote 2.2.0 is not backward compatible but it now flags all the expression with a sub_id (formerly called a sub_wfid). Previously only the expressions in a subprocess would have a subid, now all the expressions have one. It prevents some nasty issues with concurrent-iterator and forget.

The second justification for a 2.2.x are stateful participants being dropped.

stateless participants

Before 2.2.0, participants could be registered as classes or instances. From now on, only participant classes can be registered. Each time a ruote worker dispatches a workitem to a participant it uses a new instance. Such “stateless” participants cannot share info via instance variables.

Block participants like

engine.register_participant 'total' do |workitem|
  workitem.fields['total'] = workitem.fields['items'].inject(0) { |t, (i, c)|
    item = Item.find(i)
    t = t + c * item.price
  end
end

are a bit harder to make “stateless”. But thanks to the ingenious Sourcify, grabbing the source of the block is not a problem. Small reminder, it grabs the source code of the block, not its closure.

Simply put, stateful participants have been dropped.

Now for the rest of the changes.

composite conditions

Conditions in process definitions were constrained to things like “${customer.level} == gold”, which works OK if you want to keep concise process definitions. Adding a quick ‘and’ should not require too many workarounds. This is now possible :

Ruote.process_definition do
  participant 'alice', :if => "${customer.level} == 'gold'"
  participant 'bob', :if => '${customer.level} == gold and ${customer.country} == Brazil'
end

Other idioms like “${customer_list} is empty”, “${customer} in ${customer_list}”, “${x} is null” are accepted. The tests are probably a more exhaustive source of info about those idioms.

Speaking of the dollar notation, it was all about strings, whatever the values it would turn them into strings. There is a new literal way for dealing directly with non string values.

on_error, on_terminate

The Engine instance has two new setters, on_error= and on_terminate=. For example, this

engine.on_error = 'supervisor'

states that the ‘supervisor’ participant (whatever you registered under that name) will receive a notification (a workitem) each time a process instance emits an error in the engine.

on_error and on_terminate accept participant names, subprocess names or directly process definitions :

engine.on_error = Ruote.process_definition do
  concurrence do
    administrator :msg => 'something went wrong'
    supervisor :msg => 'houston, we have a problem'
  end
end

filters

Thanks to a collaboration with Raphael filters are [back] in ruote.

They come in two forms : the filter expression (one-way filtering of passing workitems) and the :filter attribute (placing a filter around a process region).

The filter attribute may also point to participants (registered like any other participants), focused on workitem filtering (whereas regular participants pass work to the underlying, real, participant).

history

Prior to 2.2.0, ruote had no history, the worker activity was not ‘archived’. A ruote engine will now have a default history keeping in memory the most recent worker operations.

If you really need such a history, you’d better use/customize the StorageHistory class.

(well, you probably don’t need to keep record of all these operations).

sequel

There is a new storage implementation, ruote-sequel. As the name implies, it’s based on the excellent Sequel.

A nice addition to the list of storage implementations.

participants (again)

You register participants, they are all stateless, why not get a copy out for certain interactions ? Engine#participant is the counterpoint to Engine#register_participant.

Participants may provide their own timeout value by implementing the rtimeout method (the timeout given in the process definition, if any, will take precedence though).

expressions

The listen expression now reacts to a process instance entering or leaving a tag (a process instance region). See ruote and tags for an explanation. The Workitem class now has a dedicated __tags__ field containing a list of tag names the process instance (that emitted the workitem) is currently in.

Ruote now has a “switch” statement, it’s a given (also covers the new let expression).

The cancel_process expression has got a new alias “terminate”, concurrent_iterator can be shortened to “citerator” while the when expression can be written “once” (or “as_soon_as”).

There is a new let expression. The main usage is to isolate a set of subprocess definitions in a new scope (case like).

The registerp and unregisterp expressions let you register participants from process definitions (granted, you could do that from the consume method of a participant too).

It was present in ruote 0.9.x but got lost in the way. The lose expression and the :lose attribute are back (thanks Claudio). ‘forget’ and ‘lose’ are ways to put aside certain execution branches of a process instance.

next steps

Next steps ? Workers registering in the storage, pausing individual processes, having process supervise other processes, improved ruote-redis, …

 

Many thanks to all who helped along the way.

 

website : http://ruote.rubyforge.org
source : https://github.com/jmettraux/ruote
mailing list : http://groups.google.com/group/openwferu-users
irc : freenode.net #ruote

 

ruote, process state and tags

A : “What state is this process in ?”
B : “Well, it’s running”
A : “I know, that’s not what I meant. Where is it now ?”
B : “Let’s check which participants have workitems (tasks) about this process now”
A : “No, I mean, the document is it still being reviewed ? What’s its state ?”
B : “Ah you mean, the document state, not the process state ?”
A : “Is there a difference ?”
B : “Well, if a business process deals with the state of multiple resources, you can’t equate process state and resource state”

Initially, for ruote (a ruby workflow engine), the “state of a process” was limited to “the set of visible workitems for that process”.

Ruote.process_definition do
  cursor do
    production
    concurrence do
      qa1
      qa2
    end
    rewind :unless => '${qa_ok}'
    packaging
    delivery
  end
end

For this process, the states are “nil” (not running), “production”, “qa1” and/or “qa2”, “delivery”, and “packaging”.

(note that there is no “terminated” state, since, out of the box, ruote doesn’t do ‘process archiving’ for you).

Asking all the participants about the workitems could get expensive, especially if they’re remote participants. It’s easier to ask the engine :

p engine.process(wfid)
  # =>
  # == Ruote::ProcessStatus ==
  #    expressions : 3
  #      0!69176db85a0651e7a8d8a16426bd93df!20110107-betesuguto : define {}
  #      0_0!be7ac163b2c6ba47d6d4b24bbb83fd8c!20110107-betesuguto : cursor {}
  #      0_0_0!3b19bdf68a953597969bac507229bcf1!20110107-betesuguto : participant {"ref"=>"production"}
  #    schedules : 0
  #    stored workitems : 1
  #    variables :     {}
  #    all_variables : {"0!69176db85a0651e7a8d8a16426bd93df!20110107-betesuguto"=>{}}
  #    errors : 0

p engine.process(wfid).position
  # =>
  # [
  #   [ "0_0_0!3b19bdf68a953597969bac507229bcf1!20110107-betesuguto",
  #     "production",
  #     {} ] ]

(full gist at https://gist.github.com/769362)

What if the “state”, business-wise, has a different granularity than the simple one we derive from the participants ? Something more like a “stage”.

The “tag” attribute could help :

Ruote.process_definition do
  cursor do
    sequence :tag => 'production-stage' do
      production
      concurrence do
        qa1
        qa2
      end
    end
    rewind :unless => '${qa_ok}'
    sequence :tag => 'delivery-stage' do
      packaging
      delivery
    end
  end
end

Our process [instance] can be in “production-stage” or “delivery-stage” (or nowhere).

We can then ask for the tags of a process status :

p engine.process(wfid).tags
  # =>
  #  { "production-stage" => {
  #      "engine_id" => "engine",
  #      "wfid" => "20110107-hopakeze",
  #      "subid" => "0f62bbb3a0a19411aaef1524ebde657c",
  #      "expid" => "0_0_0" } }

(full gist at https://gist.github.com/769371)

Tags were originally meant to be used in conjunction with the undo/cancel and the redo expressions.

They later were used within cursors for jumps (along with participant names) and when “piloting” a cursor from outside.

Asking the engine about the ‘process state’ / ‘stage’ is fine, but what if a [remote] participant wants to know about the stage is in ?

Let’s add a ‘qa_stage’ tag.

Ruote.define do
  cursor do
    sequence :tag => 'production-stage' do
      production
      concurrence :tag => 'qa_stage' do
        qa1
        qa2
      end
    end
    rewind :unless => '${qa_ok}'
    sequence :tag => 'delivery-stage' do
      packaging
      delivery
    end
  end
end

The upcoming ruote 2.1.12 adds a #tags method to its workitems (a shortcut for workitem.fields[‘__tags__’]).

With a participant implementation that looks like

class MyParticipant
  include Ruote::LocalParticipant

  def consume(workitem)
    p [ workitem.participant_name, :tags, workitem.tags ]
    workitem.fields['qa_ok'] = true
    reply_to_engine(workitem)
  end
end

a run of our process will output

["production", :tags, ["production-stage"]]
["qa1", :tags, ["production-stage", "qa_stage"]]
["qa2", :tags, ["production-stage", "qa_stage"]]
["packaging", :tags, ["delivery-stage"]]
["delivery", :tags, ["delivery-stage"]]

(full gist at https://gist.github.com/769406)

There is a difference between asking the engine for the tags of a process intance and the list of tags returned by workitem#tags. The former returns all the tags currently active for the process instance, while the latter returns the list of tags that were traversed to reach the participant expression that emitted the workitem.

Another novelty brought in by ruote 2.1.12 is the possibility to listen to tag events (entering and leaving tags).

Until now, the http://ruote.rubyforge.org/exp/listen.html expression could only listen to participants (workitems reaching and returning from participants). The upcoming ruote 2.1.12 lets us listen to process instances entering and leaving tags.

For example, we could have a monitoring process that listen to process instances entering the ‘final-stage’ tag. Each time it happens, the participant ‘supervisor’ receives a copy of the workitem :

Ruote.process_definition do
  listen :to => 'final-stage', :upon => 'entering' do
    participant 'supervisor'
  end
end

By setting the :wfid attribute of the listen expression to true, we can limit the listening to the process instance to which the listen expression belongs.

It can be useful to approximate the ‘milestone’ workflow pattern :

Ruote.define do

  concurrence :count => 1 do
    # will terminate as soon as 1 branch replies (and cancel the other)

    sequence do
      participant 'a'
      participant 'b', :tag => 'milestone'
      participant 'c'
    end

    listen :to => 'milestone', :upon => 'entering', :wfid => true do

      concurrence :count => 1 do
        # will terminate as soon as 1 branch replies (and cancel the other)

        listen :to => 'milestone', :upon => 'leaving', :wfid => true
          # as soon as the tag 'milestone' is left, this listen will
          # exit (reply to the parent concurrence) and d will get cancelled

        participant 'd'
      end
    end
  end
end

The participant ‘d’ will only receive a workitem when the milestone is reached. As soon as the milestone is left, the workitem of participant ‘d’ is cancelled (removed from him).

Note how this implementation relies on “concurrence :count => 1”, a concurrence that exits as soon as 1 of its branches replies (and cancels the remaining branches).

In conclusion, tags have multiple uses, designating process state / stages, letting undo / redo segments of processes. The next ruote (2.1.12) adds tag information in workitems and lets the listen expression observe tag events (entering or leaving a tag).

 

* web : http://ruote.rubyforge.org
* source : http://github.com/jmettraux
* mailing list : http://groups.google.com/group/openwferu-users
* irc : freenode.net #ruote

 

ruote and switch

A programming language usually has some kind of super “if”, a switch statement.

Ruote is nothing more that an interpreter, a very patient one, one that can get stopped and restarted. (it also runs multiple process instances concurrently). Since it interprets some kind of high level business process gibberish, a switch statement is nice to have.

This post explores switch statement implementations for ruote, it starts with solutions for ruote 2.1.x and then shows solutions that use two new expressions in the upcoming ruote 2.1.12.

 

before ruote 2.1.12

For ruote [2.1], I wasn’t sure if a switch statement was explicitely needed. With a workitem field or a process variable, it’s easy to switch to a given subprocess.

Suppose we have an item pickup process. The system is onboard, the drivers leaves the depot knowing his immediate pickup point, but not the next, as the system asks after each pickup what to do (via the ‘get_next_task’ subprocess).

Ruote.process_definition do

  define 'coast_pickup' do
    # ...
  end
  define 'mountain_pickup' do
    # ...
  end
  define 'get_back_to_depot' do
    # ...
  end

  sequence do # body of the process

    cursor do
      subprocess 'get_next_task'
      subprocess '${next_task}'
      rewind :unless => '${next_task} == get_back_to_depot'
        # cursor rewinds unless next task is getting back to depot
    end
  end
end

All is well, our switch occurs at “subprocess ‘${next_task}'”. Note that if we wished to, we could use ref instead of subprocess if we wanted to point to participants and/or subprocesses.

But wait… Those subprocesses are defined for the whole process, they are not limited to the switch.

We can go a bit further and isolate the switch and its cases into its own subprocess.

Ruote.process_definition do

  define 'perform_next_task' do

    define 'coast_pickup' do
      # ...
    end
    define 'mountain_pickup' do
      # ...
    end
    define 'get_back_to_depot' do
      # ...
    end

    subprocess '${next_task}'
  end

  sequence do # body of the process

    cursor do
      subprocess 'check_tasks'
      subprocess 'perform_next_task'
      rewind :unless => '${next_task} == get_back_to_depot'
    end
  end
end

This is nice, we have a ‘perform_next_task’ subprocess wrapping the cases, the body of that subprocess calls the right case given the value of the “next_task” workitem field. If there is another subprocess named ‘mountain_pickup’ in the same process definition, the one in the switch subprocess will only shadow it within the case, in other words, the scope of the cases is limited to the switch subprocess.

But, you’ll say, with a real programming language not some toy process definition language, the switch and its cases are all wrapped neatly inline, they are not set apart.

Placing the switch and its cases in the main flow is OK, but it binds subprocesses… It can override subprocesses with the same name.

    cursor do
      subprocess 'check_tasks'
      sequence do
        define 'coast_pickup' do
          # ...
        end
        define 'mountain_pickup' do
          # ...
        end
        define 'get_back_to_depot' do
          # ...
        end
        subprocess '${next_task}'
      end
      rewind :unless => '${next_task} == get_back_to_depot'
    end

We need to look at the next version of ruote to solve that issue.

 

with ruote 2.1.12

For the upcoming ruote 2.1.12, I introduced a let expression :

    cursor do
      subprocess 'check_tasks'
      let do # let's have a new scope, just for our cases
        define 'coast_pickup' do
          # ...
        end
        define 'mountain_pickup' do
          # ...
        end
        define 'get_back_to_depot' do
          # ...
        end
        subprocess '${next_task}'
      end
      rewind :unless => '${next_task} == get_back_to_depot'
    end

The ‘let’ introduces a new scope, where our case subprocesses can get defined without overrides ones with the same name outside of the let block.

Now, how about something that really looks like a switch statement ?

    cursor do
      subprocess 'check_tasks'
      given '${next_task}' do
        of 'coast_pickup' do
          # ...
        end
        of 'mountain_pickup' do
          # ...
        end
        of 'get_back_to_depot' do
          # ...
        end
      end
      rewind :unless => '${next_task} == get_back_to_depot'
    end

With some Ruby/Perl -like magic, all the pickups could get covered by the same case :

    cursor do
      subprocess 'check_tasks'
      given '${next_task}' do
        of /^.+_pickup$/ do
          # ...
        end
        of 'get_back_to_depot' do
          # ...
        end
      end
      rewind :unless => '${next_task} == get_back_to_depot'
    end

The upcoming ruote 2.1.12 has this given expression. Look at its description, it not only covers “given an x of” scenarii but also “given that” ones.

 

conclusion

I still like the first version, it’s vanilla ruote “trigger the subprocess whose name is found in workitem field x”, it’s nice to have subprocesses that can be used as cases or called from other parts of the process definition.

Calling subprocesses doesn’t limit us to processes bound within the same process, processes bound at the engine level (engine variables) or external processes (given by their URI) are callable as well (see the subprocess expression doc for more information).

The given expression is interesting because it covers “given an x of” and “given that” scenarii (see the doc). It also has a “default” part which the first version doesn’t have.

Try to write processes that are concise and that read like english. And test them.

 

rufus-jig 1.0

By the end of 2007 I had written a gem sitting on top of net/http. It was called rufus-verbs, this extra layer added a mini cache for conditional GETs, basic auth and digest auth, cookie jar and more.

I used it for a while, I got surprised by people using it (IIRC the digest auth was their reason for using it). And then I forgot it.

Since last year, I am working with things like CouchDB and ruote-kit. I need an HTTP client that groks JSON. I still need it to understand conditional GETs (etags and co).

So I built rufus-jig, something on top of Ruby’s net/http, net-http-persistent, patron or em-http-request. It uses rufus-json to select the best JSON library available (in the order yajl-ruby, json, activesupport, json-pure).

A GET would look like :

require 'rubygems'

#require 'net/http/persistent'
  # http backend

require 'yajl' # gem install 'yajl-ruby'
  # json backend

require 'rufus/jig'

h = Rufus::Jig::Http.new('http://twitter.com')

p h.get('/users/jmettraux.json')['description']
  # => "another fool"

You specify the HTTP and the JSON backend before requiring rufus-jig and that’s it.

Rufus-jig comes with a class to deal with some of CouchDB specifics.

require 'net/http/persistent'
require 'yajl'
require 'rufus/jig'

c = Rufus::Jig::Couch.new('http://127.0.0.1:5984', 'rufus_jig_test')

# PUT and GET

c.put('_id' => 'coffee0', 'category' => 'espresso')
c.put('_id' => 'coffee1', 'category' => 'instantaneous')

coffee1 = c.get('coffee1')
coffee1['brand'] = 'nescafe'
c.put(coffee1)

# attaching

coffee0 = c.get('coffee0')

c.attach(coffee0, 'picture', File.read('espresso.jpg'), :content_type => 'image/jpeg')

# fetching all docs

p c.all
p c.all(:skip => 100, :limit => 100)

# fetching a batch of docs

p c.all(:keys => %w[ coffee0 coffee2 coffee7 ])

# querying views

p c.query('my_design_doc:my_view', :key => 'Costa Rica')
p c.query_for_docs('my_design_doc:my_view', :key => 'Colombia')

# bulk operations

docs = c.all(:keys => %w[ doc0 doc1 doc3 ])
c.bulk_delete(docs) # deleting in one go

docs = c.all(:keys => %w[ doc0 doc1 doc3 ])
docs.each { |doc| doc['status'] = 'copied' }
c.bulk_put(docs) # updating in one go

# ...

# listening to CouchDB activity

c.on_change do |doc_id, deleted|
  puts "doc #{doc_id} has been #{deleted ? 'deleted' : 'changed'}"
end

c.on_change do |doc_id, deleted, doc|
  puts "doc #{doc_id} has been #{deleted ? 'deleted' : 'changed'}"
  p doc
end

 

Rufus-jig just reached 1.0. I’ll probably go on with extending it. It’ll probably need digest authentication at some point, gzipping, why not. I’ll leave the CouchDB function in it for now. We’ll see.

 

source code : https://github.com/jmettraux/rufus-jig

 

resource lifecycle tuple

I came across this passage :

Moving from design to implementation, we need to think about the protocol in a slightly different way. In a resource-oriented distributed application, an application protocol can be thought of as a function of one or more resource life cycles and the business rules that connect these resources. Because of its resource-centric nature, the Restbucks service does not host an application protocol state machine. In other words, there’s no workflow or business logic for the application protocol as such. Rather, the service governs the life cycles of the orders and payments participating in the application protocol. Any workflows in the service implementation relate to resource life cycles, not the application protocol life cycle. While we’ve been explicit in modeling the business process as an application protocol state machine, we’ve been diligent in implementing it wholly in terms of resource state machines.

REST in practice, page 131, (emphasis mine).

 

I had to link that passage to this blog post :

The bottom line is that a business entity lifecycle represents some business logic that is “business process independent”. This is probably news to many of you and some of you might say that a BEL is a “long running process” but it is not. It is long running. It is a process in the sense of an operating system point of view (nearly), but it is not and will never be a “business process”. The business process is represented by the activities (human or automated) that advance the lifecycle of one or more business entities.

Lifecycle example, from Carnets de Bord

 

volute

Vos luttes partent en fumée
Vers des flûtes enchantées Et de cruelles espérances
Me lancent Des dagues et des lances En toute innocence

J’cloue des clous sur des nuages Un marteau au fond du garage
J’cloue des clous sur des nuages Sans échafaudage

volutes Alain Bashung (funny auto-translation)

 

This started out in a notebook. I wanted to do something about ‘state’ and ‘lifecyle’, something like a state machine for multiple objects or a rule system for families of resources.

It ended up as something that feels like the subset of an aspect oriented framework. Subset because in its most vanilla usage it only cares about calls to set methods, and there is no “before” (at first glance).

This Ruby gem is named volute. Here is an example of its usage :

require 'volute'

class Package
  include Volute

  attr_accessor :location
  attr_accessor :delivered
end

volute Package do
  # filters non Package instance out

  volute :delivered => true do
    # applies when delivered switches to true
    object.emit_invoice
  end

  volute :location do
    # filters out changes that are not for the :location attribute

    volute 'SFO' => :any, 'FRA' => :any do
      # SFO and FRA are our international hubs, for any package transiting from there,
      # initial 'international' tracking
      TrackingSystem.track_international(object)
    end
  end
end

The “include Volute” reworks the attr_accessor, and triggers the evaluation of the volutes on each set.

It’s not always necessary to include Volute. Here is a example of [business] rules derived from Ruleby (volute is very dumb compared to Ruleby, the example doesn’t do justice to that project).

Here is the diagnosis example with volute :

require 'volute'

class Patient

  attr_reader :name
  attr_accessor :fever
  attr_accessor :rash
  attr_accessor :spots
  attr_accessor :sore_throat
  attr_accessor :innoculated

  attr_accessor :diagnosis

  def initialize(name)
    @name = name
    @symptoms = {}
    @innoculated = false
  end

  def diagnose!
    Volute.apply(self) # trigger evalution of volutes
    return @diagnosis
  end
end

volute Patient do

  volute :fever => :high, :spots => :true, :innoculated => true do
    object.diagnosis = 'measles'
    over # prevent further evals
  end
  volute :spots => true do
    object.diagnosis = 'allergy (spots but no measles)'
  end
  volute :rash => true do
    object.diagnosis = 'allergy (rash)'
  end
  volute :sore_throat => true, :fever => :mild do
    object.diagnosis = 'flu'
  end
  volute :sore_throat => true, :fever => :high do
    object.diagnosis = 'flu'
  end
end

pat = Patient.new('alice')
pat.rash = true

puts "#{pat.name} : diagnosed with #{pat.diagnose!}"

pat = Patient.new('bob')
pat.sore_throat = true
pat.fever = :high

puts "#{pat.name} : diagnosed with #{pat.diagnose!}"

 

The readme for volute is quite extensive. There are a few examples included.

You’ve already seen the diagnosis one, it aimed at using volute for some mini rule engine (nothing fancy at all).

There is an example about a simple (kph = km/h) equation where each time an attribute is changed, other attributes get adjusted.

The light example simply keeps tracks of the last time an attribute got modified.

There is a book in a bookshop state machine example with a variation, to explore state volutes and transition volutes.

Note that state machines built with volute are incomplete, they know nothing about further transitions (could I even call them state machines ? Rather a ‘state transition system’).

Traffic could be fun : it tracks the state of two “lines” and adjust the colour of their lights according to the number of cars waiting. It’s more in line with the “multiple objects” goal I had initially (still here somehow, but examples with 1 class are smaller).

Not sure if it’s worth moving logic out of objects, but it is a fun experiment.

 

http://github.com/jmettraux/volute

 

ruote 2.1.11 released

ruote is an open source workflow engine implemented in Ruby.

It takes as input process definitions and interprets them. It routes work among participants according to the flow described in those process definitions. It can orchestrate ping pong matches as well.

The main motivation behind this release is Torsten being tired of advising people to use ruote edge (thanks Bundler) for their ruote-kit installs, ruote was more than ripe for a new release. This version includes the result of the feedback of numerous people, as the changelog can attest.

I’ll address here two aspects, one in ruote itself, and one on its periphery.

participant in a workflow

Up until now, participants were registered one by one (order of registration matters).

engine.register_participant 'reception', Acme::ReceptionParticipant
engine.register_participant 'support', Acme::SupportParticipant
engine.register_participant '.+', Ruote::StorageParticipant

Torsten came up with an idiomatic solution in ruote-kit, that was soon promoted to ruote itself :

engine.register do
  reception Acme::ReceptionParticipant
  support Acme::SupportParticipant
  catchall
end

All is well in a vanilla world were participants are all known at system startup. People were asking about adding participants on-the-fly. Ruote has always been able to do that :

engine.register_participant 'reception2', Acme::ReceptionParticipant, :site => 'two'

but our reception2 gets registered after the catchall participant, and thus might never receive any workitems. A solution would be to place the participant as second to last :

engine.register_participant(
  'reception2', Acme::ReceptionParticipant, :site => 'two', :position => -2)

But blindly inserting participants isn’t good. Ruote 2.1.11 has a new method for setting the participant list in a single batch :

engine.participant_list = [
  [ '^reception$', 'Acme::ReceptionParticipant', {} ],
  [ '^reception2$', 'Acme::ReceptionParticipant', { 'site' => 'two' } ],
  [ '^support$', 'Acme::SupportParticipant', {} ],
  [ '^.+$', 'Ruote::StorageParticipant', {} ]
]

Very rough, but full control.

ruote in kit

ruote-kit was kind of left behind at version 2.1.8. Torsten and I resumed its development and brought it to 2.1.11.

Ruote-kit is a web administration console for ruote. Kenneth Kalmer’s genius idea was to make it a rack middleware (component), so that it works standalone or fits nicely in a rails application, under /_ruote/

Ruote-kit is more than a web administration console, it’s also an HTTP/JSON based interface to ruote, for example : http://gist.github.com/611044. The entry point of the interface is /_ruote/ all the resources are linked to from this “root”. The links are annotated with “rel” attributes that indicate to clients what stands behind the link.

There is more to it, we’re working on conveying the capabilities of both interfaces HTML / JSON from the HTML (the one that is easily navigated by a human and his browser).

storages got updates

ruote-dm, ruote-redis, ruote-couch have been upgraded to 2.1.11.

Note that there is a piece of documentation on how to implement storages that is in the works as it was requested by people who want to have a MongoDB backend.

next steps

Still have to release an updated ruote-amqp, and bring ruote-beanstalk to 2.1.11. Then it will be time to think about 2.1.12. As said, there is still work to do on ruote-kit, and since it’s used by many ruote people, it deserves full attention.

Many thanks to Eric Platon, Nathan Stults, Asier, Hartog de Mik, Brett Anthoine, Eric Smith, Kaspar Schiess, David Greaves, Rich Meyers, Kenneth Kalmer, Don French and David Goodlad (and many others) for their contributions !

source : http://github.com/jmettraux/ruote
mailing list : http://groups.google.com/group/openwferu-users
doc : http://ruote.rubyforge.org
irc : #ruote on freenode.net