about processes and engines

how does Ruote work ?

(note: this blog post is from 2008, ruote evolved a lot since, for an up-to-date “how does ruote work”, see sample workflow run)

Well, as “ruote” means “wheel”, it just spins…

Ruote being a workflow/BPM engine, interprets process definitions and runs them as process instances. But how does that happen ?

The few graphs I’ll use in this post were meant as “embedded documentation” in the HTML rendition of ruote-rest. But the final result, a mini-slideshow in the /expressions collection wasn’t very convincing.

I’ll walk through those graphs, detailing what’s happening and giving information about the engine, its components and how they cooperate to interpret process definitions and run process instances smoothly.

Let’s consider this very simple process definition, expressed in Ruby for readability :

class MyProcessDef < OpenWFE::ProcessDefinition
  sequence do

A sequence routing work from participant “alpha” to participant “bravo”.

When Ruote is asked to run an instance of that process, usually via an incoming LaunchItem containing the process definition or a pointer to the process definition somewhere off the web, it will first create an initial RawExpression with an expression id of “0” (and some unique process instance id) containing the process definition as a tree of s-expressions.

You may think of those “raw expressions” (the ones with the dotted circle) as a cell containing some DNA for further evolutions.

Expressions in Ruote are atomic pieces of process instances. Expressions can be passed two messages : “apply” and “reply” (well, there is also the “cancel” message, but that’s another story). When a raw expression gets passed the “apply” message (along with a workitem), it will expand itself into a full expression (with the same id) complete with children expressions (currently all of them as “raw expressions”).

So our initial raw expression contains a tree whose root is describing a “process-definition” expression. Once expanded we thus get a “process-definition” expression, an extra “environment” expression (containing the top level variables of the process instance) and a unique child expression, a raw expression with its tree of s-expressions, a branch of the parent’s tree.

All those expressions are managed by an expression pool component, which relies on one or more expression storage components for persistence. The classical setting involves one in-memory cache storage plus one file based persistence storage.

Expressions are linked together by a kind of pointer named a “FlowExpressionId”, each expression has a pointer to its parent expression, to the environment expression where the process variables (if needed) are stored and a list of pointers to its children (if any). There is no direct link from one expression to the other (as the other might be off memory for a while).

The two main expression storages used are one storing each expression in a file and one storing expressions in a database (ActiveRecord based). Other implementations are possible (in one of my projects, expressions (and workitems) are stored in an Apache JackRabbit repository).

Expression storages can be viewed as hashes, mapping FlowExpressionIds to expressions. (Yes, a Tokyo Tyrant based expression storage would be cool).

The apply message that triggered the expansion of the root expression from raw to full expression is then passed to the unique child of the process-definition expression, which is turned from raw to a “sequence” expression.

Our sequence expression has two children (currently still raw expressions), that it will apply one after the other. When the last one will have replied, the sequence expression will reply to its parent expression (well, any expression, once its job is done, replies to its parent expression (if there is one)).

The first participant gets applied. Participant expressions sit on the border between the engine and the participants. The most common participant will store the workitem that the participant expression gave into some kind of worklist and then wait for a reply (modified workitem generally). Once the reply comes, the participant will pass the “reply” message (with the modified workitem) to the participant expression.

This wait can last a long time, thus persistence is essential for a workflow engine, something wrong may happen, it has to be able to restart gracefully and resume its wait.

Sometimes, with participants such as this block participant, the wait doesn’t last very long :

engine.register_participant "kilroy" do |workitem|
  workitem.some_message = "kilroy was here"
  puts "Kilroy left his mark"

Participants are actors/agents sitting outside of the engine. From an implementation point of view, they are pieces of code detailing how to dispatch a workitem to that agent and how to fetch it back or where to fetch it back from the agent. Participants are registered in a participant map, another component of the engine, are are looked up via regular expressions. It’s thus OK to have a participant registered as “^user-.*”, it will be handed all the workitems from participant expressions whose participant name starts with the string “user-“.

Once the participant expression has received the workitem back from the participant, it will reply to its parent expression.

In our case, it’s the sequence expression.

The sequence expression contains a logic describing how to apply children expression one after the other. Other expressions, like ‘concurrence’, ‘loop’ or ‘cursor’ have different ways of coordinating the flow among their children expressions.

Expression implementations all extend a FlowExpression class. They have to play by certain rules, but not many.

The apply/reply coordination between the expressions is regulated through a work queue next to the expression pool. The queue worker triggers one apply or reply at a time.

Once the participant “bravo” will have replied to the participant expression 0.0.1, this expression will reply to the sequence expression and be removed from the expression pool. The sequence expression, having no more child expressions to apply, will reply to its parent expression, the “process-definition” expression.

Since this expression has no parent expression and is a root expression (expid is “0”), it will be removed from the expression pool (along with its companion, the environment expression) and the process instance will be considered terminated.

In summary, a last illustration showing the apply/reply operations as they drove the expansion of the process instance into a tree, traversing it as it grew.

Each apply arrow represents in fact a “raw to full” transformation plus a regular “apply”.

Once again, this is a very simple process definition, but it shows well the philosophy behind Ruote. A more advanced subject would explain how subprocesses are dealt with. It’s not complicated, but that’s for another post.

Some observations :

  • it’s OK to modify the tree inside of a RawExpression to alter a process instance in-flight (the latest ruote-rest event provides an integrated editor to facilitate that)
  • (it’s even OK to replace an expression (raw or full) by a modified version of it)
  • those trees inside of the RawExpression objects are very easily represented in JSON. Ruote-rest and ruote-fluo abuse that fact
  • with this “embedded tree” model, it’s OK to have multiple versions of a process definitions running in the same engine
  • I haven’t talk about cancel, basically, it’s just a mean to cut off a branch of a tree and cancelling all the activities in that branch, expression implementations are supposed to provide those three methods, apply, reply and cancel. Participant implementations are supposed to handle cancel messages as well.

Ruote is open source, which is no feat at all, but it strives to be open (for short), even for the process admin.

Hope that wasn’t too boring. If you have any question about Ruote, feel free to ask on the Ruote mailing list.

I will probably release ruote 0.9.19 this coming week, along with ruote-web 0.9.19 and ruote-rest 0.9.19.

Written by John Mettraux

September 7, 2008 at 2:21 pm

Posted in bpm, dev, openwferu, ruby, ruote, workflow

4 Responses

Subscribe to comments with RSS.

  1. […] can be expressed as XML or as Ruby code (or simply as a JSON string). I posted yesterday about how it works and there’s a quickstart […]

  2. i am interested to learn how does the participant listener work ? i have tested the jabber listener, and it works well to receive response message from my google talk client.

    But i can not decode the response into an workitem. i just want to add the response to the workitem then reply the handle to the core engine.
    such as when i received the response “{my_comments => ‘i would like the work to be done till tomorrow morning.’}”

    wi =
    wi.attributes = response_hash

    did not work for me. you did not mention the listener in the above picture. i’d like to know more about it. pls help! thanks a lot.


    June 10, 2009 at 5:05 pm

Comments are closed.

%d bloggers like this: