I watch (at least) two slices of the blogging cloud (no “sphere” today), the “programmers” one and the “business process management” one.

The term “workflow” appears more in the “programmers” slice these days, see for example “My Git Workflow”.

If you take a look at what’s currently popular in the “workflow” bookmarks, you’ll see that in the top ten there are 4 items that are not about workflow in the ‘pure’ business process management sense. But they qualify as managed business processes in John Reynolds’ terms, where

If a Process has multiple Participants, at least one of the Participants is Human, and the transitions between the Activities are controlled by an automated system, then it’s a Managed Business Process.

But a person going through the routine of his personal photography workflow is probably not going to call it a “business process”.

So what I do is probably more about BPM, the term workflow sliding towards “my own personal work flow”.

It may seem as an exercise, writing something restful. In the first take, I used Ruby on Rails to wrap OpenWFEru with a RESTful interface.

In this second take, I considered the alternatives among the Ruby web frameworks and went for Sinatra by Blake Mizerany.

Ruote-rest is mainly the result of a collaboration with a company which is integrating OpenWFEru (ruote) among its .NET applications. Its software artifacts speak XML over HTTP with ruote.

Ruote-rest, as a “take two”, is also the refinement of the concepts explored in the take one.

So what’s this RESTful workflow/BPM engine ?

It’s named “ruote-rest” and it’s available from GitHub at http://github.com/jmettraux/ruote-rest.

It provides for now 3 resources, /processes, /participants and /workitems (and the special /expressions resource).

• /processes - running [business] process instances
• /participants - participants in the processes (out of the box there are two ‘worklist’ participants provided ‘alpha’ and ‘bravo’, but you can POST others and DELETE those)
• /workitems - the workitems held by ‘worklist’ participants (for human consumption generally)

These resources are sufficient for

• launching processes
• fetching workitems, update their payload and let them resume in their business process
• canceling part or all of a process
• modifying a process on the fly
• adding or removing participants to business processes

Ruote-rest currently provides two representations of its resources, an XML one and an HTML one. The XML representation is meant for automated consumption, the main case, while the HTML one is meant for learning / debugging.

The installation instructions are in the README.txt, and there is also a packaged release available.

— warning : boring technical post ahead —

a resource tour

Let’s have a look at ruote-rest once it’s running, through its ‘HTML interface’, which demonstrates the API ruote-rest exposes, via hyperlinks and forms.

/processes enumerates the processes currently running in the engine.

New process instances can be launched (POSTed) with the form at the bottom of the HTML representation of /processes.

The form shows two ways of providing a process definition to the engine, via an URL (first input box) or by providing the process definition directly (text area).

Let’s just hit POST /processes.

Ruote-rest answers with an HTTP 201 reply containing a link to the newly created process instance. (The current HTTP status code is indicated on the top right of the HTML page, most of the time it’s at 200 OK).

Getting /processes/20080507-buparijitsu yields an HTML representation of the running [business] process instance (the workflow instance id being the 20080507-buparijitsu mentioned in the URI).

This page features links to all the expressions (/expressions/:wfid) that make up a process instance and to the JSON representation of the process definition itself.

It’s this representation that is interpreted in ruote-web when rendering graphically a process definition to some subset of BPMN. (Note that if the process instance got modified “en route”, it’s the process definition as running, not as launched which is returned).

There are two fat submit buttons, one for pausing (or resuming) the process instance (a PUT), and one for terminating (canceling) it altogether (a DELETE).

There is also a link to the “currently active expression” in the process (in other words, where the process is currently waiting). If there are concurrent execution paths for the process, there will be multiple “currently active expressions”.

A process instance is made up of expressions linked in an execution tree. The most common expressions are participant, sequence and concurrence.

The participant expression is used to link the execution of a workflow with an [external] participant, be it a service, a human, a small snip of code, …

OpenWFEru allows to modify the expressions in a running process instance. It may be necessary for administrative or business purposes (missing plan B maybe).

Expressions are stored in two states : unapplied and applied. Unapplied expressions have not yet been reached by the execution flow, they mainly are stored “raw”, uninterpreted (the whole branch they form with their unwound children expression).

The /expression/:wfid/:expid page features links back to the process and to the list of all expressions. There are also links to the parent expression, the environment expression (scope) and the children expressions, if any.

At the end of the page, there is a submit button for terminating the expression (cancelling it) (DELETE). If the execution path is located in the branch being terminated, the flow will be resumed after the cancelled branch.

By hitting the show link above the delete button, a text area containing the YAML representation of the expression will be displayed, along with a submit button for PUTting the updated expression back in the workflow engine.

That shows how to update business process instances, on the fly (this subject has been addressed for ruote-web in process gardening).

Ruote-rest, like ruote-web integrates a worklist for storing workitems for human consumption (not comestible though).

From the /workitems, the page for each individual workitem can be reached. It is also OK to query for the workitems belonging to just one process (/workitems?wfid=20080507-buparijitsu for example).

The HTML representation of a workitem has a text area containing the current payload of the workitem, as a JSON string, ready for being PUT back. If the “proceed” box is checked, the process will resume (with the updated payload).

The /participants page allows for addition / deletion of participants.

It currently only provides the way to add “active participants”, ie participants that place workitem in the integrated worklist. Other classes of participants can be added via the conf/participants.rb configuration file.

now and then

Ruote-rest is not finished. For example the /errors resource is not yet implemented (replaying a business process after an error is something important).

The initial representation is XML, along with HTML for educational purposes. JSON is a must. Why not some AtomPub like Kisha (the “take one” approach) included ?
Maybe YAML would be interesting too, for Ruby clients at least.

I’ve chosen Sinatra because I adhere to its convention :

get "/things/:tid" do
    "<p>I'm thing and my name is #{params[:tid]}</p>"
end

On the todo list is also some support for etags and if-modified-since, let’s make ruote-rest a good, respectful HTTP citizen.

The security stuff is rudimentary for now, here is how it looks :

# called before each request gets processed
#
before do

    throw :halt, [ 401, "get off !" ] \
        if request.env['REMOTE_ADDR'] != '127.0.0.1'
end

The company for which ruote-rest is being specifically developed for doesn’t require any fancy auth mecha. But with Sinatra, they aren’t that far (see the so project for example or these posts).

Talking about security, the readers will certainly have noticed that the process definition can be fed directly into the engine, that’s very practical but you have to trust your auth mechanisms…

For persistence, ruote-rest currently uses the filesystem for the engine itself and a database (via Ruby on Rails’ ActiveRecord) for its worklist.

I’m thinking about integrating the automatic graphical process rendering found in ruote-web (ex-Densha) into the HTML “educational pages”, it would look good and help people understand what’s going on.

at the github

All this happens at http://github.com/jmettraux/ruote-rest, feel free to fork. Find the mailing lists at google groups.

Sandy almost called the security to get Pat bounced out of Architecture & Process 2008. I’m joking, she just wondered what was Pat doing there.

If you consider the fact that “Architecture and Process” website is hosted at http://www.transformationandinnovation.com, then RESTful BPM is a nice fit for this event.

If you’re still not convinced, go and read Jason’s reaction and Pat’s reply.

Congrats to Pat Cappelaere for his success with RESTful WfXML.

I’m working on it as well (with my limited vision), after Kisha, I’ve moved on to Ruote-Rest.

Too many links.

There is this hidden gem in Ruby land, the webrick webdav handler.

I’ve been using it to power some webdav accesses to an application here. When the time arrived to upgrade from JRuby 1.1 “pre from last fall” to JRuby 1.1[.1], this webdav handler broke.

jmettraux@sonora:~/tmp$ ~/jruby-1.1.1/bin/jruby dav.rb
[2008-04-24 10:09:24] INFO  WEBrick 1.3.1
[2008-04-24 10:09:24] INFO  ruby 1.8.6 (2008-04-22) [java]
[2008-04-24 10:09:24] INFO  WEBrick::HTTPServer#start: pid=3998 port=10080
[2008-04-24 10:09:37] ERROR ArgumentError: wrong number of arguments (4 for 3)
        /home/jmettraux/tmp/./webdavhandler.rb:135:in `chk_utf8′
        /home/jmettraux/tmp/./webdavhandler.rb:135:in `detect’
…

Two small changes to webdavhandler.rb fixed this issue and a subsequent issue.

The updated webdavhandler.rb is available.

I’m just one in the crowd of people moving to Git. More specifically I have moved all my Rufus gems to github.

They are all under http://github.com/jmettraux

Other pieces of software should follow.

What I especially like about Git are forks. It’s not a crime to fork. That suits well the BSD / MIT licences I favour for my work. Instead of saying “no”, I can say “please fork”. For the rest, time will tell.

I’m planning a release of the rufus-verbs gem very soon.

I had a pleasant surprise today, Thibaut mailed me about his use of the Rufus::Scheduler in his Paris bike geolocation application, for calling his image generation code every twenty minutes.

Read more about how Thibaut leverages Hpricot and RMagick in his post entitled Data Visualization with Ruby and RMagick - Where Are Those Bikes ?

(Seems like I’m using geolocation quite liberally, are those bikes internet devices ? No the docking stations are, they feed their info about bikes availability to their center, which then provides the information via a website)

It’s not a post about the ‘cron’ aspect of the Rufus::Scheduler (ex-OpenWFEru::Scheduler), it’s about the cron expression of OpenWFEru’s process definition language.

A vanilla use case for the ‘cron’ expression, sending a reminder (every 10 minutes) to participant “toto” until he finishes the required activity (BPM for slavery) :

  class SampleDefinition < OpenWFE::ProcessDefinition

    # body of the process
    #
    sequence do
      activity_1
      activity_2
    end

    # subprocess definition
    #
    process_definition :name => "activity_1" do

      concurrence :count => 1 do
        # expecting just 1 branch to reply

        participant "toto"
        cron :tab => "10 * * * *" do # every 10 mn
          send_reminder :to => "toto@localpub.co.uk"
        end
      end
    end

    # ...
  end

Up until now (OpenWFEru 0.9.17 included), the cron expression relied on the variable system for storing its schedule (and eventually get rescheduled in case of engine restart). Schedules bound at the engine level are meant to last longer than the business process that initiated them.

The initial implementation worked fine. Two problems though :

1) this “bound at the engine level” was implemented as a special case, that made the code more complex, more prone to errors

2) the biggest problem : engine level “crons” were thus sitting in the engine environment as variable values and were not cancelable without a tedious manipulation (basically launching a process that erased the engine variable or stopping the engine, editing the engine env and restarting the engine).

The improved process administration console of Densha revealed acutely this “can’t cancel cron” issue. That called for a change.

This initial implementation of ‘cron’ was immediately replying to the parent expression after “hiding” itself in the among the local variables (or the engine variables).

The new implementation blocks until cancelled, hence the example process definition at the top of this post, where the concurrence expression, upon receiving a reply from its “participant toto” branch cancels the remaining “cron” branch.

If you want a cron that outlasts its initiating process, you now have to wrap it in a [sub]process. This subprocess will be visible to the process administrators, like any other process, thus easily manageable.

(That reminds me I have to dig the BPMN spec to find decent graphical representations for the cron, the when and the sleep expressions).

Writing documentation can be painful, but it’s a necessity. So my main task these days about OpenWFEru is writing down an example, it’s about the Tea Testing Team. This piece of documentation is not yet complete, but it might be a good start.

The example is based on the latest Densha, a Ruby on Rails based web application wrapping a workflow / BPM engine. It adds a small worklist implemented thanks to ActiveRecords, that makes a quite decent BPMS (business process management system). So this eye-candy Densha is used in the Tea Testing example to demonstrate what OpenWFEru is about.

(Somehow, supporting this example is the main motivation behind this release of OpenWFEru + Densha 0.9.17)

There is an online demonstration of Densha at http://difference.openwfe.org:3000.

I’ve written before about the main feature this 0.9.17 Densha brings, I labeled it process gardening, cancelling pieces of business process instances. Well, the 0.9.17 goes a bit further, you can download as yaml a part of a process instance, modify it, then reload it in the engine… Business process modifications on the fly (these are admin features, you don’t want your users to directly modify process instances, whatever their IT literacy level is).

Densha has a ‘fluo’ component, it’s a javascript library that turns the process definitions into graphical process representation (the graphical notation used is BPMN, as far as possible). It’s implemented in javascript so it can be reused in other web interfaces, whatever the web framework (or the language).

It’s one of my goals to provide soon a web process designer leveraging this ‘fluo’ javascript library.

For the edition of workitem payloads, Densha uses a very limited forms system, just string and hashes. It should be enhanced a bit in the upcoming releases, but I shall write soon documentation on how to use custom forms (up to one custom form per activity / participant). It would be nice to have xforms as well.

Densha itself is not meant to be a state of the art Ruby on Rails application, it’s rather an example, a proof of concept and others have already used pieces of it (or simply as an inspiration) for building their own systems.

OpenWFEru shall reach 1.0 soon. I’d like to work on a few simplifications before that, then it will be ready.

The quickstart for OpenWFEru is available at : http://openwferu.rubyforge.org/quickstart.html

The quickstart for Densha is at : http://openwferu.rubyforge.org/rquickstart.html

As always, feedback is welcome on the OpenWFEru users mailing list.

Incidentally, this is my 300th post. Long way.

Just released rufus-scheduler 1.0.5 (the sequel to the “openwferu-scheduler”), it features a best effort drift correction and a constraint on the scheduler precision (0.0 < p <= 1.0), for precisions other than the default 0.250 (4 times per second).

Thanks to Michael Goth for his feedback.

Now back to my usual workflow / BPM stuff.

My current objective is the release of OpenWFEru 0.9.17. There are many changes in the engine itself, but not much on its “interface”. There is however an interface that changed a lot, it’s the one of the [demo] web application around OpenWFEru.

The 0.9.16 release of this Densha web application features limited process administration capabilities : pausing/resuming a process instance, cancelling it.

“this business process instance has gone wrong, we have to cancel it”

The upcoming 0.9.17 release (already deployed on the demo web site) adds batch pausing / cancellation of processes and also some “process gardening” tools.

Process instances being execution trees, I’ve made it easy to chop off some branches (currently executing or not yet executed).

The [process] administrator can browse the execution tree of a running process instance and chop off (cancel) branches of it.

“this segment of this business process instance is not required any more, let’s remove it”

The web interface also allows modifying the process variables directly.

Workitems waiting in participant expressions are editable as well (though they could get overriden by the workitem coming back from the participant itself). Modifying the workitem waiting in a sleep expression might be more useful.

The next step for these process gardening tools will be the edition (instead of pure removal) of process branches. It’s already possible programmatically, but packaging that into a web interface is a hard task (will probably have to implement that process designer before). 0.9.17 will probably sport a primitive interface for process modification on the fly.