processi

about processes and engines

amqp, ruote, pong

Hartog de Mik just wrote an investigation into ruote and AMQP. In this blog post, he makes the long jump from zero to ruote + RabbitMQ participants via the vaguely charted ruote-amqp territory.

But right before clicking away to Hartog’s blog post, please consider this pure ruote pong solution :

require 'rubygems'
require 'ruote'

pdef = Ruote.process_definition do
  repeat do
    ping # mister ping, please shoot first
    pong
  end
end

class Opponent
  include Ruote::LocalParticipant

  def initialize (options)
    @options = options
  end

  def consume (workitem)
    puts @options['sound']
    reply_to_engine(workitem)
  end
end

engine = Ruote::Engine.new(Ruote::Worker.new(Ruote::HashStorage.new))

engine.register_participant :ping, Opponent, 'sound' => 'ping'
engine.register_participant :pong, Opponent, 'sound' => 'pong'

wfid = engine.launch(pdef)

sleep 5 # five seconds of ping pong fun

engine.cancel_process(wfid) # game over

 

http://ruote.rubyforge.org (home)
http://groups.google.com/group/openwferu-users (mailing list)
IRC #ruote on freenode.net

 

Written by John Mettraux

September 27, 2010 at 2:24 pm

Posted in ruby, ruote, workflow

quaderno

My task as a programmer seems mostly to be throwing forms at knowledge workers. I fool myself in thinking that a workflow engine is the panacea when deciding when and what form to present the user.

Workflows are supposed to be adaptable, the forms at the emerged tips even more so. “Can you add a text field here ?”.

In workflow technology, there seems to be this grail where the business user is knowledgeable enough to modify a process definition, and knowledgeable enough to modify a form. Defining a process has a strong taste of programming, and form building hints at hierarchical data structures. It’s a struggle to get right either of them.

I came up with a javascript library named quaderno that takes as input a form template written in a DSL (minilanguage is perhaps a better appellation) and a data dictionary, renders the form, and then, when the user is done, produces the updated data dictionary.

This template :

tabs
  tab "general"
    text_input name
    text_input age
    select gender [male, female]
  tab "appointments"
    box appointments.*^
      date .date
      text_input .goal
      text_area .comment

Will produce a form that looks like :

The second tab is gathering data in an “appointments” array. The suffix .*^ indicates that elements of the array, can be added and removed (+ and – merged into a *) and reordered at will (^).

This is a sample output, once the user has entered his values and the produce() function of quaderno somehow got called :

{ "name": "Toto",
  "age": "33",
  "gender": "male",
  "appointments": [
    { "date":"2010/9/18", 
      "goal":"general checkup", 
      "comment": "(and tell him to quit smoking)" } ] }

This example is visible at http://ruote.rubyforge.org/quaderno/viewer.html?sample=3, there is also another sample at http://ruote.rubyforge.org/quaderno/viewer.html.

I use quaderno for highly adaptable forms, whether they are part of a workflow or not. Perhaps I should not say “adaptable”, but “I trust other people enough to let them modify the forms”.

(at first, I had wanted to have three modes, form use, form view and form edit, but the last mode was a bit hard to maintain and the result was awful to use and to look at. I had come up with something better in terms of edition with the initial version, but I couldn’t get to show the link between data and fields. This 3rd version of quaderno is a back to the basics version, with this tiny DSL that moves away from WYSIWYG and exposes the data/form structure)

Quaderno is a MIT licensed javascript library. It’s not a solution, it’s a tool.

http://ruote.rubyforge.org/quaderno/viewer.html
http://github.com/jmettraux/quaderno

 

Written by John Mettraux

September 22, 2010 at 5:28 am

Posted in dsl, javascript, ruote, workflow

rocking the enterprise with ruby

“rocking the enterprise with ruby” is the name of the talk given by Munjal Budhabhatti and Sudhindra Rao at the RubyKaigi 2010. Munjal and Sudhindra are two ThoughtWorkers and their customer is Rackspace. Maybe I shouldn’t say customer, but rather “partner”.

While browsing the RubyKaigi’s program, I had noticed that their talk mentioned “business rules” and I was thinking to myself “there are things in this talk to learn for me as a ruote developer”. I attended the presentation and was quite surprised to learn that they’re using ruote. They especially indicated they had a provision process driven by ruote that they shrinked from 4 days to 18 minutes.

The talk enumerates the many details in the application which covers IP (4 and 6) automation and DNS automation. They’ve been delivering continuously, they’re building on Ruby, Rails, Ramaze and many other lesser known gems (ruote included) and they’re releasing open source libraries.

They had a funny anecdote : they were looking for a gem about a certain aspect of IP addresses and they found it on github, … and the developer was in the next cubicle.

The video of “rocking the enterprise with ruby” is available as well as the presentation slides.

When the talk was over, it was good fun to go over to the two speakers and introduce myself. They explained to me what they liked about ruote and how they sold it to their team. Ruote seems to fit nicely in their services and DSLs. I hope to meet them again to learn more.

It was then time to attend the following talks of the kaigi. It was a great event, too much to cover.

 

Written by John Mettraux

August 30, 2010 at 1:49 am

Posted in ruby, rubykaigi, ruote, workflow

Houston.rb ruote presentation by Wes Gamble

During the last Houston.rb meeting, Wes Gamble talked about ruote. Here are his slides :

 

Wes does an excellent job at presenting the main concepts in ruote, process definitions, participants, workflow vs state management and so on. I’m lucky to have people evangelizing ruote as I’m a too much a “you don’t understand workflow ? You certainly don’t need ruote” type.

There was a funny coincidence : my friend Alain Hoang was visiting Houston at that time and he joined the meeting. Alain and I were co-workers when I started working on ruote. He helped me with the initial Rakefile and tamed rote to build an initial website for the project. He also came up with the initial suggestion for the rufus namespace idea.

Many thanks to Wes for his great presentation (and to Alain for all his help).

 

Written by John Mettraux

August 16, 2010 at 12:46 am

Posted in ruby, rufus, ruote, workflow

Methodismus

“Method and Routine” is the title of the chapter 4 of the book 2 of Clausewitz On War. It was written between 1816 and 1830, the author was taken away by cholera before he could complete the final revision of this work. Only the first chapter of the first book was considered finished.

I found a few passages in “Method and Routine” that could be put in perspective with business process management / workflow. In our age, it’s common to draw parallels between the conduct of war and the conduct of business, so let me reproduce those passages here. I’m very interested in the rise of ACM and its relation to BPM (see Mastering the Unpredictable), those quotes bring forth valuable reflections (for me at least).

I took them from the excellent Howard and Paret translation, for those of you who read German, here is a link to an online version of the book/chapter.

“Method,” finally, or “mode of procedure”, is a constantly recurring procedure that has been selected from several possibilities. It becomes routine when action is prescribed by method rather than by general principles or individual regulation. It must necessarily be assumed that all cases to which such a routine is applied will be essentially alike. Since this will not be entirely so, it is important that it be true of at least ‘as many as possible’. In other words, methodical procedure should be designed to meet the most probable cases. Routine is not based on definite individual premises, but rather on the ‘average probability’ of analogous cases. Its aim is to postulate an average truth, which, when applied evenly and constantly, will soon acquire some of the nature of a mechanical skill, which eventually does the right thing automatically.

(Routines) As such they may well have a place in the theory of conduct of war, provided they are not falsely represented as absolute, binding frameworks for action (systems); rather they are the best of the general forms, short cuts, and options that may be substituted for individual decisions.

Finally we have to remember that as the number of officers increases steadily in the lower ranks, the less the trust that can be placed on their true insight and mature judgement. Officers whom one should not expect to have any greater understanding than regulations and experience can give them have to be helped along by routine methods tantamount to rules. These will steady their judgment, and also guard them against eccentric and mistaken schemes, which are the greatest menace in a field where experience is so dearly bought.
Routine, apart from its sheer inevitability, also contains one positive advantage. Constant practice leads to brisk, precise, and reliable leadership, reducing natural friction and easing the working of the machine.
In short, routine will be more frequent and indispensable, the lower the level of action. As the level rises, its use will decrease to the point where, at the summit, it disappears completely. Consequently, it is more appropriate to tactics than to strategy.

But any method by which strategic plans are turned out ready-made, as if from some machine, must be totally rejected.

The chapter ends with warning about poverty of imagination and examples where routines where re-used out of context and led to [military] disaster. (Could the re-use of those quotes out of their original context led to [some kind of] disaster ?).

When it comes to strategy and tactics, here is how Clausewitz put it in the same book :

Tactic teaches the use of armed forces in the engagements.
Strategy teaches the use of engagements for the object of war.

 

Written by John Mettraux

August 6, 2010 at 6:52 am

Posted in acm, books, bpm, workflow

old process, knowledge worker

I’ve started reading Mastering the Unpredictable by Keith Swenson et al. I’d like to weave two links from from the first chapter to two things I have been exposed to.

“Mastering the Unpredictable” is a collection of Essays by Keith Swenson, Max J. Pucher, Jacob P. Ukelson and others. Those essays are reflections on “Adaptive Case Management” (ACM) and its rise alongside “Business Process Management” (BPM) (or where the BPM dream falls short). The title of the book derives its name from the unpredictability of the flow of work ACM would like to support.

the old process

The first chapter of the book is entitled “The Nature of Knowledge Work”, it starts with an example of a night rescue operation. How unfitting a BPMS (Business Process Management System) is for supporting the planning of such an operation. That reminded of a process I’ve been taught in officers’ school. You might summarize it as “assess, plan, execute” (as the first chapter of ‘MtU’ does), but let’s take the time to re-read that process definition.

I’m translating from my native language to english, here is the transcript :

The military method of decision taking acknowledges that
– only a systematic and rational process allows the integration of several people in the decision process,
– most of the time, the commander and the staff are under time pressure, are faced with doubt, only get imprecise, wrong or outdated pieces of information.

The commanding activities gather all the activities of the commander and of the staff, from the arrival of a task or the occurrence of a situation requiring an action, until the achievement of the task.

By respecting always the same sequence of commanding activities, the commander and the staff acquire the confidence necessary to lead in extraordinary and crisis situations.

The flow of commanding activities, performed successively :

1. planning the action
1.1 grasping the problem
1.2 urgency measures triggering
1.3 setting the schedule
1.4 situation assessment
1.5 decision taking
1.6 engagement plan conception
2. leading the action
2.1 issuing orders
2.2 control and conduct of the engagement

The evolution of the situation, as followed by the commander and his control and intelligence teams are the basis for direction the engagement. A change in the situation triggers a new commanding activities process.

During the action planning phase, it might be wise or it is simply necessary to start planning ‘reserved decisions’. The result of the planning of those decisions can generally be used during the action leading phase.

Apart from the direction of the ongoing engagement, the staff has to take care of the planning of subsequent potential task, this is called subsequent planning. The result of this planning is a new engagement plan or new engagement orders.

The first paragraph is interesting : “only a systematic and rational process allows the integration of several people in the decision process”. This process is a convention, shared by the members of a military organization, meant to blossom between a commander and his staff.

When I got taught this process, it often got referred as “commanding rhythm”. We were meant, as young officers, to already breath it, follow its flow. When a new mission arrived or a new situation occurred, we had to quickly grasp the problem at hand, trigger urgency measures (in order to preserve [what was left of] our initiative), determine when the decision would be taken, and all sorts of “when”, then assess, decide, plan… With the staff in our heads.

We were taught to learn and apply this process on our own, so that a) our decisions would be produced efficiently b) we could later join staff and participate in the process collaboratively for a commander. With some luck and lots of skills we could c) command, leveraging the people + the process.

the knowledge worker

The first chapter of “Mastering the Unpredictable” goes on with the distinction between routine work and knowledge work. The character of the “knowledge worker” is evoked, and that reminded me of this passage, from another book :

When an architect sits down with pen and paper to determine the strength of an abutment by a complicated calculation, the truth of the answer at which he arrives is not an expression of his own personality. First he selects the data with care, then he submits them to a mental process not of his own invention, of whose logic he is not at the moment fully conscious, but which he applies for the most part mechanically. It is never like that in war. Continual change and the need to respond to it compels the commander to carry the whole intellectual apparatus of his knowledge within him.

He must always be ready to bring forth the appropriate decision. By total assimilation with his mind and life, the commander’s knowledge must be transformed into a genuine capability.

This is taken from Carl von Clausewitz On War (Howard and Paret edition, p. 147). There is the opposition between routine work and knowledge work. Clausewitz goes on and speaks about “natural talent” for the crafty knowledge work… commander.

conclusion

These were my two links. An old process and an even older (~1830) quote about knowledge work.

The tool of the commander is his staff. What is the tool of the knowledge worker ?

For those of you lucky enough to be in Japan this friday, Keith Swenson will be speaking about Mastering the Unpredictable in Yokohama. Feel free to register at http://atnd.org/events/5954.

 

Written by John Mettraux

June 28, 2010 at 2:16 am

Posted in acm, bpm, workflow

ruote demos and examples

Recently, contributions to ruote have been on the rise.

I’ve listed them on the ruote source page, but here is a small recap :

Torsten is maintaining ruote-on-rails : “A simple example Rails app for demonstrating the usage of ruote in Rails using RuoteKit”.

Eric Dennis has gone a step further with ruote-rails-example, he extended Torsten’s work to add ruote-amqp participants.

David Greaves, in the context of his BOSS project has announced a minimal-ish ruote ruote-amqp setup.

Special kudos as well to David for his ruote overview page in the wiki.

Two other “in the wild” examples of ruote : David Austin‘s qips-wfmgr and Chris Beer‘s fedora-workflow.

Many thanks to all of these people, they are making ruote better.

Written by John Mettraux

June 21, 2010 at 1:43 am

Posted in opensource, ruby, ruote, workflow

ruote 2.1.10 released

ruote is an open source workflow engine implemented in Ruby.

It takes as input process definitions and interprets them. It routes work among participants according to the flow described in those process definitions.

Version 2.1.10 is out and contains lots of small improvements and features requested and developed with the help of various people.

I have also updated/upgraded the various storage libraries around ruote and added to their ranks two new gems.

 

in ruote

The changelog has all the details, but let’s look at the LocalParticipant module which saw the most improvements. Following suggestions by Oleg and Avishai, I added the on_reply, re_dispatch, reject and put/get methods
to it.

A participant has a consume(workitem) which is called as the participant gets handed a workitem. When dealing with a remote participant, it’s sometimes necessary to have logic alongside the consume method to deal with the workitem when it comes back. That’s the role of the on_reply(workitem) method.

re_dispatch(workitem) forces a re-delivery from the participant expression to the ‘real’ participant.

The reject(workitem) method is about participants that reject a workitem, forcing the worker to let another worker deal with the dispatching to the participant.

LocalParticipant now has a pair put(k, v) / get(k) for stashing data between consume / on_reply. The data is stashed directly in the ParticipantExpression.

around ruote

Since CouchDB 0.11 is out and it allows ?feed=continuous&include_doc=true, I could drop the polling work from ruote-couch and simply keep connected and “react” on incoming changes in messages and schedules. It makes for a robust ruote-couch. I also made sure ruote-couch 2.1.10 can be shared by multiple workers.

This ruote-couch isn’t very fast, but I’ve come to appreciate CouchDB and use it in 2 or 3 projects. I like the comfort of having Couch as a storage for ruote.

I have upgraded ruote-dm as well. It’s a DataMapper backed ruote storage gem. I made sure ruote-dm could be shared by multiple workers (though I still have an issue with one of those special tests, I hope to fix that in the coming weeks).
It seems to work well with the latest DataMapper (1.0).

For the fun, I wrote a ruote-redis gem. As the name implies, it uses Redis as a persistence mechanism for ruote. It’s fast, multi-worker ready and was fun to develop. Not sure if I will use in production, but it can sit as an example at least.
There have been very interesting developments in Redis, this ruote-redis only uses a very limited set of those features. There is probably a better way.

I got interested in Beanstalk, its simplicity and minimal size are seducing. I started working with it as a kind of transient storage, but ended up writing a server/client thing. You can start a ruote-beanstalk storage linked to a beanstalk queue, a server and then start an engine and some workers that will communicate with the real storage via the beanstalk queue. Queue in the middle.

My initial purpose when investigating Beanstalk was to use it to pass workitems between the engine and the ‘real’ participants. I didn’t forget this and ruote-beanstalk comes with participant and receiver implementations.
It could become an alternative to ruote-amqp, it is at least a good example of a participant / queue / receiver set.

Finally, there’s ruote-cukes, a gem that adds extensions/examples for testing ruote process definitions with Cucumber.

next steps

I still have to improve the documentation, there are lots of weak / missing points in it.

The listen expression will be enhanced, I want it to listen (react) to process errors and to flows entering or leaving a given tag (told you).

Lots more things.

thanks

Many thanks to Torsten, Ian, Oleg, Avishai, Patrice and Matt. And also many thanks to Wes Gamble (and Sergio) for organizing a ruote BOF at the latest Railsconf.

BTW, David Greaves is writing an overview in the ruote wiki as he’s assembling his build system. Feel free to chime in.

This release is dedicated to my friend Sawada Tomoaki, who left us last friday.

 

Written by John Mettraux

June 15, 2010 at 6:57 am

Posted in ruby, ruote, workflow

workflow, bpm, selected resources

I have started to gather posts and blogs I think are worth a read in the workflow, BPM, Adaptive Case Management, rules, etc fields.

The list is at http://ruote.rubyforge.org/resources.html

I hope to list there resources that are sincere and passionate, and that challenge my point of view on workflow engines and enterprisey programming in general. I will try to avoid things that are press release like, that include words like “leading” or “fortune 500”, that are too closed, not leaving place for a slice of doubt.

 

Written by John Mettraux

March 31, 2010 at 4:35 am

Posted in acm, bpm, rules, ruote, workflow

hanabi

Hanabi is a small presentation tool.

You load hanabi.js, you create an empty canvas with a given id, you point it to the image and give it a list of points :

  var source = 'path/to/my_image.png';
  var points = [
    { x: 504, y: 340, zoom: 0.5, r: Math.PI * 2 },
    { x: 504, y: 340, zoom: 3.5, r: Math.PI * 2, stop: false },
    { x: 641, y: 260, zoom: 2.0, r: Math.PI * 2 },
    { x: 832, y: 191, zoom: 2.0, r: Math.PI * 2 },
    { x: 977, y: 173, zoom: 2.0, r: Math.PI * 2 },
    { x: 1035, y: 297, zoom: 2.0, r: Math.PI * 2 },
    { x: 1037, y: 383, zoom: 4.0, r: Math.PI * 1.75 },
    { x: 497, y: 335, zoom: 2.0, r: Math.PI * 2 },
    { x: 696, y: 513, zoom: 2.0, r: Math.PI * 2.1 },
    { x: 914, y: 501, zoom: 2.0, r: Math.PI * 2 },
    { x: 363, y: 391, zoom: 2.0, r: Math.PI * 1.9 },
    { x: 86, y: 451, zoom: 2.0, r: Math.PI * 2 },
    { x: 506, y: 336, zoom: 2.0, r: Math.PI * 2 },
    { x: 321, y: 31, zoom: 2.0, r: Math.PI * 0.2 },
  ];
  Hanabi.init('can', source, points);

Then you click your way through the image.

http://github.com/jmettraux/hanabi

 

For the real thing : http://prezi.com

 

Written by John Mettraux

March 10, 2010 at 1:51 pm

Posted in javascript