from ‘adhoc’ to ‘distributed’
I’ve already posted twice about adhoc processes and especially the eval loop which could be used to enact them.
This eval loop could also be used for another purpose : distributed processes.
Last year I supervised the work of a student, Dong Hun Kang, focusing on distributing processes among multiple OpenWFE engines. This work was a success, but now I begin to realize that we went too far as we implemented a DistributedExpressionPool transmitting expressions from the initial engine to the other engines over HTTP.
With an eval loop within each of the distributed engines, it’s pretty easy to disseminate the work among the nodes (the engines) by just sending workitems (with the classical OpenWFE dispatching mechanisms) to them. No need for a DistributedExpressionPool, no need for distributing expressions.
An engine would simply distribute the work[flow] among its fellow engine by sending them ‘code’ to eval.
It would look like :
<sequence> <!-- 'q' stands for 'quote' --> <set field="__process__"> <q> <sequence> <participant ref="operation_1" /> <participant ref="operation_2" /> </sequence> </q> </set> <participant ref="engine_alpha" /> <set field="__process__"> <q> <sequence> <participant ref="operation_3" /> <participant ref="operation_4" /> </sequence> </q> </set> <participant ref="engine_beta" /> </sequence>
In this example, the initial engine (the one executing that code) has no choice, it has to send the work to engine_alpha then to engine_beta, but one could imagine ways of determining this distribution on the fly (maybe operation 1 had better be run on the node hosting ‘engine_beta’).
Another enhancement : in this example, after engine_alpha terminates its work, the flow gets back to the initial engine which forwards it to engine_beta. It is easy to rewrite this flow so that engine_alpha directly forwards the work to engine_beta (it will then come back to initial_engine).