Documentum Stateless Process Engine
Documentum 6.7 SP1 contains an interesting feature that is somewhat hidden from sight. When installing the Process Engine 6.7 SP1 you get the ability to use the “stateless process engine” for the development of workflows.
Surprisingly this stateless process engine is not mentioned in the new features section of the Process Engine 6.7 SP1 release notes. Instead it is described in its Technical Notes. Here the following is mentioned about it:
“Use the stateless process engine to execute process instances in memory. The stateless process engine relies on a Synchronous Invoke Process activity that invokes a workflow synchronously. The synchronous transaction commits only at the end of the workflow, which ensures that if an error occurs during the subprocess, the entire workflow fails.
The stateless process engine offers the following benefits:
- Improved performance since the stateless process executes completely in the method server in one thread and in one session.
- The Synchronous Invoke Process activity template supports input and output mapping. The standard Invoke Process template has input mapping only.
- If better performance is desired and the invoked subprocess contains only automatic activities, the stateless process engine is easily applied to an existing xCelerated Composition Platform (xCP) solution. Applying the stateless process engine involves replacing the Invoke Process activity with the Synchronous Invoke Process activity.”
In the same technical notes EMC claims as an example for a workflow consisting of 20 activities a performance increase from 3 seconds to 160 milliseconds.
This all sounds pretty interesting. One of the design patterns to decrease the amount of custom coding and to increase configurability and flexibility is to develop business functionality as a process consisting of OOTB configurable activities. A drawback of this approach is that executing this functionality in a workflow can be slow. Especially when a large number of processes are running at the same time all these workflow tasks can significantly slow down the performance of the entire server.
Being able to perform a number of activities in a single thread and a single transaction can offer a huge performance gain when a large number of processes are running. Of course this concept also has a number of limitations:
- Manual activities within a stateless process are not supported. This seems kind of evident since it implies transferring control back to the client. Also timer settings, inbound initiate or step activities and trigger conditions (events) are not supported.
- Since all activities are running in a single transaction this increases the duration of the transaction. If the same object is included in multiple processes there is an increased risk of version mismatch errors.
- Because the activities are running in a transaction, actions that open their own transaction such as lifecycle actions are not supported.
- A stateless process does not populate the BAM database for reporting.
- All activities within a stateless process run as the same user as the Synchronous Invoke Process activity that triggered the process.
In my opinion these limitations are in most cases easily outweighed by the performance gain you can achieve.
Installing Process Engine 6.7 SP1
The first hurdle is the installation. The release notes of the Process Engine 6.7 SP1 mentions the fact that you can install this version of the Process Engine on a number of content server versions. Using a content server 6.7 on Windows I assumed the installation would be a breeze. But alas, almost immediately after starting the installation the installer complains about the following:
“An exception occured in: DiWAPeInitialize, beanID: PeInitialize – DiWAPeInitialize failed! – Method Server deployment descriptor does not exist at C:Documentumjboss5.1.0serverDctmServer_MethodServerdeployServerApps.earDmMethods.warWEB-INFweb.xml. The Content Server may not have been properly installed.”
This is not surprising since the 6.7 content server comes with a 4.3.0 version of JBoss. The solution I used was to upgrade the content server to 6.7 SP1 which does include a 5.1.0 version of JBoss. After that the installation of the Process Engine went smoothly.
Putting it to the test
When you open the Process Builder you will see the “Synchronous Invoke Process” activity in the set of Content Services activity templates. I created a simple process with a manual activity, a Synchronous Invoke Process activity to invoke a subprocess and another manual activity. The subprocess consisted of a DQL read activity, a Create Folder activity, a Link to Folder activity and a process Data Mapping activity. When running the process the subprocess is executed in a flash.
Curious about the duration of the subprocess I created a simple custom activity that only records current date and time to a log file. I included this activity just before and after the invocation of the subprocess and at the beginning and end of the subprocess itself. I also created the combination of the process and subprocess as a single process and included the timing activity just before and after the activities that make up the subprocess. Then I ran these processes a number of times to see what the differences were.
On average running the entire subprocess took about 500 milliseconds. Executing only the activities in the subprocess – i.e. without the overhead of starting the process and returning to the parent – took about 200 milliseconds. In comparison, running the same activities in the single process took on average about 800 milliseconds. So even including the overhead of calling the subprocess the stateless process is still faster.
Off course this is not a waterproof piece of research, its purpose is merely to get an indication whether using stateless processes will actually offer a performance gain. So far the signs are promising.
One more thing about output mapping: the Synchronous Invoke Process activity template enables you to map data from the subprocess back to the parent process. A limitation I ran into is that it is not possible to map properties from the attached object to the process data. Even though the data mapper allows you to do that when configuring the activity, the mapping will generate an error when executing the process. But since the object is already part of the parent process this should not really be a problem.