Process Microservice Description and Background

A microservice for managing BPMN processes.

Requirements:

1. Fault tolerance. If any node in the system fails, the system must remain operational.

2. Data safety. In the event of full or partial data loss on any storage node, data in the system must not be lost.

3. Horizontal scalability. As the number of processes grows, it must be possible to scale horizontally by adding nodes to the cluster, so as to avoid query performance degradation as the system ages. Completed processes must not negatively affect active ones.

MongoDB NoSQL database has been chosen as the data storage.

ECOS Process microservice connection diagram for MongoDB:

In the future, the Cassandra NoSQL database may be chosen as the data storage, as it can address several requirements at once:

1. Automatic data replication to a specified number of nodes (Data Safety)

2. All nodes in a Cassandra cluster are peers. There is no master node (Fault Tolerance)

3. A Cassandra cluster can be flexibly configured, and new nodes can be added to a running system (Horizontal Scalability)

ECOS Process microservice connection diagram for a Cassandra cluster:

Key Features:

1. In the Enterprise configuration, there are 3 levels of horizontal scalability: the storage layer, the gateway for storage access, and the cluster of stateless ECOS Process microservices

2. Since ECOS Process microservices are stateless, user requests can be routed to any of them. Hazelcast is used to synchronize actions between ECOS Process instances

3. Process definitions are stored in the external ECOS Apps microservice, which is already implemented and versions all process changes

4. Integration with external events is handled through the RabbitMQ message queue.

5. Cassandra is configured for QUORUM Read/Write (N/2 + 1), where N is the number of Cassandra nodes. This means a write is considered successful only when the majority of cluster nodes confirm it. Reads likewise require the majority of nodes to return up-to-date data. This configuration prevents situations where network issues between cluster nodes cause them to become out of sync.

6. An administrator can configure ECOS Process to connect to a Cassandra cluster via the central configuration

The NoSQL solution lacks full transactions, so to ensure data safety the process state is treated as immutable. Each process state is stored as a separate version. In a process instance, only the reference to the new state is updated, after all activities have completed successfully.

In addition to data safety, this also allows rolling back a process to any previous state, which is very helpful when unexpected situations arise.

Transaction lifecycle for a running process in ecos-process

A transaction in ECOS Process begins when an event occurs (e.g., “Contract Created”) or when a command is received (e.g., “Complete Task”).

When an event occurs, all subscribers to that event are checked, and for each subscriber the conditions (if any) are evaluated. If the conditions are not met, event processing ends. When conditions are satisfied, the required command is triggered. Processing then continues in the same way as if the command had been sent directly to the microservice.

When a command is received to perform an action in a process, the process state is read, a series of transitions and actions are performed according to the process definition, and if all operations succeed, a new version of the process state is created in the database, after which the reference in the process instance is updated to the new state. If external events or commands occurred during the transaction, they are sent to RabbitMQ upon transaction completion.