06. February 2022

8 min

Bringing Cloud Native Process Automation to the Micronaut Framework

With the release of the Micronaut-Zeebe integration, we bring cloud-native process automation to the Micronaut community. In combination with GraalVM, Zeebe Workers start in a fraction of a second, making them ideal for deployment in the cloud.

1. Process Automation is a Competitive Advantage

Being able to quickly adapt to changes in customer needs, market situations or even organizational demands is a competitive advantage for any organization. Process Automation (based on the ISO standard BPMN 2.0) addresses these challenges by having business and IT working closely together (known as Business-IT alignment) to achieve the maximum business value. A process model with its graphical representation allows business and developers to have a common language. Developers can concentrate on implementing the business requirements and trust the process engine to take care of already solved technical challenges.

We are convinced that process automation can complement modern microservice architectures to address common challenges. Embedding a lightweight process engine into an application provides some building blocks for free which you can use:

keeping state for long running processes which react to incoming events
incident handling and stateful retries
process transparency during development and runtime.

Our vision is to bring process automation to the Micronaut Community. This actually fits quite well to Camunda‘s vision which is “Automate Any Process, Anywhere”.

We believe process automation can be a success factor for many use cases. And we also believe that process automation with Camunda is a valuable asset to the Micronaut community.

We already provide a Micronaut Camunda Integration project to use Camunda’s embedded process engine in a Micronaut project. To also support Camunda’s cloud native solution we recently started the Micronaut Zeebe integration project to easily implement Zeebe job workers based on the Micronaut Framework. These workers interact with Camunda’s cloud offering (or your self-hosted Zeebe Cluster) to process the actual work.

2. Why use Camunda Cloud?

Camunda Cloud provides on-demand Process Automation Platform as a service. The BPMN (Business Process Model and Notation) standard based workflow engine scales horizontally and is based on the open source project Zeebe. The platform also includes built-in tools for collaborative modelling, operations and analytics.

The process automation platform is provided as a Software-as-a-service (SaaS) but can also be deployed on-premise using the open source components.

Within Camunda Cloud the business logic is delegated to job workers (for example implemented with the Micronaut Framework) which register to specific topics, process work, and report the result. The process engine takes care of the orchestration including escalation and error handling.

3. Architecture Overview

The Zeebe architecture is divided into two main components: Zeebe Cluster and Job Workers.

The Zeebe cluster consists of

Brokers are distributed workflow engines which keep the state of running process instances.
Gateways are the single entry point to the Zeebe Cluster.
Exporters are event streams to persist state e.g. for incidents or monitoring purposes.

Job workers are used to execute business logic and use the client to drive the process. This is where you implement your business logic. Job Workers (possibly using the Micronaut Framework) can be deployed to the cloud and scaled as needed.

Based on: https://docs.camunda.io/docs/components/zeebe/technical-concepts/architecture/

4. Why use the Micronaut Framework?

The Micronaut Framework is a JVM-based microframework to build microservices. It’s similar to Spring Boot from a developer perspective but optimised in terms of resource usage. Typically, Spring Boot applications tend to have increasing startup times and high memory usage with every Spring Boot Starter you add. The reason is mainly that those dependencies do a lot of magic at runtime by scanning the classpath, using reflection, creating proxies, creating caches, and modifying byte code.

The Micronaut Framework has approached this problem by using ahead-of-time (AOT) compilation in the build phase. The development model is actually very close to Spring Boot, i.e. you have your container-managed beans and you can inject them where needed. However, you have the advantage that the magic is done at compile time and more errors are found during the build (which takes a little bit longer but Gradle takes good care of only building deltas).

As described in the architecture overview, processes are driven by workers and depending on the project and there could be quite a few of them. If workers need to be scalable, each spin-up of a new instance consumes additional resources. With a lower footprint the costs for hosting workers can be reduced.

Micronaut Applications can be compiled to native executables and will then start within milliseconds, see details in the later section “GraalVM”.

If you have experience with Spring Boot then the Micronaut Framework will look quite familiar and easy to set up.

5. Bringing Camunda Cloud and the Micronaut Framework together

We’ve started an open source project at https://github.com/camunda-community-hub/micronaut-zeebe-client which makes it very easy to build a Micronaut Framework based Zeebe Worker. With this integration we bring process automation (based on Zeebe) to Micronaut!

Implementing a job worker is simple:

Add the dependency of the Zeebe integration project
Implement job workers for the service types defined in the process model
Configure the job worker with credentials to connect to Camunda Cloud (or your on-premise Zeebe Cluster)
Optional: configure further parameters

We will show this in detail in the following showcase.

6. Our Showcase: We’re going to the movies

We created a showcase based on the Micronaut Zeebe Client and Camunda Cloud. A customer can reserve seats for a movie and receives a message with all the details. During the process the price is calculated and withdrawn from the users bank account. If seats are no longer available, the customer is offered alternative seats which can be accepted or not. In the end tickets are created in the ticket system and a QR code is generated and sent to the customer.

The source code of this project can be found on GitHub where you will also find instructions to run the process locally or in your Camunda Cloud environment: https://github.com/NovatecConsulting/micronaut-zeebe-cloud-cinema

This is the process model:

6.1. Project Setup

The Zeebe integration works with both Gradle and Maven, but we use Gradle because it has better Micronaut Support.

We added the dependency of our integration project:

1	implementation "info.novatec:micronaut-zeebe-client-feature:1.4.0"

If you create your own project you have the following options to integrate the Zeebe integration:

Create a new Micronaut project using Micronaut Launch and check that the “zeebe” feature is selected.
Manually add the dependency to an empty Micronaut project

6.2. Creating the Process Model

Use the Camunda Modeler to create your process model. Each activity must be linked to a worker implementation using the type field. This string is later used in your worker implementation to correctly identify the activity.

6.3. Worker Implementation

There are two options to register a job worker:

Multiple workers per class: annotate a method with @ZeebeWorker (see below)
Single worker per class: annotate a class with @ZeebeWorker implementing the “io.camunda.zeebe.client.api.worker.JobHandler” interface.

To reserve a seat we implemented the following worker, which calls a service to reserve the seats and sets the ticket price as process variable:

@ZeebeWorker(type = "reserve-seats")
public void reserveSeats(final JobClient client, final ActivatedJob job) {
  logger.info("reserving seats");
  List<String> seats = Variables.getSeats(job);
  if (!seats.isEmpty()) {
    seatService.reserveSeats(seats);
    int ticketPrice = ticketService.calculateTicketPrice(seats);
    Map<String, Object> variables = Variables.empty().withTicketPrice(ticketPrice).get();

    zeebeClient.newCompleteCommand(job.getKey())
      .variables(variables)
      .send()
      .exceptionally(throwable -> {
        throw new RuntimeException("Could not complete job " + job, throwable);
      });
  } else {
    client.newFailCommand(job.getKey())
      .retries(0)
      .errorMessage("no seats found")
      .send()
      .exceptionally(throwable -> {
        throw new RuntimeException("Could not fail job " + job, throwable);
      });
  }
}

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

						@ZeebeWorker(type = "reserve-seats")
public void reserveSeats(final JobClient client, final ActivatedJob job) {
  logger.info("reserving seats");
  List<String> seats = Variables.getSeats(job);
  if (!seats.isEmpty()) {
    seatService.reserveSeats(seats);
    int ticketPrice = ticketService.calculateTicketPrice(seats);
    Map<String, Object> variables = Variables.empty().withTicketPrice(ticketPrice).get();
 
    zeebeClient.newCompleteCommand(job.getKey())
      .variables(variables)
      .send()
      .exceptionally(throwable -> {
        throw new RuntimeException("Could not complete job " + job, throwable);
      });
  } else {
    client.newFailCommand(job.getKey())
      .retries(0)
      .errorMessage("no seats found")
      .send()
      .exceptionally(throwable -> {
        throw new RuntimeException("Could not fail job " + job, throwable);
      });
  }
}

					

			

The type relates to the service task type defined in the Camunda Modeler. When the work is done, the worker informs the engine that a job is done or failed by sending a complete or fail command.

7. Testing

7.1. Unit Testing the Process Model

With the Release of Zeebe 1.3.0 Camunda published the zeebe-process-test for testing Zeebe process models. It looks promising for unit testing. However, we didn’t manage to implement integration tests in combination with real workers.

See Camunda’s blog Post Testing Processes for Camunda Cloud and Zeebe for more details.

This is how unit testing of process models looks like:

@ZeebeProcessTest
class ProcessTest {

  InMemoryEngine engine;
  ZeebeClient client;
  @SuppressWarnings("unused")
  RecordStreamSource recordStreamSource;

  @Test
  void workerShouldProcessWork() {

    // Deploy process model
    DeploymentEvent deploymentEvent = client.newDeployCommand()
      .addResourceFromClasspath("bpmn/say_hello.bpmn")
      .send()
      .join();

    BpmnAssert.assertThat(deploymentEvent);

    // Start process instance
    ProcessInstanceEvent event = client.newCreateInstanceCommand()
      .bpmnProcessId("Process_SayHello")
      .latestVersion()
      .send()
      .join();

    engine.waitForIdleState();

    // Verify that process has started
    ProcessInstanceAssert processInstanceAssertions = BpmnAssert.assertThat(event);
    processInstanceAssertions.hasPassedElement("start");
    processInstanceAssertions.isWaitingAtElement("say_hello");

    // Fetch job: say-hello
    ActivateJobsResponse response = client.newActivateJobsCommand()
      .jobType("say-hello")
      .maxJobsToActivate(1)
      .send()
      .join();

    // Complete job: say-hello
    ActivatedJob activatedJob = response.getJobs().get(0);
    client.newCompleteCommand(activatedJob.getKey()).send().join();
    engine.waitForIdleState();

    // ..

    // Verify completed
    engine.waitForIdleState();
    processInstanceAssertions.isCompleted();
  }
}

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

						@ZeebeProcessTest
class ProcessTest {
 
  InMemoryEngine engine;
  ZeebeClient client;
  @SuppressWarnings("unused")
  RecordStreamSource recordStreamSource;
 
  @Test
  void workerShouldProcessWork() {
 
    // Deploy process model
    DeploymentEvent deploymentEvent = client.newDeployCommand()
      .addResourceFromClasspath("bpmn/say_hello.bpmn")
      .send()
      .join();
 
    BpmnAssert.assertThat(deploymentEvent);
 
    // Start process instance
    ProcessInstanceEvent event = client.newCreateInstanceCommand()
      .bpmnProcessId("Process_SayHello")
      .latestVersion()
      .send()
      .join();
 
    engine.waitForIdleState();
 
    // Verify that process has started
    ProcessInstanceAssert processInstanceAssertions = BpmnAssert.assertThat(event);
    processInstanceAssertions.hasPassedElement("start");
    processInstanceAssertions.isWaitingAtElement("say_hello");
 
    // Fetch job: say-hello
    ActivateJobsResponse response = client.newActivateJobsCommand()
      .jobType("say-hello")
      .maxJobsToActivate(1)
      .send()
      .join();
 
    // Complete job: say-hello
    ActivatedJob activatedJob = response.getJobs().get(0);
    client.newCompleteCommand(activatedJob.getKey()).send().join();
    engine.waitForIdleState();
 
    // ..
 
    // Verify completed
    engine.waitForIdleState();
    processInstanceAssertions.isCompleted();
  }
}

					

			

7.2. Testing the Job Worker’s behaviour

The job worker’s behaviour can be tested with the zeebe-worker-java-testutils library. You basically provide the input variables to the job worker and verify that the result is as expected:

@Test
void test_worker_happy_path() throws Exception {
  // given
  when(service.generateValue(any())).thenReturn("myValue");

  // when
  worker.doWork(jobClient, activatedJob);

  // then
  assertThat(activatedJob).completed();
  assertThat(activatedJob.getOutputVariables()).containsExactly(entry("myVariableKey", "myValue"));
}

				
					
				1
2
3
4
5
6
7
8
9
10
11
12

						@Test
void test_worker_happy_path() throws Exception {
  // given
  when(service.generateValue(any())).thenReturn("myValue");
 
  // when
  worker.doWork(jobClient, activatedJob);
 
  // then
  assertThat(activatedJob).completed();
  assertThat(activatedJob.getOutputVariables()).containsExactly(entry("myVariableKey", "myValue"));
}

					

			

7.3. Integration Test with Testcontainers

Alternatively, you can run an integration test with your real workers against a Zeebe Broker running in Docker by using Testcontainers. The project zeebe-test-container provides the framework which can then be used in the test.

In the following test we’re verifying that a job worker is writing an expected log entry:

@Testcontainers
class TestcontainersIntegrationTest {

  @Container
  ZeebeContainer zeebeContainer = new ZeebeContainer(DockerImageName.parse("camunda/zeebe:1.3.2"));

  @Test
  void workerShouldProcessWork() throws InterruptedException {

    Logger logger = (Logger) LoggerFactory.getLogger(GoodbyeHandler.class);

    ListAppender<ILoggingEvent> listAppender = new ListAppender<>();
    listAppender.start();
    logger.addAppender(listAppender);

    try (ApplicationContext applicationContext = ApplicationContext.run(
      Collections.singletonMap("zeebe.client.cloud.gateway-address", zeebeContainer.getExternalGatewayAddress())
    )) {
        while (listAppender.list.stream().noneMatch(e -> e.getFormattedMessage().contains("Retrieved value"))) {
        System.out.println("Waiting another second...");
        Thread.sleep(1000);
      }
    }
  }
}

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

						@Testcontainers
class TestcontainersIntegrationTest {
 
  @Container
  ZeebeContainer zeebeContainer = new ZeebeContainer(DockerImageName.parse("camunda/zeebe:1.3.2"));
 
  @Test
  void workerShouldProcessWork() throws InterruptedException {
 
    Logger logger = (Logger) LoggerFactory.getLogger(GoodbyeHandler.class);
 
    ListAppender<ILoggingEvent> listAppender = new ListAppender<>();
    listAppender.start();
    logger.addAppender(listAppender);
 
    try (ApplicationContext applicationContext = ApplicationContext.run(
      Collections.singletonMap("zeebe.client.cloud.gateway-address", zeebeContainer.getExternalGatewayAddress())
    )) {
        while (listAppender.list.stream().noneMatch(e -> e.getFormattedMessage().contains("Retrieved value"))) {
        System.out.println("Waiting another second...");
        Thread.sleep(1000);
      }
    }
  }
}

					

			

Do you have any input, ideas, or know libraries we’re missing? Let us know!

8. Fast Startup and Low Memory Footprint with GraalVM

Micronaut applications can be compiled as native applications using GraalVM. With GraalVM the Micronaut Framework based external worker will start up and connect in about 35 milliseconds and use even less memory.

There are some drawbacks: the compilation takes a few minutes and the application will only run on a single platform (you could put this in a Docker image to run it anywhere).

A good use case for minimal start up times and memory usage is the cloud where you can scale your army of workers up and down as needed. The workers will be available almost instantly and use less memory – thereby reducing costs. So even if the startup time is not critical for you, the reduced memory consumption should convince anyone deploying to the cloud.

When creating native images, a static code analysis determines which code is reachable. However, there are cases in which usages cannot be predicted, e.g. reflection or dynamic class loading. If code paths are not detected this will result in runtime errors. Luckily, you can configure an agent which will intercept all calls of a regular JVM execution to generate a configuration file which can subsequently be used to generate the native image.

With this setup our native external task worker starts and connects to a local broker within 35 milliseconds!

You can find more details regarding setup, configuration, and build at https://github.com/camunda-community-hub/micronaut-zeebe-client#graalvm

9. Try it Out!

Are you now eager to try this out?

Let’s get started:

Give us a ⭐ on GitHub: https://github.com/camunda-community-hub/micronaut-zeebe-client/stargazers
Thanks You!
You can register for a 30 day free trial license at https://camunda.com/products/cloud/ to get a Zeebe Cluster running in the Camunda Cloud. Alternatively, you can set it up yourself on-premise based on Zeebe’s Deployment Guide.
Jump to the “Getting Started” section of https://github.com/camunda-community-hub/micronaut-zeebe-client to implement your first job worker!

You’re welcome to try this out and we’d love to hear your feedback and discuss your ideas!

And as always: we’re happy to support you with our BPM Technology Consulting.