Introduction to Microservices Testing and Consumer Driven Contract Testing with PACT

Background Information

In the world of software engineering we frequently face change. The arrival of the concept of “Microservices” is one of those recent events, which changes not only the architecture of software, but also the way teams are organized and how they work together.

Here is one of the definitions of Microservices, introduced by M. Fowler and J. Lewis: “…is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.” In addition to that, the concept of microservices gives us the flexibility to deploy services independently of each other.

In this blog post we would like to give an overview of how testing changes in the new world of microservices. We will also introduce the details of Consumer-Driven Contract Testing and the frameworks that support it. Looking back to the well known concept of the Agile Testing Pyramid, we want to do a re-check if the idea in it is still applicable and valid when we speak about microservices testing.

For better understanding, we will use the following example model to describe the concept behind microservices testing:

On Figure 1 above, we can see that we have two microservices, communicating between each other via REST. The first service is in the role of a Consumer, while the second is in the role of a Provider. The Provider or Product service in our case provides all product relevant information such as name, type and description, by a given product identification number. The Consumer, on the other hand, fetches the data and consumes the relevant information. There can be of course as many consumers as you need. Later on, we are going to show an example with two consumers, but those should be enough for building up an understanding.

Microservices Testing Approaches

Unit Testing

Should we still have unit tests when we speak about microservices? – Yes, of course unit tests have proven to be a reliable, fast and also not so costly approach for testing your business logic. But here it can guarantee only the functionality of your own service. If you work on the consumer microservice, unit tests cannot guarantee that the Provider service works properly.

End-to-End (System) Testing

Should we still have end-to-end tests when we speak about microservices? –  Yes, having end-to-end tests is important, but when we speak about microservices the level of complexity could be extremely high. Imagine having more than five microservices that have to be deployed up and running in order for the end-to-end tests to be executed.

Integration Testing

Moving up to the next level in the pyramid (we have intentionally left integration testing last)- should we still have integration tests when we speak about microservices? – Yes, definitely. There are a couple of approaches such as service virtualization that can help you simulate the microservices you are communicating to. Some of the known frameworks that provide such functionality are WireMock, Hoverfly. One of the downsides of doing integration tests here is their complexity and also the number of combinations you need to test and simulate, if you have more than two microservices (which we could assume is the case in a real-world project). And the integration tests can not still guarantee that the service you use, does the job it is supposed to do. WHY? Let’s look at a simple example:
You have an integration test, where the Provider service is mocked. Before deploying your consumer service, you would like to prove that it works properly. On the first run all tests are green and then you can deploy your service (see Figure 2).

However, if you run your service against the production Provider, not the mocked version, it fails. In this example the Provider has changed the data format, und updated the name of a field from “name” to “names“. The integration tests could not catch this problem, because they are being run against an outdated version of the Provider (see Figure 3).

HOW can you fill this gap in your test definitions? Here comes the time to introduce the concept of Consumer Driven Contract Testing.

Consumer Driven Contract Testing (CDC Testing)

Consumer-Driven Contracts is an approach to ensure service communication compatibility, in which Consumer and Provider make an agreement about the format of the data they transfter between each other. This agreement forms the so called Contract. Normally, the format of the contract is defined by the Consumer and shared with the corresponding Provider. Afterwards, tests are being implemented in order to verify that the contract is being kept, called Contract Tests. CDC Testing requires both sides to be involved, thus one of the prerequisites for successful Contract Testing is the possibility to have a good, at best case close communication with the Provider service team (for example when you are the owner of the Consumer and Provider). Sharing those contracts and communicating the test results is important part of implementing proper CDC tests.

PACT

PACT is an open source CDC testing framework. It also provides a wide range of language supports like Ruby, Java, Scala, .NET, Javascript, Swift/Objective-C. We will go through the two major steps with code examples.

Getting started

Demo Project on Github: Pact Demo Github Repo
For following the guide, you first need to have at least two microservices implemented. Our demo project is based on the product case, introduced earlier. In addition, we use Spring boot for the microservices implementation and Gradle as dependency and configuration management. On the Provider side, pact-jvm-provider-junit5 is being used  together with pact-jvm-provider-spring. For the Consumer, the pact-jvm-consumer-junit5 is being used. The guide consists of two major steps that contain smaller steps. The first step focuses on what should be done on the Consumer side and the second step groups the activities on the Provider side.

Step 1 Actions Overview

1.1. Define the expected result on the Consumer side service

We start by implementing the ConsumerDemoTest, where we specify the expected result from the Provider. In the presented implementation, we are using JUnit5. PACT has already provided integration with JUnit5 with a corresponding extension: PactConsumerTestExt.

Firstly, we have to build the request and the expected response for it. In the expectations part, we say that given a GET request on a specific URL path is being performed, a response with status OK, headers and json body is being expected. In the runTest() the actual verification takes place.            

1.2. Generate the PACT file

Now, let’s run the test. The result is positive as expected:

During the test execution pact mock server is being started with the provider stubs installed on it. The Consumer requests are performed against the server and compared with the defined expectations. In case of a match, a contract file is being generated. 

If we open the /target folder, we will be able to see that a new subfolder /pacts with a JSON file have been created. This is actually our contract with the Provider:

1.3. Share the generated PACT file with Provider service

Lastly, the Consumer should share the contract with the Provider. You can use either a simple file sharing, cloud service or the PACT Broker. In our demo project, we have used file sharing, but as a more advanced technique it is worth looking at the Pact broker.

Step 2 Actions Overview

2.1. Start the Provider service

Moving on to the Provider service. Let us assume that we have received the Consumer contract and we want to verify it against the real provider implementation. Firstly, the Provider service should be up and running. There are a couple of possibilities in there. You can start it from your preferred IDE (IntelliJ or Eclipse). In our showcase, the service is started within the Test itself due to the SpringBootTest annotation.

2.2. Execute requests against the Provider

Afterwards, you can run the verification test by either using JUnit (see ProductApiContractTest implementation) or with gradle: ./gradlew test. Resulting output:

2.3. Check verification results

In this case, everything seems to be working as expected. The results should be shared of course with the Consumer and then given a green light for the Consumer to deploy its service in production.

However, we also want to monitor what would have happened in the case of a problem. Therefore, for the demo purposes, we are going to inject a failure on the Provider Service and add a Second Consumer service. The new Consumer uses only the product name and type, whereas the first Consumer uses product name, type and description. The idea is presented on Figure 6 below.

We change the Provider property description by removing it:

And then we run our pact verification step (note that now the configured consumers are two). Resulting output:

As a result, we can see that we broke the contract, but only with the first consumer. This is because it uses the description property, while the second one does not use it, thus its contract passes.

Summary

The introduction of microservices opens new challenges for testing. How one can actually test this combination of distributed microservices, that to the user look as one service. The minimum that we can do on top of the well known approaches in the Agile Testing Pyramid is to start defining our contracts. Those contracts can be surely written before the provider service is implemented based on the consumer API requirements. The Consumer Driven Contract Testing creates the need for frameworks supporting it. In this blogpost we have made an introduction to PACT, in the following one we are going to discuss Spring Cloud Contract.