We already talked about the basics of Contract Testing in a broader range, now we will go deeper and talk about one of the frameworks used for contract testing: Pact.
Provider-Driven Contracts vs Consumer-Driven Contracts
When we talk about contract tests, we have basically two ways to think: Provider-Driven Contracts and Consumer-Driven Contracts. The basic difference is who defines the contract.
In a provider-driven contract, the contract is defined by the provider (the API owner or the message sender) and the consumer (the API consumer or the message receiver) validates if its use is according to the contract.
In a consumer-driven contract, the consumer defines the contract with the information used by the service and the provider verifies if it is supplying the data as needed.
A popular framework for provider-driven contract testing is Spring Cloud Contract. And Pact is a popular consumer-driven.
Meet Pact
Pact is a consumer-driven testing tool that was originally developed at realestate.com.au but is now open source. Pact supports multiple languages and platforms, such as the JVM, JavaScript, Python, and .NET, and can also be used with messaging interactions.
It has a sophisticated approach to writing tests for the consumer and the provider side, gives you stubs for separate services out of the box, and allows you to exchange CDC tests with other teams.
With Pact, you start by defining the expectations of the consumer using a DSL in any of the supported languages. When you run the test, it launches a local Pact server and runs this expectation against it to create the Pact specification file. The Pact file is just a formal JSON specification.
A really nice property of this model is that the locally running mock server used to generate the Pact file also works as a local stub for downstream microservices. By defining your expectations locally, you are defining how this local stub service should respond.
On the provider side, you then verify that this consumer specification is met by using the JSON Pact specification to drive calls against your microservice and verify responses. For this to work, the producer needs access to the Pact file. This means that we need some way for this JSON file, which will be generated to the consumer build, to be made available by the provider.
The best way to store this JSON file is to use the Pact Broker, which allows you to store multiple versions of your Pact specifications. This could let you run your consumer-driven contract tests against multiple different versions of the consumers if you wanted to test against, say, the version of the consumer in production and the version of the consumer that was most recently built.
The Pact Broker actually has a host of other useful capabilities. Aside from acting as a place where contracts can be stored, you can also find out when those contracts were validated and a lot more. Also, because the Pact Broker knows about the relationship between the consumer and producer, it’s able to show you which microservices depend on which other microservices.
Pact is a consumer-driven contract testing tool that generates explicit contracts by using a test double that records requests and expected responses to a contract that is called a “pact”.
How Pact Works
As was said before, we have the consumer, which is the one that is consuming the APIs or messages, and the providers, as the name says, it provides the API and messages.
This is an image from pact.io that illustrates in a broad view how it works.
The first step is to define de contract.
How Pact Works from Pact Docs
This animation from pactflow.io shows a bit better.
How Pact Works animation from Pact Flow
One important point is that it not only works for HTTP calls but also messages (RabbitMQ, Kafka, etc) are also supported by Pact.
Consumer side
As we said before Pact is consumer-driven so let us start with the consumer part.
The Pact Docs defines a consumer as:
“An application that makes use of the functionality or data from another application to do its job. For applications that use HTTP, the consumer is always the application that initiates the HTTP request (eg. the web front end), regardless of the direction of data flow. For applications that use queues, the consumer is the application that reads the message from the queue.”
As a consumer-driven contract framework, the first step is to create a test for the consumer. The below consumer sample is from the Pact Docs.
describe('Pact with Order API', () => {
describe('given there are orders', () => {
describe('when a call to the API is made', () => {
before(() => {
return provider.addInteraction({
state: 'there are orders',
uponReceiving: 'a request for orders',
withRequest: {
path: '/orders',
method: 'GET',
},
willRespondWith: {
body: eachLike({
id: 1,
items: eachLike({
name: 'burger',
quantity: 2,
value: 100,
}),
}),
status: 200,
headers: {
'Content-Type': 'application/json; charset=utf-8',
},
},
})
})
it('will receive the list of current orders', () => {
return expect(fetchOrders()).to.eventually.have.deep.members([
new Order(orderProperties.id, [itemProperties]),
])
})
})
})
})
One important thing to keep in mind is that in the consumer you will add in the willRespondWith only the fields from the provider that you are really using. The minimal response. In this way, the provider can change the fields that are not used by your consumer without breaking anything.
Also according to the Pact Docs, you should follow these best practices for matching:
Request matching
“As a rule of thumb, you generally want to use exact matching when you’re setting up the expectations for a request (upon_receiving(…).with(…)) because you’re under control of the data at this stage, and according to Postel’s Law, we want to be “strict” with what we send out.”
Response matching
“You want to be as loose as possible with the matching for the response (will_respond_with(…)) though. This stops the tests from being brittle on the provider side. Generally speaking, it doesn’t matter what value the provider actually returns during verification, as long as the types match. When you need certain formats in the values (eg. URLS), you can use terms (see docs below). Really really consider before you start introducing too many matchers however - for example, yes, the provider might be currently returning a GUID, but would anything in your consumer really break if they returned a different format of string ID? (If it did, that’s a nasty code smell!) Note that during provider verification, following Postel’s Law of being “relaxed” with what we accept, “unexpected” values in JSON response bodies are ignored. This is expected and is perfectly OK. Another consumer may have an expectation about that field.”
Random data - avoid it
“If you are using a Pact Broker to exchange pacts, then avoid using random data in your pacts. If a new pact is published that is exactly the same as a previous version that has already been verified, the existing verification results will be applied to the new pact publication. This means that you don’t have to wait for the provider verification to run before deploying your consumer - you can go straight to prod. Random data makes it look like the contract has changed, and therefore you lose this optimization.”
The contract
When we ran the consumer test and it is green, a “pact” (contract) will be generated, usually on /pact folder. This contract file is a JSON with all the expectations that we defined for the consumer tests.
{
"consumer": {
"name": "GettingStartedOrderWeb"
},
"provider": {
"name": "GettingStartedOrderApi"
},
"interactions": [
{
"description": "a request for orders",
"providerState": "there are orders",
"request": {
"method": "GET",
"path": "/orders"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json; charset=utf-8"
},
"body": [
{
"id": 1,
"items": [
{
"name": "burger",
"quantity": 2,
"value": 100
}
]
}
],
"matchingRules": {
"$.body": {
"min": 1
},
"$.body[*].*": {
"match": "type"
},
"$.body[*].items": {
"min": 1
},
"$.body[*].items[*].*": {
"match": "type"
}
}
},
"metadata": null
}
],
"metadata": {
"pactSpecification": {
"version": "2.0.0"
}
}
}
This contract has to be shared with the provider (usually using Pact Broker) so that the provider can verify it.
Provider side
The final step is to verify if the provider is providing (who would guess?) the API or Message with all the fields that the consumer needs.
The Pact Docs defines a provider as:
“An application (often called a service) that provides functionality or data for other applications to use, often via an API. For applications that use HTTP, the provider is the application that returns the response. For applications that use queues, the provider (also called producer) is the application that writes the messages to the queue.”
// Verify that the provider meets all consumer expectations
describe('Pact Verification', () => {
before(async () => {
port = await getPort()
opts = {
provider: providerName,
providerBaseUrl: `http://localhost:${port}`,
pactBrokerUrl: 'https://test.pact.dius.com.au/',,
publishVerificationResult: true,
providerVersion: '1.0.' + process.env.HOSTNAME,
}
server.listen(port, () => {
console.log(`Provider service listening on http://localhost:${port}`)
})
})
it('should validate the expectations of Order Web', () => {
return new Verifier()
.verifyProvider(opts)
.then(output => {
console.log('Pact Verification Complete!')
console.log(output)
})
.catch(e => {
console.error('Pact verification failed :(', e)
})
})
})
The above sample is also from the Pact Docs. In there, the endpoint returns the data as follows.
[
{
id: 1,
items: [
{
name: 'burger',
quantity: 2,
value: 20,
},
{
name: 'coke',
quantity: 2,
value: 5,
},
]
}
]
In a user case, I would personally add a mock to the test.
Summary
This was a bit about the Pact framework for consumer-driven contract tests. I would strongly recommend you to read the Pact Documentation which has a lot of information and a lot of examples. Very nice documentation.
Happy testing!
References
This post would not be possible without the great texts that came before. I am only standing on the shoulder of giants.
- Book Building Microservices
- What is contract testing? PACTFLOW
- Pact Docs