Contract Testing: Yay or Nay?

Software Development
Published Jul 12, 2023 ยท 4 min read
Tests are an essential part of the development lifecycle. They ensure good code quality and meet the business value behind the unit under test. Since different types of tests target different objectives, there comes a need for a fast "offline" test that targets the communication principles amongst different parties, hence contract tests.

Motivation

While working with a distributed systems architecture gives you a significant advantage regarding scalability and de-coupling unrelated domains, it has some drawbacks. It is true that systems are independent of each other. Yet, in most cases, some consume others, thus having a hidden dependency on them. This dependency is summarized in the three bases of communication: protocol, structure, and value. These three bases make up the communication agreement or the "contract".

Scope of Contract Testing

Contract testing can be applied to any sort of communication that happens between two services. It's mostly common on REST APIs, but it can also be used for messaging, calls over gRPC, etc.

Why Contract Tests?

Someone can argue that the value of validating contracts can be achieved with end-to-end tests or on-edge types of integration tests, where you can spin up a container of the provider you're validating against its contract. However, this approach is usually costly in time and storage resources and it usually slows down your development lifecycle if you're running these tests before pushing to the mainstream. Thus the need for an efficient type of testing where we are only concerned about simulating the communication (be it request/response, events, etc.) and validating its main principles.

Types of Contract Tests

Provider-Driven:

The first approach for contract testing is when it's initiated by the provider (a.k.a server). In this context, the provider writes the contracts that describe its APIs, then publishes them to a remote shared server.


 

On the other hand, the client downloads the contracts of the provider they're interested in and runs tests against them to validate that they're getting back the response they expect.


 

Provider-driven contract testing is featured in Spring Cloud Contract where the contracts are written either as YAML or groovy files and are packaged as a stub JAR and published to an artifact manager.
 

Consumer-Driven:

The second approach for contract testing is when the consumer (a.k.a. the client) is the one to initiate the process, where each consumer writes the contract that they expect the provider to fulfill and publishes these contracts to a remote shared server.


 

On the other hand, the provider has to download all the contracts assigned to them and do the needed development to fulfill their requirements.


 

To follow consumer-driven contract testing, PACT offers a nice solution where the contracts are written in a DSL and are published as "pacts" to a broker (pact-broker). It provides a nice way of versioning contracts and documenting them.
 

Provider-Driven vs. Consumer-Driven Contract Testing:

Provider-Driven Contract Testing

Pros:

  • the provider does not have to be aware of the consumers (recommended when working on a public API; e.g. Azure APIs)

Cons:

  • it is not preventative: we cannot prevent the provider from changing the contract in a way that breaks the consumer
  • the consumer is not aware of the change of contracts before it happens

Consumer-Driven Contract Testing

Pros:

  • it is preventative: any change suggested by the provider cannot be merged if it breaks an existing contract

Cons:

  • gives more power to the consumer: the provider will stay blocked until the contracts of all of its consumers are fulfilled
  • contracts coming from different consumers can be conflicting, and that is a dead-lock

Note: this is mostly recommended when working in internal APIs, where the provider and the consumer are aware of each other, so they would have the privilege of communicating while writing the contracts.