Contract testing

Let's imagine the following scenario...
We are working in distributed system with lots of applications.
The developers understand the importance of avoiding coupling amoung componets, so they decide to create restful applications to communicate via xml and json,
instead of building applications that are binary dependant with other applications.

During the development of a feature, the development team, did a change to the API, and unconciously they broke one of the consummer apps.
Unfortunately, this bug was really expensive, since the company just managed to discover it in its replica, pre-production environment by a long running
end to end functional test, after determining that what was broken was actually a marshaller of xml, there was no quick fix and they had to roll back.

In the root cause analysis meeting, developers from each of the teams, that own the apps that failed realised that the API change was the reason for the bug
and that there was no aditional work done in one of the unmarshallers.
The developers were told to fix the bug and also to come up with a solution that would avoid this from happening again.

After fixing the bug the developers toke some time to think how they could catch this kind of bugs before the pre-production environment where the expensive
integration tests run. One of them said, "What we need is consummer contract testing!"...

Consumer contract testing, allows consumers and providers of an API knowing if their latest changes on their marshallers or unmarshallers, could potentially be
harmful for the other party, without the necessity of performing an integration test. This is how it works:


1- The provider of the API, publishes an example of the API somewhere where he knows the consumer can access it(e.g publish it in a repo, sending it via email...).
2- The consummer takes the API example and writes a test that tolerantly accesses the values of interest.
   This in-document path(e.g xpath,jsonpath...) used to retrieve the values from the API example, is known as the contract.
3- The consummer publishes the contract in a place where knows the provider has access to it(e.g publish it in a repo, sending it via email...).
4- The provider will take the contract, and will use it in a test, against the generated output of the application. If the test fails when being run, the provider will know that they could potentially be breaking the the consumer, if they were to release the current version under test(a negotiation can take place).

Let's now have a look at a practical example of each of the steps above.

1- The developers that own the provider app, take from their passing acceptance test the output that the application is sending back to the consumer and they save
it into a file called "apiexample.xml", which looks like this:

 <output>  
      <content>  
           <partA>A</partA>  
           <partB>B</partB>  
      </content>  
 </output>  

They send this file over email to the team that owns the consumer application.

2- The developers that own the consumer app, will take the exampe and will write queries to it, to determine the contract they need. A unit test against the example, could be fine.

 @Test  
    public void apiExampleGeneratesValidatesToContract() throws Exception {  
     XPath xPath = XPathFactory.newInstance().newXPath();  
     String value = xPath.evaluate("/output/content/partB", getSource(readExample("apiexample.xml")));  
     assertThat(value,is(notNullValue()));  
    }  

3- Now that the developers know that the contract to access what they are interested in is:
 "/output/content/partB"
They can save it in a file called "contract.txt" and send it over email to the other team for they to make sure they will always be outputing according to the contract. Note that this tolerant
paths, allow to the provider to change any part of the API they want to change, as long as the contract is respected.

4- The provider will read the "contract.txt" file and will write a test where the contract will be applied to the applications output.

 @Test  
    public void apiExampleGeneratesValidatesToContract() throws Exception {  
     XPath xPath = XPathFactory.newInstance().newXPath();  
     String value = xPath.evaluate("/output/content/partB", getSource(readExample("apiexample.xml")));  
     assertThat(value,is(notNullValue()));  
    }  

Now when any of the teams run their builds, they will know if they are in breaching the contract and they will avoid the bug going further than the development environment.

You can find the complete source code of this example here.