It is a good practice to exercise the whole system(end to end) when we do an acceptance test(
goos book page 8). Unfortunately sometimes we don't control 100% of the pieces that compose a system(they belong to other department, other company...), so many times we have no choice than to assume how those parts behave...
Acceptance testing its an important part of the software development process.
This types of tests are focused in testing the scenarios that are valuable for the business. Often acceptance tests are written with a life specification framework such as JBehave, Fittnesse, Cucumber...
When testing business value the developer, needs to make sure that has understood the acceptance criteria that the business has interested in having tested.
In some companies, the acceptance criterias/specifications are prepared in a planning session prior to the development cycle, in others it is up to developers,testers and business analysts to on spot decide what needs to be acceptance tested and why.
The important thing when acceptance testing, is to express
"the whats" and not
"the hows". In other words focus on the overall functionality under the part of the system that is under test and not in the deep detail.
about their use and scope
Sometimes development teams forget that the acceptance tests are there not just to be evaluated automatically at build time, at the end of the day it will be Business Analysts, Quality Assurance teams or other development teams who will read them, to understand what the software does. That's why they need to be concise.
In my opinion acceptance testing should not involve integration with parts out of our control, unless its really a must. Instead should serve itself from plenty of Mocks, Stubs, Primers, Emulators, etc... in order to be able to focus in the main functionality described in the specification, that needs to be tested.
sometimes is not easy
Acceptance testing requires dextry for develop, maintain and enhance our own domain specific text harness.
Also, just in my opinion, as per my personal professional experience(part of it in the gambling industry) in many occasions the non deterministic nature(e.g probabilities & statistics) in which software behaves could make acceptance testing very complex. That is why, it is key to pick wisely the scenarios to test and also the edge cases.
example
Next I will show a trivial example where I will isolate and acceptance test just a part of an application which is believed to hold some business valuable. To do so I will stub all its external dependencies. We should not integration test dependencies, we should stub them and assume they work.
Let's first look at the project structure and understand what is that we are testing:
In this example it is "
SomeServiceAdapter", that holds business value and we decided to write an acceptance test for it. As we will soon see the other two adapters, represent access to remote systems which are out of our control.
The
"UpstreamSystemAdapter", could for example be a controller for a GUI or maybe a Rest endpoint that is used to gather data for processing.
The
"TargetSystemAdapter" could for example be the entrance to a persistence layer or rest client that forwards the result of processing to another system... Whatever those dependencies are, we don't care.
Initially when we write an acceptance test nothing exists, and we need to draft our requirements by creating new classes that will represent what we want to test, and also what we want to stub.
Many developers and also frameworks, like expressing the acceptance test in a common format, known as
"The Given, When, Then format". It is just a more visually friendly way of understanding a well known testing pattern called
"Arrange, Act, Assert". In other words, this pattern what they try to do is helping the developer writing the test, think about the Inputs/Premises(Given/Arrange) that are passed to some action in the code under test(When/Act) and the expected results(Then/Assert).
But we not necessarily need to follow that pattern, the important thing is that we make a concise and readable test. By the way, note that I am doing this in plain Java, without any framework, my goal is just to show a demo of how acceptance test could be written, but in real life you probably would like to write that code using your favourite live spec tool so you can get a beautiful output in some html page(e.g frameworks: Yaspec, Cucumber, Spock, JBehave, Fit, Fitnesse...). Also if you use a build tool such as Jenkins or TeamCity you should be able to nicely visualise your tests.
In the following simple example, we can see an acceptance test that tests
"SomeServiceAdapter" and at the same time, stubs the dependencies.
public class SomeServiceAcceptanceTest {
private UpstreamSystemStub upstreamSystemStub = new UpstreamSystemStub();
private TargetSystemStub targetSystem = new TargetSystemStub();
private SomeServiceAdapter someService = new SomeServiceAdapter(upstreamSystemStub, targetSystem);
@Test
public void shouldCalculateTheResultGivenTheReceivedDataAndPassItToTheTargetSystem() throws Exception{
upstreamSystemStub.sends(asList(1, 2, 3));
someService.calculate();
targetSystem.hasReceived(6);
}
}
Note how the class under test has the dependencies passed to its constructor. Also note that the types defined as parameters in the constructor are also interfaces, which are implemented by both the real classes that represent the dependencies, and their respective stubs(This way we make sure that the stub fulfils the contract with what the dependency does in reality).
One of the stubs:
public class TargetSystemStub implements TargetSystem {
private Integer result;
@Override
public void receivesData(Integer result) {
this.result = result;
}
public void hasReceived(int expected) {
assertThat(result, is(expected));
}
}
The other stub:
public class UpstreamSystemStub implements UpstreamSystem {
private List<Integer> data = new ArrayList<Integer>();
@Override
public List<Integer> data() {
return data;
}
public void sends(List<Integer> values) {
data.addAll(values);
}
}
Once the test is red, we can start implementing the production code. Important to mention, that this is just a very trivial example where the production code is so simple that the production code does not require to enter a TDD cycle but in many cases, getting to see the acceptance test green, would require also to TDD each of the bits and pieces that enable the function called from the
"when" to be properly tested. Just as a side note, when that is the case we also refer to that approach as ATDD(Acceptance Test Driven Development), it involves multiple TDD cycles prior to the completion of a business valuable acceptance tests.
Here just the production implementation of the class:
public class SomeServiceAdapter implements SomeService {
private final UpstreamSystem upstreamSystem;
private final TargetSystem targetSystem;
public SomeServiceAdapter(UpstreamSystem upstreamSystem, TargetSystem targetSystem) {
this.upstreamSystem = upstreamSystem;
this.targetSystem = targetSystem;
}
public Integer calculate() {
Integer result = upstreamSystem.data().stream().reduce(0, (n1, n2) -> n1 + n2);
targetSystem.receivesData(result);
return result;
}
}
I guess each developer has its own technique when writing acceptance tests, I just want to mention that I recently show somebody who starts writing his acceptance tests from the
"then" and I thought that was a very interesting approach, because he said that doing it that way can focus more in what exactly is expected from the system that is about to be developed, but as I said it is up to each to decide how you like writing your acceptance test, just remember that it is about the
"what" and not about the
"how" also pick your battles and build test harnesses(avoid integration testing as much as you can)
Here the link to the complete source code:
git acceptance testing example
YOLO! :)
