Does dependency between test execution imply lack of abstraction layers?

TLDR; I try to write tests at an appropriate semantic level so I never need to create dependencies between @Test methods, I re-use abstraction calls instead of dependencies between @Test methods.

People often email me the question “How do I make a test run after another test?”

I ask “Why do you want to do that?”

“Because I have a test that creates a user, then I want to use that user in another test.”

I generally answer “Don’t do that. Write some abstraction code to ‘create a user’ then use that in your ‘create a user test’ and in your ‘another test’.”

In the examples below I have made up an API for HTTP requests.

API Abstraction vs HTTP Direct with Test Dependencies

So instead of:

@Test
public void canCreateAUser(){

     // Send request to create a user
     Response r =
        myHttpLibrary.post("http://myurl/api/users").
                      withFormContent(
                      new Form().with().
                        field("username","bob").
                        field("password","dobbspass").
                        field("email","bob@mailinator.com").
                        .urlencoded()
                      );
                                   
                         
     Assert.assertEquals(201, r.statusCode);
     
     // URL to access created user is in location header
     // e.g. "http://myurl/api/user/12"
     String userURL = r.getHeader("location");                         

     // get the user and check created properly
     Response u = myHttpLibrary.get(userURL);

     Assert.assertEquals(200, userURL.statusCode);

     Assert.assertEquals("bob",
                         u.parseJson("user.username"));
     Assert.assertEquals("bob@mailinator.com",
                         u.parseJson("user.email"));
}

@Test
public void canAmendAUser(){

     // write the code that amends user
     // created in the test canCreateAUser
}

I would be more likely to write code that looks like the following:

@Test
public void canCreateAUser(){

     Response r = myApi.createUser("bob",
                                   "dobbspass",
                                   "bob@mailinator.com);
    
     Assert.assertEquals(201, r.statusCode);
     
     String userId = ResponseParser.getCreatedUserId(r);

     Response u = myApi.getUser(userId);

     Assert.assertEquals(200, userURL.statusCode);

     Assert.assertEquals("bob",
                         u.parseJson("user.username"));
     Assert.assertEquals("bob@mailinator.com",
                         u.parseJson("user.email"));
}

@Test
public void canAmendAUser(){

     Response r = myApi.createUser("bob",
                                   "dobbspass",
                                   "bob@mailinator.com);
    
     Assert.assertEquals(201, r.statusCode);
     String userId = ResponseParser.getCreatedUserId(r);

     Response a = myApi.amendUser(userId,
                                  "email","newbob@mailinator.com");

     Assert.assertEquals(200, userURL.statusCode);
     
     Response u = myApi.getUser(userId);

     Assert.assertEquals(200, userURL.statusCode);
     Assert.assertEquals("bob@mailinator.com",
                         u.parseJson("user.email"));
}

Again - I’ve made up the API, so it isn’t ‘real’ code, so it might have syntax errors, and it will not work if you try to use it.

But it illustrates the creation of an abstraction layer to access the API which you can re-use, rather than trying to use the @Test method as an abstraction layer.

For me this builds on a quote from Dijkstra from “The Humble Programmer”

“…the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise.”

@Test methods are an attempt to abstract a small set of preconditions, actions and assert on a subset of postconditions to justify the title of the @Test method.

We wouldn’t want to reuse ‘canCreateAUser’ we want to re-use ‘create a user’. We don’t need all the additional assertions in the ‘canCreateAUser’ when we ‘create a User’. We only need those additional assertions when we test the process of creating a user rather than when we create a user for use with other actions.

Why would people avoid creating abstractions?

Since I don’t avoid creating abstractions, I have to ‘suspect’ why people might not code like this. And remember the above is an example of an approach, not an example of ‘great code that you should follow’. There are many ways to implement an abstraction layer around an API to support re-use and made your @Test code readable and maintainable. This was a first attempt that I thought illustrated the point, and remained readable with only one extra layer of semantics.

Possible reasons:

I want to avoid repeating code
I don’t want to hide the implementation of calling the API

I’m sure other reasons exist - feel free to comment if you have experience of a good reason, or comment with a link to an alternative experience report.

I want to re-use `@Test` methods to avoid repeating code

The main code we are trying to avoid repeating is:

     Response r =
        myHttpLibrary.post("http://myurl/api/users").
                      withFormContent(
                      new Form().with().
                        field("username","bob").
                        field("password","dobbspass").
                        field("email","bob@mailinator.com").
                        .urlencoded()
                      );

I don’t think increasing the coupling and dependency between tests worth avoiding the repeated code since I can avoid repeating the code by moving it to an API class. And as a side-effect my @Test method concentrates on the assertion of postconditions rather than the actions.

One other strategy that ‘avoid repeating code’ (while also avoiding abstraction layers) creates is ’large @Test methods’

Which might mean:

@Test
public void canCreateAmendAndDeleteAUser(){

     // insert lots of direct API
     //        calls and assertions to
     // create a user
     // assert on the users creation
     //        for all post conditions related to create
     // amend the user
     // assert on the users amendment
     //        for all post conditions related to amendment
     // delete the user
     // assert on the users deletion
     //        for all post conditions related to deletion
}

Note, I removed the code because it would be too long, so imagine what the code would be extrapolating from the first example with the myHttpLibrary.

i.e.

a single test instead of multiple tests for ‘create’ ‘amendment’ and ‘deletion’
if this ‘single’ test fails I don’t know if it was the ‘creation’ or ‘amendment’ or ‘deletion’ by reading test names I have to debug the long test

Instead of an equivalent test using abstractions:

@Test
public void canDeleteAUserAfterAmendment(){

     Response r = myApi.createUser("bob",
                                   "dobbspass",
                                   "bob@mailinator.com);    
     Assert.assertEquals(201, r.statusCode);

     String userId = ResponseParser.getCreatedUserId(r);

     Response a = myApi.amendUser(userId,
                                  "email","newbob@mailinator.com");
     Assert.assertEquals(200, userURL.statusCode);
     
     Response d = myApi.deleteUser(userId);
     Assert.assertEquals(200, userURL.statusCode);

     Response d = myApi.getUser(userId);
     Assert.assertEquals(404, userURL.statusCode);

}

In the ‘abstraction’ example, I only assert on the minimum postconditions necessary during the execution i.e. the status codes
because I have other tests which check ‘creation’ and assert on more postconditions in the creation and I have other tests which check ‘amendment’ and assert on more postconditions in the amendment, so if there is a problem with ‘creation’ I would expect a ‘creation’ test to fail rather than have to debug a ‘canCreateAmendAndDelete’ @Test
I have minimal repeated code because I’m using an abstraction layer
I don’t need as many comments because the code is readable

I don’t want to hide the implementation of calling the API

What semantic level is the @Test working at?

Sometimes I have an API abstraction layer which I use for most @Testmethods because I’m not ’testing’ that specific API call, I’m using it.

I might be testing a flow through the system
I might be testing the response, not the call

The following code is not at an API semantic level, it is at an HTTP semantic level:

     Response r =
        myHttpLibrary.post("http://myurl/api/users").
                      withFormContent(
                      new Form().with().
                        field("username","bob").
                        field("password","dobbspass").
                        field("email","bob@mailinator.com").
                        .urlencoded()
                      );

Therefore if we are testing the HTTP semantics of the API then this is an appropriate level e.g.:

what happens if I add extra headers?
what happens if my form fields are in a different order?
what happens if it is a PUT instead of a POST
etc.

Summary

It never occurs to me to try to make @Test methods dependent on each other.

I generally refactor to abstraction layers very quickly to ensure my @Test methods are written at the semantic level necessary for that @Test.

I combine multiple layers of abstraction to make the semantics in the @Test clear.

I suspect that, if you want to make your @Test methods run in a specific order, or make your @Test methods dependent on other @Test methods, you may not have the correct level of abstraction and are using ‘dependency’ as a mechanism to solve a problem that ‘refactoring to abstraction class’ might solve as quickly, and with more benefit.

Benefits:

Don’t worry about dependencies
Readable code
Easy to maintain when the API changes
@Test are not dependent on an HTTP library so you can use different libraries if necessary
Re-use the API abstractions for different types of testing - functional, integration, exploratory, performance
Fewer issues with parallel execution of @Test code