TLDR: Code Coverage can help with code review when we focus the coverage metrics on the code we want to review and format it to support coverage metric analysis.
I use Code Coverage as a tool when reviewing code. In the example for this post, I’m using code coverage of a live project I’m working on, prior to refactoring code. This helps me understand gaps, refresh my memory about the code, and make an assessment about the risk of something not being picked up during my refactoring (if I accidentally make a change in behaviour).
Video
In this video I demonstrate the use of Code Coverage to help me review code. You’ll learn:
- How Code Coverage Metrics can change due to code formatting.
- How to configure code coverage to target specific classes.
- Situations where High (100%) Code Coverage can still require more tests.
- Coverage refers to ‘model’ coverage, not just Code Coverage.
Code
The project that I am working in:
And the main test code I’m looking at:
I have a Unit Test here which is targetting a specific class.
ChallengerSparkHTTPRequestHookTest
Which is targeting the class
ChallengerSparkHTTPRequestHook
Test Covers Package
If I run the test then by default it runs against all the classes in the package.
Which is great, but that can give me an inflated sense of coverage.
To review code, I want to zoom in on a specific class, so I change the run config
- from package
uk.co.compendiumdev.challenge.challengehooks.*
- to class
uk.co.compendiumdev.challenge.challengehooks.ChallengerSparkHTTPRequestHook
When I run the test with coverage now, I only see coverage for the test I want to review.
Numbers
If I went by numbers, then I might not expand any further.
76% is pretty good.
But, because of the way the code has been written, I have an inflated coverage metric.
I ’exercise’ a lot of code, but none of it was covered by the test.
Formatting
Formatting of the code might make a difference.
if(method.equals("DELETE") &&
path.equals("/heartbeat")){
challengers.pass(
challenger,
CHALLENGE.DELETE_HEARTBEAT_405);
}
I don’t have a test that issues a DELETE method.
But the first line with the ‘if’ method has been ‘covered’ because a ‘GET’ test hits all the if statements in my code, and the code says does "GET".equals("DELETE") no, then ignore this if.
I can achieve increased code coverage if I change the fomatting of the code even though I don’t amend any tests.
e.g. my code coverage looks higher when I review the code if I format it as follows:
if(method.equals("DELETE") && path.equals("/heartbeat")){
challengers.pass(
challenger,
CHALLENGE.DELETE_HEARTBEAT_405);
}
In the IDE, when I look at the coverage, it shows the full if condition as ‘covered’, but actually I only evaluated the first Boolean expression, I didn’t need to evaluate the second because it is an AND (&&) condition.
But, visually, in the IDE, it looks like it might have ‘covered’ the full condition.
I have to be really careful when reviewing condition or predicate coverage to make sure the combination of boolean conditions has been exercised, because I don’t see that from the coverage report.
Formatting can help a little with that, but I still need to review the test in detail to see the actual coverage.
Line Coverage
When I review the code I can see that some of these lines are pretty important.
If the condition passes then
exercise the actual line of code I want to test
I can achieve a lot of reported coverage through exercising the code because the condition lines are exercised but very few of the tests actually exercise the actual line of code I want to test and we can visually see that in the report.
So even though I have a high code coverage number.
By looking at my executable code, I haven’t covered very much at all.
I tested:
- a null state
GET /challenges
But I haven’t covered
GET /heartbeatDELETE /heartbeatPATCH /heartbeatTRACE /heartbeat
This is why I don’t really like ’limits’ on the line coverage, e.g. “76%” in this case is clearly not good enough.
I have high code coverage, but low requirement coverage.
Code Coverage is not Test Coverage
Looking at the code though, I can see that I don’t need to have a lot of combinations of condition coverage because all the if statements are AND (&&).
If they were OR (||) statements then it increases the amount of tests I need to add, even though it would not increase the code coverage metrics.
People hear different things when we talk about ‘coverage’.
- Coverage refers to ‘model’ coverage
- Coverage does not mean “Code Coverage”
- “Code Coverage” does not mean “Test Coverage”
Coverage is always about ‘model’ coverage. In this case the model is the code, and the coverage is achieve by running Unit Test code.
My “Test Coverage” is really referring to the ‘model’ that results in the creation of the tests. Not the Code. The Code Coverage is a side-effect.
If I haven’t covered all my code, with my tests, then perhaps:
- the code isn’t related to the model that I created the tests to cover
- I have a gap in my test coverage
Gaps
In this example:
- I expect my Test class to be the only set of code I have to exercise to achieve coverage of this class.
- The code is simple enough that if a line is not covered, then I have a missing test in my Test class
- The conditions are simple enough that I do not have to review the Tests to see if I cover all the boolean conditions in the Conditions
- A 100% coverage metric for this class, looks like an essential aim
I filter my Run Configuration to target the individual class to support the review.
Other classes would require additional tests, even once 100% coverage is reported, because of the condition coverage we need to aim for.
The condition coverage, should not be dictated by the code, it should be dictated by the model the code was designed to implement. Because the code might be wrong.
This content was released early, and ad-free, to Patreon supporters with a full transcript.
