This is a fantastic phrase, it has been popular and it has worked; testing now starts earlier and testing has a higher profile than ever before, but despite all that, it isn’t what we meant or even what we really needed.
Back in the depths of despair that was the Software Crisis. Testers managed to bellow a rallying cry “Testing should start early!” Testers could be found picketing management and development meetings with ‘V-model’ placards and T-shirts with ‘cost of defect’ bar charts emblazoned.
And the message got through. Testing started earlier, testing was funded, fewer bugs made it into production, testing became a career and all was well with the software development cycle. Eventually the software crisis disappeared. And yet, years later, we still produce software with defects and we still hear “Testing must start earlier”.
So the question is why? What do you mean? Earlier than what? What kind of testing?
If testing doesn’t start early enough then obviously it won’t be as effective as it could be, and therefore should start earlier, but this is true of every process in the software development cycle. Every process needs to start early enough to be effective and needs to be considered important enough to get funding.
But once we start early enough to be effective and in control of the process, do we get the results that we thought we were going to get? Something must be missing if we still get defects in the system even though we started testing early.
If we examine what happens when we do start testing early we can see that:
- Testers get more time with specification documentation and can produce test condition models that help them identify incongruent documentation.
- Testers get the time to build flow and process models that help them identify errors of omission e.g. “what happens when we press 1 on screen 3?”
But ultimately, most of the defects that testers are going to find still come from the execution of tests, and the only way that you can move that process forward is to deliver the system earlier and change the development process. e.g. prototyping, extreme programming, iterative development.
What we are really seeing is that when we start testing early, we increase the amount of review work that is done earlier in the project. Testers are essentially finding defects that are found through review work.
So what is the difference between reviewing and testing? Well, quite a lot, the testing process itself is subject to the review process. A better question might be: what are the similarities between reviewing and testing?
- can help identify defects associated with requirements and documentation.
- Both start early in the life cycle
- Both continue throughout the life cycle
- Both are forms of quality control.
It is quality control that should start early and it is quality control that will help detect bugs throughout the life cycle of the project and to do that successfully we have to implement many forms of quality control effectively throughout the project.
The current popular quality control technique is testing. Why? It isn’t because testing is cheap. But in some development environments this means that only the testing team tests, with minimal input from development and management. Of course it costs more money that way, a lot more, (compared to a review programme, testing is really expensive) but it is someone else’s responsibility and by the time testing find the problems it can be argued that it is too late and too expensive to change the offending documentation.
Let us hope you never have to work in an environment like that, but as long as software development stresses testing rather than the gamut of quality control techniques, mass responsibility and involvement, we will have to face the consequences of our lazy partitioning.
Be careful what you wish for.
This essay originally appeared on CompendiumDev.co.uk on 1st April 2002 as an ’essay’ this was in the days before I had a ‘blog’ setup and still had a ‘web site’. It seems more like a blog post than an essay so I’ve moved it over to EvilTester.com.
