TLDR; Imagine never having to worry about saying “Verification” when you meant “Validation”. Well you can. Write down what the V-words imply, and then use those sentences you wrote down instead.
I’ve been in meetings where people have argued about the words “Verification” and “Validation”. For some reason these words have a high weighting of importance in “The World of Software Testing”.
I have the personal issue that I forget which one is “meeting requirements” and which one is “meeting needs” and have to reverse engineer the words every time I need to use or interpret them.
I’m not the only one, because often I see the questions: “Am I building the product right?” (Verification), and “Am I building the right product?” (Validation).
And I asked myself, “What if I don’t use the words verification and validation?”
Imagine “A World of Clarity”
Imagine:
- never being asked to “verify it, but don’t validate it yet”
- you could say what you mean rather than use ambiguously re-defined english words
- you could say what you mean rather than hope that the person you are communicating with has embraced the glossary of the PMBOK version 4, or ISO 9000, or IEEE STD 610 (which superceded IEEE STD 729)
I actually thought that IEEE STD 610 was the up to date version but when I went to IEEE, I found that 610.12-1990 has been superceded by IEEE 24765:2010. I don’t actually know what 24765:2010 says about “Validation” and “Verification” because it costs $450 and I haven’t seen anyone directly quote 24765:2010. I do wonder what Validation and Verification are specified to mean now.
Unclear words?
I find the words “Verification” and “Validation” unclear.
Historically the words verification and validation have taken on meaning within Software and Project Management.
You can find a discussion on wikipedia, but I think the two word summary definitions read as follows:
- verification - meeting requirements
- validation - meets needs
These words probably mean more than this to some people. I’ve summarised and paraphrased.
I quite like dictionary definitions to help me understand words:
- verification - noun “The process of establishing the truth, accuracy, or validity of something.”
- validation - noun - “The action of checking or proving the validity or accuracy of something.”
In this case the dictionary definition didn’t help me distinguish between the two words very well since they are both defined in terms of the same underlying words e.g. validity, accuracy.
And they are both nouns to describe processes (things we do).
Clearly common ‘usage’ of these words provides the difference for Software Development and Testing where such a difference is important.
And the difference betweeen things we do often arises from what the outcomes are, and the context that surrounds them (where, when, who).
Difference that makes a difference
The important difference of the IEEE /PMBOK usage of Verification and Validation does not appear to come from “process/action” or “truth/accuracy”. It appears to come from the distinction between requirements and needs.
But this can also lead to ambiguity since some requirements covers the ’needs’.
Need:
- I need the TV to show me a clear picture and have audible sound
Requirement:
- TV should show clear picture
- TV should have audible sound
We proceed to refine the requirement to define what ‘clear’ means: refresh rate, refresh frequency, colours, resolution, etc. And the same for sound.
But we may have refined it for a specific person’s need, rather than everyone’s need i.e. people hear sound differently
Therefore, are we actually splitting Requirement and Need across two different words to remind ourselves of gaps and ambiguities?
- our requirements might not completely cover all identified needs
- needs may exist that we haven’t represented as requirements
Our requirements may not be complete. Do the words “Verification” and “Validation” remind you of that?
If we concentrate on ’needs’ then we selectively choose which needs to meet and we recognise that we have interpreted the statement of need, and we may have ambiguity in our interpretation.
And we know that the needs and requirements don’t just come from ‘users’. They come from stakeholders as well. And sometimes stakeholders supply them because of regulatory requirements. etc. etc.
We have a rich set of source input models which contain ’needs’ that we convert into ‘requirements’. Do the words “Verification” and “Validation” remind you of that?
Testing Sources
I found the following on page 13 of the ISTQB foundation certificate syllabus version 2018.
“Another common misperception of testing is that it focuses entirely on verification of requirements, userstories, or other specifications. While testing does involve checking whether the system meets specified requirements, it also involves validation, which is checking whether the system will meet user and other stakeholder needs in its operational environment(s).”
To paraphase:
- “verification - checking whether the system meets specified requirements”
- “validation - checking whether the system meets needs in operational environments”
This maps on pretty well with the PMBOK definition quoted on Wikipedia
Unfortunately I think we’ve been led to a point where we have raised the importance of the terms “Verification” and “Validation” such that we might try to twist our understanding of Software Testing around these words.
Rather than using notions implied by the definitions as inputs to our test approach. e.g. notions of: explicitly specified requirements, implicit requirements, usage needs, environment specific scenarios, stakeholders, specific data used, risk of ambiguity in requirement specification, risk of gaps in requirement gathering, etc.
I wonder if a danger exists that, an industry might be trying to justify the continued use of the words Validation and Verification despite the lack of clarity offered by the words, rather than evolve clearer ways of communicating the underlying issues and concepts.
Quality
Verification and Validation, often go hand in hand with discussions of quality
Rephrasing by Ourselves
I could ignore the words “verification” and “validation” and instead say.
“During the development process we want to check that the system meets specified requirements and meets needs of users and stakeholders”
But I don’t want to just ‘check’.
If I just ‘check’ then I find out that it “can” or it “can’t”. But really I want to build it so that it does. And then I might want to double check that I really did build it, but I might choose not to, it becomes a Risk informed decision what I choose to check and what I don’t.
Note: I originally wrote “does” or “doesn’t” but this seemed a tad too definite and absolute. i.e. it always does. “Can” suggests the possibility that under some specific circumstances it can do it, but we have no way to be absolutely sure. And that lack of absolute certainty feels more like the world of Software Testing to me.
I might want to ’ensure’ or another word that implies that we will try and direct the process such that we consider the end result a quality product. Rather than just cycle through: make it and check it then fix it, and then check it and fix it, and then etc.
I might then choose to avoid the words and describe my process in terms of:
“During the development process we want to ensure that the system meets specified requirements and meets needs of users and stakeholders”
I can build a Test Approach that wraps around this, helping mitigate Development Process gaps or risks. And avoiding certain approaches to testing which we believe we have tightly controlled.
Other People Rephrases
I have seen people rephrase Verification and Validation as:
- Are we building the right thing?
- Are we building the thing right?
see also Software Verification and Validation for more details.
If I try to directly map these questions as people seem to use them:
- Are we building the right thing?
- Validation
- meets usage needs
- Are we building the thing right?
- Verification
- meets requirements
I don’t think the questions map directly on to the words Verification and Validation as I think the questions cover more ground than the V words.
My twenty second attempt at mapping:
- Are we building the right thing?
- Validation and Verification
- meets usage needs
- meets requirements
- solid business case?
- is there a demand for this?
- are we targetting the appropriate platform?
- etc.
- Are we building the thing right?
- do requirements match needs
- are we continually checking this?
- good enough process?
- are we introducing errors?
- avoiding waste?
- cost effective?
- team skills?
- etc.
I appear to have interpreted these questions in an an unconventional way according the the normal mapping on to Validation and Verification that I find on the Internet. This may have resulted from my personal inability to use the words Validation and Verification as per the standardised industry norms.
I think the interpretation above offers me more value than a constrained mapping to Validation and Verification.
I’d rather use the questions for expanded definitions and concept exploration instead of proxies for Validation and Verification, as the expansion guides me towards a better development approach.
I think that this also moves me away from Verification and Validation and into the realm of “Quality”.
Juran
“Quality” is a big and meandering subject.
I own a 4th Edition of Juran’s Quality Control Handbook. At over 1000 pages, it isn’t a “Rollercoaster Thrill Ride Page Turner” of a book.
On page 2.3 of the 4th Edition “Quality” was defined as having multiple meanings, two of which “dominate the use of the word”:
- “Quality consists of those product features which meet the needs of the customers and thereby provide product satisfaction”
- “Quality Consists of freedom from deficiencies”
Juran’s book then goes on to define these terms in more detail and continues to explore the concept of Quality in much more detail.
Juran also discusses the concepts of Quality Control and Quality Assurance.
I find a lot of goodness buried in this book but to follow that now would provide a distraction too far from the V words.
To be fair to ISTQB Foundation Syllabus they have a section that touches on Quality - assurance and control. The Syllabus section doesn’t attempt to map back to Validation and Verification.
I don’t plan to critique this section here. I just mention it exists in case people think I’m picking on the Syllabus. I’m not picking on the Syllabus here. I’m using the Syllabus because it is one of the few free publicly available ‘standard’ type documents. People will use it to learn about testing. If they do, they may develop a high importance weighting on the terms Verification and Validation.
Overlapping Pairs
Why did I pick Juran? Because he concentrated on two main meanings.
And people rephrase Verification and Validation into two questions.
And Validation and Verification are two words.
Do they all map directly?
No. I don’t think so.
But we can clearly find a lot of overlap between:
- Are we building the right thing?
- Are we building the thing right?
- verification - meeting requirements
- validation - meets needs
- quality is meeting needs to provide product satisfaction
- quality is freedom from deficiencies
I personally think it is a useful exercise to work through the bullets and expand them for yourself as to what they might mean and how you interpret them.
Recommended Exercise: work through the bullets and expand them for yourself to identify what they might mean to you, and how you interpret them.
As a short example:
- Freedom from deficiencies
- how would we know?
- How could “deficiency” be defined such that we would know?
- Requirements specified to the level that can be reliably checked without ambiguity?
- could deficiencies still exist?
- requirements don’t cover everything then
- etc.
Juran summarised 1000+ pages and several other books into his ’two main meanings’. I don’t think he intended them to be complete. I think he wanted them to be explored and interpreted on a contextual basis. I’m less convinced that Validation and Verification were presented in PMBOK and IEEE standards as ’things to be interpreted’.
Think Through Your Vocabulary
Validation and Verification seem to be default words that The World of Software Development and Testing uses to describe ways of evaluating things.
Despite recognising that the words are very similar and the usage they are being put to is very nuanced, The World of Software Development and Testing continues to use those words, rather than finding different ways of communicating.
To clarify the use of the words, The World of Software Development and Testing, write articles to explain those words so the nuances can be understood more clearly by people in that world. I guess I’ve tried to do that too.
What I do
I don’t use the terms Verification and Validation. I haven’t used them for some time. I didn’t find that they helped me think about Software Development and Software Testing.
I make a special effort to take responsibility for how I think about and communicate testing.
I don’t think I should just use glossary based terms by default. If I do use them, I need to make sure that the people whom I say them to, actually understand them in the same way that I do.
Why I wrote this
I started down this writing path because:
- In my head, Validation and Verification had a high importance flag in my model of testing, but I avoided using those terms because of ambiguity and lack of clarity.
- I don’t want unquestioned defaults in my model - particularly if I avoid and work around them.
I think they had a high importance in my model because:
- I’ve read a lot of written Software Testing material over the years and I think that material has stressed Validation and Verification.
- I find these words all over the internet in relation to Software Testing
- I read articles telling me that the nuance between these words is very important for me to understand.
- It’s a default that has been conditioned into my model of Software Testing
I wrote this out because:
- Other people might have received the same conditioning
- There might not be a lot of material on questioning and exploring this basic conditioning. Most of the material seems to reinforce the conditioning.
I stopped using Validation and Verification quite a long time ago but never challenged the default in my model. I created a workaround, rather than a fix. This is my attempt at a fix.
My workaround was:
- avoid using the noun form i.e. validation or verification
- use a verb form - validate, validating, verify, verifying (based on the dictionary definition)
- pick one form. Because the definitions in the dictionary are so similar I tended to use the word “validate” i.e. “I will validate that…”
- replace validate with a more common word that is used in its definition of e.g. “checking”, “check”
- i.e. “I will check that I can login”, “I am checking that an incorrect password/username combination can give me an error message and that the error message doesn’t tell me that the password is incorrect”
This workaround made me uneasy because:
- I felt at odds with “The Testing World”
- I felt I might be missing a ’nuance’
This workaround provided me with the ability to:
- Communicate in normal english
- Communicate in simple words
- Take responsibility for my communication
I think I also have a fix now.
I don’t feel that Verification and Validation are terms that I will use in relation to Software Development and Testing despite their prominent Glossary positioning and industry writing reinforcement.
Feel free to feel differently.
