TLDR; Claims, and Expectations can be used as a high level description of Software Testing. Given a set of claims, check if claims are met. Create a model based on expectaions, then use and evaluate expectations. Further explore to identify scenarios where claims fall apart and expectations are unsatisfied.
High Level Model of Testing
There is a lot to learn about software testing, but it is still possible to present a high level set of generalizations that anyone can understand, implement, and build on.
And that’s what I’m going to do now.
Given Models
So the first thing is that we have often been told things about a product, and we can test those claims.
We’ve been told something, possibly verbally, or possibly written down in marketing material, or we found it in instructions.
We’ve been given a model about the software and we can test using it.
“This software can do X”. To test at a most basic level, we try to make it do X, and if we succeed in making it do X, we don’t say “function X works”.
We say, “When I did this list of actions, I observed the following results, which suggests, under the conditions I described, that the product can do X”.
Which is slightly different from saying “the product can do X” because we don’t know if it can always do X, or if it handles all the situations for X or all the input data.
So we can’t say “it works”. But we can say that it ‘seems to do what it claims, under this set of conditions’.
We might want to continue testing to find out more scenarios where it can do X, and possibly find somewhere that it does not do X.
We can say that we observed results which were expected.
We can say what the results were, and other people can interpret that how they will. And clearly, if we find a problem, then we can say it does not work, and then explain what we mean and point to the claim, the original claim that we were given as an authoritative source or an Oracle, which gives our report more weight.
But we can’t say that it does work.
Created Models
We may deduce, the other thing we can do is, we may deduce or infer expectations from the behavior and appearance of the product. We can test from those expectations, so we don’t need to be told everything in advance.
We don’t need to be given a model. We can create our own model of the application and test using it.
So if a field in a form said that it accepts an email address, then we can deduce that an email address should be acceptable in that field, so we can try it out. We could try different combinations of email addresses. We could try valid and Invalid email addresses and see if the system can identify between them.
We can test based on our deductions and our inferences.
And when we do, we have to be careful that we don’t draw hasty conclusions, because we might be tempted to report that the system is broken because it does not handle Invalid emails.
But we inferred or deduced that it should handle Invalid emails. We had no authoritative source telling us, so we have to report our observation and why we expected a different result. And we may learn, on reporting, that the validation feature has not been implemented yet.
So we learned and expanded our model about the system but we didn’t really expand anyone else’s model about the system.
And those two points claims and expectations, or a Given Model and a Created Model, is a pretty good place to start if you want to explore and experiment with software testing.
This was originally recorded on Racket in 202106
