Software Testing Definitions and Models - How and Why to Create your own Definition

TLDR; I avoided creating a definition of software testing for a long time because I thought it would constrain me. Instead I built a model of what I did. Eventually I generalised it to a ‘definition’ - “Testing is building and comparing models of a system to the system.”

[Digital image courtesy of the Getty’s Open Content Program.]

From 29/11/2016 to 6/12/2016, every weekday, I used Instagram as a platform for creating a ‘daily thought’ on the topic of “Definition of Software Testing”. That was a useful process in its own, this blog post is to document the output.

What follows:

text description of the daily thoughts with additional commentary (keep reading)
the youtube video has annotated slides and the videos
you can find the slides on slideshare
raw instagram videos (see list at the end)

Testing is all about testing

Is Testing all about bugs - no. Testing is all about testing. Only nothing is all about anything else - Nothing is all about the absence of anything else. The only thing that that a thing is all about is the thing. Testing is not all about bugs because testing is all about testing.

Testing is what testers do

Testing is all about testing. But do we know what that means? Do we know what testing actually is?
One way to figure that out is to come up with a definition, but definitions are hard.

I remember attending a training course with James Bach and he recommended that we create our own definitions of testing. And I tried and failed.

So what could I do?

Here is one thing I did when I was trying to get my head round my definition of testing. I created a list. Because, if Testing is what Testers do. Then I create a big set of these are the things that are part of testing. I have a model of my testing every day life. I stuck an etc. at the bottom.

And then I can generalise that into a definition of testing.

That might be a useful thing to try if you haven’t yet got your own definition of testing or are still working on a model or definition of testing.

A limiting belief about “Testing is…”

There were a couple of things that stopped me creating definitions for myself.

The first one - I found it hard. I found it hard to create a definition in the first place so that’s why I created a list.

And the second thing was I had a limiting belief about definitions because I thought that if I decide what testing is then I’m going to be constrained to what that definition of testing is, the definition is going to stop me being flexible, and if something doesn’t fit into my definition then I’m not going to do it because its not testing. By definition its not testing because I’ve defined what testing is.

And the way I got round that was by realising that definitions are just models. We can have multiple models, we can use multiple definitions and our models can be wrong, and our models change over time. That is what allowed me to create a definition of testing. So if your approaching it and one of the things your worried about is “I have to only do THIS because that’s what TESTING IS”, that’s just a model and a belief which can be changed.

And we can have multiple models - and all at the same time.

“Alice laughed: “There’s no use trying,” she said; “one can’t believe impossible things."

“I daresay you haven’t had much practice,” said the Queen. “When I was younger, I always did it for half an hour a day. Why, sometimes I’ve believed as many as six impossible things before breakfast.”

Alice in Wonderland. by Lewis Carroll (Charles Lutwidge Dodgson)

My definition of testing?

So have you tried creating a model of testing for yourself yet?

A definition of testing?

I’m going to tell you my definition.

My current working definition or model of testing is…

“Testing is building and comparing a model of a system to the system.”

And since we use multiple models…

“Testing is building and comparing models of a system to the system.”

So see how that works for you. I mean, what does that not cover, in what you do?

That will give you a hint and indication in what you value as important for testing. And if it does sit well for you then see where you can take it. Because these definitions are supposed to empower us and help us to think about testing, rather than limit how we view and think about testing.

If there are parts you don’t like then that’s a good indicator of your own definition.

Process held me back not definitions.

I was concerned that a definition would hold me back.

But in reality, what usually caused us issues was not our definitions but our processes. And our strict compliance to processes.

Sometimes processes are a result of someone’s model or definition, or strong belief, but sometimes they are a result of people not having their own model or definition. They just pick whatever definition, or their interpretation, of something they found in a book. And they don’t have a proper model to guide them. So whatever we do, we can’t change the process, because we don’t understand the process, and we don’t understand how it fits in to the particular context that we’re working.

When your boss, or the person who defines your process, doesn’t have a definition, or a model of testing of their own. They look around and take someone elses, without understanding it, and without being able to justify it.
When you don’t have a definition, you can’t compare the current model to your’s, you can’t fight back or offer alternatives.

When you’re in that situation and you don’t have a model or a definition then you really don’t have any basis to question the process that you’re using to have a good way of moving forward.

In the early days I had problems because I had no definition or model of testing. I just knew we had problems, and I couldn’t find the answers in books, and I didn’t trust my own ideas for alternatives.

Models are very important for us to look at our own processes and construct our own way of working within our environment.

How does my definition help me?

When you build a model or definition of your testing, how does that help you?

Strategy

Well if you’re looking at strategy you starting thinking:

why would I build this model?
why would I model this part of the system?
why would I compare this model to the system?

That gives you clues to your strategy.

Your strategy starts with why.

Your strategy deals with why are you building and conceptualising certain models and why they are important to compare.

Approach

And your approach could be how.

How am I going to compare this model?
What techniques am I going to use to expand the model into concrete instances of conditions that we’re going to use.

Planning

Your planning comes in with who and when.

Who is going to compare this?
Who is going to build this?
Who has the skills?
When are we going to do this?
Which models are more important to build before others? When are we going to do it?

A Summary?

So it all feeds into your general strategy/approach/plan part, and your model or definition of testing can help guide you through that.

Strategy - Why would we build that model, why would we compare that.
Approach - How will we compare this?
Plan - When will we compare this and who will compare it?
Techniques - help me expand the model.
Exploration - find gaps in the model.

Recommended Exercises

write a list of things that you do
- are they all ’testing’?
seek out other people’s definitions
- write them down
- do they cover your ’list of things that you do’ that you classified as testing?
create a sentence that summarises your model of testing
- does it cover everything you do? Does it need to?