Wherein the TOTE (Test Operate Test Exit) model is used to explore the nature of feedback and abstraction of test phases and map the TOTE model on to TDD, Exploratory Testing, Design processes, Analysis, Learning, Decision Making and Problem Solving.
I originally wrote parts of this in 2002 (20020815), and 2017 (20170508). I merged them together on 20211101 and expanded the text (at the risk of confusing things, I left in some bad jokes)
TOTE
In 1960, George Miller proposed a model of Goal driven behavior which he titled T.O.T.E (Test operate test exit). We can map the T.O.T.E model on to Software Testing.
In 1960, George Miller, presented a model of problem solving which he called the T.O.T.E model
- Test, Operate, Test, Exit
The notion being that you loop around a [Test, Operate]* cycle and when your Test is complete, then you have done enough Operations and you can Exit.
It was a model of problem solving, or decision making.
I wrote about it in my NLP papers
You can find George Miller’s book Plans and the Structure of Behavior on archive.org. He describes the TOTE model in that book.
Fig 1: The TOTE Model as a Graph
An Understanding of TOTE
My understanding of TOTE is very simple. TOTE models the situation where the stimulus behind behavior is the achievement of a Goal. In order to achieve the Goal, that goal has to be defined thoroughly enough to allow us to recognise when the goal has been achieved so that as we move towards the achievement of that goal (operate) we can assess (test) if that goal has been achieved and then Exit.
George Miller intended this to be a model of behavior, not a model of IT. A model of directed behaviour where decisions are made based on information achieved by testing.
Combining models from two disciplines - Psychology/Software Testing - is a useful outcome from inter-disciplinary research. During the combination of models we may learn something about the difference between the two models. We may learn something that makes one model unique. We may learn a little more about how we view Software Testing and Development.
Software Development can be viewed as a sequence of TOTE cycles. In that we first decide that we want a system to allow us to do things, so we list those things and assess how close we are to having the system we want (test), we build the system (operate), and if the system allows us to do the things we want (test) then we call it complete and release it (exit). Then, we access if there are more things we want the system to do, and we enter another TOTE cycle.
Fig 2: System Development TOTE Sequence
In fact every single aspect of software development, because humans do it, becomes subject to behavioral analysis and can be viewed using the TOTE model.
Avoiding Testing
Could TOTE explain why many companies have tried to avoid testing?
- We specify the system (test)
- We build the system (operate)
- We Unit Test the system (test)
- Why can’t we then ship it? Surely adding System Testing to the mix can be perceived as being over the top (OTT).
Possibly, because people use one word “Test” and ignore the fact that Testing operates at multiple logical levels.
System Testing executes the system from a different level than that of Unit Testing. Unit Testing works from within and system testing from without.
Unit Testing can start executing before the system has been put together, System Testing cannot start executing, without flagging up a lot of ’this is not implemented yet’ warnings.
Unit testing must know how the system has been constructed, System testing doesn’t need to. Both have different sets of goals and different sets of operations and tests.
User Acceptance Testing waits for the development testing (Unit and System Testing) to be complete. The goal being to fit the system into their business process.
An IT development model that has Unit Testing, followed by System Testing, followed by User Testing isn’t OTTT, from a distance it is simply OT. We, as IT personnel, have simply taken the T chunk and split it into UT, ST, and UAT to make the most of the parallelism that is available with multiple levels of validation conducted by different people.
NOTE: (20211101) Reading the above text in 2021 makes me slightly uneasy because I no longer user the same terms to describe testing that I did back in 2002, concepts like Unit Testing, System Testing, UAT now feel like marketing terms to me. Terms that don’t ’explain’, but are widely enough used that I can communicate a rough approximation of what I mean when I use them. I left the terms because they seemed like an easy way to talk about multiple logical levels.
Test operates at multiple levels: nested, sequential and in parallel.
Fig 3: The Parallelism of the development cycle
Note: I model TOTE cycles as nested as well as sequenced. An “Operate” can consist of nested TOTE cycles which might be a better way to view ST and UAT. Sequences can operate in parallel.
TOTE in Action
Showing T.O.T.E in action. I drew this dynamically to make the point that it is a cyclical process and we Test to decide if we continue to Operate, to Exit the process, and to change what we will operate.
You can see the 9 second version on Instagram
Test to build a model.
Sometimes we exit because things are Good Enough, but we still need criteria to determine what Good Enough means.
Sometimes we Operate to learn if something is Good Enough.
TOTE for learning
At the time that I explored the TOTE model previously I didn’t make the connection that arc from Operate -> Test was also a feedback process.
In the TOTE model the Test learns from Operate, which we can easily map on to Exploratory Testing.
- we come up with an idea to explore (
Test) - we explore (
Operate) - we learn from that (
Operate -> Test) - we derive new things to explore (
Test) - etc.
And we Exit when our time has finished or we have covered our ideas or whatever other ’exit’ criteria we started our testing with.
TDD TOTE
I’ve also written a lot more code using TDD. And I know that my TDD process very often resembles a TOTE process.
- write some
@Testcode (Test) - see if fail and write some code to make the test pass (
Operate) - write more
@Testcode to flesh out the design (Test) - and repeat
Until our design is complete, or our review of our @Test and code can’t come up with anything new, and we have compared it with our statement of intent, etc.
What might “Test” mean in the context of TOTE for Software Testing?
“Test” is a verb, not a noun.
“Test” here seems to map on to an Evaluation process to make a decision to determine the next step, either “Operate” or “Exit”.
The ability to make decisions and take responsibility for our Testing is a key skill.
A verb consumes nouns as input and produces nouns as output.
“Test” Input:
- Evidence
- Plans
- Questions
- Answers
- etc.
“Test” Output:
- Decisions
- Communicated Information
- etc.
Since TOTE cycles can be nested, most of our decisions will be noted down in our Exploratory Testing logs and they form the bulk of our logs:
- what data we used
- what path we picked next
- what we chose to defer until later
- what we chose to interrogate
- what we chose to observer and where we placed our attention
- what evidence we chose to gather
- what question we formed and left for later investigation
- what we chose to prioritise in the moment
- etc.
Each of these decisions would have been the result of a nested and sequenced TOTE cycle within our Exploratory Testing Session.
What might “Operate” mean in the context of TOTE for Software Testing?
Again “Operate” is a verb, not a noun.
“Operate” seems to map on to the actual Testing process: exploration, experimentation.
Input:
- Test Ideas
- Test Conditions
- Acceptance Criteria
- Questions
- Models
- etc.
Output:
- Answers
- Evidence
- More…
- Test Ideas
- Test Conditions
- Questions
- Amended Models
- etc.
As we Operate, we make decisions:
- should I go ahead with my plan?
- should I be distracted by this new idea?
- should I investigate this defect?
- etc.
Each nested decision is a new TOTE process.
One reason why Testing and Development works well in short bursts or cycles is that we set out to achieve a goal. We make many decisions during the pursuit of that goal, we learn information that may put into question the original goal.
Short bursts of activity, allow us to adapt to the learning that results in Amended Models, new Evidence and new Questions.
e.g. 1 hour of testing, half a day of development, one week of a product development cycle
TOTE Learning
We could view TOTE as a completely well defined process of evaluation where at every ‘Test’ point we know exactly what we are deciding upon and use the pre-defined evaluation criteria. This doesn’t seem to map well on to high level, long term goals, this works better for very short term goals.
The Operate process acts as a learning process which feeds into the Test process and explains the cycle. Each time we ‘do’ something (operate), we learn something which we feed into the Test (Evaluation) process.
I missed the learning process inherent in the Operate -> Test arc first time around back in 2002.
TOTE: a model containing three simple words… still seems worth investigating.
See also
- instagram.com/p/BTnqhiQAGr0
- www.instagram.com/p/BTpdkSSAz0K
- NLP for Testers
- George Miller’s book Plans and the Structure of Behavior
PS. If you want your own T.O.T.E model diagram then feed this into Graphviz or WebGraphviz
digraph G {
node [shape = "rectangle"];
Test -> {Operate Exit}
Operate -> Test
subgraph { rank = same; Test; Exit}
Exit [shape = "ellipse"];
}
End Notes
TOTE is just a model. But Quality Software Development is a goal that most development teams set out to achieve, they cannot do that without knowing what they mean by Quality and without checking to ensure that they have achieved Quality. Their view of Quality will be limited to the logical level and manner in which they evaluate Quality.
Do we want to produce programs that are fault free or do we want to produce programs that allow people to do what they want to do in a quality manner? Ideally I’d prefer both, but humans make mistakes so we should aim for a product that functions in a quality manner. When we achieve that we do it by assuring ourselves of the quality of the models in the software development process as they are produced, and before they are fed into a downstream process that will build new models using the information contained in those models.
How do we assure ourselves that the model is a quality model? We have to know what we want from a model, control the production process as much as possible, check the model when it is done and periodically throughout its production.
This is the fundamental assertion of the T.O.T.E model in psychology [1]. The T.O.T.E model is a sequence of steps: Test, Operate, Test, Exit. With a goal in mind we Test to see if the goal has been achieved, if not then we operate to change something which will hopefully bring us closer to our goal, we test and if the test is satisfied we exit that process as our goal has been achieved.
The T.O.T.E model describes a process of refinement.
An Aside:
Some Variants of TOTE used in IT
BAD (malformed paths)
- OE (oh) - no testing involved, code it then ship it
- OTEA (oh dear), operate, test, then exit anyway
GOOD (well formed paths)
- TE -( tche!) - no funding project canned and deemed unfeasible, bad idea do not pursue
- *(TOT)E TDD - write the failing test, write the code till test passes, refactor? more tests? done?
[1] Modeling with NLP, Robert Dilts, 1998, Meta Publications
