Modelling for Naming and Understanding

TLDR; Often the process of modelling is more important than the models, we carry forward what we learn rather than remodel consistently.

I was looking through old folders of notes to feed to the shredder and paused to comment on some graphical models prior to the shred.

Over the years I’ve kept too many old notes and it is now shredding time.

Photos of Scraps

As I worked through a “Test Modelling” folder I found a scrap of paper with the following diagrams:

Code focused testing:

System focused testing:

Process focused testing:

I was going to bin this, but I thought that the diagrams weren’t a bad reflection of how I viewed things at the time. And more important they were a step on my learning and understanding of Software Testing process through the use of models. I thought I’d revisit them to see how I felt about them.

I didn’t add any text to the models other than names.

This offers a level of ambiguity that allows me to re-interpret them over time without a lot of narrative getting in the way.

One reason I don’t make written notes in books as I read them is because during a re-read I don’t want to be dragged back to the level of understanding or interpretation that I once had, I want to be free to re-interpret. So if I do make notes when reading books, I do it on the side in a notebook, possibly ripping out the note book pages and adding them to the back of the book.

Exploring Naming

Looking at the diagrams I think I was exploring naming. I tried to give the diagrams names that correspond to existing labels for test scoping:

• Unit Testing
• Integration Testing
• System Testing
• etc.

I wasn’t trying to ‘define’ the names. I was trying to model what they might mean in terms of a scope or focus for the Testing process.

The dashed lines represent the ‘scope’ that the name implied.

i.e. if doing System Testing then the diagram of the scope covers the input -> process -> output but does not include the downstream or upstream system. Although the diagram shows that it might include responses to messages to allow flows within the system scope to complete.

Redrawing the photos

I spent a little time redrawing the photos in Miro to make them clearer to see what I think about them now.

During the course of this I did some renaming.

Integration became System Integration.

And I extended the scope of the System to make it clear that I was interested in the responses to external messaging.

System Focused Testing

The System focus here is scoped from inputs coming in to the system, to the outputs and responses from the system outputs.

This implies to me that everything would be driven from a System input.

The diagram opens up the concept that it could be performed in an integrated environment with multiple systems, or those systems could be mocked or stubbed because the scope here is the flows within the system and the responses to the outputs.

I might be tempted to view the model as suggesting that every ‘step’ in the testing is a clean step. e.g. given a ’new’ system environment, I send in an input, trigger the processing, the responses, and wait till the process is finished before checking any results.

But the model doesn’t say anything about how we approach the testing other than what the focus for testing might be when ’named’ System Testing.

We could send in multiple inputs, simulate responses in multiple orders and see the impact on the functionality.

Also nothing here dscribes the infrastructure or where we might be observing or interrogating to check results. My reading of this diagram is that anything changed during the system processing is fair game for observation and interrogation.

System Integration Focused Testing

The focus in this diagram is the communication between systems.

I think the arrow between systems should be double headed to cover messages and responses.

But here the diagram suggests narrowing our focus to just enough system processing to generate a message and just enough to trigger a response.

This narrowing of focus means that we are relying on other testing models and activities covering the actual system processing and here we are exploring if systems can communicate effectively.

With this model, there is no stubbing or faking of messages or responses. The actual systems would be generating messages and would actually be processing them.

The System Focused model would allow the stubbing and faking, so Contract Based Automating would probably fit into the System model rather than the System Integration model.

UAT Focused Testing

I don’t think I’ve worked with an organisation that has peformed UAT for well over 10 years.

But the “UAT” model is interesting to me because the scope of testing partially covers the System functionality. Some of the functionality is excluded when it is purely automatic, or purely system to system and doesn’t materially impact the Real World Interaction for the user.

The Real World Interaction is represented by the “Manual Processes” elipse.

So this focus is less about ‘acceptance’ and more about exploring if the system is fit for purpose when the user tries to conduct their work in the Real World and all the supporting communication activities that take place outside the system but the system must support.

When I drew the model I was giving them ’names’ that were in common use at the time but I really think I was exploring the scope that testing can cover.

This scope, with the Real World interaction and partial system coverage is the scope that most projects seem to miss out.

Unit Testing

The Unit Testing diagram is interesting to me.

There is nothing in the diagram to limit the scope.

Does ‘code’ refer to a class? A collection of classes? A function?

There is nothing to prefer or limit the use of mocking or stubbing.

And what counts as a ‘Module’? I suspect I used Modle to mean a logical collection of classes. These might map on to partitions of logical designs like the domain objects, or API, or persistence mechanisms?

But what I take from it is the ‘internal’ nature of the focus. Code and the code interactions, at whatever layer of abstraction or interface is required, using whatever scaffolding is important to allow the code to operate.

Link Testing

Is Link Testing even a term?

I suspect I was concerned that Unit Testing might forget about the logical layer interactions and I’d still want to see the code integrate, but I’d want to see it integrate outside of the concept of the System.

Phases of Testing

The Phases diagram is interesting to me because I don’t see any ‘phases’. Looking at it now I’m wondering if I was trying to show that naming phases was not important, but making sure that coverage of all the different focal views was more important.

Certainly this is more like the view of Testing that I have now. Phases are not a concept that particularly concern me. Coverage is much more of a concern for me:

• making sure that we have tested at the right levels to explore and identify problems
• making sure that we have automated at the right levels to identify changes
• making sure that we have automated at the right level of abstraction to make the automated execution maintainable and allow refactoring such that it can evolve as the system evolves

What’s missing?

Lots of things.

There is no representation of the Infrastructure here, but I would assume that would be considered:

• with a “System Integration” Focus
• during the “Link” Focus where we have a subset of coverage that operates at a logical level, in memory level and at an integration level (e.g. using TestContainers to spin up Postgres, MySQL or LocalStack)

There is no representation of Performance testing, or Security testing, but again these can be done with a Unit, System or Integration focus. Depending on the tooling we use and the risks we are prepared to accept.

Also the names aren’t that important to me anymore.

Looking at it now, the concepts that are important to me are:

• testing in isolation
• testing with scaffolding
• testing with integration (with the ‘real’ things and not scaffolding)
• expanding the integration outwards to remove as much scaffolding as possible and eventually testing withouth any scaffolding
• ecompassing the ‘real world’ and ‘real processes’ as part of the testing, not just focus on the system

The order in which we do things is no longer imporant to me. But stepping back and looking for gaps in our coverage and identifying risk is important. Such that at the arbitrary end of our process we can see that we included all of it.

Feel free to critique the diagrams if you wish. I have no sense of self associated with them. I think they were created by me-15-20 years ago, so they are a historic document and are really here to provide an example of the types of models and diagrams that I used to think through testing.

Rather than critique them, I’d rather you shared your own models, or just built your own models and revisit them years later, hopefully you’ll see that you are not your previous self.

As Wiliam Blake wrote in “Jerusalem: The Emanation of the Giant Albion”

I must create a System or be enslaved by another Man’s,

I will not reason and compare: my business is to create.

If our business is create, then as we do, we will naturally change and evolve the models that underpin our own Systems. When we look back at our earlier work we will hopefully see that we have changed our models and how we interpret them. Otherwise we risk being enslaved by our own System.