TLDR; Observation in real time. Interrogation after the act. Bringing Interrogation closer to Observation can help detect issues during a process. The depth of Observation and Interrogation changes depending on our knowledge of the system and technology. And we may not be done, if our observation was limited.
When I test I make a distinction between Observation and Interrogation and I’m going to explain what that means, and show you hands on example of how that distinction helps me improve my testing and the scope of of my testing.
Supporting Video
Explanation
I think it is very important to analyse our own approach to testing and identify our own models and unique perspectives.
One distinction that I noticed from my testing was Observation in contrast to Interrogation.
When I act, I Observe, in real time, and spot confirmation or divergent signals as I act.
Having acted, I then want to check if something happened, at which point I pause and Interrogate the system to identify conformance or divergence.
Both processes can use the same tools, require the same technological understanding and both can increase the amount of information available to me to communicate or build on.
But, because of the point in my process that they happen. Interrogation offers me the ability to go deeper into the system and impact. And Observation requires tools that help ‘divergence’ stand out, and potentially filter more noise than Interrogation.
Making the distinction also helps me realise that I sometimes want to bring closer together the points where I Interrogate so that I don’t let too much time elapse before Interrogate, otherwise there is an increased probability that signals escape my notice.
Video Notes
Observation vs Interrogation
- a distinction I use as part of my test process model MORIM - modelling, observation, reflection, interrogation, manipulation - which I cover in my Technical Web Testing course and various blog posts
- observation means what we perceive in real time
- interrogation covers what we investigate after
- this is subjective - ‘after’ might be ‘almost immediately’
Example:
- https://thepulper.herokuapp.com/apps/pulp/
- v010
- the pulper has books, and authors, when I delete all authors for a book, I expect the book to be deleted
- look at books, lester dent, kenneth robeson - 364 total books
- look at authors, delete lester dent, delete kenneth robeson (observation - see authors removed)
- look at books, (interrogation some books deleted, 231 books count)
- we can use the distinction between observation and interrogation to help us improve
- interrogation makes me think - what else should I be looking at to check for expected impact?
- reports, database? api calls? what is the sphere of influence of this functionality?
- interrogation makes me think - what else should I be looking at to check for unexpected impact?
- this is much harder to limit because the answer is everywhere, so I rely on my test coverage for additional functionality to cover this.
- observation makes me think how else can I observe this?
- what did we observe?
- we saw authors being deleted from the GUI
- did we observe deep enough? database? file system? memory?
- what can we observe? http traffic (GUI calling server to delete)
- can I bring observation and interrogation closer together to spot any problems more quickly?
- have books screen open at same time as author screen, refresh book screen as delete author
- see it using the api through observation
- interrogation shows me the books deleted after deleting lester dent, is that expected? kenneth robeson is a ‘house author’ but I’m not sure I’d expect that to delete the book, that might be a bug.
When I test I’m thinking am I observing enough, and there are times when I want to increase the scope of my observation. To do that I also need to expand my technical knowledge of the application to identify tools and we might even need additional hooks into the application to support that observation process.
At times I also try to bring my interrogation process closer to my observation process to spot any issues faster.
When I’m still building my model of the application then I am less likely to do this. So if I’m just starting my testing and I’m not familiar with the flow of the GUI, I’m less likely to incorporate deeper levels of observation. But because I have the distinction Observation and Interrogation, I know that at some point I’m going to have to increase the depth of my observation.
This distinction also impacts questions of ‘when is testing done?’ because if I have only tested using interrogation, and my interrogation has been at a surface level, or has been at the batch end to end process, I may want to do more testing to increase the number of inspection and evaluation points as I test.
I’ve found a lot of value in exploring that distinction Observation (more real time), and Interrogation (more batch oriented), and considering the scope that my observation and interrogation covers and adding more technical tooling and knowledge into the process to expand my ability to evaluate the system operation.
Recommended Next Steps
- Use https://thepulper.herokuapp.com/apps/pulp/ and repeat the observation and interrogation process
- Go further, there are more bugs in The Pulper than I describe, practice your Observation and Interrogation
- When you next test the applications you work on, think “What am I observing?”, “What could I be observing that I am not?”. “Are those things I am not observing useful to observe?”, “What tooling could I introduce to extend my observation?”
- After testing, think “What can I interrogate?”, “What other parts of the application might relate?”, “What side-effects have I not checked yet?”, “Are these assertions I could automate?”, “What tooling would help be better interrogate?”
