TLDR; Testability is for humans. Automatability (Automatizability) is for applications.
I was doing some research for my upcoming Eurostar webinar and I encountered a few videos and posts of people who were using ’testability’ to refer to the ability for the application to support automated execution.
I didn’t think that was appropriate. I’d rather distinguish between Testability and Automatizability. The more popular form of Automatizability seems to be Automatability.
When we look at application features associated with testability we may find overlap with the application features we associate with automatizability. But humans can often work with a less formal implementation. What works for humans does not always work for other applications that want to automate your application.
I’m mainly concentrating on application support for testability here, rather than: development process testability, infrastructure testability, test ability.
But I do also touch upon technology testability.
Testability
Basically - If we can test an application then we can consider it testable.
We can increase our ability to test an application by using tools and understanding the technology.
Some technologies seem easier to test than others, they have a higher intrinsic testability.
- This is a subjective assessment.
- I find a web app more testable than a mobile app or a green screen app.
- more tools support my testing of the web technology and mobile technology
- ergonomics and usability of the web technology are easier for me to test than the mobile app
- etc.
We can increase the testability of an application by adding features to help us test it more easily.
- This makes the application more testable.
- Some of the features might help usability.
Testability is not Automatizability
- Testability - does the application have features that make it easier for a human to test?
- Automatizability (Automatability) - does the application have features that make it easier to control and interrogate by another application.
The features we add to make it more testable are not necessarily the same features that we add to make it more automatable.
e.g. logging
- Improved testability by logging that shows hard to observe messages or internal state transitions or variables
- the logging might improve Automatizability if it is easy to parse and accessible by another program
Testability is not Free - consider risk
The features we add might come with Risks.
- It might impact Usability if it clutters the screen or confuses the user so the user needs to be able to choose if they see it or not
- It might impact security if it gives away too much information so we need to be able to switch it off the ability to see it for specific users, or environments
- It might impact performance, so we might need to only have it run on specific environments
More resources
Testability is a broad subject and I’m really only concentrating on features we can add to applications to make them more testable here.
If you want to explore testability in more detail then consider:
- James Bach’s Heuristics of Software Testability
- James Bach’s testability blog category
- Maria Kedemo - Dimensions of Testability and video
Summary
- Testability is not Automatizability (Automatability)
- Testability relates to Human interaction
- Automatizability relates to application interaction
- Some technologies are more testable than others because of tool support and usability
- Application testability can be increased by adding specific features to support testing
- Testability features might overlap Automatizability features
- Application automatizability supports programmable observation, interrogation and manipulation
