This episode covers the Test Automation Pyramid, created by Mike Cohn in 2008-2009 in the book “Succeeding With Agile”. We will go beyond the diagram and look at the model that supports it. Then deep dive into the model to explore it’s meaning in relation to Automated Execution Coverage, not Testing.
This was released to Patreon supporters early, with full transcript and no ads.
Introduction
Subtitle: The Test Automation Pyramid: Demystifying a Simple Model
I want to talk about the test automation pyramid, or as some call it, the test pyramid. We are going to dive into its origins, the misconceptions surrounding it, and how we can interpret it in today’s tech landscape.
The test automation pyramid was first introduced in the book “Succeeding With Agile” by Mike Cohn back in 2008-2009. During that time, automation tools like Selenium were still in their early stages, and automating GUIs was challenging. The pyramid model was a reflection of the development landscape at the time, where there were process issues, outsourced testing, and proprietary automation tools.
However, over the years, the model has been misinterpreted and misrepresented in various ways. Different variations have emerged, using terms like end-to-end tests, integration tests, manual tests, and acceptance tests: all deviating from Cohn’s original model. This has led to confusion and disagreements about the meaning of the test automation pyramid.
To make matters worse, some people have focused on the testing aspect of the model, rather than the automation part. This interpretation has caused unnecessary debates about the role of testers and the value of exploratory testing. But if we shift our perspective and view the model as an “automated execution using Xunit coverage model,” we can more easily understand the model.
The essence of the test automation pyramid lies in achieving coverage at different levels of an application. The model suggests targeting the most appropriate level for automation, rather than trying to automate everything at the UI level. By doing so, we can avoid redundancy and duplication, reduce maintenance efforts, and speed up the automated execution process.
Each application will require a unique approach to achieve the desired coverage. The focus should be on understanding the architectural layers, interface points, and integration risks within your specific application.
View the test automation pyramid as a set of heuristics for achieving coverage through automated execution. Rather than getting caught up in the misinterpretations and debates, create your own model for your applications and processes.
Summary
- The model was created by Mike Cohn in 2008-2009 in the book “Succeeding With Agile.”
- The original model focused on UI, service level, and unit level automation.
- Over the years, different interpretations and variations of the model have emerged.
- The term “service level” in the model has led to ambiguity and different interpretations.
- The diagram in the model is a simplified representation of a deeper underlying model.
- The focus should be on achieving coverage at the most appropriate level in the system.
- The model addresses the importance of avoiding duplication and redundancy in automated coverage.
- The process and team structure can impact the effectiveness of the model.
- The model can be reframed as an automated execution coverage pyramid.
Related Links
- Simon Stewart https://www.youtube.com/shorts/GNO6iHlYxzg
James Bach presented a “round earth test strategy”
- https://www.satisfice.com/blog/archives/4947 which deals with the “Test Automation Pyramid” from the perspective of “Test” rather than “Automation”.
- There is a video of this presentation here https://www.youtube.com/watch?v=gy93j7LE6sA
Discussion Questions
- What is the main takeaway from this episode on the test automation pyramid?
- What is important about the term “unit process” in relation to unit testing?
- Compare your opinion to the speaker’s on using the test automation pyramid model to describe test automation strategies?
- Why is understanding the underlying model important when looking at diagrams?
- What are some possible gaps between the UI, service, and unit automating?
- Why is there a confusion surrounding the scope of a unit?
- What is the difference between automating through API and testing the service layer made up of classes?
- How can tooling and architecture become different when using API automation compared to UI automation?
- What process and communication challenges can lead to problems with automated execution?
- How can teams balance UI level testing with unit level testing to achieve better coverage?
- Has your understanding of the test automation pyramid changed after listening to this episode?
- What are some advantages and disadvantages of using the test automation pyramid as a model for test strategies?
- What do you think are some common misconceptions about the test automation pyramid?
- How can teams effectively split up responsibility for achieving automated test coverage between different roles or teams?
- How can teams ensure that their processes support effective test automation coverage and execution?
- What are some alternative models or heuristics that can be used to design effective test automation strategies?
- How can teams identify gaps and overlapping components in their test automation strategies?
- What are some good practices for achieving effective unit testing with Xunit frameworks?
- How do you prioritize early, fast, and smaller points in automated testing while still achieving comprehensive coverage?
- What steps can teams take to ensure proper communication and collaboration among team members when designing and executing test automation strategies?
