On the 21st of March I hosted a “Dev Ops and QA” session at the Test Focus Groups in London.
This turned into a small 2 page article for Test Magazine in the May 2017 issue.
The write up is based on the discussions over the day.
I try to summarise what DevOps meant to the people in the meetings, their concerns and some of the experiences we discussed.
Some of the features people associate with DevOps being:
- faster delivery into production
- continuous deployment into production
- automated ’everything’ (deployment, testing, monitoring, alerting, rollback, etc.)
- cheaper environments (use of cloud infrastructure)
Devops we described as a culture where the Software Development process encompasses the entire scope of System Development from requirements through to Operational Deployment and Support.
It became clear from the discussions that one highlight of DevOps was that QA could finally be used to describe the process. QA does actually mean Quality Assurance. Because Devops brings together Software Development and Operations from the start point of requirements through to Operation. Possibly for the first time a team can have full requirement life-cycle and the term QA implies assuring quality at every stage in the process from requirements gathering through to operational production support.
We discussed if testers would still be required. If we take the view that a team has full requirement life-cycle responsibility then that worry would seem unfounded. But companies do interpret Devops as programmers and operations staff merging, and relying on automated assertions to provide assurance, in this situation Testers may find themselves at risk. Those of us in a testing role should not be complacent. When teams work more closely together and collaborate there is an expectation that we can add value to more processes.
An adoption of a Devops culture does mean overlap of abilities in the different roles and testers will need to increase their technical understanding of the systems they work on, and their ability to interact with the system and support tools on a more technical level.
Testers also need to be able to communicate the value that their approach to testing adds to the development process, and how it differs from an automated acceptance checking and TDD process in order to help organisations develop a culture that does engrain both testing and QA into the full lifecycle.
DevOps does offer challenges to the World of Testing but also offers opportunities to System Development as a whole and Testers, as ever, need to adapt their testing approach to the needs of the organization. You can find more information in the actual article in “Test Magazine” May 2017
Article Text Content
First Draft Article content added here in case the online version of Test Magazine is removed.
Devops is an easy buzzword to consider applying to your software development process, but what does it mean and how does it impact your QA and testing process?
This was the basic starting discussion point in the “QA in Devops” discussions at the Test Focus Groups.
Devops itself has multiple definitions. Some of the features people generally associate with the Devops buzzword are listed below, clearly there will be many more:
- faster delivery into production
- continuous deployment into production
- automated ’everything’ (deployment, testing, monitoring, alerting, rollback, etc.)
- cheaper environments (use of cloud infrastructure)
The above features and benefits are not exaggerated they have been experienced by many teams and organisations adopting Devops, but the change required in the organisation and development approach can be massive and the current start point for an organisation may be far distant from the end point. People often want to adopt Devops as a quick win, without realising that Devops requires a culture change rather than process or tool adoption.
Many of the participants in the focus group discussion had experienced Devops, and all of the experiences were different - as were the processes that were described as Devops, but we can find commonality that might help understand what make Devops different.
The name Devops suggests a merging of Software Development and Operations. Software Development includes analysis, requirements, programming and testing, but often misses out the operational deployment. Devops then can be viewed as a culture where the Software Development process encompasses the entire scope of System Development from requirements through to Operational Deployment and Support.
Some companies interpret Devops as a merging of developers (often just programmers) and operations staff. When this happens the people with a Developer role need to learn all the skills associated with deploying and supporting the system in production, similarly the Operations staff need to learn how to program release quality code. In practice this would mean a fairly lengthy period of training and inefficiency. A more effective interpretation might view the merge as a vastly improved collaboration.
Collaboration between the Software Development team and the Operations team is often a source of contention. We do want to keep people away from production environments because: of data protection issues, tinkering with production increase risk of down time, general security and stability. We might have regulatory requirements which we implement via strict control over environment access and permissions. All of this tends to push Operations into a very separate and isolated position, with the negative side-effect that, despite being a major stakeholder in the system, their system requirements are often the lowest priority. Operations teams have system requirements including: ease of upgrading systems, deploying systems without downtime, backups without downtime, monitoring system performance and functionality, automated deployment to prevent mistakes.
With a move to Agile Software Development we very often see improvements in meeting the business requirements, but often still don’t include the operational requirements. Adopting Devops can help improve that situation.
We often misuse the term QA, in Software Development processes to mean Testing, rather than Quality Assurance. A misuse of the term since Quality Assurance applies to every sub process and task within Software Development and not just Testing. When we explore the world of Devops, QA does actually mean Quality Assurance. Because Devops brings together Software Development and Operations from the start point of requirements through to Operation. Possibly for the first time a team can have full requirement life-cycle and the term QA implies assuring quality at every stage in the process from requirements gathering through to operational production support.
People do seem to worry that Testers may no longer be required when teams adopt Devops. If we take the view that a team has full requirement life-cycle responsibility then that worry would seem unfounded. But companies do interpret Devops as programmers and operations staff merging, and relying on automated assertions to provide assurance, in this situation Testers may find themselves at risk. Organisations may not realise that this particular Devops interpretation makes the development process higher risk, with increased risk that error infused scenario and data combinations are released to production because they were not tested to a professional standard. The systems were tested and the tests were coded to allow them to be executed quickly and continuously on every system change, the analysis behind the tests may have been superficial compliance to acceptance criteria or limited to the TDD coding used to design the implementation code.
Given the prevalence of the view that Testers are not required, although some form of testing is still required, those of us in a testing role should not be complacent. When teams work more closely together and collaborate there is an expectation that we can add value to more processes. For example a tester might not be a specialist in Unix, shell scripts and deployment of environment and applications, we will have to develop an understanding of that area to add value in collaboration and develop basic skills in that area so that we can contribute to those processes.
An adoption of a Devops culture does mean overlap of abilities in the different roles and testers will need to increase their technical understanding of the systems they work on, and their ability to interact with the system and support tools on a more technical level.
Testers also need to be able to communicate the value that their approach to testing adds to the development process, and how it differs from an automated acceptance checking and TDD process in order to help organisations develop a culture that does engrain both testing and QA into the full lifecycle.
During the forums we also discussed sources of learning about Devops and the main resources recommended were:
- “The DevOps Handbook” describes the philosophy and practices behind a typical DevOps culture
- “The Phoenix Project” a novelisation of a team adopting and adapting to a DevOps approach
- “The DevOps Adoption Playbook” a more management focussed book about adopting DevOps
I do recommend reading the above books as these are more likely to form the basis of a team’s shared view of what Devops might mean. I also recommend reading as many blogs and experiences of DevOps as you can because the implementation of DevOps is a very environmental specific endeavor.
Sadly when organisations move to implement DevOps they do so because of the benefits that they have heard arise e.g. faster delivery, cheaper environments (use of cloud infrastructure), etc. I write “sadly” because Devops requires a culture change of even closer collaboration than that required for Agile development, it could potentially require deep architectural changes int the systems to achieve all the benefits envisioned. And when the desired benefits do not manifest quickly enough because the work it takes to change so much: the culture, the development approach, the systems, the monitoring, the environments. Organisations often jump to the next newer buzzword in the hope of quick wins.
There is a lot to learn from the Devops community, regardless of your current approach to Software Development, but we should be wary of announcing a strategical shift to Devops and expecting to instantly having a cutting edge Software Development process.
Devops requires a culture change and ultimately means building systems such that we can quickly deliver on requirements and monitor them in production to ensure they give the value we expected when delivered. Devops is a full lifecycle approach that can take years to develop. Organisations may find value in adopting some of the tools and tactics of Devops teams e.g. more automated assertion of acceptance criteria, continuous integration, increased monitoring of live systems, closer collaboration of different disciplines in the development process.
Some organisations will describe that adoption of Devops because they concentrate on the buzzwords and techniques rather than adopting the Devops philosophy of experimenting to build software effectively and efficiently.
The Devops philosophy of responsibility and incremental improvement through experimentation can be adopted by any team, and any discipline, at any time to gradually improve your process. You do not need to adopt the buzzword of Devops to learn from practitioners sharing their experiences under that banner. I hope you do read some of the books listed above to experiment with approaches they describe and apply them to your existing processes.
Books Mentioned:
- The DevOps Handbook by Gene Kim, Jez Humble, Patrick Debois, and John Willis
- The Phoenix Project by Dene Kim
- The DevOps Adoption Playbook by Sangeev Sharma
