- Keynote: Helping Testers Add Value to Agile Projects
- Tutorial: Technical Testing in an Agile Environment
- Workshop: Black Ops Testing
Keynote: Helping Testers Add Value to Agile Projects
I presented the closing keynote to the conference.
Blurb
Every Agile project is different, we know this, we don’t do things ‘by the book’ on Agile projects. We learn, we interact, we change, we write the book we go along. Throughout all of this, testing needs to remain viable, and it needs to add value. Remaining viable in this kind of environment can be hard.
Fortunately, we can learn to add value. In this keynote, Alan will describe some of the approaches and models he has used to help testing remain viable. Helping testers analyze the ‘system of development’ so the test approach can target process risks. Helping testers harness their own unique skills and approaches. The attitudes that the testing process often needs to have driving it, and the skill sets that teams need to ensure are applied to their testing.
At a simple level, this is just Systems Thinking and Modeling. In practice this can prove highly subversive and deliberately provocative. Because we’re not talking about ‘fitting in’, we’re talking about survival.
About
The keynote was underpinned by the notion that ‘Agile’ is not a ’thing’, instead ‘Agile’ provides the context within which our project operates and therefore is part of the weltanshauung of the project. Or as I referred to it, the “System Of Development”.
Because really I concentrate on ‘Systems’. I think that I view ‘context’, as an understanding of the System. And remember that, we, the tester, form part of that system and the context as well. Therefore our ‘beliefs’ about testing become important as the impact how we interact with the people, and the system, and the process, in place on the project.
As ever, I made a lot of notes before the Keynote, and I present those below.
A few things to note. Many of the keynotes overlapped: talking about role identification and beliefs, communicating from your own experience and models, building models of your process and testing, etc.
And prior to Agile Testing Days, Michael Bolton presented a Eurostar Webinar on ‘Agile’, which is worth watching, and James Bach presented “Skilled Testing and Agile Development” at Oredev 2014 which is worth watching. I include links to both of those for your delectation because they represent other testers applying Systems Thinking to create a model of ‘Testing in Agile’, watch them both.
Keynote Slides
I uploaded the slides to slideshare:
Watch the Keynote
Keynote Notes
Notes:
Warren Zevon, wrote “Ain’t that pretty at all”, in 1982
Warren Zevon wrote a song called “ain’t that pretty at all”
Warren Zevon was one of those singers who when he comes on, I pretty much have to listen, his voice and songs drag me in, rather than sitting as background music.
In this song, Mr Zevon describes a character who is pretty jaded.
Well, I’ve seen all there is to see
And I’ve heard all they have to say
I’ve done everything I wanted to do . . .
I’ve done that too
I know what that feels like. I’ve done management, performance testing, exploratory testing, agile testing, security testing, acceptance testing, UAT, automation, etc.
I’ve worked on Agile, waterfall, heavy weight documentation, lean, yada yada yada. None of it has ever fully worked or been perfect.
Feels like I’ve done everything. the danger is I become jaded or fixed in my ways.
People want Agile to be perfect.
And this Warren Zevon character And doesn’t like what he sees.
And Agile isn’t perfect.
Reality doesn’t seem to match up with his expectations, or desires, or wants.
And it ain’t that pretty at all
Ain’t that pretty at all
Agile can be messy. It doesn’t always match the books or talks. And when you are new to it sometimes you don’t like what you see.
I went to the grand canyon, took a bus couple of hours to get there. When I got out, everyone else seemed to see a wonder example of mother nature.
I saw some cliffs.
We may not have the strategies to cope
I was back on the bus in 10 minutes. I might be jaded but I can avoid the sunk cost fallacy.
But we might not have the strategies we need to deal with the situation.
So I’m going to hurl myself against the wall
Cause I’d rather feel bad than not feel anything at all
If you come from a waterfall background you might not know how to handle the interactions on an agile project. And if your prep was books and blogs, you might find they described an ideal where the strategies they used are not in place.
Some of our strategies for coping might be self destructive and we might not notice, and other people might not tell us. Because systems are self-healing and they can heal by excluding the toxic thing in the system.
Without the right strategy we make the wrong choice
And when you don’t have a lot of strategies you end up making choices and taking actions that aren’t necessarily the most appropriate for the situation.
Testers telling developers that the project would be better if they just did TDD and paired. Or if we all worked on automation acceptance tests together. Might not get the outcome that you want.
You might fall back on strategies that worked on other projects. But don’t fit in this one.
So I’m going to hurl myself against the wall
‘Cause I’d rather feel bad than not feel anything at all
You end up wanting to write lots of documentation up front because you’ve always done it, or you want to test in places where the stories don’t go, or you want to remind everyone that ‘Agile’ isn’t done like that.
Whatever…
So very often, my job….
I’ve been to Paris
And it ain’t that pretty at all
I’ve been to Rome
Guess what?
Is to help people, with their beliefs and expectations. To work with the people and system around them.
Because when I walked away from the Grand Canyon it wasn’t a reflection on the grand canyon, it was a reflection of me. My expectations were different from the reality. My belief in being wowed, because of all the things I’d heard about it stepped in the way of seeing what was on the ground in front of me. Or in the ground in front of me.
So they don’t do something stupid
I’d like to go back to Paris someday and visit the Louvre Museum
Get a good running start and hurl myself at the wall
Going to hurl myself against the wall
‘Cause I’d rather feel bad than feel nothing at all
And it ain’t that pretty at all
Ain’t that pretty at all
So they don’t do something stupid on the project that stops them maximising their effectiveness, and puts blockers in the way to working effectively with the team.
I help testers survive in Agile projects
That’s never my role title. Test Manager, Test Consultant, Automation Specialist. Agile Tester. Blah Blah Blah.
But I seem to help testers survive, and help testing survive. So that we don’t fall prey to just automating acceptance criteria, we actually test the product and explore its capabilities.
And I say ‘survive’, because that’s what I had to learn to do.
We survive when we add value
I think we survive when we add value, learn, and make ourselves a viable part of the project in ways that are unique to us.
In “The Princess Bride”, the hero Westley is caught by The Dread Pirate Roberts, whom he offers to be his valet for 5 years.
The Dread Pirate Roberts agrees to try it but says he will most probably kill him in the morning. He’s never had a valet before so doesn’t know how a valet would add value to him, or how he could use him.
And when the morning arrives and The Dread Pirate Roberts comes to kill poor Westley. Westley thanks him for the opportunity to have learned so much about the workings of a pirate ship the previous day.
Westley explains to the dread pirate roberts how much he learned about the workings of the ship, and how he had helped the cook by pairing with him, and how he had reorganised the items in the cargo hold.
And every day Westley survives his capture by the Dread Pirate Roberts by working and adding value during the day. And every day he demonstrates the value that he can add to the operation of the pirate ship. And every day he learns more about piracy and the skills of pirates.
Until, years later, The Dread Pirate Roberts explains that The Dread Pirate Roberts is a role, not a person, and so the dock into a port, take on a new crew and Westley adopts the role of Dread Pirate Roberts.
And testers need to act such that people don’t ask “What does testing do on Agile” because they know what their testers do on the project, and they see value in those activities. They know what specifically “Bob and Eris, or Dick and Jane or Janet and John” their testers, actually do.
This isn’t fluffy people stuff
When I look online and see the type of strategies that people describe for surviving and fitting in on Agile projects, it isn’t quite what I do or recommend, so I know there are alternative paths and routes in.
I’m not here to make you look good
I’m really not here to make you look good.
I mean I’m not here to make you look bad… but I might.
And I’m not here to make the product look bad… but I might.
And I’m not here to make the process, or the testing, or the ‘whatever’ look bad… but I might.
If it aint that pretty, then we can do something about it when we recognise that, and negative feedback can help.
I’m really here to help raise the bar and improve the work we all do together. If that makes you look good, then that’s great, if that makes you look bad and you improve as a result, then that’s great too. Looking good is a side-effect.
Survival does not mean ‘fitting in’
I’m not talking about fitting in. Buying Lunch. Making people look good. yada yada.
I don’t think you get bonus points for doing that.
But if that’s you, don’t stop. I think its great if people want to buy doughnuts for everyone.
Its not me. So there are other ways to survive on projects. I don’t have many strategies for ‘social success’
Personally I think team work means helping other people contribute the best stuff they can, and the stuff that only they can, and help pass that knowledge around the team. And everyone needs to to hat.
Testers survive by doing testing stuff
Its pretty much as complicated as that. Learn to test as much as you can, and drop all the bureaucracy and waste that adds no value. Then you’ve covered the basics.
Then you learn how to help others improve by passing on your knowledge and mindsets.
And we are taught how to do ’testing stuff’ so if we do that well and improve our testing skills then we have a good chance of survival.
Testers are taught to work with the System Under Development
Essentially we are taught how to understand and work with the System Under Development
We learn techniques to analyse the system and model it and then build questions from the model which we ask of the system.
We might build a data domain model, then build questions around its boundaries and then ask the system those questions by phrasing them in form the system understands.
We learn how to do that as testers with techniques and analysis and blah de blah.. testing stuff.
We survive when we adapt to the System Of Development
Testing survives when it learns to work with the Systems in place. And two obvious systems are the System under development, and the System of development. We have to adapt to both.
And fortunately testers are often good at analysing systems. Modeling them. Viewing them from different perspectives. Breaking them into chunks. Viewing the flow of data. working out what data becomes information to subsystems. Looking for data transformations. Looking for risk in process and communication. Then figuring out different injection points.
When we work with systems under development we don’t just input data at one end and see what happens out the other. Same with Systems of development, we don’t just work at the stories that come in, and then check the system that comes out. We learn to look for different injection points and create feedback earlier.
So one of the main things I help testers do, is analyse the System Of Development, and adapt to it.
Many of the testing survival tricks and techniques we learn relate to Waterfall projects.
Annie Edson Taylor, thought she would become rich and famous if she could survive a trip over the Niagra falls in a barrel.
She survived. She wasn’t rich and famous.
She survived by padding out her barrel and increasing the air pressure in the barrel.
You can get yourself ready before you hit Agile.
I survived waterfall by removing all padding, and taking responsibility for what I did.
I analysed the system of development and did what this specific implementation needed, so I could survive, and I took responsibility so that I could add value.
So I was in a better place than many people on waterfall projects when we started working on Agile.
Blake wrote “I must create a system…”
I must create a system or be enslaved by another man’s. My business is not to reason and compare, my business is to create.
This is one of the first texts I ever read on system design and modelling, and I read it when I was about 19 or 20, and stuff stays with you when you encounter it early.
But this is my meta model. of modelling. And I know I have to own and take responsibility for my view of the world and the systems that I work with. Otherwise I’ll fall prey to ‘other’ peoples’ strategies.
I remember the first time I worked on an ‘Agile’ project
I had to learn the strategies to survive. And that’s how I can recognise the same blockers in other testers or projects new to Agile.
I fell prey to the Agile Hype
I had a whole set of beliefs about what agile was going to be like:
- the pairing
- the TDD
- the ATDD
- the BDD
- the ADBDBTBD - I might have made that one up
But there was a lot of stuff.
And it ain’t that pretty at all
Reality wasn’t as pretty as I imagined.
I got stuck.
Remember I knew how to handle Waterfall. I had that down.
I could work the system of development and work around the blocks and annoyances that it threw at me.
But here was a new thing. A new system.
Stuck on…
I knew what we were ‘supposed’ to do. But people weren’t really doing that. and I didn’t know how to do this… hybrid thing.
I realised that people didn’t know what to do.
And I realised I simply didn’t have the basic skills to work in this new system of development.
I was the worst ‘pair’ in the world. You know when deer get caught in headlights. That was me the first time I was given the keyboard in a pairing session.
So I did what I always do…
Try to take over the world.
No - of course not. I’ve tried that before. Trying to impose your model on top of everyone else’s and making it work, is hard.
Its easier to treat the system as a unique system and model it. Observe it. Understand it.
Look at the parts, the subsystems, the relationships, the feedback flows, the amplifiers, the attenuators.
See I think this is what I do, I work with systems
That’s why my social strategies are… different.
I see the people systems in front of me. The processes. The politics. etc.
And that led to me changing… me
And much of what I did, I teach testers to do on projects.
I worked on my coding skills so I could pair
Every Agile Project is different.
So we learn to analyse the system that is in place. TheWeltanschauung. The context.
System’s have common concepts and elements, in general: entities, relationships, data flows, subsystems etc.
But each specific system, is different, and we can view it in different ways, with different models.
Tutorial: Technical Testing in an Agile Environment
I have listed the blurb for this tutorial below:
Let’s adopt the view that when we test on Agile projects we want to add value as often and quickly as possible.
On this tutorial we will use a mix of discussion, instruction, exercises and debriefs to explore what it means to work in an Agile Environment and how we can push our testing beyond user stories.
We will test an application together, and you will be challenged to add your unique value to the group. You will draw upon your experience as you test, share your thoughts and ideas, and learn nuances and new approaches from the group, and your own explorations.
Bring a laptop, or other electronic computing device which you think can use for testing, because you will be testing. Load it with whatever tools and resources you think you need. You can use them, and share them. We want to learn from each other and explore what it means to conduct Technical Testing in an Agile Environment.
Some stuff we will cover:
- How to use Systems Thinking to test technically and understand the “System under development”
- How to use System Thinking to understand the “System of development” and adjust your test approach accordingly
- What does “Technical Testing” mean?
- How far to push the boundaries of user stories
- How to improve your technical skills
- Technical Testing does not mean Automation
- What are your existing “technical skills” - you probably have more than you think.
- Risks and Issues with technical testing, and strategies to counter them
- How to document and describe your technical testing to others
Because it was a tutorial, I have not made the slides available.
Workshop: Black Ops Testing
Tony Bruce, Steve Green, and myself hosted a double track workshop under our Black Ops Testing banner. Where we used redmine as the target application.
These workshops are fun to do, basically we go off and test an application, then we use that as prep for the workshop, figure out what we want to draw people’s attention to, what we tested, how we made notes, what techniques and tools we used. Then we identify hints, tips and areas of people to target.
During the workshop, we have our plan for what we want to cover, but we also see what the people in the workshop do, we help ‘pull’ that out of the attendees in de-briefs so everyone can learn from each other. We also coach the attendees individually, or in groups, as they test.
The workshop feels very fluid and adhoc, but in reality, that’s because we have done enough prep to allow it to flow.
Regardless, the attendees, and we, enjoy it.
Workshop Slides
I uploaded the slides to slideshare:
Workshop Notes
At Agile Testing Days 2014, I presented a full day workshop on “Technical Testing” in Agile and was part of the Black Ops Testing Workshop with Steve Green and Tony Bruce.
In the tutorial I present examples of Technical Testing, and how to integrate Technical Testing into Agile, the participants also test a real live application and apply the techniques, mindsets and tools that I describe.
Since it describe Technical Testing in an Agile system, we also spent time discussing the injection points for the technical testing process and thought processes.
The Black Ops Testing workshop took a similar approach but with less ’talking’ since it was a much shorter time period with more people.
We started with a 5 minute lightning talk from myself, Tony, and Steve. During this we emphasized something important that we hoped the participants would focus on during their testing. We then let the participants loose on the system as we mingled. We coached, asked questions and observed. Then during the debrief we extemporize on our observations and ask questions about what we saw to pull out the insights from the participants. We repeat this process over the session.
Both the Black Ops Testing Workshop and my Tutorial used Redmine as the application under test.
We picked Redmine for a number of reasons, and I’ll list some below:
- Virtual Machines are available which make it easy to install
- The Virtual Machines can be deployed easily to Amazon cloud servers
- It is available as an Amazon Cloud Marketplace system, making it easy to install
There are some words in there you might notice “easy to install”, “deployed easily”.
Wouldn’t it be great if all the apps we had to test were easy to install and configure and gain access to?
Yes it would. And as testers, when we work on projects, we can stress this to the project team, or work on it ourselves so that we don’t spend a lot of time messing about with environments.
I used bitnami and their dashboard to automatically deploy and configure the environment. Tony used the amazon aws marketplace and worked with their dashboard. James Lyndsay helped us out, and went all old school, and deployed the base install to the machine.
I learned from this, that my exploratory note taking approach has permeated all my work. As I was installing the environment and configuring it, I made notes of ‘what’ I wanted to do, ‘where’ I was finding the information I needed, ‘how’ to do the steps I took, what data I was using (usernames, passwords, environment names, etc.). And when I needed to repeat the installation (I installed to a VM in Windows, on Mac, and in the cloud), I had all my notes from the previous installations.
When something went wrong in the environment, and I didn’t have access to the parts of the system I needed, I was able to look through my notes and I could see that there were activities I had not performed. I thought I had, but my notes don’t lie to me as much as my memory does.
It should come as no surprise to you then, that I stress note taking in my tutorial, and in the Black Ops Testing Workshop.
