National Software Testing Conference 2016

May 2016 in London, at the National Software Testing Conference 2016, I presented a talk entitled “The Art of Questioning to improve Testing, Agile, and Automating”

Slides

Details

With only 25 minutes to present, I had to make very sure I was conscious of the main points I was trying to drive home. This was always going to be a topic overview, but I wanted some very obvious and actionable insights for attendees. And hopefully spur them on to future research.

The presentation draws on lessons learned from various forms of fast, brief and systemic psychotherapy. With a few simple points:

Why? is a question that targets beliefs
How, What, Where, When, Who - all target structure and process
We all have models of the world and our questions reflect that model
Answers we give, reflect our model
Responses to answers give information on how well the models of the question asker, and answering person, match up
Testing can be modelled as a questioning process
Improving our ability to ask questions improves our ability to test, manage, and change behaviour.

You can read some early work I did in this area (2004) on my ‘NLP For Testers’’ papers.

Conference Notes

I spent a lot of time between talks speaking to people and networking, so I didn’t get a chance to see many talks. But those I did get to see I made some notes on that I will have to think about.

A few of the talks overlapped - particularly Paul Gerrard, Daniel Morris, and Geoff Thompson. At least, they overlapped for me.

Paul Gerrard from Gerrard Consulting provided a good overview of how important modeling is for effective testing, and he mentioned his ‘New Model of Software Testing’ to illustrate how the ‘checking’ or ‘asserting’ part of testing is a very small subset of what we do. Paul also described some work he is doing on building some tool support for supporting exploratory testing. I’m looking forward to seeing this when Paul releases it.

Daniel Morris overlapped with Paul when he was describing the various social networks and online shopping tools. Daniel was drawing attention to the multiple views that social networks and shopping sites provide. They have a rich underlying model of the products, and the customers, and the shopping patters, what people buy when they buy this, the navigation habits of the users. etc. All very much aligned to the content Paul described and the tool support that Paul was building up.

Both Daniel and Paul described some of the difficulties in visualising or collating the work of multiple testers, e.g. when testing how do you see what defects have already been raised in this area - if we were navigating a shopping site, we would see it on screen as a navigate, also ‘*****’ starred reviews of ‘how good is this section of the application’. I found value in this because I’m always trying to find out how to better visualise and explore the models I make of software and I found interesting parallels here, and obvious gaps in my current tool support.

Geoff Thomson from Experimentus described some ‘silent assassins’ for projects and stressed that companies and outsourced providers seem to be moving to a focus on ‘cost’ rather than ‘quality’. Geoff also provided different views of project progress and cost, again demonstrating that ‘model’ of the project can be represented in different ways.

I also saw David Rondell provide an overview of various technologies and the rate of change that testing has to deal with. Container based technologies and rapid environment configuration tools like Docker, Mesos, Ansible, Vagrant, Chef, etc. were mentioned in many of the talks. Very often we don’t have time at a management level to really dive deep into these technologies but it was good to see them being discussed at a management level. (There is a good list of associated technologies on the XebiaLabs website)

The Gala in the evening gave us a chance to network further and I received an excellent masterclass in Sales from Peter Shkurko from Parasoft, it is always good to augment book learning with experience from real practitioners. I asked Peter a lot of questions over dinner and Peter’s experience helped me expand my model of sales with tips I hadn’t picked up from any Sales book or training.

For any conference organisers - if you can get the vendors to present, not just, ’their tools’, but also their experience of ‘selling those tools’ e.g particularly in selling software testing, or selling software tools. I think participants would find that useful.

I tried to pay Peter, and the rest of the table back by contributing testing knowledge and experience in the Gala Quiz. (We got lucky because there was a 4 point value WebDriver question that we were able to ace.)

The result of our combined sales and testing knowledge meant that our table won the Gala Quiz and received ‘golden tickets’ which will grant us access to the European Software Testing Awards in November. Because sales, marketing training, development and testing can all work together.

Preparing the Talk

The Pitch

I don’t often respond to ‘calls for papers’. I’m often invited to speak. For this conference I was invited to speak. But I still go through a call for papers process. i.e. we have to negotiate what I’m going to talk about, and I have to convince the organiser that my talk is relevant to their attendees. I’ll describe that process in this section.

The Outline

Before being accepted to talk, I have to ‘pitch’ the talk. This is prior to knowing what I will talk about, so I generally create a break down of things I want to cover, then create a pitch from it.

My outline was ‘practical examples of asking questions to improve Agile processes and approaches to Automating.’:

Some examples of questions that testers ask. Some funny, some serious.
Questioning can lead to dependence, rather than responsibility
How to help practitioners use questions to take responsibility
Practical examples of how testers identify assumptions and presuppositions from application GUIs, and use those to create test ideas during exploratory testing
Understand that asking questions is essential for effective decision making
Understand that much of our decision making is based on belief rather than fact
Testing modelled as a ‘questioning’ activity
How management can ask questions that help practitioners improve, even when managers don’t fully understand the details of the task
A practical model of asking questions based on techniques from psychotherapy and transformational grammar
Questioning and Decision making is important for every member of the team
The types of decisions that are made in Agile projects and when Automating
The consequences of not asking questions about those decisions
Practical examples of the types and consequences in Agile and when Automating

In reality, this is the outline for a much larger talk. I didn’t realise at the time, that the NSTC talks are only 25 minutes long, 20 minutes if you leave time for questions.

So in the finished version a few things didn’t make it.

Practical examples of how testers identify assumptions and presuppositions from application GUIs, and use those to create test ideas during exploratory testing
Understand that asking questions is essential for effective decision making
Understand that much of our decision making is based on belief rather than fact
Questioning and Decision making is important for every member of the team
The types of decisions that are made in Agile projects and when Automating
Practical examples of the types and consequences in Agile and when Automating

Instead I concentrated on the general topic of questioning and how questions relate to models of the world. I didn’t spend as much time on decision making.

Pitch

The outline, becomes a pitch, to the conference organiser so that they know that thought has gone into the content, and structure - even if it was only 30 minutes - as the talk has yet to be written at this point.

I had in mind a talk covering Questions and Decision making - in general, in testing, in Agile and Automation. A more general title would probably be: “Questions and Decision making in Agile and Automation. The examples and content would span the following areas of interest mentioned in the call for papers:

A software testing case study/sharing good practices,
Changing trends,
Agile,
Test automation,
Continuous delivery,
BDD

Examples would be drawn from my experiences working in: Agile, BDD, Test Automation, Continuous Delivery.

Much of my work as a consultant involves quickly building a model of the team system, identifying risks and gaps, and identifying the most appropriate questions which help the team take responsibility for addressing the problems we expose in their answers.

The rest of my work often involves me getting hands on to help the team implement fixes, and ’lead by example’ the process of asking questions to continually improve and to responsibility for our decisions while implementing those decisions in code.

This talk will describe experiences of doing that, examples of the questions and decisions, and a model of how I do that. Rather than going into detail of specifics of approaches used when automating and showing code, which I don’t think is suitable for the target audience listed on the ‘Who Should Attend?’ page.

Since I haven’t written the talk yet, I’ll describe a rough outline of what I think I would cover.

Some examples of questions that testers ask. Some funny, some serious.
Questioning can lead to dependence, rather than responsibility
How to help practitioners use questions to take responsibility
Practical examples of how testers identify assumptions and presuppositions from application GUIs, and use those to create test ideas during exploratory testing
Understand that asking questions is essential for effective decision making
Understand that much of our decision making is based on belief rather than fact
Testing modelled as a ‘questioning’ activity
How management can ask questions that help practitioners improve, even when managers don’t fully understand the details of the task
A practical model of asking questions based on techniques from psychotherapy and transformational grammar
Questioning and Decision making is important for every member of the team
The types of decisions that are made in Agile projects and when Automating
The consequences of not asking questions about those decisions
Practical examples of the types and consequences in Agile and when Automating

The blurb was as follows:

Asking questions is essential for effective decision making. Unfortunately, when we don’t ask the right question, we often make decisions based on belief rather than fact. Questions run through our entire software development process. Testing can be modelled as a ‘questioning’ activity where testers identify assumptions and presuppositions in a system, and use those to create test ideas which ask questions of the system as exploratory testing. Automating can be modelled as a ‘questioning’ activity where we codify questions and expected answers. In this talk Alan will provide practical examples of using questions to improve Agile processes and approaches to Automating and testing. Alan will also provide a model of questioning. A model that management can apply to help practitioners improve, even when they don’t fully understand the technical details of the task. Attend this session to learn how effective questioning helps you take responsibility for improving your process.

A few people wrote some kind words about the presentation on twitter that I spotted:

@eviltester the best talk at this year's national conference by far. Thank you for the insights.
— Oliver R Smith (@OliverRSmithUK) May 18, 2016

@ROQTALK - Great talk on questionning in testing by @eviltester - provoke a response #SoftwareTestingConference
— David Moore (@ROQIT_DM) May 17, 2016

@eviltester it was brilliant having you there!
— TEST Magazine (@testmagazine) May 19, 2016

I have even been featured in the adverts for the National Software Testing Conference. Modeling at its best. Which catalogue wants to contact me first?