I wrote an article for the Testing Circus April 2016 issue. With the controversial and thought provoking title of “Join the Anti-“Test Automation” Brigade”. It has substance and is not just click bait.
Testing Circus Magazine april 2016
Testing Circus Magazine asked me to write an article. And my response was ‘Join the Anti-“Test Automation” Brigade’. This article describes a technique that I have tried to develop and refine for the last year or so and has paid big dividends when I now explain and work on projects that automate as part of their test process.
The title had a click-bait-esque ring to it ‘Join the Anti-“Test Automation” Brigade’.
The title, however, misleads. The article really does describe one of the most powerful strategies I know for improving the incorporation of automated execution into your Test Strategy.
You can read the article, starting on page 6 of the April 2016 edition.
Join The Anti-“Test Automation” Brigade
While it might seem like a click bait title. I really do have content. You’ll have to read the article to see, but here are some snippets.
“Test Automation” doesn’t mean anything. Or rather, it means too much. It could mean many things. And as a result, doesn’t really mean anything.
“Test” and “Automation”. Two words that are endemic in the Software Testing world. And what do they mean?
“Test” is too ambiguous. I try to limit my usage of the word.
Is “Automation” a verb or a noun? Is it a verb? “I will automation”, “I automation”. Not really, but we can make it part of a verb phrase - we tend to say “I will do automation”, “I am doing automation”. We do use it to imply action. We do use it as a verb.
Dictionaries typically define “Automation” as a noun. “Run the automation”, “Fix the automation”, “How much automation has passed”, etc.
My problem with “automation” started when I realised that no-one knew what I meant when I said the word.
And now… to find out the solution to the problem. Turn to page 6 in your copy of Testing Circus Magazine April 2016.
Article Text
Full text added below just in case the pdf download from Testing Circus becomes unavailable.
In this article I’m going to show you a very quick way to fix all your “Test Automation” problems. This will be especially useful to you if you have ever asked either of the following questions:
“What test automation should we do?” and “How much test automation should we have?”.
Both of these are questions that I am asked at conferences, when I’m onsite consulting with clients and via email. They seem to be common questions in the testing industry. My answer to both questions has changed over the years.
I used to find these questions hard to answer because I wanted my answers to fit the process and approach of the person asking the question. Inevitably I would have to ask them follow on questions like: “Are you using an Agile process?”, “What technology are you working with?”, “Do you have programming skills?”.
But I don’t need to ask those clarifying questions any more. My answers are simple. And, I’ve reached the point now, where my answers are the same:
“None. You can’t.”
Over the past few years my attitude to “Test Automation” has changed. It has now changed such that I am vehemently anti-“Test Automation”.
Immediately, some people reading this will be nodding their heads in agreement. At this point it would be presumptuous for you to do so because I have not explained why I now take this stance, or what I meant. Those of you nodding may have interpreted my statement as:
- we can not automate testing
- testers should not program
Or possibly some other interpretation. In fact I deliberately led you to interpret it in ‘some other’ way, because I want to make the point that one reason I am vehemently anti-“Test Automation” arises from the ambiguity present in the phrase.
“Test Automation” doesn’t mean anything. Or rather, it means too much. It could mean many things. And as a result, doesn’t really mean anything.
“But Alan you’re so involved in Test Automation!”, some of you might be saying if you are aware of my work, because I have books and online training courses relating to: Java, Selenium WebDriver, Tools. I also have blogs and web sites dedicated to those topics. I will hastily explain. I am not vehemently anti:
- using tools as you test
- automating the execution of applications and asserting on state conditions
- programming as part of your test approach
- ‘specialising’ in more technical aspects of the testing process
- and other technical activities
I don’t like the words.
I so, don’t like the words, that for the last year, and possibly longer, I’ve been training myself not to say them, not to write them, and not to use them.
I think they encourage lazy thinking, lead to poor communication, and create unnecessary arguments between people.
What led to this point of view?
Thanks for asking. I shall tell you what has led to my changing my views.
I used to use the phrase “Test Automation”. I happily described much of my work as “Test Automation”. It can probably still be found in old blog posts and articles because I haven’t edited my history to remove it. I am self editing going forwards.
Start by examining the words themselves. “Test” and “Automation”. Two words that are endemic in the Software Testing world. And what do they mean?
“Test” is a verb. “I will Test”, “I Test”. A verb that has multiple meanings and we rightly argue about its meaning in the test community since it seems that it could describe the essence of what we are all about.
“Test” is also a noun. “I will review this Test”, “I have written a Test”, “I will execute this Test”. When people say these statements, I don’t know what they mean, because “Test” is ambiguous. I’m sure there are standards with definitions of the term. I’m also sure that when people use the word “Test” they don’t always mean those definitions, and I know that I didn’t when I used the word.
“Test” is too ambiguous. I try to limit my usage of the word.
Is “Automation” a verb or a noun? Is it a verb? “I will automation”, “I automation”. Not really, but we can make it part of a verb phrase - we tend to say “I will do automation”, “I am doing automation”. We do use it to imply action. We do use it as a verb.
Dictionaries typically define “Automation” as a noun. “Run the automation”, “Fix the automation”, “How much automation has passed”, etc. “Automation” as a noun is problematic because it leads to counting (“we need more automation”) rather than understanding. Hopefully our aim when we automate is not to increase the amount of things we have automated, in order to count them.
Hopefully, when we automate, our aims is to make our test approach better, possibly by:
- removing manual effort that is better delegated to a tool,
- observing at lower levels of the system,
- executing a pre-defined path with more data combinations, and faster, than a human could do etc.
My problem with “automation” started when I realised that no-one knew what I meant when I said the word.
And my problem was resolved when I realised that “Automation” was a very new word in the English language.
“Automation” was coined in the 1950’s, it is jointly credited to John Diebold, and Delmar S. Harder. Delmar S. Harder, a vice president of Ford Motor Company is reputed to have said “we need more automation” as he toured one of the Ford factories and examined the way cars were produced. Because he wanted to increase the speed of production and reduce the variation in the production process in order to compete more effectively in the car manufacturing business.
John Diebold was writing a book on the increasing use of tooling and process automating in the manufacturing industry and its impact on society in general. But he found the word ‘automatization’ hard to spell, so he used the word ‘automation’ and it became popular as a result. We had not automated the spell checking process in the 1950’s.
“Automation” is a new word.
There are other words we could use:
- automatization - the process of making something more automated (a word which no longer appears in some automated spell checking dictionaries)
- automaton - a machine which performs functions according to coded instructions
- automating - to convert to automatic operation
- automated - having converted something to operate automatically
- etc.
These words are less ambiguous.
They can become ambiguous if we then couple them with “Test”.
“Test Automating” suggests a process, but I don’t know what part of your test process you are automating, so it remains ambiguous. And doesn’t really help us think about the intent behind our time spent automating.
Some people describe themselves as “Test Automators”. I have no idea what they do. I imagine that they automate tests, but since I don’t know how they have defined “test”, I have no idea what they do. As a role description it is not one that I see value in. I don’t know how they do it. Are they a tool expert? Do they use a specific tool? Can they program? I don’t know.
I describe my ability to use code to automate applications as programming. I can program in a number of different languages - Java, Ruby, .Net C#, JavaScript; although I am most proficient in Java. For example, I use several libraries that can automate web applications, and HTTP based APIs. I can describe my skills in words that are easy to understand and ask simple questions about.
We don’t need to use the word “Automation” and we don’t need to create “Test Autosomething” phrases.
If I revisit the earlier questions in light of this approach:
- “What test automation should we do?” and
- “How much test automation should we have?”
“What should we automate?” I don’t know. Where do you perceive risks in your current test approach?
If you perceive risk because you do not have time in the release process to revisit a lot of the application and check that basic functionality still works, then you might want to try and automate some of that work.
If you perceive a risk that you don’t have time to explore the system thoroughly because you spend a lot of time maintaining the environment, uploading data, clearing down data, installing software etc. Then perhaps you might want to automate some of that work.
“What should we automate?” I don’t know. What skills do you have?
If you have no ability to program, and don’t have anyone on you team who can program then you are going to have to rely on existing tools. Either commercial or open source. But your ‘what should you automate’ will be dictated to you by the functionality of the tools that you can use. If your team can program, then your team can write its own tools, and write them to augment your test approach and help you investigate risk.
“How much should we automate?” How much ‘what’? Now you have to be specific about what you want to automate. Do you want to increase the coverage of paths through the system? Do you want to increase the amount of randomised data that you use? You have to decide ‘what’ before you ask ‘how much’.
I appreciate that suggesting that you stop using the word “Automation” is a step too far for some people.
That’s fine. Its a step I have taken, and it has worked wonders for me. If we have a conversation and you use the word “automation” then remember that I won’t know what you mean, and I’ll probably have to ask you to explain what you mean, and you’ll probably have to use different words in your explanation anyway.
“Automation” is so ingrained in the language and psyche of the testing world that it will be hard to remove. But that doesn’t mean we should not remove it.
Ultimately we want test processes that are effective, tailored for our environments, and which we have taken responsibility for. We should feel that our testing process can contain and use any combination of: tools, programming languages, technologies, libraries, frameworks, techniques, documentation styles, skills, etc. Any combination that works for us.
Avoid using the phrase “Test Automation” and more specifically the word “Automation”. This will solve your “Test Automation” problems because you will never have any “Test Automation” problems again. You will still have problems. Very specific problems with tools and technology. You will still have to decide which parts of your process you automate, and where automating will add value.
You will will have to think about your approach carefully. But I guarantee that by avoiding the phrase “Test Automation” you will use words that help you think more carefully. People will still understand you. In fact they many understand you better.
Lead by example. Join the anti-“Test Automation” brigade, and automate as much of your working practice as you need to.
