TLDR; Condemn? No. Support? I do like parts of the paper. Censor? No.
And now I’m doing something I don’t like: writing a blog post in response to ‘something on the internet’. I’m writing a blog post which I don’t think has any practical value. I warn you now. I don’t think you will find much humour herein either. I have added a comedy punchline at the bottom though, if you want to skip ahead.
Chris McMahon has written an open letter to named reviewers of James Bach and Michael Bolton’s paper on “Context Driven Approach to Automation in Testing”. I reviewed the paper.
In the Open Letter blog post Chris McMahon asks the reviewers three things:
- condemn publicly “both the tone and the substance of that paper”
- If you do support the paper, I ask you to do so publicly.
- “regardless of your view, I request that you ask the authors” … “to remove that paper from public view” because it “is an impediment to reasonable discussion and it has no place in the modern discourse about test automation”
I find a few things personally difficult here:
- I don’t feel strongly about the paper. There are parts I like, parts I agree with, parts I disagree with, parts I don’t like as much. Much like any paper.
- I don’t like censorship
I’m more annoyed by the “censor this” call to action, than the paper and the surrounding twitter storm.
Normally I ignore these public ‘add your voice to my voice’ calls for action. But since I was named, and since I was offended by the request to back a call for censorship I wrote this post.
I was surprised that other people reading and commenting on the post didn’t view Chris’s post as a call for censorship. I guess people have different definitions of censorship and free speech.
Some people view censorship as something that only governments or controlling bodies can do.
I don’t:
Censorship: the action of preventing part or the whole of a book, film, work of art, document, or other kind of communication from being seen or made available to the public, because it is considered to be offensive or harmful… dictionary.cambridge.org/dictionary/english/censorship
Background
Chris McMahon has written a review of the paper on his blog . He doesn’t like it.
In the paper James and Michael provide 3 case studies. 1 and 2 show some exploratory testing with tools being used.
Case 3 is an example of how not to automate something. I viewed case study 3 as an investigation approach.
Chris didn’t like it. I think everyone can (and should) write ‘case studies’ about what they tried, what worked, and what failed.
And…
If I were trying to automate the GUI of the application that James and Michael chose to automate. I would probably have started with AutoIt or AutoHotKey. Chris disagrees and seems to dislike AutoHotKey based on what he read about it online. I have used both AutoIt and AutoHotKey successfully in the past. I have even written applications in AutoIt, and I wrote a Java Library around AutoIt DLLs.
Chris McMahon writes that “C# and VisualBasic.NET are really the only reasonable choices for such a project”, so Chris may be reviewing the paper from the point of view of a strategic and long term modelling exercise rather than a tactical testing exercise.
I may well have tried AutoIt or AutoHotKey first because I was creating a short term hack to get something done. It is possible to approach automating initially as a tactical exercise, we don’t need to jump straight into a strategic solution.
If I were writing code to automate it longer term and more strategically I would have investigated Appium Desktop drivers winappdriver or winium possibly White or Sikuli. I don’t know what other options there are now.
I’d have to investigate libraries available and what programming languages they work best with, before jumping straight into C# or VisualBasic.NET as Chris suggets. Plus, I don’t really know C# or VisualBasic.net so I’d probably look at JavaScript, Python of Java options because that would fit with my skillset and knowledge. And if I were automating strategically I’d probably be working in a team so I’d have to consider the skillsets and knowledge of the team, the long term maintenance people were prepared to put up with and the development language being used by the team.
As James and Michael say in their paper automating “requires learning not only about the application and the tool, but also about how they will interact”. I have not had much success automating QT apps in the past, I try to find alternative approaches.
My “Open Letter Answer”
I wrote a blog comment, at least twice, but the blog comments were deleted when added, I was informed this was due to a bug in the Blogger software platform. I had to write a blog post in order to be able to respond.
My open letter answer to Chris below:
“I ask you to join me in condemning publicly both the tone and the substance of that paper.”
No.
“Condemn - to express complete disapproval of.”
I don’t think so.
James and Michael asked me to review the paper. I read it. I wrote comments. I sent the comments back to James and Michael. Some of my comments they took on board some of them they chose not to (or the paper does not exhibit all the type of changes I described). That is their right. They wrote the paper. It is theirs.
I’ll point out some features of the paper that I agree with. The simple fact that I agree with some of it, means I won’t condemn it:
- “There are many wonderful ways tools can be used to help software testing.”
- Tools applied incorrectly add waste
- “Let’s break down testing further…” section
- “Let your context drive your tooling”.
- Case 1 and Case 2 are better than I remember them
Edit: I wrote more words here, but had to edit them out to fit the max comment length of this blog. Mainly I’m editing out longer examples of things I thought were good, and some I thought were bad, but it doesn’t actually matter what I thought. People can read it and make up their own mind.
All of the above was written to say:
- I won’t condemn it, because there are parts I agree with
- The parts I disagree with, I can ignore, and I don’t have to condemn
- I gave comments to James and Michael, they may have reworked some of the text because of it, some of it they may not have. Its their paper, that’s their choice.
“If you do support the paper, I ask you to do so publicly.”
I think there are useful parts in this paper. I think some parts are less useful.
There are parts I liked. Some parts I didn’t.
There are parts I agree with. Some parts I didn’t.
Dear blog reader: As with everything you read. Read it yourself and make up your own mind. My views of this paper are not important. Other peoples’ views of this paper are not important. Only your views of the paper are important. If you feel you are not qualified to ‘pass judgement’ on the paper. Then don’t pass judgement. Read it for value.
And regardless of your view, I request that you ask the authors of the paper bearing your names to remove that paper from public view as well as to remove the copy that Keith Klain hosts here. For the reasons I pointed out, this paper is an impediment to reasonable discussion and it has no place in the modern discourse about test automation.
No. Absolutely not.
I will not knowingly engage in, or condone, censorship.
No.
The fact that I was asked this. Is the reason I responded at length.
“this paper is an impediment to reasonable discussion”.
Hardly.
You pointed out what you thought was wrong with the paper. I see nothing in your comments that demonstrate “this paper is an impediment to reasonable discussion”. On the contrary, having pointed out the flaws, you can then go on to say what you think people can do instead. That is a prompt to reasonable discussion, not an impediment.
“it has no place in the modern discourse about test automation”.
People are entitled to describe their experiences of automating software. Even if they describe situations that they failed in the process. I might prefer to see some different conclusions drawn, or different generalisations made. But its not my paper.
Strong Feelings
Clearly Chris feels strongly about the paper.
When I encounter ’things on the internet’ that I feel strongly about I try to create, or link to material, that offers alternative choices and demonstrates alternative views, which might open up options for myself or my readers.
I don’t think I ‘attack’ as much as I used to. And I think I’ve taken down a lot of the old ‘book reviews’ I wrote which did that, since I don’t think they added value.
If I read a criticism, I like to read what the author thinks I should do instead. Then I can see balance.
People Read Differently
People will read the paper differently.
There were parts of the paper I found value in:
e.g.
- “There are many wonderful ways tools can be used to help software testing.”
- Tools applied incorrectly add waste
- “Let’s break down testing further…” section
- “Let your context drive your tooling”.
- Case 1 and Case 2 are better than I remember them
And there are more than that, but you can read the paper for value and see what you take from it.
Some people read with pre-existing view of the authors either positive or negative and their view of the material can sometimes reflect that.
Choose your own reading approach prior to reading.
I read to pull out value for myself to move forward.
Related Reading
I have some papers and conference talks related to this topic:
The Paper
James and Michael have released the paper. For free. For anyone to read.
They continue to edit it, to clarify their position in the paper.
You could read it. You might disagree with it. You might find value in it. You might want to send comments to James and Michael to clarify sections. You might want to write a critique yourself, you might come out in favour with the paper, you might not.
Your choice. Your reaction. Your learning.
If you want to build a ‘movement’ against it to prevent it “from being seen or made available to the public, because it is considered to be offensive or harmful” then that’s your choice. Leave me out of it. Free speech is important.
When we discover something ‘bad’ on the internet, it can be helpful to create something that offers an alternative to guide people towards something that we think is more positive rather than trying to rally people to tear it down. For instance, create your own case study or write your own paper that offers a useful starting point for other people.
