I attended the TestWorks Conference in Amsterdam on the 2nd of October 2015 to present a Keynote on “Real World Automation Challenges”. Based on my experience writing code to automate applications, and from working with clients world wide, I described challenges faced and the thinking behind the solutions chosen.
So if you want more options for how you approach automating executing then this might be the talk for you.
Slides
Blurb
The theme of the conference was Hands-on experience with different test automation tools. So I drew from my experience to consider different approaches I’ve adopted when automating, considering them as solutions to challenges.
The initial blurb I submitted read as follows:
Sometimes automating an application seems like a search for workarounds. At times, the application appears to conspire against us, effortlessly defeating our approaches. And other times we need to work around limitations in the tool that we have chosen. Alan will present challenges experienced when automating, complete with explanations of the solutions adopted and reasons for choosing them. Including: how to choose a tool and a programming language. How to prioritise and add value while automating quickly, without introducing future maintenance issues. Also, using abstraction layers to make code readable and avoid intermittent execution issues. All of this from hard won hands on experience of automating in the real world. After this presentation you will have more options to use when you face similar challenges in your own work.
I planned to look through my experience for the most important lessons I’ve learned and present them as solutions to the challenges we faced, with an description of the reasons why we used them.
The blurb eventually became:
“Automating application execution isn’t easy, even with a lot of experience. Applications find ways of making themselves hard to automate. And if it isn’t the application, then it’s the tools we use. And if it isn’t the tools, then it’s the people and the processes. Automating can seem like a hunt for workarounds which sometimes fly in the face of ‘best practice’ advice. Alan will describe solutions he’s used on real world challenges. And he offers to help with real life dilemmas including: How do you build abstraction layers that work? If BDD tools aren’t test tools why do we use them test? How technical do you need testers to become? And how do you find time to test when you are automating so much?”
About the Talk
This was a follow on from the Test Automation Day conference because the organisers of TestWorksConf liked the content of the Test Automation Day conference.
I don’t like doing the same talks again, but the organisers liked some of the concepts so I repeated some of those concepts in here. But I reduced the overlap and did not cover all the history of automation stuff again.
The Test Automation Day conference was aimed at managers. The TestWorksConf talk was aimed at practitioners. Which meant I could show some code and dropped down to consider the thought processes required for effective automated execution.
And I do think practitioners need to have an understanding of the history of what we do. Then we don’t have to feel that we are always breaking new ground and that we have to learn all the lessons from scratch.
So I wanted to identify some of the high level concepts that allow practitioners to create great automated solutions, under pressure, and still have the ability to ’test’ systems, not just automate them.
About The Conference
The conference (#testworksconf) was probably the most ‘hands on’ conference I have ever attended.
Unfortunately there was too much on for me to go to every talk. A very interesting conference, very hands on, with lots to learn.
Hopefully I’ll be able to attend on future years as well.
Because of the information provided at the conference, I have had my first exploration of Gatling. I was also able to spend some time with Erik Swets and he helped me use Gatling.
Machiel van der Bijl pointed me towards a lot of Model Based Testing resources that I need to find time to investigate in more detail. During the conference Machiel provided a quick overview of Model Based Testing and demonstrated the tool that Axini have been working on.
Ivan Shubin provided a really good interactive demo of his Galen Framework visual testing tool. Galen has now bumped its way a little higher up my investigation todo list and I think is worth checking out.
Maarten van den Ende and Adé Mochtar provided an overview of Selenium Grid in containers using the 2.47.1 Docker images and expanding it further with Mesos.
John Smart provided the closing keynote with a demo of his Serenity tool, and more interesting the ‘Journey Pattern’ Actor approaches he uses to cut down on the amount of code in page objects and other abstraction layers. It was different enough from the type of approaches that I covered in my Abstractions talk that I need to look through John’s example code in more detail. Fortunately, John also used the TodoMVC app as his example target.
Also at the conference; an overview of updates to FitNesse, an intro to the Robot Framework and Cucumber.JS/Angular and Protractor. And a very interesting introduction to the Mox Angular mocking library.
After the conference I was able to spend some time with the Xebia team for knowledge sharing.
#TestAutomation is fun! @eviltester @viktorclerc @Xebia pic.twitter.com/MzrfU1Uwm6
— Herald Biewenga (@Herald4) October 6, 2015
Wow, the day with @eviltester was very cool. Inspired, and learned new stuff. Played with Gatling for the first time 😮
— Maaike Brinkhof (@Maaikees) October 6, 2015
What did Twitter say?
We were fortunate that this was a very social media active crowd, so there were quite a few quotes and reactions from the audience.
@testworksconf @eviltester is on stage pic.twitter.com/DNFIH7uBAx
— Herald Biewenga (@Herald4) October 2, 2015Survival is a way of thinking #automation @eviltester #testworksconf pic.twitter.com/EH7HXFczcF
— cirilo wortel (@sietstweets) October 2, 2015@eviltester shows that Testing is a team effort. #testworksconf pic.twitter.com/Ls9nMYKgYT
— Erik Zeedijk (@ErikZeedijk) October 2, 2015How the genius #HenryFord coined the term #automation in 1948, told by @eviltester. #testworksconf pic.twitter.com/IJtr09SLOu
— Jordann Gross (@JordannGross) October 2, 2015Automate or Die! Real world automation survival with @eviltester #TestworksConf pic.twitter.com/7bd7LfhXwX
— TestWorks (@testworksconf) October 2, 2015The house is pakked #pakhuisdezwijger #testworksconf @eviltester pic.twitter.com/VQScN4wIyn
— Viktor Clerc (@viktorclerc) October 2, 2015The first speaker started! @eviltester #TestworksConf pic.twitter.com/q3yzzKZRJ9
— Xebia Nederland BV (@Xebia) October 2, 2015Now guru @eviltester about how manual testing is a form of slavery 🙈👍🏼😈, why we should automate everything pic.twitter.com/rJXVNNAT6g
— Caleb Repkes (@calebrepkes) October 2, 2015#xebia team are listening with full attention to @eviltester #TestworksConf pic.twitter.com/kDkesczZLp
— Herald Biewenga (@Herald4) October 2, 2015Real world automation survival @eviltester at #testworksconf pic.twitter.com/9bTTMvfPPG
— Erol (@erolstt) October 2, 2015Thanks to @eviltester for his interesting and inspiring talk! @testworksconf #testworksconf @viktorclerc pic.twitter.com/im7R3Xfol1
— Xebia Nederland BV (@Xebia) October 2, 2015Systems, abstractions, requisite variety helps you survive in a world of automate or die #testworksconf @eviltester
— Herald Biewenga (@Herald4) October 2, 2015The solution to problems when automating is rarely more automation. @eviltester #testconfworks #verytrue
— Tom Langerhorst (@tomdetester) October 2, 2015Testing.., observe and manipulate the system @eviltester #testworksconf
— cirilo wortel (@sietstweets) October 2, 2015How technical do testers need to be? Enough to observe deeper levels of the system. - @eviltester @testworksconf
— Marjana Shammi (@mshammi) October 2, 2015#selenium was the solution for giant space aliens, back in '60 @eviltester #testworksconf
— Herald Biewenga (@Herald4) October 2, 2015Be very specific at using the word #TestAutomation or #Automation, but tell what to do! I agree With @EvilTester at #TestWorksConf
— Marcel Hogenhout (@marcelhogenhout) October 2, 2015Be specific what exactly you are automating, when you talk about test automation, says @eviltester in his talk #testworksconf
— Maaike Brinkhof (@Maaikees) October 2, 2015@mshammi @eviltester @testworksconf @workroomprds That's because it doesn't exist.
— QualityExcellentness (@kinofrost) October 2, 2015"We should learn to build our application in such a way that we can easily automate the tests of them." Quote @eviltester #TestWorksConf
— Caleb Repkes (@calebrepkes) October 2, 2015Don't compete with slave labour #automation #testworksconf by @eviltester
— cirilo wortel (@sietstweets) October 2, 2015In the fifties they nailed the problems that come with automation. Yet we still struggle with them as if they're brand new. #testworksconf
— Maaike Brinkhof (@Maaikees) October 2, 2015
