March 2016 in London, at the SIGIST March 2016 Conference, I performed the closing keynote with a talk entitled"Push your technical testing further - into technology and security"
Slides
The slides have been released to slideshare:
The Blurb
The blurb read:
As testers we learn how to functionally test systems. We learn to analyse requirements and test ‘What’ a system should do. We can take our functional testing further. We can test ‘How’ the system does what it does, by understanding the technology used to build the system. We will find defects and issues that we would otherwise miss. Some of the defects would normally be associated with security testing, but we will find them without learning the techniques used for security testing. This approach to testing is applicable to any Software Development methodology and doable by any tester. Alan will explain the specific steps he used to learn to test web applications and push his functional testing further. He will provide examples of tools he uses, and why he uses those tools.
Alan also describes the thought process used to find the tools so that you can identify tools for your technology stack. After this talk you will know how to increase the potential that your testing can identify deep system issues, and steps you can immediately take which will push your functional testing further.
Three key points:
- Interact with the system at a deep technological level to find more bugs. Many classified as security bugs and missed by security testing approaches.
- Tools are necessary to observe and manipulate the system, learn about some important web testing tools and how to find new tools for your technology stack.
- These skills are open to anyone prepared to put in the work to learn. Specific steps and approaches are provided as examples for learning to test web systems.
Notes
During the talk the Sigist kindly provided two copies of “Java For Testers” that we gave away as prizes, and I brought along a unique ‘proof’ copy of “Dear Evil Tester”, which I also gave away as a prize.
In the talk I was basically providing some case studies of using technical knowledge and skills to inform your testing. And gave some examples of the overlap between this style and security testing.
I also mentioned the Usborne computer books from the 80’s, many of which Usborne have released as free pdfs. There are a few on the website that I do not own, so I will read those later.
These books were a major influence on my career. I learned to write adventure games using “Write your own Adventure Programs”
A book that I previously mentioned over on Selenium Simplified where I describe the relationship between Keyword Driven Test Frameworks and Text Adventure Verb Noun parsers.
I still use the lessons I learned in this book, to this day.
I later augmented this information with another ‘Dragon’ book:
Social Media
Lisa Crispin provided a valuable service and live tweeted a summary of the talk.
Thank you Lisa.
Check out @eviltester's work & techniques. #SIGiST pic.twitter.com/z4Rya2sBgb
— lisacrispin (@lisacrispin) March 15, 2016
Do the work! Research, experiment, learn, take small steps. Motivating talk from @eviltester at #sigist! Check out https://t.co/5VoFlcUAJH
— lisacrispin (@lisacrispin) March 15, 2016
Learn to use proxy tools & their features, learn browser dev tools, user profiles. How can each help you test? @eviltester #sigist
— lisacrispin (@lisacrispin) March 15, 2016
Some interesting challenges from @eviltester #sigist pic.twitter.com/Nb3Yscblpk
— lisacrispin (@lisacrispin) March 15, 2016
if you don't understand something, add it to your model and learn. @eviltester #sigist
— lisacrispin (@lisacrispin) March 15, 2016
Learn by modeling what you already know. HTML? HTTP? Browsers? Use browser dev tools. #sigist @eviltester
— lisacrispin (@lisacrispin) March 15, 2016
Make choices about what testing to do based on risk. Have to learn some technical things, training is available. @eviltester #sigist
— lisacrispin (@lisacrispin) March 15, 2016
'methodology' doesn't matter. Process/social context might matter. Any tester can use @eviltester's techniques, need tech skills/knowledge
— lisacrispin (@lisacrispin) March 15, 2016
You can use functional testing techniques to find security issues - use tools to observe at deeper levels. #sigist @eviltester
— lisacrispin (@lisacrispin) March 15, 2016
Observe & interrogate a web app - can use http proxies, dev tools. @eviltester #sigist pic.twitter.com/KiGAkFZ5j4
— lisacrispin (@lisacrispin) March 15, 2016
Does manipulating at lower level mean more risk? No test approach mitigates all risk, so we need multiple approaches. @eviltester #sigist
— lisacrispin (@lisacrispin) March 15, 2016
We may need technical knowledge to observe and interrogate and learn more. @eviltester shows java app tool example. #sigist
— lisacrispin (@lisacrispin) March 15, 2016
Tail log files and observe what is happening. Interrogate, manipulate, observe. #sigist @eviltester Tools impact ability to observe, test
— lisacrispin (@lisacrispin) March 15, 2016
#sigist @eviltester likes words w negative associations like 'interrogation' & 'manipulation' for testing activities. (I'm not sure why?)
— lisacrispin (@lisacrispin) March 15, 2016
Presupposition analysis - what has to be true for this to make sense? Assumptions that are made, we can use 4 testing. @eviltester #sigist
— lisacrispin (@lisacrispin) March 15, 2016
Precondition analysis - testers remove pre-conditions to see what happens. #sigist @eviltester
— lisacrispin (@lisacrispin) March 15, 2016
It's key to work with intent. Have a goal - what were you trying to achieve? @eviltester #sigist
— lisacrispin (@lisacrispin) March 15, 2016
Model different systems of testing - feedback (+ and -), learning as we test, visualize learning, test as questioning. @eviltester #sigist
— lisacrispin (@lisacrispin) March 15, 2016
Now learning from @eviltester at #sigist pic.twitter.com/a1SR6CBqz4
— lisacrispin (@lisacrispin) March 15, 2016
