Tabara de Testare 2015

I attended the Tabara de Testare testing group on 1st April 2015 to present an introduction to Technical Testing. A little different from a normal conference talk in that the venue was spread over 4 Romanian cities, and I was in the UK.

“Most inspiring webinar I’ve ever seen! I really enjoyed and highly recommended Alan’s online study material way before this, but the webinar was simply amazing! Thanks a lot for the opportunity!”

- Mihai S.

The Content

The talk/webinar was intended to provide an overview of my views of Technical Testing.

I tried to explain the models I use, the sources that helped me create those models, and how it ultimately impacts my test approach in a way that I call ‘Technical Testing’.

The Tech

I presented virtually over Google Hangouts. The Tabara De Testare testing group is spread over four cities in Romania, each of which live streamed the webinar to a room filled with their members. I could see the room via the presentation machines web cam.

We also had 70+ people watching from the comfort of their own homes and workplaces.

Each of the test group locales was represented on the hangout as a participant and a webcam on their end showed the room I was presenting too, and hooked up to a projector, allowed the people present to view the presentation.

This also meant that we were able to take live Q&A by unmuting the locale with the question, and chatting over Hangouts to Q&A.

This worked well, and allowed 70+ people to view from other locations as well as the four main venues for the evening.

Hangouts allowed me to screenshare and demo some technical testing. But I do not recommend demonstrating an http debug proxy while presenting a webinar as some of my browser plugins crashed as soon as I proxified my connection.

All in all, a useful experiment in remote live presenting and a good way to reach testing groups that I would otherwise never be able to attend.

The Slides

During the webinar I ran through the slides, then provided a short demo of Browser Dev tools supporting technical testing investigations on the redmine.org demo application.

Dangerously, I then tried to demo proxy tools to help answer a question from the audience.

Clearly - using a proxy tool, while conducting a live webinar through a browser isn’t the least dangerous option. And lo, I lost the Q&A chat window as a result. But I think we covered most of the questions during the live Q&A which followed.

The Preparatory Notes

My initial notes are presented below.

Intro: What do I mean by Technical Testing?

“going beyond requirements and using Technical Information about the implementation and an understanding of the technologies used in the building of the system to add to the risk profile and use to help derive test approaches”

• using Web Testing as an example
• how approaching testing from a technical perspective changes how you view the system and how you test
• how a technical understanding leads to a different use of tooling and automation

Quick Surface Structure Examples:

• Non-Technical Testing of a Web App
  • tester -> browser
• Technical Testing of a Web App
  • tester -> browser -> proxy -> server -> DB

What do I mean by Testing?

• All the traditional stuff we learned
  • e.g. Testing Techniques, Testing Object Oriented Systems
• All the Lessons we’ve learned over the years
  • e.g. Testing Computer Software, Lessons Learned in software testing
• Over the years we build up our own models of what we do
  • Heuristics, Rapid Software Testing
• My Model - MORIM
  • Modelling - Rethinking Systems Analysis & Design, Domain Driven Design, Diagnosing the System For Organisations
  • Observation
  • Reflection
  • Interrogation
  • Manipulation
• For ‘Testing’ I took a lot of this from Psychoanalysis
  • Modelling, you build a model of the client
  • Observation, what language patterns do people use? Structure of Magic
  • Interrogation, what questions should we ask? Provocative Therapy
  • Reflection, how we build a better model, figure out what questions to ask next
  • Manipulation, if you want to effect change, you have to manipulate - Hypnosis

What do I mean by Technical Testing?

• A lot of ’technical’ stuff in the previous section: Hypnosis, testing techniques and approaches, psychological models, linguistics, etc.
• But now I mean ’technical’ in the sense of Technology
• How you test a command line app differs from how you test a web app because the technology is different.
  • There are different risks
  • You have a different interface
  • You have different scope of inputs
• You could have a command line app, and a web app, with essentially the same requirements (they’d be high level, but could read the same)
• You could use the requirements to build much the same test plan (you’d be wrong if you did this, but you could)
• But the technology is different, you will want to test it differently.
• Some things I want to understand from a ’technical perspective’
  • what language is it written in?
  • what tools did the developers use?
  • what approaches did the system of development use? TDD. BDD, JUnit, etc.
  • what libraries is it using?
  • what is the architecture?
  • a command line tool which connects to a server offers different risks than a tool which works with flat files or one which connects to a database
• The more I understand, the richer my model, the more variety I an introduce into my testing.

MORIM Technical Testing

• Modelling - the richer we model the more options we have

  • coverage - is about models

• Observe

  • the more I understand about the technology the more I can figure out what and how to observe
    • e.g. web app, it isn’t just some magic in a browser
  • what does a browser render?
    • HTML, CSS, JavaScript
    • all of this as a DOM
    • This isn’t static, it can change
    • How can I observe that?
      • view source
      • dev tools
      • proxy to see the files as they are loaded, prior to representation in the DOM
      • I can use the debug console to see variables and elements etc.
  • A Web app isn’t just in the browser, there is traffic
    • HTTP - POST, GET
    • Ajax - XMLHttpRequest - XML, JSON
    • Headers, Cookies
    • How can I observe that?
      • dev tools
      • proxies
  • A Web app also exists on the server
    • HTTP Server
    • Edge Servers
    • App server
    • How are requests routed internally?
    • How can I observe that?
      • log files
      • process monitoring
        • But where is the data persisted?
        • Database?
        • Flat Files?
        • How can I observe that?

• Reflection

  • The more I understand the technology the more I research, and the more questions I have
    • Is it load balanced? How? How can I observe that? What risks are there now?
    • Questions guide my intent

• Interrogation

  • How can I deep dive into the system and its components, and not just observe it at a surface level
  • Dev tools sometimes lose the traffic data when you navigate between pages, proxies don’t
  • I can store proxy traffic for later viewing and analysis
  • etc.

• Manipulation

  • I know the system has all these different parts
  • How can I manipulate them - add new data into the database? Change the HTTP traffic
  • Why would I want to? Bypass the JavaScript validation on the front end and send requests to the backend
    • we could switch javascript off, but that might mean the full processing on the front end is not performed and a ‘different’ message might be sent, we might not send XMLHttpRequests, etc.
    • I need the ability to maintain the application context as much as possible but still manipulate it as much as I need to

And we rinse and repeat

But note that:

• I don’t start with ‘what tools can I use?’

I proceed from model and understanding through:

• Observe, Interrogate and Manipulate

Then I find tools to help me.

• I try and find multiple tools to help me.
  • Proxies:
    • BurpSuite,
    • ZAP,
    • Fiddler

Why?

Because tools are ’technology’ they have features and functionality that others might not, when I understand this then new approaches for manipulation might open up e.g. Fuzzing - repeating requests while varying data.

Resources List:

• Testing:
  • Testing Techniques (Beizer),
  • Testing Object Oriented Systems (Binder)
  • Testing Computer Software,
  • Lessons Learned in software testing
  • Heuristics lists - http://www.qualityperspectives.ca/resources_mnemonics.html
  • Rapid Software Testing by James Bach & Michael Bolton http://www.satisfice.com/info_rst.shtml
  • Principles of Experimentation And Measurement - Gordon M. Bragg
• Modelling
  • Rethinking Systems Analysis & Design by Gerald M. Weinberg
  • Domain Driven Design by Eric Evans
  • Diagnosing the System For Organisations by Stafford Beer
  • Software Requirements & Specifications by Michael Jackson
• Observation
  • The Structure of Magic Volumes 1 & 2 by Richard Bandler and John Grinder
• Interrogation
  • The Web Application Hackers Handbook
  • Provocative Therapy by Frank Farrelly
  • NLP For Testers, by Alan Richardson https://www.compendiumdev.co.uk/page.php?title=nlp
• Reflection
  • Quantum Psychology by Robert Anton Wilson
• Manipulation
  • How to Break Software, How to Break Web Software, How to Break Security by Whittaker (and others)
  • Patterns of the Hypnotic Techniques of Milton Erickson by Richard Bandler & John Grinder
  • Changing with Families by Bandler, Grinder & Virginia Satir