I have always been fascinated with Confidence Men, Grifters, Tricksters. And all of that spills over into this Black Friday Special of The Evil Tester Show.

In this Black Friday Special of The Evil Tester Show Podcast we consider lessons we can learn from discount sales, retail under pressure, and confidence tricksters and expert provocateurs.

We discuss:

  • The Black Friday Software Testing Game
  • Oracles
  • Observation
  • Evidence
  • Belief
  • UK Sale of Goods Act - what is a sale?
  • What is an expert?
  • How to communicate like an expert
  • Automating through the GUI
  • Options for Automating
  • Testing for Confidence
  • Communicating Confidence
  • You can be confident that something is wrong
  • Confidence doesn’t mean Goodness

Black Friday

  • the Black Friday Testing game
    • extension
      • who’s observing whom?
        • find a product ona site - check if amazon have price matched - often without it being marked as black friday
      • are they really deals?
        • do you know how to check?
          • camelcamelcamel for amazon
          • possibly way back machine
  • lessons
    • urgency of communication
      • this option expires - countdown timer
      • encourages, not looking for evidence
    • perceived good value
      • trust involved, without evidence

UK Law sale of goods act “to claim that products are on sale, you should show the previous price and should have been selling at that price for a meaningful period of time”

Expert opinions

  • definition of an expert?
  • someone who’s voice is likely to be heard
  • statements are often made with confidence

Expert opinions on GUI Test Automation

I saw online recently were around Automating applications and particularly automating through the GUI

  • its hard
  • problem with automating through the GUI is not that you are automating through the GUI
    • if that was the problem the solution would be to stop automating through the GUI
  • but expert opinion in terms of tweets, supporting tweets, likes, replies and comments were “stop automating through the GUI”
  • expert opinions spoken with confidence
  • sense of humour - assume they didn’t mean it - and were playing devils advocate
    • if you stopped, you’d have to find a different way to automate
    • examples: APIs, app as api, Unit tests
    • automating at different places mitigates different risks
  • if the app under test has been architected such that the functionality is actually in the GUI and only makes sens in that live integrated environment then you’ve built something that is harder to automated in a different way and your problems might well be better solved there, than stopping automating.
  • but its up to you and how you interpret the words of experts

Expert opinions on confidence

  • paraphrase “testing is to boost the confidence in the business that the software can go live”
  • confidence goes both ways - confident that this works, confident that this does not work
    • one has evidence, one has probability and belief
  • QA - is the process of building confidence. Testing is part of the process of attempting to shake our confidence and test our belief
    • measure twice, cut once - but we still test the strength of the stitching, try it on, look at it in a mirror - we still test it
    • the interpretation of test results might boost confidence, its not a guarantee and I don’t think its an aim
      • unless we find a problem and then bug advocacy is about increasing confidence that something is wrong

Bug Advocacy: The art of communicating to build confidence that problems exist

Confidence

  • con tricks
    • perceived value
    • urgency
    • hard to evaluate their truth
    • con man is an expert - tailors their message for the mark
  • expert opinions often don’t see through
    • scientists taken in by magicians posing as fake psychics

You can be fooled by your own experimental protocols

Confidence doesn’t mean truth.

You can be confident that something is false.

You can state beliefs confidently.

Testing - confidence from evidence and understanding the scope of what has been investigated