Automating Calculator From Console

I was asked how I might go about automating an online calculator application. I have a calculator app of my own so I was interested in comparing the two apps and experimenting with a canvas based JS App.

I was asked how I might go about automating an online calculator application.

e.g.

https://www.online-calculator.com/

Specifically - how would I use despatch event to send mouse clicks to the application to automate it?

How Would I Automate It?

Video Overview

Watch on YouTube

First Things First

The first thing I do is look at the risks, the possibilities and the options.

To find options, I interact with the application to see how it works:

use mouse to click on the buttons
use keyboard to issue commands

What are the possiblities?

the app is canvas based, which limits many possibilities i.e. finding DOM elements and interacting with them
I might be able to send keyboard events
I might be able to send mouse events
I might be able to interact with the application code directly

What are the risks?

the ads might interfere, they change the DOM when they switch
the screensize would make canvas automating more difficult because I would have to calculate relative sizes of the screen.
the main app is running in a frame

Experiment - iframe

I decide to try the “Use the online calculator full screen” link to see if that takes it out of the iframe.

It doesn’t so I open the link which is the url of the iframe to make it easier to review and work with.

/html5/online-calculator/index.php?v=10

Experiment - DOM breakpoint

I want to check some of my possibilities, and they all revolve around… how does the code work.

So I have to check that out.

The first thing I do is right click on the canvas element in the DOM view and add a breakpoint.

break on attribute modification

Then, when I interact with it, I will see what code is triggered.

And when I hover over the canvas I see that I’m put into some library code for createjs

http://createjs.com/

That isn’t the level I want to work at so I delete the breakpoint and review the code directly. This is easier now that I’m no longer working with it in a frame.

Experiment - read source in the HTML file

Looking at the index.php I can see that there are some high level JavaScript functions enabled and a top level object main

So I could probably work with the JavaScript directly.

The main object has the three main methods I can see bound in the code for handling keyboard events.

So keyboard events looks like the easier way to work with this app.

I need the ability to create keyboard events.

https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent

Experiment - keyboard event

I know that I can use the number keys to create numbers in the calculator so I’ll try and trigger a number keyboard event.

I’m going to use one of the console built in methods called monitorEvents to get me started.

monitorEvents(window, 'keydown')

Now, when I press keys in the window I can see the generated events.

I could just replay those if I wanted.

Right click on one of the events and store as global variable.

It is stored as temp1

Then I can ‘replay’ that event

iframeKeboardEvent(temp1);
iframeKeboardEventDown(temp1);
iframeKeboardEventPress(temp1);

These are the three functions listed in the html that take events.

The 3rd one - iframeKeboardEventPress triggered a ‘5’ in the calculator.

So my first attempts at automating would be around keypress events.

But, what I want to do now, is trim the event down to the bare minimum.

unmonitorEvents(window, 'keydown')

I remove the monitor to clear up my environment.

By looking through the documnetation for keyboard events

I can see the basic structure for a keyboard event:

new KeyboardEvent("keypress",{key: "5"})

So I try that:

iframeKeboardEventPress(new KeyboardEvent("keypress",{key: "5"}))

And that doesn’t work.

I know that there are a bunch of attributes which can be examined by the code to identify a key

key
code
keyCode
charCode
which

So I Ihave a quick look in the code to see what it is using.

This isn’t the html file now, this is the js file. main.js

I search for .code etc. and then see that .which is being used.

which using the keycode number.

So I can find that quickly at keycode.info

https://keycode.info/

“5” is code 53

So I try that.

iframeKeboardEventPress(
  new KeyboardEvent("keypress",
     {which: 53}))

But I’m on Chrome, and forgot that on Chrome I need to use keyCode, which will be interpreted as which

iframeKeboardEventPress(
  new KeyboardEvent(
     "keypress",{keyCode: 53}))

To be safe, I’m going to use both:

iframeKeboardEventPress(
  new KeyboardEvent(
     "keypress",{keyCode: 53, which: 53}))

But now I have the ability to trigger a ‘5’ into the calculator.

What Next - raw events

At the moment I’m using the application against itself, by passing the event into a method.

But I might want to use the events directly. so I try that.

document.getElementById("canvas").
dispatchEvent(new KeyboardEvent(
   "keypress",
   {which: 53, keyCode: 53, bubbles:true}
))

I always put bubbles:true in there because I am never sure if I’m dispatching the event to the correct level.

So now I have code to build on that can trigger events to trigger functionality in the calculator.

What Next - model an engine

Now I would have to model the application i.e. what keys do I want to use, and how do I want to represent that in code?

I’ll cheat for the moment and assume that most of the calculator functions are based on character codes, and JavaScript has a charCodeAt function for strings.

"1".charCodeAt(0)

Above gives me 49 which is the character code for the character at position 0 in the string “1”, i.e. the code for ‘1’.

So I prototype a quick automation engine

"12+48/10=".split('').forEach(
    function(item, index){
      console.log(item.charCodeAt(0))
    }
)

This prints out:

Which are the character codes for each of the numbers and symbols in the string.

What if? Instead of logging them, I despatch them?

"12+48/10=".split('').forEach(
  function(item, index){
   var keycode = item.charCodeAt(0);
    document.getElementById("canvas").dispatchEvent(
       new KeyboardEvent(
         "keypress",{which: keycode, keyCode: keycode, bubbles:true}
       )
    )
  }
)

take a string “12+48/10=”
split it into the individual characters
for each character
use charCodeAt to convert it into the ascii keycode value
dispatch that as an event to the calculator

Which gives me the answer 16.8

To be honest, I wasn’t expecting that, I was expecting ‘6’.

But if I type those keys in in myself.

I get the same answer.

To get ‘6’ as I was expecting I have to write “12+48=/10=”

This is as far as I went with the automating. I’m not sure what the mappings for the other button functions are.

If you want a slightly different calculator to experiment on, then I have one that I created here:

https://testpages.herokuapp.com/styled/apps/calculator.html

It is DOM based rather than canvas based, and might be easier to automate from JavaScript.

Or feel free to try the online-calculator app, they offer different challenges and automation approaches.

What About Checking Results?

We can automate the application.
We can automate the application in support of our testing e.g. don’t have to type the values.
We can’t automate the application to assert on the results.

Why can’t we check the results.

The results are rendered on the canvas, we would have to ‘scrape the image’ and OCR the data in order to automate result checking.

The application only exposes 3 methods, none of which allow us access to the internal variables.

The application would have to be amended to support assertion checking during automating.

Ideally I would like an internal object to be public e.g. main.state.getDisplay(), main.state.getMemoryContents() etc.

Could we do that ourselves?

Yes. Not reliably from the console, but we could amend the code of the application and use a proxy to supply the amended file with more automation hooks.

And sometimes on projects, testers do amend the application to make it more automatable, and sometimes they ‘hack’ the application to make it more automatable.

Hopefully, most teams would work together to create an application that supports assertion checking.

Automating vs Testing vs Asserting

Is this application automatable? Yes. I can automate it.
Is this application testable? Yes. I can test it.

What I can’t do, is automate it for the purpose of asserting on conditions that I want to repeatedly check are true.

This is an example of an application which we can ‘automate’, and we can automate to support our testing, but can’t incorporate the automating into our assertion strategy.

Any automating we perform would likely be tactical, until we viewed it as important enough to make changes to the application to allow automated assertion to form part of our test strategy.

This video and text was released to patreon supporters on 14th August: