TLDR; Don’t rule out headless browsers like HTMLUnit, just because no-one uses them in the real world. They might make automating your task easier and faster.
Have you ever been told that HTMLUnit and PhantomJS (and insert any other headless browser in here) “are not real browsers, customers don’t use them, so we shouldn’t use them for testing”.
I don’t agree with that statement because I think the statement makes the overly broad assumption that “when we automate the browser to trigger functionality, we want to ’test that it works in the browser'”
Not Always Customer Focus
When we automate as part of our test process, we don’t have to do everything as though a customer did it.
Sometimes:
- I don’t automate to ’test that it works in a browser'.
- I automate a task, not testing.
- I don’t want to ’test’ with the browser, I want to send a message, to the backend, that a browser would send.
- I want to check that the server responds appropriately to a browser generated message.
- I want to test the backend, not the frontend/backend integration in a specific browser.
- I use the browser because I want to trigger a ‘form’ submission, and automating the form submission without a browser requires that I treat the “App as API” and that requires more work to mitigate the risk that: the ‘form’ I submit artificially matches the ‘form’ submitted by a browser.
What Technology Level are we Testing?
The appropriateness of the automated manipulation approach depends on what part of the system we want to test.
When I want to check that Firefox submits the form correctly to the backend, and the backed processes it correctly and returns a message to Firefox, then I automate Firefox.
But when I want to check that the backed processes a submitted form correctly and returns a message to the sender, then I can automate using a headless browser, if that makes the job easier and faster.
I don’t always automate for ’testing’
Sometimes I don’t automate to ’test’. Sometimes I automate tasks.
Tasks that I would normally do through the GUI, but I only want to ‘use’, not ’test’, the GUI :
- setup data
- bulk verify data in the system, prior to testing
- etc.
When might I choose a headless browser?
Why might we choose a headless browser?
If some of the following evaluated to true and you viewed them as important:
- faster to start
- more reliable over repeated runs
- easier to setup
- easier to run as part of the CI process than a real browser
- fewer side-effects to tidy up in the event of failure e.g. browser windows left open
- the functionality uses such basic HTML that it doesn’t need a ‘full’ browser
- run task in background without a browser popping up
- etc.
…then a headless browser might meet your needs.
But I could do ’this’ instead
Good.
You don’t have to use a headless browser to achieve these benefits:
- you could use the API if it exists and does what you need
- you could use an HTTP library if it doesn’t introduce more risk and more work
- you might be able to chain cURL or Wget (or other) commands in a shell script
- you might identify other approaches
Do consider other options.
Do not assume that you always have to use a ‘real’ browser.
Summary:
You will have good reasons to run specific scenarios through a ‘real’ browser, but don’t rule out headless browsers for all tasks.
- Adopt the belief that ‘headless browsers’ fit some circumstances well.
- We automate ‘web’ interfaces to test the back end server processing as well as the front end and specific browser front end interaction with the backend.
- Use tools that support the most appropriate manipulation interface for the functionality under test.
- You may not need to use a ‘real browser’, but could use a Headless browser with the ‘real browser automating’ tools that you already know how to use
