TLDR; Identify Oracles, automate observation of changes, understand GUI/Mobile differences, harness tool support.
At the moment Nintendo have initiated a voluntary viral distributed denial of service attack which hits retailers on demand.
Every time a retailer announces pre-order availability of a SNES Mini Classic they are hit by 1000s (100,000s?) of people simultaneously trying to buy the product.
Thus far this week I have seen it destroy the Game web site entirely, and on Argos I received the Red Circle of Ajax Doom when trying to add the SNES Classic Mini Console to my basket.
But I was ready for this.
And you can be too.
In this post I’ll describe how to use your testing skills to pre-order a SNES Classic Mini, or at the very least - how to monitor web sites automatically, and why API, mobile and GUI apps can differ.
Failure Driven
I failed to managed to pre-order earlier in the week from Game, Amazon, Nintendo because:
- I always received notification of the pre-order availability too late.
Either I arrived when it was sold out, or when the site was crashing and burning under the load.
Put your Tester hat on now. What do you need to do?
- Find a way of receiving announcements instantly?
- But from whom? Each announcement comes from a different site.
- Find an oracle that collates all the announcements.
- Get notification from that oracle.
- But from whom? Each announcement comes from a different site.
I found the TechRadar site has a page announcing the pre-orders.
I have an oracle, but no notifications.
Automated Observation
I could write a script to monitor the website and look for changes, or I could add a tool to support my observation.
Since I only really want notifications when I’m at my computer and able to take advantage of the notification, I looked for a browser plugin that would monitor web sites.
I installed Page Monitor, a handy chrome plugin that will periodically monitor websites for changes.
And set it up to automatically observe Tech Radar and notify me of page changes.
I also set it up to monitor: Amazon, Game and Nintendo, just in case more pre-orders become available.
Avoiding False Positives
And the first thing that happened was… I received notifications within minutes… because Tech Radar has adverts and other stuff on the page that change.
I needed my alerts to be more specific.
Fortunately Page Monitor allows you to restrict the monitoring to specific parts of the page using the advanced custom mode.
And using my Super Advanced Technical Testing abilities, I looked at the DOM and found the ID for the HTML segment I wanted to monitor, then wrote the CSS selector for that section #article-body
Yup, I didn’t really need super advanced technical testing abilities to figure that out. Basic HTML skills and the ability to use Browser Developer tools would have sufficed.
Woo Hoo
Yesterday my setup worked. I received very quick notification for a site that had listed them but I hadn’t used the retailer before, they looked like they were offering credit and the price was more than anyone else.
So I stoppedthe purchase, but I was fairly sure my notification process was working.
Today
Today Argos decided to voluntarily step up to the Distributed Denial Of Service Attack and listed the console for pre-order on their website.
I duly added the console to my basket and received the mighty AJAX Circle of Doom.
A div showing an image that pretends something is happening, but it isn’t.
Do I give up?
No, for I have Testing Skills.
Observe More Deeply
Opening the network tab in the Development tools shows me that an XHR request is made when I try to add it to the basket, so I look to see if any information is returned.
/*
{
"errorCode": "CMN0420E",
"errorMessage": "The following command exception has occurred
during processing: \"java.lang.NullPointerException\".",
"errorMessageKey": "_ERR_COMMAND_EXCEPTION",
"errorMessageParam": ["java.lang.NullPointerException"],
"correctiveActionMessage": "",
"correlationIdentifier": "-634834g2b:62cj5372441:-57c2",
"exceptionData": null,
"exceptionType": "1",
"originatingCommand": "ArgosProductPurchaseJSON",
"systemMessage": "The following command exception has
occurred during processing: \"java.lang.NullPointerException\"."
}
*/
Yeah, that doesn’t look good. I guess the Argos backend is written using Java - but why send back a commented out error message? Very odd.
I’m pretty sure the API isn’t meant to return that.
It then seems to go into an infinite series of callback requests which fail and fire another one.
Cross Browser Issue Ruled Out
This also means that I don’t try cross browser.
Very often when something fails, particulary with an AJAX or JavaScript trigger, we have to test it cross browser because retail sites are notorious (at least in our house) for failing on Chrome, but working in Internet Explorer.
I’ve decided to discount this possibility after looking at the network traffic. If the XHR had timed out, or some other error reported then I might have tried to purchase it on different browsers.
I Must Give Up Now
Surely I must give up now.
No.
For looking at the URL used for the AJAX request I see it says /webapp/.
Perhaps there are other URLs for API and Mobile App? And perhaps they are on servers which have not been subjected to the same level of traffic?
We often see this, where the Web GUI has a set of backend server calls, which are different from the mobile application server calls, and which are different from the API server calls.
Often these backends are on different servers because they have different load requirements e.g. the Web site is hit more often so needs to scale more quickly than the mobile app. *cough* take note Argos
When testing an application, this increases the scope of testing and makes our life more difficult. But knowing that this is how applications are often written and deployed gives us additional workarounds when interacting with the real world.
Install the Mobile App
I downloaded the Argos mobile app on to my phone and tried to order from there.
And the transaction went through smoothly.
Mostly smoothly
I had one issue where I entered my postcode and allowed the app to prefill my address, which it did correctly, but then the app told me that the town it had entered was incorrect so I had to amend it. But other than that functional defect the process went smoothly and I received a confirmation email suggesting that my pre-order has been successfull.
And, as I write this, over an hour after ordering on the mobile app, the web site still provides its users with an AJAX Circle of Doom and no human readable error message that the server communication failed.
Risks
As a tester I’m used to looking for risks, even after ‘success’.
- Pre-orders are notorious for being cancelled. I do have confirmation email but - pah, it isn’t worth the paper it isn’t printed on.
- Credit card payments for pre-orders can fail at the point of delivery resulting in a cancelled pre-order.
I’m taking the risk that by giving you my secrets, such that you too may experience pre-order joy, that if my pre-order fails there are no more consoles left to pre-order because of your success.
But, see - testing skills can help you in the real world after all.
