Some API Testing Basic Introductory Notes and Tools

Some applications provide an API. Some websites provide an API. This post provides some information on API testing, since that appears to have consume a lot of my time in January 2015.

As preparation for our Black Ops Testing Workshop I performed a lot of API testing. And co-incidentally the January Weekend Testing session chose API testing as its topic. There should be enough links in this blog to provide you with the tools I use to test APIs.

API - Application Programmer’s Interface

The name suggests something that only programmer’s might use. And indeed an API makes life easier for software to interact with other software.

Really an API provides one more way of interacting with software:

By sending messages in an agreed format, to an agreed interface and receiving. an agreed response format back.

APIs tend to change less frequently, or in a more controlled fashion, than GUIs because when an API changes, all consumers of that API have to change as well.

Software tends not to have the requisite variety that a human user exhibits:

If you change the GUI then a human can probably figure out where you moved the button, or what new fields you added to the form that they need to type in.
Software won’t do that. Software will likely break, or fail to send the new information and so the interaction will break.

If you read this blog through an RSS reader then the RSS reader has used this blog’s API. The API consists of a GET request on a URL to receive a response in XML format (an RSS feed).

You, as a user could GET the same URL and read the XML in the browser, but the API tends not to offer the same user experience, so we don’t often do that. Or we use tools, like an RSS Reader, to help us.

Manually Testing an API

Just because the API calls itself a Programmer’s Interface, does not mean that all our interaction with the API has to involve programming.

We can issue requests to an HTTP API with minimal tooling:

Use a browser to issue GET requests on an HTTP API
Use an HTTP Proxy to issue HTTP requests to an HTTP API
Use command line tools such as cURL or WGet to issue HTTP requests to an HTTP API
Use specific tools e.g. Postman to issue HTTP requests to an HTTP API

Preparation for Black Ops Testing

When choosing software for Black Ops Testing and Training workshops, I like software that has multiple methods of interaction e.g. GUI, API, Mobile Apps/Sites

This way testing can:

compare GUI against API, as well as underlying database
use the API to load data to support manual testing
check GUI actions by interrogating and manipulating the system through the API
test directly through the API

Prior to the Black Ops Testing workshop I had tested a lot of APIs, and I generally did the following:

Create confirmatory automation to check the API against its documentation.
Manually testing the API using HTTP Proxy tools
- to create tools and observe responses
- edit/replay messages and observe responses
- use the fuzzing tools on proxies to create messages with predefined data payloads
Use the browser for simple querying of the API and Firebug, with FirePath to help me analyse the responses

During the run up to the Black Ops Testing workshop I was re-reading my Moshe Feldenkrais books, and a few quotes stood out for me, the following being the first:

“…learning means having at least another way of doing the same thing.”

I decided to increase the variety of responses available to me when testing an API and learn more ways of sending messages to an API and viewing the responses.

So I learned cURL to:

send messages from the command line,
feed them through a proxy,
change the headers
create some data driven messages from the command line with data sets in a file

I used 3 different proxies to experiment with their features of fuzzing, message construction and message viewing.

I experimented with different REST client tools, and settled on Postman.

I now had multiple ways of doing the same thing so that when I encountered an issue with Postman, I could try and replicate using cURL or Proxies and see if my problem was with the application or my use of Postman.

similarly with any of the tools, I could use one or other of the tools to isolate my problem to the app or my use of that specific tool
this helped me isolate a problem with my automation, which I initially thought was application related

Weekend Testing

During the Weekend Testing session, we were pointed at the songkick.com API

I wanted some additional tool assistance to help me analyse the output from the API.

Because while Postman does a very capable job of pretty printing the XML and JSON, I needed a way to reduce the data in the message to something I could read more easily.

So instead of viewing the full XML tree. I used codebeautify.org/Xpath-Tester to create simplified XPath queries which rendered a subset of the data, i.e. if I wanted to read all the displayNames for events. I could click on the events in the tree and find the displayName attribute, or I could use XPath to show me only the displayNames for events

//event/@displayName

Looking back over my Feldenkrais notes, I can see a relevant quote for this:

“… he said something about learning the thing I already know in a different way in order to have free choice. For that I must be able to tell differences. And the differences must be significant. But I can distinguish smaller differences, not by increasing the stimulus, but by reducing the effort. To do this I must improve my organization.”

Summary

My approach to API testing has changed. Because I spent the time increasing the variety of responses I have to the task of testing an API.

I didn’t really mention automation in the above, although I use RestAssured for much of my API automation at the moment.

The above is a subset of what I have learned about API testing.

I plan to continue to increase the variety of responses I have to testing APIs and increase my experience of testing APIs. I will over time collate this information into other posts and longer material so, do let me know in the comments if there are any questions or topics you would like to see covered in this blog.