Test Driven JavaScript using QUnit

TLDR: A 2008 case study in using QUnit.

TDD JavaScript Using QUnit 2008 Style

Introduction

I have a little project that I started writing in OpenLaszlo. I chose OpenLaszlo because I thought I could script it in standard JavaScript but during the development of my minor app I found a few errors with the OpenLaszlo JavaScript implementation and the project stalled. Now, having started a new job, where the development team use splendid new technologies like QUnit, and JQuery, I resurrected my app.

So I present, to you, some notes on my experience of Test Driven JavaScript using QUnit.

I class myself as a real novice with JavaScript.

Fortunately the development team gave me a quick demo of QUnit at work so I understood a little about the technology, and no better teacher than experience can help me understand, so time to dabble.

As a novice, I followed the instructions on the QUnit site and created a basic test page (called test_eprime.htm).

<html>
<head>
<script src="jquery-latest.js"></script>
<link media="screen" href="testsuite.css" type="text/css" rel="stylesheet">
<script src="testrunner.js"></script>

<script>
  
  $(document).ready(function(){
    
});

 </script>

<h1>E-Prime JavaScript Tests</h1>
<h2 id="banner"></h2>
<h2 id="userAgent"></h2>
<ol id="tests"></ol>
<div id="main"></div>
</body>
</html>

And when viewed in a browser this gives me the joyous news that I have 0 tests of 0 failed.

Woo Hoo!

In order to support this my code folder has:

jquery-latest.js
- testrunner.js
  - testsuite.css
test_eprime.htm (the code above)

So now I’ll create my first test. I do this by adding all my tests between the magic lines in the above test frame work html

<script>
  
$(document).ready(function(){

// **** I add all my code in here ****
    
});

</script>

QUnit has the concept of modules which ‘chunk’ tests together. So I create an html formatting module, for all my HTML formatting tests.

module("HTML Formatting");

And then I’ll write my first test…

test("html char conversion", function(){
    equals("&gt;",convertToHTMLIfRequired('>'));
});

I added a new ’test’ to the set of tests. The test had the name “html char conversion”. I created the code of the test as an unnamed function. The purpose of the test? To check the function convertToHTMLIfRequired function returns > when called with the parameter >

So now I save my test_eprime.htm file and reload it into FireFox.

And voila…

…my test failed. Hopefully as you expected, after all I didn’t write any code.

So off I go to my trusty text editor and create a new file called eprime.js which looks like this

 function convertToHTMLIfRequired(c){
   return '&gt;';
 }

Run the tests again by refreshing the page in the browser. And…

Wha! Oh, I forgot to include eprime.js into my test_eprime.htm, OK, easy fix.

...
 <script src="testrunner.js"></script>
 <script src="eprime.js"></script>
...

Refresh to run the tests.

_**

Alright! High Five! Watch as I do my “successful test passing” dance.

And so we move on.

Back into the tests and make that a little more robust…

test("html char conversion", function(){
    equals("&gt;",convertToHTMLIfRequired('>'));
    equals("&lt;",convertToHTMLIfRequired('<'));
});

Refresh the browser and my tests fail. As we would hope.

Eagle eyed readers will note that I got the ’expected’ and ‘actual’ parameters the wrong way around. I should have used the format…

equals(<actual>,<expected>);

I could blame my error on the QUnit front page which shows the following example code…

equals( "hello", value, "We expect value to be hello" );

But I’d feel petty and, since I didn’t read the manual, I take the blame.

So I go away and add more code, like a good TDD practitioner.

function convertToHTMLIfRequired(c){
  if(c=='<')
   return '&lt;';

  if(c=='>')
   return '&gt;';
 }

Refresh the browser…

Woo hoo! Cue “Successful test passing” dance… and so on, and so on…

I then follow this with more TDD iterations, which worked out rather well. But so as not to bore you, I shall conclude.

Initial Thoughts

The eagle eyed among you will have said “Why does it keep saying 0 tests of 0 failed, even when 1 failed?” and the answer I respond with reads “I don’t know, I haven’t read the manual yet”.

And despite the “0 tests of 0 failed”, I like:

this unit test harness,
- how easy I found writing JavaScript using TDD with QUnit,
  - TDD JavaScript,
the fact that my IDE consists of a free text editor notepad++, and a free Browser,
- the fact that I haven’t had to read the manual yet, and have managed to make a lot of progress,
  - that I can keep running my tests easily,
by loading my test page into different browers I can check for browser compatibility

Adding Code Coverage with JS Coverage

Continuing on our adventures in TDD JavaScript land and we reach for the code coverage tool. JSCoverage was the first hit for JavaScript code coverage, which means it must ‘be’ good, right? Let’s find out how well it plays with QUnit.

You can download JSCoverage as a precompiled Linux or Windows build.

So I downloaded the pre-built Windows build.

And upon opening the zip we get a jscoverage.exe and a whole bunch of documentation.

In the grand tradition associated with the previous TDD JavaScript blog post, I won’t read the documentation I just downloaded. Instead I’ll have a quick scan of the online manual.

JSCoverage takes two arguments, a source directory and a destination directory. Anything in the source directory gets instrumented. Since I don’t want to instrument the QUnit and JQuery libraries I …

move those into a folder structure of their own,
and put my code in its own folder structure,
then I run jscoverage eprimecode c_eprimecode

Err…OK, now I have a new folder called c_eprimecode… I better read the instructions…

Oh, OK, so in c_eprimecode I now have a file called jscoverage.html, and I can put the path of test_eprime.htm and when I click GO it will load that page, thereby running all the tests against the instrumented version of eprime.js

easy…

And… it worked!

100% coverage. You can see I’ve added a few more tests and a few more lines of code since you last saw eprime.js and I felt pleased that my TDD efforts had led to 100% coverage.

I can click on the eprime.js link and see the code coverage metrics.

All covered lines have a green count next to the line number, and the number in the green box shows how many times that line ran during the testing. If we did not cover a line then we see a read box with the ominous 0 in it. So scanning through the source for the big bad red becomes very easy.

And that only took about 5 minutes. What can I do now? Ah, Inverted mode.

If I add a line into my test_eprime.htm like the following

<button onclick='window.open("path/to/jscoverage.html");'>Coverage report</button>

Then I can run my test_eprime.htm like normal, but click to the coverage report.

So I do that…

But now I get mixed up. I start clicking on the coverage button when in the non-instrumented version and get a 404, so I want to fix that.

I know… I’ll add some JavaScript so that the button only displays on the instrumented code page…

<button id="coveragebutton"
        onclick='window.open("jscoverage.html")'>Coverage report</button>
<script>   
if (typeof(_$jscoverage)=='undefined')      
document.getElementById("coveragebutton").style.display="none";
</script>

And now, just to finish off, a little cmd script to automate it all for me…

instrument_eprimecode.bat

jscoverage eprimecode c_eprimecode
c_eprimecode\test_eprime.htm

So now I have QUnit Unit tests and JScoverage all running together.

Code Coverage Summary

I do not know how well JScoverage copes with more complicated JavaScript. As my coding skills in JavaScript improve I guess I’ll find out. But certainly for the moment it does the job for me.

I use code coverage as an ‘after check’. So I do my TDD and get it all as good as I can, then I just double check, but examining the code coverage:

did I miss anything?
do I care about the code that got missed?

JScoverage does that for me just now.

Other coverage tools for JavaScript exist: ProtoCov, the commercial JavaScript coverage Validator, and some sort of FireBug add on. But these are noted for future investigation as I have not used these.

SideNote: not really related to the above, but I didn’t want to lose it. I saw this JSMock (mocking library) that I need to try out at some point.

Comment from Ed

I’d just like to point out that the jscoverage program accepts a --no-instrument option so that you don’t have to rearrange the folder structure of your code to avoid instrumenting certain files. For example, to avoid instrumenting JQuery you could add something like --no-instrument=jquery-latest.js on the command line.

Functional Testing With QUnit

With the JavaScript I have written - I describe the tests which exercise the functions in the .js file and manipulate the variables in the .js file as Unit tests. Then when I test functions which interact with the html page that I import the .js file into - I class that as Functional testing. In this exciting episode of “Adventures in JavaScript” I describe my initial attempts at Functional testing with QUnit.

For a while I thought how could I make test the functions which interact with the html page as Unit tests? Perhaps I should learn how to use JSMock to mock out the html elements that I want to use in live? But I decided to take the easy route and in the test html file, just ‘mock up’ a crude version of the form I want to use when I go live. By my definition, my Unit test suite started to include some functional tests.

So how does that help me? Well, suddenly I have a suite of ‘unit’ tests which I can use to test the browser interaction for cross browser compatibility. And did they find cross browser bugs - yes indeedy, but if I tell you now, then I get ahead of myself so… one step at a time.

Create a test to drive the creation of the ‘mock’ GUI.

module("functional testing");

test("html input output elements exist",function(){
 expect(2);

 ok(document.getElementById("inputtext").innerHTML!=undefined, "input text field exists");
 ok(document.getElementById("eprimeoutput").innerHTML!=undefined,"output text field exists");

})

This test should only pass when I have some elements with IDs “inputtext” and “eprimeoutput”. So I create a small form at the bottom of my test page to represent my ‘production’ environment.

<label for="inputtext">Text: </label>
<textarea cols="60" rows="5"></textarea>

Now I did a little more than the test suggested I should, but… hey ho.

My test passed, so now I can start writing code to output the variables stored in my .js file.

But I can’t do that without a test, so…

test("display output text in html element",function(){
 expect(1);
 outputText = "<p>boo</p>"
 writeTheOutputTextToTheScreen();
 equals(document.getElementById("eprimeoutput").innerHTML,
        "<p>boo</p>");
});

Then I can write the code for this:

 function writeTheOutputTextToTheScreen(){
  document.getElementById("eprimeoutput").innerHTML = outputText;
 }

And so on, until I have a nice suite of tests and covered code, all of it running in FireFox - so off I duly go to try it in Internet Explorer… wha! That test failed. OK, so now my JavaScript knowledge has let me down and I have written code which does not run on both browsers. The JavaScript Gurus among you can probably spot the error here.

IE decides to uppercase all the html I inject so that

<p>boo</p>

becomes

<P>boo</P>

Which obviously helps my functional testing no end! And worse - if I change the

to make the test pass in IE, then FireFox lowercases everything so the tests fail in FireFox. Sheesh!

What to do?

OK…hack around in the DOM the old fashioned way, rather than compare the innerHTML…

test("display output text in html element",function(){
 expect(2);
 outputText = "<p>boo</p>"
        
 writeTheOutputTextToTheScreen();
        
 equals(document.getElementById("eprimeoutput").
          getElementsByTagName("p").length,
		1,
        "contains 1 para");
 equals(document.getElementById("eprimeoutput").
            getElementsByTagName("p")[0].innerHTML,
	    "boo",
		"contains boo");
});

Woo hoo - a cross browser functional test - however minor.

But lesson learned - I have to use the DOM functions rather than relying on innerHTML. (don’t worry - I shall get around to using JQuery eventually - I have to learn the basics the hard way first.)

This does make some of my tests rather large though…

test("check and output eprime text",function(){
 expect(21);

 document.getElementById("inputtext").value=
 "So Isn't this being some bad text or rather Bob's WASn't he's being good";
 
 checkThisTextForEprime(document.getElementById("inputtext").value);
 writeTheOutputTextToTheScreen();
 
 equals(outputText, "<p>" + "So " + inEPrimeOutputFormat("Isn't") + 
                                   " this " + inEPrimeOutputFormat("being") +  
                                   " some bad text or rather " + 
                                   inPossibleEPrimeOutputFormat("Bob's") + " " +
                                   inEPrimeOutputFormat("WASn't") +  " " + 
                                   inPossibleEPrimeOutputFormat("he's") + " " + 
                                   inEPrimeOutputFormat("being") + " good</p>");

        para = document.getElementById("eprimeoutput").getElementsByTagName("p");
 equals(para[0].childNodes.length,13,"contains 1 elements");
        equals(para[0].childNodes[0].nodeValue,"So ");
        equals(para[0].childNodes[1].className,"ep_violation");
        equals(para[0].childNodes[1].innerHTML,"Isn't");
        equals(para[0].childNodes[2].nodeValue," this ");
        equals(para[0].childNodes[3].className,"ep_violation");
        equals(para[0].childNodes[3].innerHTML,"being");
        equals(para[0].childNodes[4].nodeValue," some bad text or rather ");
        equals(para[0].childNodes[5].className,"ep_warning");
        equals(para[0].childNodes[5].innerHTML,"Bob's");
        equals(para[0].childNodes[6].nodeValue," ");
        equals(para[0].childNodes[7].className,"ep_violation");
        equals(para[0].childNodes[7].innerHTML,"WASn't");
 equals(para[0].childNodes[8].nodeValue," ");
        equals(para[0].childNodes[9].className,"ep_warning");
        equals(para[0].childNodes[9].innerHTML,"he's");
        equals(para[0].childNodes[10].nodeValue," ");
        equals(para[0].childNodes[11].className,"ep_violation");
        equals(para[0].childNodes[11].innerHTML,"being");
        equals(para[0].childNodes[12].nodeValue," good");
    });

But that will teach me for writing non-TDD tests. You can tell I did not create this using a TDD cycle because some lines of the test did not lead to the generation of lines of application code - I wrote a “well I’ll just check a few conditions in the one test just to be sure” test.

Writing functional tests like these (I wrote a few more like this) allowed me to discover other differences between IE and FireFox that caused me issues. e.g. To display some stats I created another FORM with some labels to display the information:

<div id="statistics">

<form lpformnum="2">

<ul>
<li>
      <label for="wordCount">Word Count: </label>
   <text id="wordCount" readonly="readonly"></text></li>
<li>
      <label for="discouragedWordCount">Discouraged Words: </label>
   <text id="discouragedWordCount" readonly="readonly"></text></li>
<li>
      <label for="possibleViolationCount">Possible Violations: </label>
   <text id="possibleViolationCount" readonly="readonly"></text></li>
</ul>
</form>
</div>

I do not know exactly, what possessed me to write the GUI like that, but I did. And I discovered that my code to write to the innerHTML of the TEXT worked on FireFox but not in IE, so because of my failing functional tests I rewrote this to the far simpler:

<div id="statistics">

<ul>
<li> Word Count: <span id="wordCount"></span></li>
<li> Discouraged Words: <span id="discouragedWordCount"></span></li>
<li> Possible Violations: <span id="possibleViolationCount"></span></li>
</ul>
</div>

So “Hooray” for simple functional tests.

I do believe that I could simplify much of the code I have written above by using the JQuery library, and that I could also use JQuery to help me test my event code, and probably create the GUI I want from within the code itself. So I shall probably try that next time in the ever more exciting “Adventures in JavaScript TDD with QUnit”.

Note: TDD does not guarantee a bug free product. I know that I have bugs in the code I have written because I find them with additional exploratory testing. But I make sure that for each bug I find - I create a failing test so that when I fix it, I know I did enough… at least until the next bug.

And for anyone wondering - “What On Earth kind of app does this code form part of?” - well you can find it at eviltester.github.io/TestingApp/apps/eprimer/eprimer.html
And if you want to learn about e-prime you can get more information on wikipedia

Clicking Buttons

Now to use JQuery to ‘click’ on a button. So that I can ‘functionally test’ the button click side-effects. I have a button on my GUI which I want to click and make sure the side-effects of calling the function attached to that button display the results I expect on screen:

<button>Check For E-Prime</button>

The following code only works in IE and generates a click event to trigger my button:

test("process eprime text by clicking on button",function(){
expect(3);
document.getElementById("inputtext").value=
              "Surely and \nIsn't this being" +
      " some bad text or rather Bob's WASn't he's being good";
        
var clickevent=document.createEvent('MouseEvents')
clickevent.initEvent('click', true, true)
document.getElementById("CheckForEPrimeButton").dispatchEvent(clickevent)
 
equals(document.getElementById("wordCount").innerHTML,
       15,"displayed wordCount = 15");
equals(document.getElementById("discouragedWordCount").innerHTML,
       4,"displayed discouragedWordCount = 4");
equals(document.getElementById("possibleViolationCount").innerHTML,
       2,"displayed possibleViolationCount = 2"); 
 
});

Sadly it fails in FireFox, but when I introduce the JQuery

test("process eprime text by clicking on button using JQuery",function(){
expect(3);
document.getElementById("inputtext").value=
         "Surely and \nIsn't this being" +
 " some bad text or rather Bob's WASn't he's being good";
 
$("#CheckForEPrimeButton").click();
         
equals(document.getElementById("wordCount").innerHTML,
       15,"displayed wordCount = 15");
equals(document.getElementById("discouragedWordCount").innerHTML,
       4,"displayed discouragedWordCount = 4");
equals(document.getElementById("possibleViolationCount").innerHTML,
       2,"displayed possibleViolationCount = 2"); 
 
});

Voila, cross-browser functional testing.