Develop Reference

How to test code that uses browsers APIs

I'm using Node for running my unit-tests.
I have a JavaScript module run in the browser I'd like to test.
My code is "isomorphic", i.e. it avoids language features not available in Node, like exports.
But it uses pure browsers APIs: XMLHttpRequest, FormData and File.
I have found Node's implementations for each of them.
But the one of XMLHttpRequest does not support upload.
So I'm looking for the simplest way to unit-test this code in an environment with these APIs.
The code does not need DOM or other browsers APIs, "only" these three.
I've already used PhantomJS for other needs but:
this will create another test workflow (minor issue),
it supports an older JavaScript version and it would force a complete rewrite of the code to test (major issue),
the code has a lot of NPM dependencies that probably won't be compatible (blocking issue).
As the code is Browserified all these issues may disappear but before going along this way I'd like to be sure.
Is there any chance to get it work with PhantomJS, CasperJS or the like?
Which other alternatives are available?

This is not how you test code that runs in a browser. If it runs in a browser, it needs to be tested in a browser.
You need to look into solutions based on the webdriver spec. The big hairy monster in this ecosystem is Selenium. I'm currently researching this topic because of some issues we've had with using selenium-server. You should also look into Nightwatch and Leadfoot. Webdriver.io is the first recommendation a lot of people recommend, as its a node-based client that wraps (poorly) around Selenium. But the documentation is all over the place and we've run into frequent bugs using it.

Testing browser extensions

I'm going to write bunch of browser extensions (the same functionality for each popular browser). I hope, that some of the code will be shared, but I'm not sure about this yet. For sure some of extensions will use native API. I have not much experience with TDD/BDD, and I thought it's good time to start folowing these ideas from this project.
The problem is, I have no idea how to handle it. Should I write different tests for each browser? How far should I go with these tests? These extensions will be quite simple - some data in a local storage, refreshing a page and listening through web sockets.
And my observation about why is it hard for me - because there is a lot of behaviour, and not so much models, which are also dependent on a platform.

I practise two different ways of testing my browser extensions:
Unit tests
Integration test
Introduction
I will use the cross-browser YouTube Lyrics by Rob W extension as an example throughout this answer. The core of this extension is written in JavaScript and organized with AMD modules. A build script generates the extension files for each browser. With r.js, I streamline the inclusion of browser-specific modules, such as the one for cross-origin HTTP requests and persistent storage (for preferences), and a module with tons of polyfills for IE.
The extension inserts a panel with lyrics for the currently played song on YouTube, Grooveshark and Spotify. I have no control over these third-party sites, so I need an automated way to verify that the extension still works well.
Workflow
During development:
Implement / edit feature, and write a unit test if the feature is not trivial.
Run all unit tests to see if anything broke. If anything is wrong, go back to 1.
Commit to git.
Before release:
Run all unit tests to verify that the individual modules is still working.
Run all integration tests to verify that the extension as whole is still working.
Bump versions, build extensions.
Upload update to the official extension galleries and my website (Safari and IE extensions have to be hosted by yourself) and commit to git.
Unit testing
I use mocha + expect.js to write tests. I don't test every method for each module, just the ones that matter. For instance:
The DOM parsing method. Most DOM parsing methods in the wild (including jQuery) are flawed: Any external resources are loaded and JavaScript is executed.
I verify that the DOM parsing method correctly parses DOM without negative side effects.
The preference module: I verify that data can be saved and returned.
My extension fetches lyrics from external sources. These sources are defined in separate modules. These definitions are recognized and used by the InfoProvider module, which takes a query, (black box), and outputs the search results.
First I test whether the InfoProvider module functions correctly.
Then, for each of the 17 sources, I pass a pre-defined query to the source (with InfoProvider) and verify that the results are expected:
The query succeeds
The returned song title matches (by applying a word similarity algorithm)
The length of the returned lyrics fall inside the expected range.
Whether the UI is not obviously broken, e.g. by clicking on the Close button.
These tests can be run directly from a local server, or within a browser extension. The advantage of the local server is that you can edit the test and refresh the browser to see the results. If all of these tests pass, I run the tests from the browser extension.
By passing an extra parameter debug to my build script, the unit tests are bundled with my extension.
Running the tests within a web page is not sufficient, because the extension's environment may differ from the normal page. For instance, in an Opera 12 extension, there's no global location object.
Remark: I don't include the tests in the release build. Most users don't take the efforts to report and investigate bugs, they will just give a low rating and say something like "Doesn't work". Make sure that your extension functions without obvious bugs before shipping it.
Summary
View modules as black boxes. You don't care what's inside, as long as the output matches is expected or a given input.
Start with testing the critical parts of your extension.
Make sure that the tests can be build and run easily, possibly in a non-extension environment.
Don't forget to run the tests within the extension's execution context, to ensure that there's no constraint or unexpected condition inside the extension's context which break your code.
Integration testing
I use Selenium 2 to test whether my extension still works on YouTube, Grooveshark (3x) and Spotify.
Initially, I just used the Selenium IDE to record tests and see if it worked. That went well, until I needed more flexibility: I wanted to conditionally run a test depending on whether the test account was logged in or not. That's not possible with the default Selenium IDE (it's said to be possible with the FlowControl plugin - I haven't tried).
The Selenium IDE offers an option to export the existing tests in other formats, including JUnit 4 tests (Java). Unfortunately, this result wasn't satisfying. Many commands were not recognized.
So, I abandoned the Selenium IDE, and switched to Selenium.
Note that when you search for "Selenium", you will find information about Selenium RC (Selenium 1) and Selenium WebDriver (Selenium 2). The first is the old and deprecated, the latter (Selenium WebDriver) should be used for new projects.
Once you discovered how the documentation works, it's quite easy to use.
I prefer the documentation at the project page, because it's generally concise (the wiki) and complete (the Java docs).
If you want to get started quickly, read the Getting Started wiki page. If you've got spare time, look through the documentation at SeleniumHQ, in particular the Selenium WebDriver and WebDriver: Advanced Usage.
Selenium Grid is also worth reading. This feature allows you to distribute tests across different (virtual) machines. Great if you want to test your extension in IE8, 9 and 10, simultaneously (to run multiple versions of Internet Explorer, you need virtualization).
Automating tests is nice. What's more nice? Automating installation of extensions!
The ChromeDriver and FirefoxDriver support the installation of extensions, as seen in this example.
For the SafariDriver, I've written two classes to install a custom Safari extension. I've published it and sent in a PR to Selenium, so it might be available to everyone in the future: https://github.com/SeleniumHQ/selenium/pull/87
The OperaDriver does not support installation of custom extensions (technically, it should be possible though).
Note that with the advent of Chromium-powered Opera, the old OperaDriver doesn't work any more.
There's an Internet Explorer Driver, and this one does definitely not allow one to install a custom extension. Internet Explorer doesn't have built-in support for extensions. Extensions are installed through MSI or EXE installers, which are not even integrated in Internet Explorer. So, in order to automatically install your extension in IE, you need to be able to silently run an installer which installs your IE plugin. I haven't tried this yet.

Testing browser extensions posed some difficulty for me as well, but I've settled on implementing tests in a few different areas that I can invoke simultaneously from browsers driven by Selenium.
The steps I use are:
First, I write test code integrated into the extension code that can be activated by simply going to a specific URL. When the extension sees that URL, it begins running the tests.
Then, in the page that activates the testing in the extension I execute server-side tests to be sure the API performs, and record and log issues there. I record the methods invoked, the time they took, and any errors. So I can see the method the extension invoked, the web performance, the business logic performance, and the database performance.
Lastly, I automatically invoke browsers to point at that specific URL and record their performance along with other test information, errors, etc on any given client system using Selenium:
http://docs.seleniumhq.org/
This way I can break down the tests in terms of browser, extension, server, application, and database and link them all together according to specific test sets. It takes a bit of work to put it all together, but once its done you can have a very nice extension testing framework.
Typically for cross-browser extension development in order to maintain a single code-base I use crossrider, but you can do this with any framework or with native extensions as you wish, Selenium won't care, it is just driving the extension to a particular page and allowing you to interact and perform tests.
One nice thing about this approach is you can use it for live users as well. If you are providing support for your extension, have a user go to your test url and immediately you will see the extension and server-side performance. You won't get the Selenium tests of course, but you will capture a lot of issues this way - very useful when you are coding against a variety of browsers and browser versions.

Jasmine dependency on browser

From Last few days I am reading and working on Jasmine.After reading it I am able to write specification (test cases) and also able to run it successfully.
I also debugged internals flow of jasmine.
As jasmine is embeded in a html file and this html file will run on browser.
I want to know any type of dependencies of jasmine on browser (any browser).
Is jasmine is using browser's script engine ? If yes how it is configured?
If any one has an idea or tutorials then please share.
Thanks in advance.

Jasmine itself does not depend on a browser or the DOM. Often it is run in some kind of browser because the code under test has a dependency on the DOM or simply because its a convenient way to get started. In that case then, yes, it is using the browser's JavaScript engine. It can be used in most modern browser that way.
However Jasmine works fine in JavaScript environments outside a browser. For example, it is one of several testing frameworks that you can use with Node.js.
Jasmine can also be used with a 'headless' browser like Phantom.js. This can speed up your tests and allow you to integrate them into a continuous integration process.
The browser itself doesn't need to be configured in any special way to run Jasmine. How you setup your tests may depend on the kind of server-side framework you are using and the way your JavaScript files are organised. Check the list of environments on the Jasmine Wiki.

Official page of Jasmine says that
"It (Jasmine) does not depend on any other JavaScript frameworks. It does not require a DOM."
Jasmine needs to run inside a JavaScript interpreter/VM in order to work. Browsers are one way to do this. Jasmine will run in any browser that supports JavaScript. Jasmine has no external dependencies.
If you are using the standalone Jasmine release, you can run your specs just by loading the spec runner web page.

Help on testing with Selenium

I'm trying to make some tests on a JavaScript application and someone advised me to use Selenium. I visited its site but I cannot understand what is it and how can I use it for testing. Can someone help me understand?

There are a lot of options and it can be quite daunting to start.
Start with the IDE. It is a Firefox plug-in and would get you writing tests in no time. This is good for semi-automated tests running only on Firefox. And good to get some scripts generated for you to kick-start your tests.
Setup RC. It is a Java program that runs on 'a' box (could be localhost) spawning browsers and running your tests and you can connect to it using variety of languages and program your tests. RC is your friend if you want to automate your testing completely.
As for Grid, it is yet another Java program that manages different RCs on your network which makes it all distributed from browser, load and functionality perspectives. You don't need this initially and when the time comes your work on RC would be reusable 80-100%.

If you're using the Firefox plugin, all you have to do is record a "test". Then generate the testing code in the language you want to run the scripts in. They have an option for Java - and the test can be run standalone (outside of a browser on any platform). The test will attempt to replicate what you did in the browser. If it is able to complete the same steps, your test passes.
Selenium replicates what the browser does when running it's tests and does an admirable job (though not perfect)

JavaScript unit test tools for TDD

This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.
I've looked into and considered many JavaScript unit tests and testing tools, but have been unable to find a suitable option to remain fully TDD compliant. So, is there a JavaScript unit test tool that is fully TDD compliant?

Karma or Protractor
Karma is a JavaScript test-runner built with Node.js and meant for unit testing.
The Protractor is for end-to-end testing and uses Selenium Web Driver to drive tests.
Both have been made by the Angular team. You can use any assertion-library you want with either.
Screencast: Karma Getting started
related:
Should I be using Protractor or Karma for my end-to-end testing?
Can Protractor and Karma be used together?
pros:
Uses Node.js, so compatible with Win/OS X/Linux
Run tests from a browser or headless with PhantomJS
Run on multiple clients at once
Option to launch, capture, and automatically shut down browsers
Option to run server/clients on development computer or separately
Run tests from a command line (can be integrated into ant/maven)
Write tests xUnit or BDD style
Supports multiple JavaScript test frameworks
Auto-run tests on save
Proxies requests cross-domain
Possible to customize:
Extend it to wrap other test-frameworks (Jasmine, Mocha, QUnit built-in)
Your own assertions/refutes
Reporters
Browser Launchers
Plugin for WebStorm
Supported by NetBeans IDE
Cons:
Does not support Node.js (i.e. backend) testing
No plugin for Eclipse (yet)
No history of previous test results
mocha.js
I'm totally unqualified to comment on mocha.js's features, strengths, and weaknesses,
but it was just recommended to me by someone I trust in the JS community.
List of features, as reported by its website:
browser support
simple async support, including promises
test coverage reporting
string diff support
JavaScript # API for running tests
proper exit status for CI support etc
auto-detects and disables coloring for non-ttys
maps uncaught exceptions to the correct test case
async test timeout support
test-specific timeouts
growl notification support
reports test durations
highlights slow tests
file watcher support
global variable leak detection
optionally run tests that match a regexp
auto-exit to prevent "hanging" with an active loop
easily meta-generate suites & test-cases
mocha.opts file support
clickable suite titles to filter test execution
node debugger support
detects multiple calls to done()
use any assertion library you want
extensible reporting, bundled with 9+ reporters
extensible test DSLs or "interfaces"
before, after, before each, after each hook
arbitrary transpiler support (coffee-script etc)
TextMate bundle
yolpo
This no longer exists, redirects to sequential.js instead
Yolpo is a tool to visualize the execution of JavaScript. JavaScript API developers are encouraged to write their use cases to show and tell their API. Such use cases forms the basis of regression tests.
AVA
Futuristic test runner with built-in support for ES2015. Even though JavaScript is single-threaded, I/O in Node.js can happen in parallel due to its async nature. AVA takes advantage of this and runs your tests concurrently, which is especially beneficial for I/O heavy tests. In addition, test files are run in parallel as separate processes, giving you even better performance and an isolated environment for each test file.
Minimal and fast
Simple test syntax
Runs tests concurrently
Enforces writing atomic tests
No implicit globals
Isolated environment for each test file
Write your tests in ES2015
Promise support
Generator function support
Async function support
Observable support
Enhanced asserts
Optional TAP o
utput
Clean stack traces
Buster.js
A JavaScript test-runner built with Node.js. Very modular and flexible. It comes with its own assertion library, but you can add your own if you like. The assertions library is decoupled, so you can also use it with other test-runners. Instead of using assert(!...) or expect(...).not..., it uses refute(...) which is a nice twist imho.
A browser JavaScript testing toolkit. It does browser testing with browser automation (think JsTestDriver), QUnit style static HTML page testing, testing in headless browsers (PhantomJS, jsdom, ...), and more. Take a look at the overview!
A Node.js testing toolkit. You get the same test case library, assertion library, etc. This is also great for hybrid browser and Node.js code. Write your test case with Buster.JS and run it both in Node.js and in a real browser.
Screencast: Buster.js Getting started (2:45)
pros:
Uses Node.js, so compatible with Win/OS X/Linux
Run tests from a browser or headless with PhantomJS (soon)
Run on multiple clients at once
Supports Node.js testing
Don't need to run server/clients on development computer (no need for IE)
Run tests from a command line (can be integrated into ant/maven)
Write tests xUnit or BDD style
Supports multiple JavaScript test frameworks
Defer tests instead of commenting them out
SinonJS built-in
Auto-run tests on save
Proxies requests cross-domain
Possible to customize:
Extend it to wrap other test-frameworks (JsTestDriver built in)
Your own assertions/refutes
Reporters (xUnit XML, traditional dots, specification, tap, TeamCity and more built-in)
Customize/replace the HTML that is used to run the browser-tests
TextMate and Emacs integration
Cons:
Stil in beta so can be buggy
No plugin for Eclipse/IntelliJ (yet)
Doesn't group results by os/browser/version like TestSwarm *. It does, however, print out the browser name and version in the test results.
No history of previous test results like TestSwarm *
Doesn't fully work on windows as of May 2014
* TestSwarm is also a Continuous Integration server, while you need a separate CI server for Buster.js. It does, however, output xUnit XML reports, so it should be easy to integrate with Hudson, Bamboo or other CI servers.
TestSwarm
https://github.com/jquery/testswarm
TestSwarm is officially no longer under active development as stated on their GitHub webpage. They recommend Karma, browserstack-runner, or Intern.
Jasmine
This is a behavior-driven framework (as stated in quote below) that might interest developers familiar with Ruby or Ruby on Rails. The syntax is based on RSpec that are used for testing in Rails projects.
Jasmine specs can be run from an HTML page (in qUnit fashion) or from a test runner (as Karma).
Jasmine is a behavior-driven development framework for testing your JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM.
If you have experience with this testing framework, please contribute with more info :)
Project home: http://jasmine.github.io/
QUnit
QUnit focuses on testing JavaScript in the browser while providing as much convenience to the developer as possible. Blurb from the site:
QUnit is a powerful, easy-to-use JavaScript unit test suite. It's used by the jQuery, jQuery UI, and jQuery Mobile projects and is capable of testing any generic JavaScript code
QUnit shares some history with TestSwarm (above):
QUnit was originally developed by John Resig as part of jQuery. In 2008 it got its own home, name and API documentation, allowing others to use it for their unit testing as well. At the time it still depended on jQuery. A rewrite in 2009 fixed that, now QUnit runs completely standalone.
QUnit's assertion methods follow the CommonJS Unit Testing specification, which was to some degree influenced by QUnit.
Project home: http://qunitjs.com/
Sinon
Another great tool is sinon.js by Christian Johansen, the author of Test-Driven JavaScript Development. Best described by himself:
Standalone test spies, stubs and mocks
for JavaScript. No dependencies works
with any unit testing framework.
Intern
The Intern Web site provides a direct feature comparison to the other testing frameworks on this list. It offers more features out of the box than any other JavaScript-based testing system.
JEST
A new but yet very powerful testing framework. It allows snapshot based testing as well this increases the testing speed and creates a new dynamic in terms of testing
Check out one of their talks: https://www.youtube.com/watch?v=cAKYQpTC7MA
Better yet: Getting Started

Take a look at the Dojo Object Harness (DOH) unit test framework which is pretty much framework independent harness for JavaScript unit testing and doesn't have any Dojo dependencies. There is a very good description of it at Unit testing Web 2.0 applications using the Dojo Objective Harness.
If you want to automate the UI testing (a sore point of many developers) — check out doh.robot (temporary down. update: other link http://dojotoolkit.org/reference-guide/util/dohrobot.html ) and dijit.robotx (temporary down). The latter is designed for an acceptance testing.
Update:
Referenced articles explain how to use them, how to emulate a user interacting with your UI using mouse and/or keyboard, and how to record a testing session, so you can "play" it later automatically.

Chutzpah - A JavaScript Test Runner
I created an open source project called Chutzpah which is a test runner for JavaScript unit tests. Chutzpah enables you to run JavaScript unit tests from the command line and from inside of Visual Studio. It also supports running in the TeamCity continuous integration server.

The JavaScript section of the Wikipedia entry, List of Unit Testing Frameworks, provides a list of available choices. It indicates whether they work client-side, server-side, or both.

BusterJS
There is also BusterJS from Christian Johansen, the author of Test Driven Javascript Development and the Sinon framework. From the site:
Buster.JS is a new JavaScript testing framework. It does browser testing by automating test runs in actual browsers (think JsTestDriver), as well as Node.js testing.

google-js-test:
JavaScript testing framework released by Google:
https://github.com/google/gjstest
Extremely fast test startup and execution time, without having to run a
browser.
Clean, readable output in the case of both passing and failing tests.
A browser-based test runner that can simply be refreshed
whenever JS is changed.
Style and semantics that resemble Google Test for C++.
A built-in mocking framework that requires minimal boilerplate code (e.g. no
$tearDown or $verifyAll) with style and semantics based on the Google
C++ Mocking Framework.
There are currently no binaries for Windows

We are now using Qunit with Pavlov and JSTestDriver all together. This approach works well for us.
QUnit
Pavlov, source
jsTestDriver, source

You have "runs on actual browser" as a pro, but in my experience that is a con because it is slow. But what makes it valuable is the lack of sufficient JS emulation from the non-browser alternatives. It could be that if your JS is complex enough that only an in browser test will suffice, but there are a couple more options to consider:
HtmlUnit: "It has fairly good JavaScript support (which is constantly improving) and is able to work even with quite complex AJAX libraries, simulating either Firefox or Internet Explorer depending on the configuration you want to use." If its emulation is good enough for your use then it will be much faster than driving a browser.
But maybe HtmlUnit has good enough JS support but you don't like Java? Then maybe:
Celerity: Watir API running on JRuby backed by HtmlUnit.
or similarly
Schnell: another JRuby wrapper of HtmlUnit.
Of course if HtmlUnit isn't good enough and you have to drive a browser then you might consider Watir to drive your JS.

YUI has a testing framework as well. This video from Yahoo! Theater is a nice introduction, although there are a lot of basics about TDD up front.
This framework is generic and can be run against any JavaScript or JS library.

You might also be interested in the unit testing framework that is part of qooxdoo, an open source RIA framework similar to Dojo, ExtJS, etc. but with quite a comprehensive tool chain.
Try the online version of the testrunner. Hint: hit the gray arrow at the top left (should be made more obvious). It's a "play" button that runs the selected tests.
To find out more about the JS classes that let you define your unit tests, see the online API viewer.
For automated UI testing (based on Selenium RC), check out the Simulator project.

We added JUnit integration to our Java to Javascript code generator ST-JS (http://st-js.org). The framework generates to corresponding Javascript for both the tested code and the unit tests and sends the code to different browsers.
There is no need for a separate server as the unit test runner opens the needed http port (and closes it once the tests finished). The framework manipulates the Java stacktrace so that the failed asserts are correctly displayed by the JUnit Eclipse plugin. Here is a simple example with jQuery and Mockjax:
#RunWith(STJSTestDriverRunner.class)
#HTMLFixture("<div id='fortune'></div>")
#Scripts({ "classpath://jquery.js",
"classpath://jquery.mockjax.js", "classpath://json2.js" })
public class MockjaxExampleTest {
#Test
public void myTest() {
$.ajaxSetup($map("async", false));
$.mockjax(new MockjaxOptions() {
{
url = "/restful/fortune";
responseText = new Fortune() {
{
status = "success";
fortune = "Are you a turtle?";
}
};
}
});
$.getJSON("/restful/fortune", null, new Callback3<Fortune, String, JQueryXHR>() {
#Override
public void $invoke(Fortune response, String p2, JQueryXHR p3) {
if (response.status.equals("success")) {
$("#fortune").html("Your fortune is: " + response.fortune);
} else {
$("#fortune").html("Things do not look good, no fortune was told");
}
}
});
assertEquals("Your fortune is: Are you a turtle?", $("#fortune").html());
}
private static class Fortune {
public String status;
public String fortune;
}
}

You should have a look at env.js. See my blog for an example how to write unit tests with env.js.

MochiKit has a testing framework called SimpleTest that seems to have caught on.
Here's a blog post from the original author.