Develop Reference

How to write integration tests for WebExtensions (loading, checking some expectations, unloading)?

Is there a way to write integration tests for WebExtension based Browser addons?
In addition to unit tests, I would like to write an integration test that fully loads an extension, performs some tests, and finally unloads it.
My own research:
I assume it is possible with Selenium, but from my experience Selenium can lead to flakey tests that are hard to maintain. I wonder if there is a lighter alternative. Could also be that Selenium is the tool of choice. I have to admit that I don't have much experience with testing Browser extensions.
For limited use cases, I have used mock-browser. But as far as I understand it, it is not possible to simulate loading and unloading extensions with it.
Example:
To get an idea what kind of tests I would like to automate, here is a small example of a manual test that we have:
Start a browser with the extension. If the extension loads correctly, it will start increase a counter periodically
(Manually) check whether the counter increases. If the counter increases, the test passes.
If the test environment support loading an extension, this manual test could be easily automated. The problem is just to setup an environment that allows to load the extension. Currently, we run our unit tests with Node and using Mocha as a test framework.

JavaScript integration testing in Ruby on Rails

I've searched a bit for this and tried to implement a self-made solution but so far haven't found to be confident with it.
What I need is to write integration tests in Ruby on Rails which interact with JavaScript and get programmatic ways to assert some behaviors. I'm using Test::Unit for the controller/models part but I'm struggling to test some jQuery/JavaScript behaviors used by my app. Mainly it consists in ajax calls and interactions in the UI which updates some sets of information.
I haven't found a solution which makes me confident and which integrates nicely with autotest and the whole red-green process, so for now most parts of my client-side code is untested and that's making me nervous (as it should be :P).
So, does anyone have suggestions for best practices on this issue? Unit testing JS is a bit tricky, as Crockford points out, because it dependes heavily on the current state of the UI and etc and as AFAIK even he hasn't found a good way to implement decent testing...
Shortly: I need to implement tests for some UI behavior which depends on Ajax, integrating with autotest or some other CI tool and haven't found a good and elegant way to do it.
Thanks all for the attention,
Best Regards

AFAIK, outside of a combination of Capybara with Selenium Web-Driver there is very few options for automated testing of JS code.
I use cucumber with capybara and selenium web-driver and because selenium-webdriver actually launches firefox or chrome to go through testing a particular page with ajax call, It does take significantly longer to run through a suite of tests.
There are some alternatives but they dont work all the time or for every situations.
For instance: Capybara with envjs

In April 2011 the thoughtbot guys updated their quest for javascript testing.
Akephalos has fallen out of favor for the following reasons:
Bugs: as previously mentioned, there are bugs in htmlunit,
specifically with jQuery’s live. Although all browser implementations
have bugs, it’s more useful if tests experience the same bugs as
actual browsers.
Compatibility: htmlunit doesn’t fully implement the feature set that
modern browsers do. For example, it doesn’t fully handle DOM ranges or
Ajax file uploads.
Rendering: htmlunit doesn’t actually render the page, so tests that
depend on CSS visibility or positioning won’t work.
Performance: when most of your tests use Javascript, test suites with
htmlunit start to crawl. It takes a while to start up a test with
Akephalos, and a large test suite can easily take 10 or 15 minutes.
So they rolled their own solution which is open source - capybara-webkit. It's still fairly new but it looks like the way to go now.

This article recommends Akephalos.

I have used cucumber and capybara with selenium. This was very frustrating because selenium did not seem to be able to see dynamically generated javascript, despite the fact that capybara was supposed to be waiting for it. That was in January 2011. Things may be different now.
Currently, I am using cucumber and capybara with akephalos. So far, it has been very difficult because 1. it is headless, so you can't see progress. Capybara's "save_and_open" call has helped to a degree. 2. jQuery and akephalos don't seem to play that nicely together. For instance, triggering on a radio button with jquery's .change() works fine in chrome, but doesn't in akephalos. Maybe this is intentional because I later heard somewhere that it doesn't work in IE. I fixed the issue by using .click() instead of .change() for the radio button but since the .change function was set up to run on a bunch of questions, I had to code specifically to get it to work for the test.
The bottom line for me is that automated javascript acceptance testing in a rails env is still immature and possibly more work that it is worth.

Executing JavaScript with Python without X

I want to parse a html-page that unfortunately requires JavaScript to show any content. In order to do so I use a small python-script that pulls the html-code of the page, but after that I have to execute the JavaScript in a DOM-context which seems pretty hard.
To make it even harder I want to use it in a server environment that has no X11-server.
Note: I already read about http://code.google.com/p/pywebkitgtk/ but it seems to need a X-server.

You can simulate a browser environment using EnvJS. However, in order to make use of it, you will have to embed some kind of JavaScript runtime (e.g. Rhino) in your program (or spawn one as an external process).

You could try using Xvfb to have a fake frame buffer, so you won't need to run X11 (though it may be a dependency of Xvfb on your system). Most rendering engines don't have a headless mode, so something like Xvfb is necessary to run them. I used this technique successfully using XULRunner to navigate web pages, though not from python.

I'm still trying to figure this out myself, so take my answer with a grain of salt.
So far, I found http://blog.motane.lu/2009/06/18/pywebkitgtk-execute-javascript-from-python/, which describes the use and the quirks of Pywebkitgtk by someone who has similar needs to what we do.
Later, however, the writer of that blogpost discovered that he can't get it to work with Xvbf, so he hunted some more and found a Qt webkit (possibly in Qt itself, if I understand correctly) http://blog.motane.lu/2009/07/07/downloading-a-pages-content-with-python-and-webkit/. Apparently it's a much better solution than PywebkitGTK.
Naturally, I'll be looking into the other solutions offered here--but I wanted to bring up the Qt solution, because to me, it seems the most likely candidate for what I want to do...and if not, then perhaps it will be for someone else, looking for an answer to this question! :-)

I use VNC or Xvfb for this purpose, combined with Firefox. After experimenting with the two, I settled on XTightVNC. We use it to create screenshots on demand for various test purposes. It's nice to use one of these because you're executing it in an actual browser, same as a user would be (though most users probably won't be using the same OS as your server).
The handy thing about using VNC is that you can connect remotely to set up and test the browser when needed.

This might help: http://code.google.com/p/pyv8/

Help on testing with Selenium

I'm trying to make some tests on a JavaScript application and someone advised me to use Selenium. I visited its site but I cannot understand what is it and how can I use it for testing. Can someone help me understand?

There are a lot of options and it can be quite daunting to start.
Start with the IDE. It is a Firefox plug-in and would get you writing tests in no time. This is good for semi-automated tests running only on Firefox. And good to get some scripts generated for you to kick-start your tests.
Setup RC. It is a Java program that runs on 'a' box (could be localhost) spawning browsers and running your tests and you can connect to it using variety of languages and program your tests. RC is your friend if you want to automate your testing completely.
As for Grid, it is yet another Java program that manages different RCs on your network which makes it all distributed from browser, load and functionality perspectives. You don't need this initially and when the time comes your work on RC would be reusable 80-100%.

If you're using the Firefox plugin, all you have to do is record a "test". Then generate the testing code in the language you want to run the scripts in. They have an option for Java - and the test can be run standalone (outside of a browser on any platform). The test will attempt to replicate what you did in the browser. If it is able to complete the same steps, your test passes.
Selenium replicates what the browser does when running it's tests and does an admirable job (though not perfect)

Selenium or Watir for Javascript Testing in Rails

We're using RSpec and Cucumber in our Rails apps with good results. Webrat is great for non-AJAX interactions, but we're getting ready to get back into writing tests for our Javascript.
Webrat has Selenium support built in, and we've used Selenium before, but I'm curious if anyone has had good results using Watir with Cucumber and what the pros and cons are of Watir versus Selenium.

As the founder of OpenQA and Selenium RC, I'm obviously biased towards Selenium as a good option. We recently just put out a 1.0 beta 2 release and are very close to a 1.0 final.
However, you couldn't go wrong with Watir/FireWatir either. Adam's comment that WebDriver will merge to form Selenium 2.0 is correct, but he's incorrect in implying that Watir doesn't use native hooks. If Watir were simply a Selenium clone and also used JavaScript injection, I'd say it wasn't worth looking at.
But because it has native hooks, it can do some things that Selenium currently can't. While it has fewer browsers supported, it goes a bit deeper in the main browser it does support (IE) and lets you control things outside of the page/canvas.
In summary: either is fine, Selenium is great, and if you hang on a little longer with Selenium you'll soon get the best of both worlds with WebDriver/Selenium 2.0.

I am having good results using Cucumber with Celerity through JRuby.
Celerity is a headless browser which wraps around HtmlUnit with a Watir-compatible API and supports JavaScript and AJAX testing.
Being headless makes Celerity faster and easy to integrate within a Continuous Integration build cycle.
Since Celerity is API-compatible with Watir, you can switch between Watir and Celerity fairly easily. There are some caveats, but it's been worth the effort.

I'd say Watir was much slicker, but less useful. It's mostly an IE automation system, with fairly flaky Firefox support. (I'll admit to not having checked out FireWatir in a while, it may have improved.)
There is experimental support for Selenium-RC on Watir. That would definitely be the best of both worlds.

You may want to try WebDriver. It will become part of Selenium in the future. Works with JRuby.
It is different than Selenium, Sahi or Watir in the way that it controls the browser natively, not using JavaScript, and is not affected by many issues other solutions have.

I have tried Watir but not with cucumber. It was for java app. Problems we faced were with was trying to determine page is loaded. We had to scrape the forums a bit before finding the solutions. Otherwise it was fine.

Watir 2.0 will also use WebDriver.

If you would like to use Watir on Rails 3.
You can use 'watir-webdriver-rails' gem (I'm the creator).

Best solution which I found its Env-Js http://github.com/smparkes/env-js
Its like webrat but with JS support

Don't overlook something like QUnit or Jasmine to unit test your javascript. If nothing else it'll cut down on the number of full-stack tests you have to write in cucumber/selenium/capybara/whatever