I've searched a bit for this and tried to implement a self-made solution but so far haven't found to be confident with it.
What I need is to write integration tests in Ruby on Rails which interact with JavaScript and get programmatic ways to assert some behaviors. I'm using Test::Unit for the controller/models part but I'm struggling to test some jQuery/JavaScript behaviors used by my app. Mainly it consists in ajax calls and interactions in the UI which updates some sets of information.
I haven't found a solution which makes me confident and which integrates nicely with autotest and the whole red-green process, so for now most parts of my client-side code is untested and that's making me nervous (as it should be :P).
So, does anyone have suggestions for best practices on this issue? Unit testing JS is a bit tricky, as Crockford points out, because it dependes heavily on the current state of the UI and etc and as AFAIK even he hasn't found a good way to implement decent testing...
Shortly: I need to implement tests for some UI behavior which depends on Ajax, integrating with autotest or some other CI tool and haven't found a good and elegant way to do it.
Thanks all for the attention,
Best Regards
AFAIK, outside of a combination of Capybara with Selenium Web-Driver there is very few options for automated testing of JS code.
I use cucumber with capybara and selenium web-driver and because selenium-webdriver actually launches firefox or chrome to go through testing a particular page with ajax call, It does take significantly longer to run through a suite of tests.
There are some alternatives but they dont work all the time or for every situations.
For instance: Capybara with envjs
In April 2011 the thoughtbot guys updated their quest for javascript testing.
Akephalos has fallen out of favor for the following reasons:
Bugs: as previously mentioned, there are bugs in htmlunit,
specifically with jQuery’s live. Although all browser implementations
have bugs, it’s more useful if tests experience the same bugs as
actual browsers.
Compatibility: htmlunit doesn’t fully implement the feature set that
modern browsers do. For example, it doesn’t fully handle DOM ranges or
Ajax file uploads.
Rendering: htmlunit doesn’t actually render the page, so tests that
depend on CSS visibility or positioning won’t work.
Performance: when most of your tests use Javascript, test suites with
htmlunit start to crawl. It takes a while to start up a test with
Akephalos, and a large test suite can easily take 10 or 15 minutes.
So they rolled their own solution which is open source - capybara-webkit. It's still fairly new but it looks like the way to go now.
This article recommends Akephalos.
I have used cucumber and capybara with selenium. This was very frustrating because selenium did not seem to be able to see dynamically generated javascript, despite the fact that capybara was supposed to be waiting for it. That was in January 2011. Things may be different now.
Currently, I am using cucumber and capybara with akephalos. So far, it has been very difficult because 1. it is headless, so you can't see progress. Capybara's "save_and_open" call has helped to a degree. 2. jQuery and akephalos don't seem to play that nicely together. For instance, triggering on a radio button with jquery's .change() works fine in chrome, but doesn't in akephalos. Maybe this is intentional because I later heard somewhere that it doesn't work in IE. I fixed the issue by using .click() instead of .change() for the radio button but since the .change function was set up to run on a bunch of questions, I had to code specifically to get it to work for the test.
The bottom line for me is that automated javascript acceptance testing in a rails env is still immature and possibly more work that it is worth.
Related
I am trying to validate the JavaScript on my website. The scripts do not throw any errors and run fine on Chrome and Firefox (latest stable version). However, the animated parts absolutely do not work on IE (9)[*1][*solved]. I have used jQuery and jQueryUI for the animation and had hoped that it would be cross browser compatible.
Usually I would not have cared abut IE users, but liked the idea of folks at anybrowser.org and thought of sticking to a standard instead of using the lowest common denominator and leave the graceful degradation to the browsers / JS engines. My page is HTML5 compliant according to the w3c validator and I wanted to do the same with JavaScript, but could not find an acceptable way to do it.
Why JSLint did not work -
It will not take the page and check it like w3c validator.
So there is no way to check jQuery referenced script. (There are a few SO posts on this).
I did find an ECMA-264 test page that would check the browser, but not my JavaScript.
The question after all the preamble is: How do I validate the Javascript on my webpage? Is there an authoritative validator for JavaScript?
*1 Added: The minification has nothing to do with the script not working on IE9. Even the unminified version does not work.
*Solved: The problem with the specific page was solved by this comment.
This started as "What is Javascript?" got closed here and migrated to Programmers. Thanks to YannisRizos for helping me to split the original question to something that was acceptable between Programmers and SO.
A test suite. No really.
If you want to ensure your javascript runs perfectly in every target environment, then you write a test suite for your code and make sure it passes in all your target environments.
A test suite helps find where your code breaks when run on different platforms. And therefore helps you fix it.
But assuming you want something that is less work, http://www.jshint.com/ is your best bet.
Or http://www.jslint.com/ if you like getting yelled at.
But honestly, no validator will ever be able to ensure that your code runs without error and exactly how you expect everywhere. Simply because your code follows best practices correctly, that doesn't mean it will work in an old version of IE.
Standards evolve, but an old browser that never updates get stuck with whatever standard was out at the time that the browser maker may or may not have even implemented successfully or accurately.
Testing, automated or manual, is the only way to ensure things work universally.
I'm looking for a way to automate some simple testing of a web application. "Clicking x causes y" sort of testing. I want to write tests that can be run in a variety of browsers. I was thinking writing an inject-able script (bookmarklet sort of thing) which invokes click events and tests that elements exists might be a good way to go.
Are there any inherent dangers to this approach or major issues? Are there better alternatives? I don't want to get too involved in creating this test (don't want to create a test server, or go through a lot of setup), I just want to automate some repetitive testing.
As Diodeus said, Selenium is probably the most popular browser automation library right now (I believe Facebook uses it). Other frameworks you may wish to investigate:
Watir
Windmill
Sahi
In addition, you'll want to consider cross-browser testing when setting up an automated suite of tests. You can roll your own for this, or if you'd rather throw money at the problem, BrowserStack now offers an API that allows your tests to run on a range of browsers.
Selenium is pretty popular: http://seleniumhq.org/
I want to parse a html-page that unfortunately requires JavaScript to show any content. In order to do so I use a small python-script that pulls the html-code of the page, but after that I have to execute the JavaScript in a DOM-context which seems pretty hard.
To make it even harder I want to use it in a server environment that has no X11-server.
Note: I already read about http://code.google.com/p/pywebkitgtk/ but it seems to need a X-server.
You can simulate a browser environment using EnvJS. However, in order to make use of it, you will have to embed some kind of JavaScript runtime (e.g. Rhino) in your program (or spawn one as an external process).
You could try using Xvfb to have a fake frame buffer, so you won't need to run X11 (though it may be a dependency of Xvfb on your system). Most rendering engines don't have a headless mode, so something like Xvfb is necessary to run them. I used this technique successfully using XULRunner to navigate web pages, though not from python.
I'm still trying to figure this out myself, so take my answer with a grain of salt.
So far, I found http://blog.motane.lu/2009/06/18/pywebkitgtk-execute-javascript-from-python/, which describes the use and the quirks of Pywebkitgtk by someone who has similar needs to what we do.
Later, however, the writer of that blogpost discovered that he can't get it to work with Xvbf, so he hunted some more and found a Qt webkit (possibly in Qt itself, if I understand correctly) http://blog.motane.lu/2009/07/07/downloading-a-pages-content-with-python-and-webkit/. Apparently it's a much better solution than PywebkitGTK.
Naturally, I'll be looking into the other solutions offered here--but I wanted to bring up the Qt solution, because to me, it seems the most likely candidate for what I want to do...and if not, then perhaps it will be for someone else, looking for an answer to this question! :-)
I use VNC or Xvfb for this purpose, combined with Firefox. After experimenting with the two, I settled on XTightVNC. We use it to create screenshots on demand for various test purposes. It's nice to use one of these because you're executing it in an actual browser, same as a user would be (though most users probably won't be using the same OS as your server).
The handy thing about using VNC is that you can connect remotely to set up and test the browser when needed.
This might help: http://code.google.com/p/pyv8/
I'm fairly new to both tools and need to go hardcore with both as I manage, monitor, and tweak a new site's design process. What sort of strategies should I ask be implemented that sets a good solid foundation for debugging, testing, and logging?
[to the degree that back-end stuff can play a role - it's .net mvc
thx
I would use Firebug to see how things are working with a few Firebug Add-ons.
I would use YSlow to check that you aren't downloading too much and it will make suggestions if you aren't minifying and gzipping your javascript.
I would also use FireQuery as that highlights jQuery very nicely in Firebug. I use it quite a lot these days to see what it should be firing.
Firebug doesn't rewrite XHRequests anymore but there is a bug in the latest Firefox/Firebug where if can block long running XHR calls. Details here
First off make sure you've read Firebug's docs. Some of the commands work cross-brower with other tools as well.
A simple search query will show you all available extensions for Firebug. As some people mentioned - some of them are really helpful.
Also it's important not to limit yourself to just a single tool since you will most likely be developing for multiple browsers. So make sure you take a look at webkits developer tools (Safari, Chrome) as well. Here's a good article which sums up the most popular development/debug tools.
You might want to research how jQuery/jQuery plugins are structured/organized so you have general idea how to organise your own JavaScript/jQuery code. It all depends how JavaScript heavy is your application. If jQuery just provides some visual enhancements and few Ajaxified pages here and there, don't bother. From other hand if it's very JavaScript heavy (as in a lot more site logic on client-side then on backend) I would suggest Prototype over jQuery, but it's just my opinion.
You could consider using automatic tools to build your JavaScript if you have a lot of code.
For example:
Sprockets
Juicer
On production server you want to end up with as few JavaScript files as possible and make sure to compress em.
If you're interested in more links to articles/tools for javascript heavy applications, drop a comment. I'm just trying to stay on topic at the moment.
I would just give a small warning using FireBug's network monitor and AJAX together. When enabled, it rewrites some HTTP headers and breaks stuff badly (well it used too, not sure anymore).
So if anything goes ape. Check that network monitoring is disabled.
I will also add for tools FireCookie, as it goes very well with $.cookie.
When I am debugging jQuery code I am using the NET panel a lot in Firebug for all ajax requests. Very helpful to see what are you sending and what are you receiving.
Also I use a lot the comand line, to test snippets of code.
You cannot do without the console. It will be very helpful. Example:
$.get( 'url.php', {},
function(data){
$.each(data, function(x){
console.log( x ); // will log each x object to see what it contains
});
}, 'json'
);
I also suggest you install FireUnit addon. It helps you work with QUnit unit tests. Of course that is if you are planning to write unit tests but in most cases that's the very good idea.
As much as you might love Firebug, Safari's developer tools are also quite powerful, and worth checking out. It's all I use when I dev.
Worth mentioning that Safari's javascript engine is still faster than FFX's, while Chrome reigns supreme. They're playing catch-up though, so this really isn't worth caring about.
We're using RSpec and Cucumber in our Rails apps with good results. Webrat is great for non-AJAX interactions, but we're getting ready to get back into writing tests for our Javascript.
Webrat has Selenium support built in, and we've used Selenium before, but I'm curious if anyone has had good results using Watir with Cucumber and what the pros and cons are of Watir versus Selenium.
As the founder of OpenQA and Selenium RC, I'm obviously biased towards Selenium as a good option. We recently just put out a 1.0 beta 2 release and are very close to a 1.0 final.
However, you couldn't go wrong with Watir/FireWatir either. Adam's comment that WebDriver will merge to form Selenium 2.0 is correct, but he's incorrect in implying that Watir doesn't use native hooks. If Watir were simply a Selenium clone and also used JavaScript injection, I'd say it wasn't worth looking at.
But because it has native hooks, it can do some things that Selenium currently can't. While it has fewer browsers supported, it goes a bit deeper in the main browser it does support (IE) and lets you control things outside of the page/canvas.
In summary: either is fine, Selenium is great, and if you hang on a little longer with Selenium you'll soon get the best of both worlds with WebDriver/Selenium 2.0.
I am having good results using Cucumber with Celerity through JRuby.
Celerity is a headless browser which wraps around HtmlUnit with a Watir-compatible API and supports JavaScript and AJAX testing.
Being headless makes Celerity faster and easy to integrate within a Continuous Integration build cycle.
Since Celerity is API-compatible with Watir, you can switch between Watir and Celerity fairly easily. There are some caveats, but it's been worth the effort.
I'd say Watir was much slicker, but less useful. It's mostly an IE automation system, with fairly flaky Firefox support. (I'll admit to not having checked out FireWatir in a while, it may have improved.)
There is experimental support for Selenium-RC on Watir. That would definitely be the best of both worlds.
You may want to try WebDriver. It will become part of Selenium in the future. Works with JRuby.
It is different than Selenium, Sahi or Watir in the way that it controls the browser natively, not using JavaScript, and is not affected by many issues other solutions have.
I have tried Watir but not with cucumber. It was for java app. Problems we faced were with was trying to determine page is loaded. We had to scrape the forums a bit before finding the solutions. Otherwise it was fine.
Watir 2.0 will also use WebDriver.
If you would like to use Watir on Rails 3.
You can use 'watir-webdriver-rails' gem (I'm the creator).
Best solution which I found its Env-Js http://github.com/smparkes/env-js
Its like webrat but with JS support
Don't overlook something like QUnit or Jasmine to unit test your javascript. If nothing else it'll cut down on the number of full-stack tests you have to write in cucumber/selenium/capybara/whatever