We're using RSpec and Cucumber in our Rails apps with good results. Webrat is great for non-AJAX interactions, but we're getting ready to get back into writing tests for our Javascript.
Webrat has Selenium support built in, and we've used Selenium before, but I'm curious if anyone has had good results using Watir with Cucumber and what the pros and cons are of Watir versus Selenium.
As the founder of OpenQA and Selenium RC, I'm obviously biased towards Selenium as a good option. We recently just put out a 1.0 beta 2 release and are very close to a 1.0 final.
However, you couldn't go wrong with Watir/FireWatir either. Adam's comment that WebDriver will merge to form Selenium 2.0 is correct, but he's incorrect in implying that Watir doesn't use native hooks. If Watir were simply a Selenium clone and also used JavaScript injection, I'd say it wasn't worth looking at.
But because it has native hooks, it can do some things that Selenium currently can't. While it has fewer browsers supported, it goes a bit deeper in the main browser it does support (IE) and lets you control things outside of the page/canvas.
In summary: either is fine, Selenium is great, and if you hang on a little longer with Selenium you'll soon get the best of both worlds with WebDriver/Selenium 2.0.
I am having good results using Cucumber with Celerity through JRuby.
Celerity is a headless browser which wraps around HtmlUnit with a Watir-compatible API and supports JavaScript and AJAX testing.
Being headless makes Celerity faster and easy to integrate within a Continuous Integration build cycle.
Since Celerity is API-compatible with Watir, you can switch between Watir and Celerity fairly easily. There are some caveats, but it's been worth the effort.
I'd say Watir was much slicker, but less useful. It's mostly an IE automation system, with fairly flaky Firefox support. (I'll admit to not having checked out FireWatir in a while, it may have improved.)
There is experimental support for Selenium-RC on Watir. That would definitely be the best of both worlds.
You may want to try WebDriver. It will become part of Selenium in the future. Works with JRuby.
It is different than Selenium, Sahi or Watir in the way that it controls the browser natively, not using JavaScript, and is not affected by many issues other solutions have.
I have tried Watir but not with cucumber. It was for java app. Problems we faced were with was trying to determine page is loaded. We had to scrape the forums a bit before finding the solutions. Otherwise it was fine.
Watir 2.0 will also use WebDriver.
If you would like to use Watir on Rails 3.
You can use 'watir-webdriver-rails' gem (I'm the creator).
Best solution which I found its Env-Js http://github.com/smparkes/env-js
Its like webrat but with JS support
Don't overlook something like QUnit or Jasmine to unit test your javascript. If nothing else it'll cut down on the number of full-stack tests you have to write in cucumber/selenium/capybara/whatever
Related
I'm looking for a way to automate some simple testing of a web application. "Clicking x causes y" sort of testing. I want to write tests that can be run in a variety of browsers. I was thinking writing an inject-able script (bookmarklet sort of thing) which invokes click events and tests that elements exists might be a good way to go.
Are there any inherent dangers to this approach or major issues? Are there better alternatives? I don't want to get too involved in creating this test (don't want to create a test server, or go through a lot of setup), I just want to automate some repetitive testing.
As Diodeus said, Selenium is probably the most popular browser automation library right now (I believe Facebook uses it). Other frameworks you may wish to investigate:
Watir
Windmill
Sahi
In addition, you'll want to consider cross-browser testing when setting up an automated suite of tests. You can roll your own for this, or if you'd rather throw money at the problem, BrowserStack now offers an API that allows your tests to run on a range of browsers.
Selenium is pretty popular: http://seleniumhq.org/
I've searched a bit for this and tried to implement a self-made solution but so far haven't found to be confident with it.
What I need is to write integration tests in Ruby on Rails which interact with JavaScript and get programmatic ways to assert some behaviors. I'm using Test::Unit for the controller/models part but I'm struggling to test some jQuery/JavaScript behaviors used by my app. Mainly it consists in ajax calls and interactions in the UI which updates some sets of information.
I haven't found a solution which makes me confident and which integrates nicely with autotest and the whole red-green process, so for now most parts of my client-side code is untested and that's making me nervous (as it should be :P).
So, does anyone have suggestions for best practices on this issue? Unit testing JS is a bit tricky, as Crockford points out, because it dependes heavily on the current state of the UI and etc and as AFAIK even he hasn't found a good way to implement decent testing...
Shortly: I need to implement tests for some UI behavior which depends on Ajax, integrating with autotest or some other CI tool and haven't found a good and elegant way to do it.
Thanks all for the attention,
Best Regards
AFAIK, outside of a combination of Capybara with Selenium Web-Driver there is very few options for automated testing of JS code.
I use cucumber with capybara and selenium web-driver and because selenium-webdriver actually launches firefox or chrome to go through testing a particular page with ajax call, It does take significantly longer to run through a suite of tests.
There are some alternatives but they dont work all the time or for every situations.
For instance: Capybara with envjs
In April 2011 the thoughtbot guys updated their quest for javascript testing.
Akephalos has fallen out of favor for the following reasons:
Bugs: as previously mentioned, there are bugs in htmlunit,
specifically with jQuery’s live. Although all browser implementations
have bugs, it’s more useful if tests experience the same bugs as
actual browsers.
Compatibility: htmlunit doesn’t fully implement the feature set that
modern browsers do. For example, it doesn’t fully handle DOM ranges or
Ajax file uploads.
Rendering: htmlunit doesn’t actually render the page, so tests that
depend on CSS visibility or positioning won’t work.
Performance: when most of your tests use Javascript, test suites with
htmlunit start to crawl. It takes a while to start up a test with
Akephalos, and a large test suite can easily take 10 or 15 minutes.
So they rolled their own solution which is open source - capybara-webkit. It's still fairly new but it looks like the way to go now.
This article recommends Akephalos.
I have used cucumber and capybara with selenium. This was very frustrating because selenium did not seem to be able to see dynamically generated javascript, despite the fact that capybara was supposed to be waiting for it. That was in January 2011. Things may be different now.
Currently, I am using cucumber and capybara with akephalos. So far, it has been very difficult because 1. it is headless, so you can't see progress. Capybara's "save_and_open" call has helped to a degree. 2. jQuery and akephalos don't seem to play that nicely together. For instance, triggering on a radio button with jquery's .change() works fine in chrome, but doesn't in akephalos. Maybe this is intentional because I later heard somewhere that it doesn't work in IE. I fixed the issue by using .click() instead of .change() for the radio button but since the .change function was set up to run on a bunch of questions, I had to code specifically to get it to work for the test.
The bottom line for me is that automated javascript acceptance testing in a rails env is still immature and possibly more work that it is worth.
In our eternal quest to integrate Javascript into our RSpec and Cucumber workflow we're evaluating Culerity and are provisionally very excited about it.
Any advice on Culerity in general or specifically as compared to Selenium?
On documentation alone, I'd go with Selenium:
Selenium documentation.
Culerity documentation.
Does Culerity have a Firefox add-on to record test-scripts (does make things easy)
Which has the better track record?
I see Selenium as being better in these regards right now.
I personally found Culerity to be a quite a bit buggier and more tempermental than Selenium. I had several failing tests that turned out to be issues with Celerity rather than something wrong with the test or app. A lot of time wasted :/
The development experience for the Palm Pre sucks, to say the least. At best you are working on JavaScript in Eclipse with the Aptana plugin. The intellisense you get is really, really dismal. You have to be looking at the documentation every 5 seconds.
I was thinking, Visual Studio really excels at JavaScript and intellisense. Has anyone succeeded at writing Palm Pre code in VS2008 with intellisense, which would require hooking in Palm's Mojo Library?
Is there a known way to hook in 3rd party non-source libraries to JavaScript in Vs2008?
I've been looking for alternatives to Eclipse, too, but so far it does not appear that anyone has done it. For myself it felt like too much work, so I just switched to working with several terminal windows open and a browser window to the documentation.
I'm a long time eclipse user but was disappointed in the eclipse environment for Pre - especially after using the Android SDK which hooks nicely into eclipse (then again the dev environment is Java there).
Have you tried Komodo? I switched over to it and the autocompletion is nice and the webos addon does a decent job. I don't have the professional version so I can't comment on the debugging capability which I'd love to have.
If you can tell VS08 you want to load the Mojo.js stuff, then it should work. I think you should be able to configure any custom tools you need for building etc. as well.
Personally I'd recommend you check out JetBrains' IntelliJ IDEA. They have really good JavaScript support and I've used it with some Pre-stuff as well.
I'd like to write some small applications for Windows and OSX.
Portable is good. For instance, the simple TclKit solution for TCL would work well if I could stand to look at Tcl for any length of time.
I'm considering using Javascript + extensions -- I really like Javascript -- seems to me there should be some way to connect a Javascript engine to WxWin or even the Tk toolset -- it's been done for Perl, Python, Ruby, etc. Why not JS?
Any comments appreciated.
I would rather recommend using Adobe Air execution container, it offers way more capabilities / OS integration features compared to HTA of Microsoft or even XULRunner of Mozilla.
If you want to write a desktop app in Javascript, check out HTA: http://msdn.microsoft.com/en-us/library/ms536496(VS.85).aspx
Here is an example of an app that I wrote a long time ago in HTA: http://www.boltbait.com/htmleditor/
Hope this helps.
If you have experience with HTML+JS web applications, I'd recommend Mozilla XULRunner. It gives you native-looking interface widgets (every piece of Firefox interface is a XUL element), and a workflow similar to building HTML-based interface.
You define the interface using XUL, an XML-based language, so it's almost like writing HTML, minus browser incompatibilities and CSS (you can use CSS, but only if you want). All application logic can be written in JavaScript, but in XULRunner you can do a lot more in JS than in a browser (read/write files, execute system commands, make cross-domain XMLHTTPRequests, and a lot of other stuff).
More: http://xulplanet.com/
A lot of example applications: http://code.google.com/p/xulapps/
Mac OS X users tend to dislike applications that don't follow usability guidelines of the platform (menu must be at top of the screen, standard shortcuts & drag'n'drop must work, "OK" must be at right-hand side of alertboxes, no "Apply" button in prefs and so on).
With one-size-fits-all toolkit this may be very difficult to achieve.
The good news is that there's JavaScript<>Cocoa bridge that can be used to create stand-alone full-featured Mac OS X application in JS – just keep front-end and back-end of your application separate enough to be able to plug different UI for each OS.
Rhino would probably give you access to swing. Though, I don't know how much work would be involved.
On the other hand, building a swing app with JRuby is so easy even a sock puppet can do it, so it shouldn't be too hard.
Definitely if you want to implement it using JavaScript, Adobe Air is the way to go. You will even have support on Linux.
Give Titanium Appceletaror a look: http://www.appcelerator.com/.