Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
Few questions:
How Karma and testing framework X (Jasmine, Mocha, QUnit) relate to each other?
What is the equivalent framework at Java world? I assume Jasmine, Mocha, QUnit equal to jUnit/TestNG. How about Karma?
Can I run testing framework X (e.g. Jasmine) without Karma?
Is Karma for unit test or integration/e2e test? This reference shows is for unit test, however this said is for e2e test.
Karma is a browser test runner.
The idea is that browsers do not have natively a concept of loading tests files, running them, and reporting results.
What karma does is (roughly) :
starting a small web server to serve "client-side" javascript files to be tested (1)
also serve the "client-side" javascript files with the tests (or Specs, as they're often called) (2)
serve a custom web page that will run javascript code for the tests (3)
start a browser to load this page (4)
report the results of the test to the server (5)
karma can then again report the results to text files, the console, anything your CI server likes, etc...
Looking at each part :
(1) Those files will be your actual js files ; you will tell karma how to load them. If you use requirejs, there is a karma plugin, and some config is needed.
(2) Those tests can be written in a variety of Javascript testing framework (Jasmine, QUnit, Mocha) ; this is JS code that is run in the browser.
(3) The custom web page will be a bit different for each testing framework ; this is why karma has plugins for different frameworks.
(4) Karma can launch the page in many browsers (FF, Chrome, or headless browsers like PhantomJs.)
(5) Reporting to karma is, again, framework-dependant, and dealt with karma plugins.
So to answer your questions :
in Java, most people use JUnit which is both a framework to write tests and run them, but does not have the problem of differentiating the environment in which tests are run and the one in which test reports are aggregated ; karma would be the missing piece between a JUnit Suite and a JUnit TestRunner
Yes, you can do everything that karma does "by hand" - pick one framework (jasmine, qunit, mocha) and follow instructions. The advantage of karma is that it provide a solution out-of-the-box, if you're in a standard setup.
Karma can be used for both unit test (with jasmine / qunit / whatever) and integration tests (which will use another API, like webdriver, to drive the browser)
One shorter way to understand the difference:
People testing with plain Jasmine / Mocha most likely are running all the code with the Node virtual machine.
Adding Karma to the mix (on top of your existing framework of choice) will run your test suite with the engine of other browsers.
By doing this you get the little extras you get with a browser environment. It will be easier to test DOM related code, but you will also be giving up to the extra resources given by the Node engine (like filesystem / shell access)
The thesis of the guy who designed Karma was very informative in describing existing solutions and comparing them, and of course describing Karma itself
https://github.com/karma-runner/karma/blob/master/thesis.pdf
Summary: Karma is a test runner. It can be used by QUnit, Jasmine, Mocha, ... Karma has advantages to other test runners to improve your TDD/BDD development cycle. It "watches" files, so when you save a change, Karma runs tests and reports back instantly, no switching context to Web Browser to run the test.
In short, perhaps question should be Karma AND Jasmine or Mocha or QUnit?
Related
Until now, I've done all my testing with just Jasmine and jQuery to automate the browser. I am starting to explore Karma but it seems to be set up only for unit tests that involve loading HTML fragments as test fixtures and I find no way to load a whole web page. Am I missing something or is this simply the wrong tool for what I am trying to do?
From the karma FAQ's section:
Can I use Karma to do end to end testing?
Karma has primarily been designed for low level (unit) testing. If it's an AngularJS app, you can use Karma with the karma-ng-scenario plugin. However, we recommend Protractor for high-level testing.
Typically, loading a whole web page goes beyond the scope of single units of code: you need to spin up a browser, load the page and make your actions and assertions. End-to-end testing frameworks like Protractor, Nightwatch.js or Webdriver.io might be more suitable for this specific case.
You can do integration testing using the following toolchain:
Karma ➔ Mocha ➔ Mocha-CasperJS ➔ CasperJS.
Check out the mocha-casperjs package.
On the Protractor site (http://www.protractortest.org/#/infrastructure), it says Protractor uses Selenium for browser automation. Then on the Grunt site (http://gruntjs.com/) it says Grunt is also for automation. If they are both for automation, I'm not sure what are the differences between the two and how they fit together. In some ways, my question is this: Why would I need Grunt if I'm using Protractor?
"Automation" itself is quite a broad term.
These are completely different tools:
protractor is a wrapper around WebDriverJS and used for end-to-end testing
grunt is a javascript task runner
You can actually run protractor tests via grunt.
I would like to end-to-end test my Meteor/React app in multiple browsers.
Karma is great for automating tests in multiple browsers, but I don't see any config options to make it load the page to test from my own server, only the option to specify which JS files contain tests and support code.
This is obviously because Karma injects its own test harness HTML and JS. But to end-to-end test my page, I would need it to load the page from Meteor instead of Karma's own embedded server, since Meteor injects a lot of its own JS dependencies into the page.
Bundling the tests into the code served up by Meteor is easy. The problem is how to tell Karma to load a specific URL, and to be able to put script tags or whatever to load and run the Karma test harness into my own HTML code.
I believe it may have been possible with the Velocity test runner for Meteor, but I found it horribly slow and wanted to implement end-to-end testing without it.
Does anyone know if this is possible with Karma or another framework?
I have got javascript tests running in visual studio using Resharper and Jasmine. It all works fine. Now however I want to run an integration test. After looking into this I just can't work out whether I should be using Karma or PhantomJs or both? And whether they link to resharper?
I then have to run all my tests on the continuous integration server using Team City. Doesn't seem like there is a clear logical way to architect front end testing from dev right through to the CI server.
For example do I really need to use Karma locally and on the CI server? I don't want to because it is not baked into visual studio and I will have to keep opening the command line and running my tests manually. This will annoy other developers and they won't bother running them I imagine. Help!
We are using Team City by the way....
Neither of these test runners are connected to Resharper in any form. I don't think there is a one and only "correct" way of running Jasmine tests in TeamCity currently - for all approaches I am aware of there is a bit of assembly required.
Personally I am using a combination of PhantomJS and the Jasmine TeamCity Reporter which works like this:
Get a list of unique test URLs from server - each of these test suites will use the Jasmine TeamCity Reporter to log results
for each test URL dynamically generate a test JS file to be used for phantom and write it to disk
use PhantomJS to run the JS file which now will load the test suite page and its test result will be written to console where they are picked up by TeamCity.
I've been writing JavaScript unit tests using Jasmine. However, those tests run inside a browser, not as part of MSTest.
I want my TFS Continuous Integration builds to break when a JavaScript unit test fails. I know there is a solution for this in Visual Studio 2012, but I'm on 2010 (and will be for a long time in the future probably).
Is there an easy way to integrate Jasmine based JavaScript unit tests with TFS Build?
The Chutzpah test runner enables you to run your QUnit and Jasmine JavaScript unit tests from the command line or Visual Studio.
Therefore you should be able to integrate it into your TFS build via the InvokeProcess activity.
I'd use nodejs + jasmine-node, but you can also look at http://phantomjs.org/ with a junit reporter. Jasmine-node provides a junit reporter with the --junitreport output. You might also look at the TFS Build Extensions (http://visualstudiogallery.msdn.microsoft.com/2d7c8577-54b8-47ce-82a5-8649f579dcb6/view/Discussions/) and the activities in there. It includes a xml transform for JUnit to TRX (test results xml) which can be pushed into TFS.