Mocha supports it.only syntax to reduce the amount of tests to run. In our team, a developer might use it.only temporally within a code base.
Now it happened that such an only test was approved and deployed by accident. This led the build server to happily run only one test and to declare that build a success.
My question now becomes: Is there a way to tell mocha:
It should only allow only on a developer's machine? Can it be make environment aware so that either all tests always run on a build machine or an only test case would also declare the job a failure.
(I know that there are different ways to reduce the amount of tests run by mocha. E.g. within WebStorm one can run a subset of tests within a project without changing code. The scope of the question is to allow for the fact that a developer might use it.only and they shall be free to do so. I want to detect if such a change might sneak its way into the codebase though.)
On your CI, you should run mocha with the forbid-only flag.
--forbid-only causes test marked with only to fail the suite
Depending on your setup it may look like:
mocha --require babel-register --recursive tests/unit --forbid-only
A developer still may chose to run the test suite without the flag.
Related
I want Jenkins to run JavaScript unit tests.
Although I came across different JavaScript unit test frameworks, like Jasmine or Tape, I cannot find any documentation on how to make them work and set-up their output to co-operate with Jenkins.
Can anyone point me to a documentation for Jasmine, Tape and other unit test frameworks, explaining on how to configure them properly to be triggered and evaluated by Jenkins?
There is nothing special to do with regards to Jenkins. You need to install the testing libraries as you normally would on the Jenkins executor nodes, and then you can use them as you normally use them from within your Jenkins job. Note that this may mean calling npm install from within your Jenkins job - this is pretty typical; you would do something similar whether you were using Python virtualenv, Ruby bundler, etc.
Is there an option in Javascript's Mocha test runner to only run tests which failed on the previous run? Is there an easy way to implement that if not? There are a lot of words written about retrying flaky tests but thats not what I want. I want to run tests, see failures, make updates to the code, then automatically run only the previous failed tests to see if my changes fixed them
Found this https://github.com/segmentio/mocha-broken but integration into WebStorm wasn't streamlined enough so dropped it...
Might fit your needs though, worth a try
As the title says, I'd like to be able to able to run Jasmine tests using Mocha on node. As an experiment I've installed Jasmine and Mocha and ran
jasmine examples
to install the examples.
Running Jasmine runs the tests as expected:
$ ./node_modules/jasmine/bin/jasmine.js
Started
.....
5 specs, 0 failures
Finished in 0.012 seconds
But running the tests in Mocha doesn't work:
$ ./node_modules/mocha/bin/mocha spec/jasmine_examples/PlayerSpec.js
0 passing (11ms)
5 failing
1) Player
should be able to play a Song:
ReferenceError: expect is not defined
at Context.<anonymous> (spec\jasmine_examples\PlayerSpec.js:14:5)
etc
Similarly if I try and add a call to jasmine.createSpy() into a test, it works fine under Jasmine but under Mocha it reports
ReferenceError: jasmine is not defined
May be I shouldn't be entirely surprised but as I'm new to this Javascript world could someone explain to me either how to get it working or why it doesn't work?
In case anyone is wondering why I want to do this, as a team we're using Jasmine but I'm using IntelliJ as my IDE. This doesn't understand Jasmine tests so I have to manually create run configurations to run specific tests. If I could get them to run under Mocha, I could use the built-in Mocha support and just click on the little arrows IntelliJ puts next to Mocha tests.
There are superficial resemblances between Jasmine and Mocha (describe, it, etc.) but there are a lot of differences that have to be bridged if you want to have a suite run under both. You cannot take a Jasmine suite and generally expect it to work in Mocha without modifications. Jasmine is not designed to run Mocha tests and Mocha is not designed to run Jasmine tests.
For instance Mocha cannot do anything with jasmine.createSpy(). It does not even have an equivalent for it built into Mocha itself. If you were to port your suite to Mocha (i.e. abandon Jasmine in favor of Mocha), you'd have to use a library like Sinon to provide similar functionality. If you want to have your suite run both in Mocha and Jasmine, then you might be able to bridge the gap with a wrapper library that detects which runner it is running under and calls jasmine.createSpy() or a Sinon equivalent as needed but with any non-trivial test suite the work required would be substantial. (And frankly, there's no project I work on where I could justify the expense.)
You'd also have to use a library like Chai to provide expect.
I am looking for a way to make karma.js crash when my tests fail so that my build process gets interrupted since that is easier to monitor in a remote building server.
I am using mocha as a reporter and jasmine as the test processor engine.
Is there any option or particular variable in the karma.conf.js file maybe that would allow me to "crash on test fail"
There is no need for a "crash on failure".
Your build process should check the exit code of the Karma process that it launches. If all test pass, the code will be 0. If there is a test failure, the code will be non-zero. (I've just checked and it was 1 when I tried.)
You may be able to get a plugin for karma for your build server that will report the results of your tests.
For example, there's a plugin for karma/teamcity (karma-teamcity-reporter) which allows you to fail the build step, preventing the application from publishing if you have it set up that way.
you can use grunt as automation tool to run your karma test case so that whenever any grunt job(test case) failed it will by default stop execution.
I'm writing a small meteor app and simultanously I'm trying to practice TDD. I've run into a small hurdle in that I can't figure out how I can run my Jasmine unit tests through a debugger.
The tests are stored in [project root]/tests/jasmine/client/unit as in the example.
The velocity docs detail how to debug the server-side tests here, but it doesn't mention client-side tests.
I can debug client integration tests; when starting meteor it logs [velocity] jasmine-client-integration is starting a mirror at http://localhost:51259/., and by going there it's possible to debug my tests. But no mirror seems to be created for the client unit tests?
EDIT: in fact it seems like I can debug the client integration tests at localhost:3000 as well. The developer console looks like this:
and by adding a breakpoint in miscSpec.js I can debug my integration tests.
Lastly, I know that Meteor-Jasmine writes the output from console.log in the unit tests to [project root]/.meteor/local/log/jasmine-client-unit.log. That is serviceable, but I would prefer to be able to run the tests through a debugger, since I find it easier to figure out what I did wrong.
I'm using
meteor 1.2.0.2
sanjo:jasmine 0.20.2
As a workaround, I have moved all my client unit tests from [project root]/tests/jasmine/client/unit to [project root]/tests/jasmine/client/integration. There does not seem to make much of a difference whether unit tests are run as unit tests or integration tests. In fact, the Velocity docs has almost the exact same description of the two modes.
It would be nicer to be able to debug the tests in a view where you can also select which tests that should be run, just when running Jasmine normally, but perhaps this is in the works.