I am new to using Jasmine and BDD testing and I am trying to test if a simple page exists.
I have the following code:
describe("when server is running",function () {
it("should serve an index page", function(){
expect(server.app).toBeDefined();
request.get('http://localhost:3000/', function(error, response, body){
expect(response.statusCode).toBeEqual(200);
done();
});
expect(server.app).toBeDefined();
});
});
When the tests are run I get:
Finished in 0.027 seconds
1 test, 2 assertions, 0 failures, 0 skipped
There are obviously 3 assertions and it's the middle one, within the function() call that is not performed (I added the third repetitive one just for checking and commented out each one individually).
I came across the guide at https://semaphoreci.com/community/tutorials/getting-started-with-node-js-and-jasmine and made not of the addition of the done() callback but still no joy.
Is this something wrong with the test, the request or the jasmine-node setup I have.
I'm running this on Ubuntu 14.04.
Thanks.
Related
I have a very basic karma/jasmine setup with one test suite containing two tests. I expect the first test to fail and the second test to pass.
describe("The system", function() {
it("should fail", function() {
expect(true).toBe(false);
});
it("should succeed", function(done) {
setTimeout(function() {
expect(1).toBe(1);
done();
}, 10);
});
});
However, when I run these tests in the browser and click the Debug button to open the Karma DEBUG RUNNER, I see both tests failing, where the second test fails with the error message of the first test. The regular test run (i.e. not in the Karma DEBUG RUNNER) works as expected.
The error message for the second test is:
Uncaught Expected true to be false.
at UserContext.<anonymous> (http://localhost:9876/base/workingspec.js:4:22) thrown
When I disable or remove the first test, the second test passes.
Why do both tests fail in this case? Why does the second test fail with the error message of the first test?
My test setup contains the following packages/versions:
+-- jasmine-core#2.7.0
+-- karma#1.7.0
+-- karma-chrome-launcher#2.2.0
+-- karma-jasmine#1.1.0
`-- karma-jasmine-html-reporter#0.2.2
The problem is indeed in the Debug.js file of the Karma Debug Runner, as #user907860 already hinted at. It is not particular to Jasmine. I have reported the issue and created a fix which has just been merged into the master branch of Karma, so the next release should fix this issue.
At first I thought it was a bug, but after some research with the brilliant Chrome devtools, it seems that this is an expected behavior, at least by Jasmine. It may be, though, a bug in the Karma framework.
In short, the node_modules/karma/static/debug.js file (which is the js-file for the debug page) has these lines (I have Karma v1.7.0):
for (var i = 0; i < result.log.length; i++) {
// Throwing error without losing stack trace
(function (err) {
setTimeout(function () {
throw err
})
})(result.log[i])
}
If you comment the throw line, and restart the karma server, you'll see only console log messages, which should be expected: first FAIL, then PASS, and then the summary.
Verbosely, the bug in Karma may be in it's behavior to report after each spec.
That is what's happening here step by step (I have version "jasmine-core": "^2.6.4", at least this is in my package.json file):
Jasmine runs the first spec and it fails;
Karma reports on this in the log and adds a function, which throws an error to the stack (I assume the reader knows the asynchronous model in JavaScript, if not, then he must read about this in something like
"Effective JavaScript: 68 Specific Ways to Harness the Power of JavaScript" by David Herman, a true gemstone book, or elsewhere). Also, this is important, though I'm not sure, since I haven't look into the code so deep, it registers sort of "globalError", since on the next step
Jasmine runs the second spec, and calls the getJasmineRequireObj().GlobalErrors function (jasmine.js:2204). The "globalError" is detected and the spec instantly becomes a failure. The asynchronous expect is added to the stack after Karma's function, the one, which throws the error
Then the first function (which throws the error) starts execution, add it throws. This function will always be called before the Jasmine asynchronous spec, since in the debug.js there is no time passed to the setTimeout call:
//notice the absent time argument
setTimeout(function () {
throw err
})
Jasmine runs the assync expect from the second spec and it passes
Karma reports on the second spec as a failure and adds the error throwing (if not commented out, then no error occurs and this spec passes) to the stack
Karma's second error is thrown
Below are screen shots with numbers, put by me to illustrate the steps from the list:
with throw not commented in the debug.js file:
and with the commented throw:
I've tried to put the descriptive error message and exit the test run via fail() function but only with a partial success - it appears that I am doing something wrong... Here is my code:
it('set up internal budget', function(done) {
var acceptBudgetButton = element(by.buttonText('Accept Budget'));
page.setInternalBudget(); //setting budget values
browser.wait(function() {
return browser.isElementPresent(acceptBudgetButton);
}, 30000, 'Error - Accept Budget button is not visible.');
acceptBudgetButton.click();
done();
done.fail('Unable to setup internal budget. Terminating test run');
});
When "Accept Budget" button is not available I expected 3 things from this script:
1) See "Accept Budget button is not visible" error, followed by wait time out
2) See "Unable to setup internal budget. Terminating test run" error
3) Expect protractor to immediately exist test run, as we got into failure.
In the reality, only first expectation is met. Script keeps on running and I don't even get "Unable to setup internal budget" error... so it looks like my done.fail statement is being ignored.
What should I change to make all of my dreams/expectations come true?
Yes, nothing would be executed after done. What I would do is to define the browser.wait()'s success and failure handlers. Something along these lines:
browser.wait(function() {
return browser.isElementPresent(acceptBudgetButton);
}, 30000).then(
function () {
acceptBudgetButton.click();
done();
},
function () {
console.log('Error - Accept Budget button is not visible.');
done.fail('Unable to setup internal budget. Terminating test run');
});
Is there a way to get QUnit.js to not run the remaining tests after a single one fails?
Using the following code as an example:
QUnit.test('test1', function(assert) {
assert.equal(1,1);
assert.equal(1,2);
assert.equal(3,3);
});
QUnit.test('test2', function(assert) {
assert.equal(4,4);
assert.equal(5,5);
assert.equal(6,6);
});
Is there some way to get QUnit to stop executing after the assert.equal(1,2)? This means that test2 should never be run.
The best way to stop QUnit after test case fail will be
QUnit.testDone( function( details ) {
if (details.failed>0){
QUnit.config.queue.length = 0;
}
});
Okay, based on my comments above I ran the code below and things to stop as I think you want them to. Again, as I said in the comments, I would really investigate whether this is a good idea. Generally you want your tests to be idempotent such that any one failure does not affect any other test.
Note that we have to set the reorder config option to false here, otherwise QUnit will attempt to run the previously failed test first to "short circuit" things, but you don't want that I'm guessing. I also added a "test0" just to see the fill effect.
QUnit.config.reorder = false;
// This is how we detect the failure and cancel the rest of the tests...
QUnit.testDone(function(details) {
console.log(details);
if (details.name === 'test1' && details.failed) {
throw new Error('Cannot proceed because of failure in test1!');
}
});
QUnit.test('test0', function(assert) {
assert.equal(1,1);
assert.equal(2,2);
assert.equal(3,3);
});
QUnit.test('test1', function(assert) {
assert.equal(1,1);
assert.equal(1,2);
assert.equal(3,3);
});
QUnit.test('test2', function(assert) {
assert.equal(4,4);
assert.equal(5,5);
assert.equal(6,6);
});
You won't get any visual feedback that the tests were canceled because this isn't really interacting with the QUnit UI. However, because we threw an Error object you can open the developer console and see the output there:
I am creating HTTP tests with frisby.js which works on top of jasmine.js.
I also have to create some mongoDB objects to test against.
The problem is when I want to clean up these DB objects. When one of the expects fail I want to intercept that and call my own cleanup function. This means that after each failed test, I won't be able to remove the test objects from the DB.
The afterEach function in jasmine does not work properly and jasmine does not have any support for afterAll or beforeAll yet.
That is why I have made the tests as they are today.
it("testing userform get with correct userID and expect correct return", function() {
var innerUserId = userID;
frisby.create('Should retrieve correct userform and return 200 when using a valid userID')
.get(url.urlify('/api/userform', {id: innerUserId}))
.expectStatus(200)
.afterJSON(function(userform){
// If any of these fail, the after function wont run.
// I want to intercept the error so that I can make sure that the cleanUp function is called
// afterEach does not work. I have tried with done()
var useridJSON = userform.UserId.valueOf();
var firstnameJSON = userform.firstname.valueOf();
var surnameJSON = userform.surname.valueOf();
expect(firstnameJSON).toMatch(testUser.firstName);
expect(surnameJSON).toMatch(testUser.surname);
expect(useridJSON).toMatch(innerUserId);
})
.after(function(){
cleanUp(innerUserId);
})
.toss();
});
I am wondering if there is a way to intercept the error for "expect" in frisby or jasmine so that I can make a call to my own cleanup function before exiting.
Full example here
The quickest solution to this problem is to wrap the error code in a try-catch.
This is because if a javascript error occurs, jasmine will NOT keep running assertions. This is different from an assertion error. If an assertion error occurs, jasmine and frisby will keep on testing all the other assertions and then do the "after"-function.
.afterJSON(function(userform){
try {
var useridJSON = userform.UserId.valueOf();
var firstnameJSON = userform.firstname.valueOf();
var surnameJSON = userform.surname.valueOf();
catch(e) {
cleanUp(innerUserId);
// Can do a throw(e.message); here aswell
}
expect(firstnameJSON).toMatch(testUser.firstName);
expect(surnameJSON).toMatch(testUser.surname);
expect(useridJSON).toMatch(innerUserId);
})
This is not the pretty way, but works.
I ended up adding the throw(e) and placed the expects in a finally scope. This way I got jasmine to present all the errors that occured in the test.
As for "before exiting", how about this:
process.on('uncaughtException', function(err) {
console.error(' Caught exception: ' + err);
});
Update: I figured out what the issue was, see my comment below.
Is there a way to guarantee state before each Jasmine test?
For example:
describe('some thing', function () {
beforeEach(function () {
doSetup();
// this expect does not evaluate :(
expect(something).toBe(inSomeState);
});
it('has some behavior', function () {
// test code
});
});
The expect inside of the setup make no difference at all. Even throwing an error in the beforeEach does nothing. I would like to have some assurance that the setup has completed correctly before running the next test.. is there a way to do this?
Okay I see what the issue is. Hoping this will help other people out there having this same issue. grunt-contrib-jasmine does not display errors or failed expects inside of beforeEach or afterEach with the display option set to short. Setting this option to full will once again display Errors and failed expects.