If I have code in a test that should never be reached (for example the fail clause of a promise sequence), how can I force-fail the test?
I use something like expect(true).toBe(false); but this is not pretty.
The alternative is waiting for the test to timeout, which I want to avoid (because it is slow).
Jasmine provides a global method fail(), which can be used inside spec blocks it() and also allows to use custom error message:
it('should finish successfully', function (done) {
MyService.getNumber()
.success(function (number) {
expect(number).toBe(2);
done();
})
.fail(function (err) {
fail('Unwanted code branch');
});
});
This is built-in Jasmine functionality and it works fine everywhere in comparison with the 'error' method I've mentioned below.
Before update:
You can throw an error from that code branch, it will fail a spec immediately and you'll be able to provide custom error message:
it('should finish successfully', function (done) {
MyService.getNumber()
.success(function (number) {
expect(number).toBe(2);
done();
})
.fail(function (err) {
throw new Error('Unwanted code branch');
});
});
But you should be careful, if you want to throw an error from Promise success handler then(), because the error will be swallowed in there and will never come up. Also you should be aware of the possible error handlers in your app, which might catch this error inside your app, so as a result it won't be able to fail a test.
Thanks TrueWill for bringing my attention to this solution. If you are testing functions that return promises, then you should use the done in the it(). And instead of calling fail() you should call done.fail(). See Jasmine documentation.
Here is an example
describe('initialize', () => {
// Initialize my component using a MOCK axios
let axios = jasmine.createSpyObj<any>('axios', ['get', 'post', 'put', 'delete']);
let mycomponent = new MyComponent(axios);
it('should load the data', done => {
axios.get.and.returnValues(Promise.resolve({ data: dummyList }));
mycomponent.initialize().then(() => {
expect(mycomponent.dataList.length).toEqual(4);
done();
}, done.fail); // <=== NOTICE
});
});
Related
I'm using supertest, chai and mocha to test my Web API application. I have the following code:
it('should return 500', function(done) {
this.timeout(30000);
request(server)
.get('/some/path')
.expect(500)
.end(function(err, res) {
done();
});
});
It should fail. The code which runs in the get request is:
// Inside method getData
try {
// Get data
// throws error
} catch (e) {
// catches error and displays it
deferred.reject(e);
return deferred.promise;
}
// Other code not in getData method
dbOps.params.getData(collection, parameter, query).then(
function (arr) {
response.send(arr);
}, function (err) {
logger.error(err.message);
response.status(500).send(err.message);
}
);
It basically does deferred.reject(e); and sends the error as the response of the API. I would like to catch the deferred.reject(e); part and in the same time continue the chain of .except(500).end(...). Something like:
catch_deferred(request(server).get('/some/path'))
.expect(500)
.end(function(err, res) {
expect(err).to.equal(null);
expect(res.body).to.be.an('object').that.is.empty;
done();
Is there some way to do it? I can't use the try-catch block because its not an exception. Also I can't chai's expect().to.throw() because there is not exception being thrown.
Disclaimer: I never use deferred.reject().
Possible solution: use sinon.
You can use spy, if you want the real deferred.reject() to run and you just want to monitor it. Monitor means: to know whether the method get called, with what argument, and also the return value. Example:
// Preparation: (Disclaimer: I never use Deferred.reject())
const spyDeferredReject = sinon.spy(Deferred, 'reject');
// Do stuff..
// Expectation phase: check whether deferred reject get called.
expect(spyDeferredReject.calledOnce).to.equal(true);
// Check whether deferred reject called with correct argument.
expect(spyDeferredReject.args[0][0]).to.be.an('error');
You can use stub, if you want to monitor it and make sure the real method not get called.
Hope this helps.
I know I could throw an error from inside the test, but I wonder if there is something like the global fail() method provided by Jasmine?
Jest actually uses Jasmine, so you can use fail just like before.
Sample call:
fail('it should not reach here');
Here's the definition from the TypeScript declaration file for Jest:
declare function fail(error?: any): never;
If you know a particular call should fail you can use expect.
expect(() => functionExpectedToThrow(param1)).toThrow();
// or to test a specific error use
expect(() => functionExpectedToThrow(param1)).toThrowError();
See Jest docs for details on passing in a string, regex, or an Error object to test the expected error in the toThrowError method.
For an async call use .rejects
// returning the call
return expect(asyncFunctionExpectedToThrow(param1))
.rejects();
// or to specify the error message
// .rejects.toEqual('error message');
With async/await you need to mark the test function with async
it('should fail when calling functionX', async () => {
await expect(asyncFunctionExpectedToThrow(param1))
.rejects();
// or to specify the error message
// .rejects.toEqual('error message');
}
See documentation on .rejects and in the tutorial.
Also please note that the Jasmine fail function may be removed in a future version of Jest, see Yohan Dahmani's comment. You may start using the expect method above or do a find and replace fail with throw new Error('it should not reach here'); as mentioned in other answers. If you prefer the conciseness and readability of fail you could always create your own function if the Jasmine one gets removed from Jest.
function fail(message) {
throw new Error(message);
}
You can do it by throwing an error. For example:
test('Obi-Wan Kenobi', () => {
throw new Error('I have failed you, Anakin')
})
Copy/pasta failing test:
it('This test will fail', done => {
done.fail(new Error('This is the error'))
})
Here are certain scenarios where some of the answers won't work. In a world of async-await, it is quite common to have try-catch logic like so.
try {
await someOperation();
} catch (error) {
expect(error.message).toBe('something');
}
Now imagine if someOperation() somehow passed, but you were expecting it to fail, then this test will still pass because it never went to the catch block. So what we want is to make sure that the test fails if someOperation does not throw an error.
So now let's see which solutions will work and which won't.
Accepted answer won't work here because the throw will be catched again.
try {
await someOperation();
throw new Error('I have failed you, Anakin');
} catch (error) {
console.log('It came here, and so will pass!');
}
The answer with true === false also won't work because, assertions too throw an error like above which will be catched.
try {
await someOperation();
expect(true).toBe(false); // This throws an error which will be catched.
} catch (error) {
console.log('It came here, and so will pass!');
}
The one solution that DOES WORK (as shown in #WhatWouldBeCool's answer) for this case is below. Now it explicitly fails the test.
try {
await someOperation();
fail('It should not have come here!')
} catch (error) {
console.log('It never came here!');
}
Update May-2022
The fail() function is not officially supported by Jest anymore. Instead, you can do a couple of things to fail explicitly.
Method-1
You can wrap your promise function within expect and tell jest the function should reject with the given error. If the someOperation() somehow passes, jest will throw an error. If the someOperation() fails for any other reason other than the one you specified, it will throw an error. There are also different methods other than toThrowError() that you can use.
await expect(someOperation()).rejects.toThrowError('error!')
Method-2
You can declare explicitly how many assertions you expect in your test. If that doesn't match because someOperation() never failed, jest would throw an error.
expect.assertions(1)
try {
await someOperation();
} catch (error) {
expect(error.message).toBe('something');
}
Dont think there is, discussed here: https://github.com/facebook/jest/issues/2129
A lot of good ideas here. Only to add extra info about testing async code which may lead to trying to make Jest explicitly fail, check the docs for Testing Asynchronous Code https://jestjs.io/docs/en/asynchronous
To test a function that returns a Promise that resolves, it's important to return the Promise, so Jest knows that the test is done only when the Promise is resolved or it'll time out:
test('the data is peanut butter', () => {
return fetchData().then(data => {
expect(data).toBe('peanut butter')
})
})
To test a function that returns a Promise that rejects, it's important to return the Promise, so Jest knows that the test is done only when the Promise is rejected or it'll time out. And also have to say how many assertions Jest needs to count or it won't fail if the Promise is resolved - which is wrong in this case -:
test('the fetch fails with an error', () => {
expect.assertions(1)
return fetchData().catch(e => expect(e).toMatch('some specific error'))
})
You can always do something like this :)
expect(true).toBe(false);
The done callback passed to every test will throw an error if you pass a string to it.
for instance
it('should error if the promise fails', async (done) => {
try {
const result = await randomFunction();
expect(result).toBe(true);
done();
} catch (e) {
done('it should not be able to get here');
}
});
In this following code if the randomFunction throws an error it will be caught in the catch and with auto fail due to the string being passed to done.
Add jest-fail-on-console npm package, then on your jest.config.js
import failOnConsole from 'jest-fail-on-console'
failOnConsole();
This will fail a test once there is a console error or warning done by jest because of an error or warning thrown in the test item.
I just ran into this one, and after some digging, I found the root of the issue.
Jest, since its inception, has been compatible with Jasmine. Jasmine provided a fail function for programmatically fail the test. This is very useful for cases where throwing an error would cause the test to pass incorrectly (overly-simplified example, but hopefully illustrates the use-case):
function alwaysThrows() {
throw new Error();
}
describe('alwaysThrows', () => {
it('should throw', () => {
try {
alwaysThrows();
// here if there is nothing to force a failure, your
// test could "pass" as there are no failed expectations
// even though no error was thrown. If you just put the
// following to prevent that, you actually force the test
// to always pass:
throw new Error('it should have failed');
// that's why instead you use Jasmine's `fail(reason)` function:
fail('it should have failed');
} catch(err) {
expect(err).toBeDefined();
}
});
)
});
So, what has happened is this:
originally Jest did have a fail() function defined, because its default test runner was jest-jasmine2, which provided fail().
In Jest version 27 (or thereabouts), Jest replaced jest-jasmine2 with jest-circus as the default test runner. jest-circus does not implement a fail() function. This was reported as a bug on July 28th 2021: https://github.com/facebook/jest/issues/11698
Jest's type definitions (maintained in DefinitelyTyped) did not remove the fail() function, so autocompletion and the TypeScript compiler still think that it exists and can be used. There is an issue going on in DefinitelyTyped as well: https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/55803
The issue with this thread is that they have decided not to remove it from the type definitions as it is marked as a "regression" in the Jest repository. Unfortunately, the Jest repository's thread has no official response about whether or not they will support this in the future, so the type definitions are in limbo.
So, long story short, Jest doesn't support fail() by default, but knowing that it's a matter of the default task runner, you can restore the fail() functionality by telling Jest to use the jest-jasmine2 runner instead of the default jest-circus runner:
npm i -D jest-jasmine2
configure the Jest config:
module.exports = {
testRunner: "jest-jasmine2"
};
P.S.: usually there is a better way than try/catch to account for errors in your actual test cases. You can see an example of different ways to handle errors without requiring try/catch in both synchronous and asynchronous contexts here: https://gist.github.com/joeskeen/d9c053b947e5e7462e8d978286311e83
You can throw an error simulating an error thrown by the application and then expect its message to be different from what it actually is.
try {
await somthingYouExpectToFail();
throw new Error("Fail!");
} catch (error) {
expect(error.message).not.toBe("Fail!");
}
I suspect based on other responses (like Jasmine: test a setTimeout function throws an error and How to test a function which has a setTimeout with jasmine?) that there's no good solution here, but I want to figure out how to better deal with the second test below:
describe('Tests', function () {
it('succeed', function () { expect(true).toBeTruthy(); });
it('have problems', function(done) {
setTimeout(function() { console.log(undefined.foo); });
setTimeout(done, 5);
});
it('fail', function() { fail; });
it('also fail', function() { fail; });
});
Jasmine's current behavior here is to run the first test and then bail out when the rogue exception-causing setTimeout is encountered in the second test; the last two failing specs never run.
My use case doesn't look like this, of course! The asynchronous error is happening somewhere down a call stack, over a river, and through the woods. And it's obviously an error that it happens and isn't caught! But it's unfortunate if such errors will always terminate Jasmine itself, rather than causing a test case failure.
I believe you want try...catch or promises alongside Jasmine's done.fail().
Promises:
it('gets some huge file', done => {
getFile('http://movies.com/holymountain.mp4')
.then(res => {
expect(res.status).toBe(200)
done()
})
.catch(done.fail)
})
Try/catch
it('gets some huge file', done => {
try {
getFile('http://movies.com/holymountain.mp4')
expect(getFile.status).toBe(200)
done()
} catch (e) {
// console.log(e) // ???
done.fail()
}
})
Issue for reference
I'm having a hard time getting meaningful failures in tests when I need to check things in a promise.
That's because most testing frameworks use throw when an assertion fails, but those are absorbed by the then of promises...
For example, in the following I'd like Mocha to tell me that 'hello' isn't equal to 'world'...
Promise.resolve(42).then(function() {
"hello".should.equal("world")
})
Complete fiddle here
With Mocha we can officially return the promise, but this swallows completely the error and is thus much worse...
Note: I'm using mocha and expect.js (as I want to be compatible with IE8)
With Mocha we can officially return the promise, but this swallows completely the error and is thus much worse...
In your fiddle, you are using Mocha 1.9 which dates from April 2013, and did not support returning promises from tests. If I upgrade your fiddle to the latest Mocha, it works just fine.
This is less of an answer rather than a suggestion? Using the before hook would be useful here.
describe('my promise', () => {
let result;
let err;
before(done => {
someAsync()
.then(res => result = res)
.then(done)
.catch(e => {
err = e;
done();
});
});
it('should reject error', () => {
err.should.not.be.undefined(); // I use chai so I'm not familiar with should-esque api
assert.includes(err.stack, 'this particular method should throw')
});
});
You could also use sinon to make synchronous mocks and then use whatever should.throw functionality your assertion library provides.
To test a failing Promise, do this:
it('gives unusable error message - async', function(done){
// Set up something that will lead to a rejected promise.
var test = Promise.reject(new Error('Should error'));
test
.then(function () {
done('Expected promise to reject');
})
.catch(function (err) {
assert.equal(err.message, 'Should error', 'should be the error I expect');
done();
})
// Just in case we missed something.
.catch(done);
});
I want to take advantage of Mocha's built in promise support, but i'm having difficulty dealing with false positives when I want to test catch branches in my promise chains.
This question gets closest to what I want, but the solution requires every developer on my project to add a then that will throw an error, and ensure that that error doesn't accidentally pass the test.
Because of this, i've had to revert to using the done style of tests, which relies on the built in timeout to catch errors instead of an assertion. This is less than ideal, but removes the chance of false positives.
var RSVP = require('RSVP');
function request (shouldSucceed) {
if (shouldSucceed) {
return RSVP.Promise.resolve('success');
} else {
return RSVP.Promise.reject(new Error('failure'));
}
}
describe('request', function () {
it('throws an error if shouldSucceed is not provided', function () {
// this test is incorrectly calling `request`
// which leads to it passing without actually testing
// the behaviour it wants to
return request(true)
.catch(function (err) {
expect(err).to.be.an.instanceof(Error)
});
});
it('throws an error if shouldSucced is not provided (manual then block)', function () {
// this test tacks a `then` onto the chain, to ensure `request`
// actually throws an error as expected
// unfortunately, it still passes since the assertion needs to do
// a little extra work to ensure the type of error thrown is the
// expected error
return request(true)
.then(function () {
throw new Error('Not expected');
})
.catch(function (err) {
expect(err).to.be.an.instanceof(Error)
});
});
it('throws an error if shouldSucceed is not provided (done syntax)', function (done) {
// this assertion fails (as it should)
// due to a timeout
return request(true)
.catch(function () {
expect(err).to.be.an.instanceof(Error);
done()
});
});
});
Output:
request
✓ throws an error if shouldSucceed is not provided
✓ throws an error if shouldSucced is not provided (manual then block)
1) throws an error if shouldSucceed is not provided (done syntax)
2 passing (2s)
1 failing
1) request throws an error if shouldSucceed is not provided (done syntax):
Error: timeout of 2000ms exceeded. Ensure the done() callback is being called in this test.
Is there a cleaner way to tell mocha that I am expecting something to happen in a catch block, and that a successful resolution of the promise should be a test failure?
You're looking for
it('request(false) should throw an error', function () {
return request(false).then(function() {
throw new Error('unexpected success');
}, function(err) {
expect(err).to.be.an.instanceof(Error)
});
});
See When is .then(success, fail) considered an antipattern for promises? for an explanation of the difference to the code that you currently have.