How does Mocha know about test failure in an asynchronous test? - javascript

I am trying to understand how the asynchronous code for Mocha (at http://mochajs.org/#getting-started) works.
describe('User', function() {
describe('#save()', function() {
it('should save without error', function(done) {
var user = new User('Luna');
user.save(function(err) {
if (err) throw err;
done();
});
});
});
});
I want to know how Mocha decides whether a test has succeeded or failed behind the scenes.
I can understand from the above code that user.save() being asynchronous would return immediately. So Mocha would not decide if the test has succeeded or failed after it executes it(). When user.save() ends up calling done() successfully, that's when Mocha would consider it to be a successful test.
I cannot understand how it Mocha would ever come to know about a test failure in the above case. Say, user.save() calls its callback with the err argument set, then the callback throws an error. None of Mocha's function was called in this case. Then how would Mocha know that an error occurred in the callback?

Mocha is able to detect failures that prevent calling the callback or returning a promise because it uses process.on('uncaughtException', ...); to detect exceptions which are not caught. Since it runs all tests serially, it always knows to which test an uncaught exception belongs. (Sometimes people are confused by this: telling Mocha a test is asynchronous does not mean Mocha will run it in parallel with other tests. It just tells Mocha it should wait for a callback or a promise.)
Unless there is something that intervenes to swallow exceptions, Mocha will know that the test failed and will report the error as soon as it detects it. Here is an illustration. The first test fails due to a generic exception thrown. The 2nd one fails due to an expect check that failed. It also raises an unhandled exception.
var chai = require("chai");
var expect = chai.expect;
it("failing test", function (done) {
setTimeout(function () {
throw new Error("pow!");
done();
}, 1000);
});
it("failing expect", function (done) {
setTimeout(function () {
expect(1).to.equal(2);
done();
}, 1000);
});
This is the output on my console:
1) failing test
2) failing expect
0 passing (2s)
2 failing
1) failing test:
Uncaught Error: pow!
at null._onTimeout (test.js:6:15)
2) failing expect:
Uncaught AssertionError: expected 1 to equal 2
+ expected - actual
-1
+2
at null._onTimeout (test.js:13:22)
The stack traces point to the correct code lines. If the exceptions happened deeper, the stack would be fuller.
When Mocha cannot report what went wrong exactly, that's usually because there is intervening code that swallows the exception that was raised. Or when you use promises the problem may be that someone forgot to call a method that indicates whether the promise is supposed to be completely processed and unhandled exceptions should be thrown. (How you do this depends on the promise implementation you use.)

It won't, it's a shame. It has no way to know that your callback is executing. It's an easier way to do asynchronous testing, where you just tell the test when you are finished. The downside, as you have noticed, is that errors in asynchronous callbacks won't be detected. Nevermind, Mocha hooks to process.on('uncaughtException',...) as mentioned by Louis. Note that if you use done instead of waitsFor and runs in jasmine, then you will have the problem.
Other frameworks like js-test-driver force to you wrap callbacks so the the testing framework can put a try catch around your callbacks (and you don't need to call done). Your test would look like the following:
var AsynchronousTest = AsyncTestCase('User');
AsynchronousTest.prototype.testSave = function(queue) {
queue.call('Saving user', function(callbacks) {
var user = new User('Luna');
user.save(callbacks.add(function(err) {
if (err) throw err;
// Run some asserts
}));
});
};

Related

Why Jest passed all test but still throw an error

I added a feature to my code, but when I run all the test, all the test runs successfully, but Jest still throws an internal error with this message:
Ran all test suites.
the_path\node_modules\expect\build\index.js:303
throw error;
^
JestAssertionError: expect(received).toBe(expected) // Object.is equality
Expected: "node-user-settings"
Received: "null"
I created another .js file and called the code I wanted to test with Jest and it runs properly.
Now, I don't understand why Jest throws an internal error.
This is the test that generates the error.
test("asynchronously gets a value, using callbacks without optional filename", () => {
settings.getState_c("moduleName", null, null, (err, data) => {
expect(err).toBe(null);
expect(data).toBe("node-user-settings");
});
});
I think you need to tell Jest that you're testing an asynchronous function: https://jestjs.io/docs/asynchronous.
Use done instead of async if your function does not return a Promise, but is asynchronous.

promise & mocha: done() in before or not?

I am reading some tutorials on promise tests in mocha. There is a piece of codes:
before(function(done) {
return Promise.resolve(save(article)).then(function() {
done();
});
});
Why done() called in the then() in the before()? What is the difference between the above codes and the following codes:
before(function(done) {
return Promise.resolve(save(article));
});
Thanks
UPDATE
My question is to compare with the following codes:
before(function() {
return Promise.resolve(save(article));
});
Sorry for the typo.
The first code snippet with the before hook returns a promise and calls done. In Mocha 3.x and over, it will result in this error:
Error: Resolution method is overspecified. Specify a callback *or* return a Promise; not both.
It used to be that it did not particularly matter if you used done and returned a promise, but eventually the Mocha devs figured that specifying both done and returning a promise just meant the test designer made a mistake and it was better to have Mocha pitch a fit rather than silently allow it.
In your 2nd snippet, you have the done argument and return a promise but Mocha will still wait for done to be called and will timeout. (It really should detect the argument and raise an error like in the 1st case, but it doesn't...)
Generally, if you are testing an asynchronous operation that produces a promise, it is simpler to return the promise than use done. Here's an example illustrating the problem:
const assert = require("assert");
// This will result in a test timeout rather than give a nice error
// message.
it("something not tested properly", (done) => {
Promise.resolve(1).then((x) => {
assert.equal(x, 2);
done();
});
});
// Same test as before, but fixed to give you a good error message
// about expecting a value of 2. But look at the code you have to
// write to get Mocha to give you a nice error message.
it("something tested properly", (done) => {
Promise.resolve(1).then((x) => {
assert.equal(x, 2);
done();
}).catch(done);
});
// If you just return the promise, you can avoid having to pepper your
// code with catch closes and calls to done.
it("something tested properly but much simpler", () => {
return Promise.resolve(1).then((x) => {
assert.equal(x, 2);
});
});
With regards to the completion of asynchronous operations, it works the same whether you are using it, before, beforeEach, after or afterEach so even though the example I gave is with it, the same applies to all the hooks.
I am not sure if I understood 100% the question, but the tests will not start until done is called.
beforeEach(function(done) {
setTimeout(function() {
value = 0;
done();
}, 1);
});
This test will not start until the done function is called in the call to beforeEach above. And this spec will not complete until its done is called.
it("should support async execution of test preparation and expectations", function(done) {
value++;
expect(value).toBeGreaterThan(0);
done();
});
You don't have to pass done in your example, just:
before(function() {
return Promise.resolve(save(article));
});
If you do pass done the test runner will expect to be called before continue, otherwise it will probably throw a timeout error.
In this particular case there is no functional difference. The callback, often called done, was introduced to handle asynchronous tests when using callbacks. Returning a Promise is sufficient, but note that you cannot define the done callback, because the test suite will wait until it's called. Use done when you can't easily return a Promise. In your case the second test will be infinite, because you define done, which you never actually call.

In Jest, how can I make a test fail?

I know I could throw an error from inside the test, but I wonder if there is something like the global fail() method provided by Jasmine?
Jest actually uses Jasmine, so you can use fail just like before.
Sample call:
fail('it should not reach here');
Here's the definition from the TypeScript declaration file for Jest:
declare function fail(error?: any): never;
If you know a particular call should fail you can use expect.
expect(() => functionExpectedToThrow(param1)).toThrow();
// or to test a specific error use
expect(() => functionExpectedToThrow(param1)).toThrowError();
See Jest docs for details on passing in a string, regex, or an Error object to test the expected error in the toThrowError method.
For an async call use .rejects
// returning the call
return expect(asyncFunctionExpectedToThrow(param1))
.rejects();
// or to specify the error message
// .rejects.toEqual('error message');
With async/await you need to mark the test function with async
it('should fail when calling functionX', async () => {
await expect(asyncFunctionExpectedToThrow(param1))
.rejects();
// or to specify the error message
// .rejects.toEqual('error message');
}
See documentation on .rejects and in the tutorial.
Also please note that the Jasmine fail function may be removed in a future version of Jest, see Yohan Dahmani's comment. You may start using the expect method above or do a find and replace fail with throw new Error('it should not reach here'); as mentioned in other answers. If you prefer the conciseness and readability of fail you could always create your own function if the Jasmine one gets removed from Jest.
function fail(message) {
throw new Error(message);
}
You can do it by throwing an error. For example:
test('Obi-Wan Kenobi', () => {
throw new Error('I have failed you, Anakin')
})
Copy/pasta failing test:
it('This test will fail', done => {
done.fail(new Error('This is the error'))
})
Here are certain scenarios where some of the answers won't work. In a world of async-await, it is quite common to have try-catch logic like so.
try {
await someOperation();
} catch (error) {
expect(error.message).toBe('something');
}
Now imagine if someOperation() somehow passed, but you were expecting it to fail, then this test will still pass because it never went to the catch block. So what we want is to make sure that the test fails if someOperation does not throw an error.
So now let's see which solutions will work and which won't.
Accepted answer won't work here because the throw will be catched again.
try {
await someOperation();
throw new Error('I have failed you, Anakin');
} catch (error) {
console.log('It came here, and so will pass!');
}
The answer with true === false also won't work because, assertions too throw an error like above which will be catched.
try {
await someOperation();
expect(true).toBe(false); // This throws an error which will be catched.
} catch (error) {
console.log('It came here, and so will pass!');
}
The one solution that DOES WORK (as shown in #WhatWouldBeCool's answer) for this case is below. Now it explicitly fails the test.
try {
await someOperation();
fail('It should not have come here!')
} catch (error) {
console.log('It never came here!');
}
Update May-2022
The fail() function is not officially supported by Jest anymore. Instead, you can do a couple of things to fail explicitly.
Method-1
You can wrap your promise function within expect and tell jest the function should reject with the given error. If the someOperation() somehow passes, jest will throw an error. If the someOperation() fails for any other reason other than the one you specified, it will throw an error. There are also different methods other than toThrowError() that you can use.
await expect(someOperation()).rejects.toThrowError('error!')
Method-2
You can declare explicitly how many assertions you expect in your test. If that doesn't match because someOperation() never failed, jest would throw an error.
expect.assertions(1)
try {
await someOperation();
} catch (error) {
expect(error.message).toBe('something');
}
Dont think there is, discussed here: https://github.com/facebook/jest/issues/2129
A lot of good ideas here. Only to add extra info about testing async code which may lead to trying to make Jest explicitly fail, check the docs for Testing Asynchronous Code https://jestjs.io/docs/en/asynchronous
To test a function that returns a Promise that resolves, it's important to return the Promise, so Jest knows that the test is done only when the Promise is resolved or it'll time out:
test('the data is peanut butter', () => {
return fetchData().then(data => {
expect(data).toBe('peanut butter')
})
})
To test a function that returns a Promise that rejects, it's important to return the Promise, so Jest knows that the test is done only when the Promise is rejected or it'll time out. And also have to say how many assertions Jest needs to count or it won't fail if the Promise is resolved - which is wrong in this case -:
test('the fetch fails with an error', () => {
expect.assertions(1)
return fetchData().catch(e => expect(e).toMatch('some specific error'))
})
You can always do something like this :)
expect(true).toBe(false);
The done callback passed to every test will throw an error if you pass a string to it.
for instance
it('should error if the promise fails', async (done) => {
try {
const result = await randomFunction();
expect(result).toBe(true);
done();
} catch (e) {
done('it should not be able to get here');
}
});
In this following code if the randomFunction throws an error it will be caught in the catch and with auto fail due to the string being passed to done.
Add jest-fail-on-console npm package, then on your jest.config.js
import failOnConsole from 'jest-fail-on-console'
failOnConsole();
This will fail a test once there is a console error or warning done by jest because of an error or warning thrown in the test item.
I just ran into this one, and after some digging, I found the root of the issue.
Jest, since its inception, has been compatible with Jasmine. Jasmine provided a fail function for programmatically fail the test. This is very useful for cases where throwing an error would cause the test to pass incorrectly (overly-simplified example, but hopefully illustrates the use-case):
function alwaysThrows() {
throw new Error();
}
describe('alwaysThrows', () => {
it('should throw', () => {
try {
alwaysThrows();
// here if there is nothing to force a failure, your
// test could "pass" as there are no failed expectations
// even though no error was thrown. If you just put the
// following to prevent that, you actually force the test
// to always pass:
throw new Error('it should have failed');
// that's why instead you use Jasmine's `fail(reason)` function:
fail('it should have failed');
} catch(err) {
expect(err).toBeDefined();
}
});
)
});
So, what has happened is this:
originally Jest did have a fail() function defined, because its default test runner was jest-jasmine2, which provided fail().
In Jest version 27 (or thereabouts), Jest replaced jest-jasmine2 with jest-circus as the default test runner. jest-circus does not implement a fail() function. This was reported as a bug on July 28th 2021: https://github.com/facebook/jest/issues/11698
Jest's type definitions (maintained in DefinitelyTyped) did not remove the fail() function, so autocompletion and the TypeScript compiler still think that it exists and can be used. There is an issue going on in DefinitelyTyped as well: https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/55803
The issue with this thread is that they have decided not to remove it from the type definitions as it is marked as a "regression" in the Jest repository. Unfortunately, the Jest repository's thread has no official response about whether or not they will support this in the future, so the type definitions are in limbo.
So, long story short, Jest doesn't support fail() by default, but knowing that it's a matter of the default task runner, you can restore the fail() functionality by telling Jest to use the jest-jasmine2 runner instead of the default jest-circus runner:
npm i -D jest-jasmine2
configure the Jest config:
module.exports = {
testRunner: "jest-jasmine2"
};
P.S.: usually there is a better way than try/catch to account for errors in your actual test cases. You can see an example of different ways to handle errors without requiring try/catch in both synchronous and asynchronous contexts here: https://gist.github.com/joeskeen/d9c053b947e5e7462e8d978286311e83
You can throw an error simulating an error thrown by the application and then expect its message to be different from what it actually is.
try {
await somthingYouExpectToFail();
throw new Error("Fail!");
} catch (error) {
expect(error.message).not.toBe("Fail!");
}

mocha test timeout fail insted of assertion fail [duplicate]

One of the things that I find frustrating about Mocha is that when tests fail, they don't give the actual error message of the failing line, Instead, they just end with Error: timeout of 2000ms exceeded. Ensure the done() callback is being called in this test.
Take this test for example:
describe("myTest", function() {
it("should return valid JSON.", function(done) {
api.myCall("valid value").then(function(result) {
console.log(result);
var resultObj = JSON.parse(result);
assert.isFalse(resultObj.hasOwnProperty("error"), "result has an error");
done();
});
});
});
The output is:
myTest
{"error":null,"status":403}
1) should return valid JSON.
0 passing (2s)
1 failing
1) myTest should return valid JSON.:
Error: timeout of 2000ms exceeded. Ensure the done() callback is being called in this test.
The assert.isFalse is failing, but the message that should be displayed ("result has an error") isn't displayed. In fact, processing seems to stop right there because done() is never called. Take that line out and the test passes because done() is called.
So, what am I missing? Why do Mocha tests behave this way? The actual test library I'm using is:
var assert = require("chai").assert;
Does anyone know what I'm doing wrong or why this behaves this way?
It looks like your API is using promises. Before trying anything else, I would suggest checking what the documentation of the API says about promises and how to deal with unhandled exceptions because this may be what is happening here. Some promise implementations require that you call .done() at the end of your call chain to ensure that uncaught exceptions are going to be processed. Some require that some global promise setting be properly configured. The Bluebird documentation gives a good discussion of the issues.
Mocha is capable of handling uncaught exceptions in run-of-the-mill code:
var chai = require("chai");
var assert = chai.assert;
chai.config.includeStack = true;
describe("foo", function() {
it("let the exception be caught by Mocha", function(done) {
setTimeout(function () {
assert.isFalse(true, "foo");
done();
}, 1000);
});
});
This will result in the output:
foo
1) let the exception be caught by Mocha
0 passing (1s)
1 failing
1) foo let the exception be caught by Mocha:
Uncaught AssertionError: foo: expected true to be false
at Assertion.<anonymous> (/tmp/t7/node_modules/chai/lib/chai/core/assertions.js:286:10)
at Assertion.Object.defineProperty.get (/tmp/t7/node_modules/chai/lib/chai/utils/addProperty.js:35:29)
at Function.assert.isFalse (/tmp/t7/node_modules/chai/lib/chai/interface/assert.js:297:31)
at null._onTimeout (/tmp/t7/test.js:8:20)
at Timer.listOnTimeout (timers.js:119:15)
I've encountered the same in my code, using Q for promises.
What happened was:
The assertion inside the then block failed.
The rest of the then block, including the done() statement, was not executed.
Q went looking for a catch block, which wasn't there.
This led to a 'hanging' promise, and thus to a Mocha 2000 ms timeout.
I worked around it by doing something like this:
describe("myTest", function() {
it("should return valid JSON.", function(done) {
api.myCall("valid value").then(function(result) {
console.log(result);
var resultObj = JSON.parse(result);
assert.isFalse(resultObj.hasOwnProperty("error"), "result has an error");
done();
})
.catch(function(err) {
console.error(err);
done(err);
});
});
});

Unit testing window.onerror with jasmine

I am fairly new to javascript and am trying to use jasmine to unit test some error-handling code.
In particular, I'm trying to write some tests that verify that our custom code (called windowHandleError) that replaces window.onerror() gets called, and is doing what we want it to.
I've tried something along the lines of:
it("testing window.onerror", function() {
spyOn(globalerror, 'windowHandleError');
globalerror.install();
var someFunction = function() {
undefinedFunction();
};
expect(function() {someFunction();}).toThrow();
expect(globalerror.windowHandleError).toHaveBeenCalled();
});
But it doesn't trigger the onerror. There are some related questions I've looked at, but they seem to ask about specific browsers, or how/where to use onerror instead of how to test it.
window.onerror not firing in Firefox
Capturing JavaScript error in Selenium
window.onerror does not work
How to trigger script.onerror in Internet Explorer?
Based on what some of those said, I thought running the spec tests in a debugger would force the onerror to trigger, but no dice. Anyone know a better approach to this?
I recently developed small JavaScript error handler with unit tests based on Buster.JS which is similar to Jasmine.
The test that exercises the window.onerror looks like this:
"error within the system": function (done) {
setTimeout(function() {
// throw some real-life exception
not_defined.not_defined();
}, 10);
setTimeout(function() {
assert.isTrue($.post.called);
done();
}, 100);
}
It throws a real-life error within a setTimeout callback which will not stop the test execution and will check that a spy was called after 100ms in another setTimeout and then call done() which is how you test async functionality with Buster.JS. The same approach is available with Jasmine by using done() in async tests.
Without knowledge of Jasmine.
All unit tests run inside a try/catch block so that if one test dies, the next test can run (True for QUnit at least). And since window.onerror doesn't catch exceptions that is already caught inside a try/catch, it will not run when testing this in a unittest.
Try calling the onerror function manually based on the exception.
try {
//Code that should fail here.
someUndefinedFunction();
} catch (e) {
window.onerror.call(window, e.toString(), document.location.toString(), 2);
}
expect(globalerror.windowHandleError).toHaveBeenCalled();
This is far from perfect as document.location is not the same as the url argument, and you need to manually set line number. A better way would be to parse e.stack for the correct file and line number.
When calling the function like this inside a unit test, it might be a better idea to simply test that your function is set and that it functions properly when called with all faked arguments.

Categories

Resources