I've written a lot of asynchronous unit tests lately, using a combination of Angular's fakeAsync, returning Promises from async test body functions, the Jasmine done callback, etc. Generally I've been able to make everything work in a totally deterministic way.
A few parts of my code interact in deeply-tangled ways with 3rd party libraries that are very complex and difficult to mock out. I can't figure out a way to hook an event or generate a Promise that's guaranteed to resolve after this library is finished doing background work, so at the moment my test is stuck using setTimeout:
class MyService {
public async init() {
// Assume library interaction is a lot more complicated to replace with a mock than this would be
this.libraryObject.onError.addEventListener(err => {
this.bannerService.open("Load failed!" + err);
});
// Makes some network calls, etc, that I have no control over
this.libraryObject.loadData();
}
}
it("shows a banner on network error", async done => {
setupLibraryForFailure();
await instance.init();
setTimeout(() => {
expect(banner.open).toHaveBeenCalled();
done();
}, 500); // 500ms is generally enough... on my machine, probably
});
This makes me nervous, especially the magic number in the setTimeout. It also scales poorly, as I'm sure 500ms is far longer than any of my other tests take to complete.
What I think I'd like to do, is be able to tell Jasmine to poll the banner.open spy until it's called, or until the test timeout elapses and the test fails. Then, the test should notice as soon as the error handler is triggered and complete. Is there a better approach, or is this a good idea? Is it a built-in pattern somewhere that I'm not seeing?
I think you can take advantage of callFake, basically calling another function once this function is called.
Something like this:
it("shows a banner on network error", async done => {
setupLibraryForFailure();
// maybe you have already spied on banner open so you have to assign the previous
// spy to a variable and use that variable for the callFake
spyOn(banner, 'open').and.callFake((arg: string) => {
expect(banner.open).toHaveBeenCalled(); // maybe not needed because we are already doing callFake
done(); // call done to let Jasmine know you're done
});
await instance.init();
});
We are setting a spy on banner.open and when it is called, it will call the callback function using the callFake and we call done inside of this callback letting Jasmine know we are done with our assertions.
I am working with v4 gulp and converting my tasks to functions. I have a simple clean function that executes a parallel task.
const clean_server = () => del('build/server/*');
const clean_client = () => del('build/client/*');
export function clean(done) {
gulp.parallel(clean_server, clean_client);
done();
}
When I call done() the way I'm calling above, which is also suggested from the docs https://gulpjs.com/docs/en/getting-started/creating-tasks, I see the task get initiated, however, it doesn't actually complete the task.
However, when I change it to:
export function clean(done) {
gulp.parallel(clean_server, clean_client)(done);
}
This way works.
So my question is, why doesn't the first way as suggested by the docs complete the async task?
Because done must be passed as a callback to parallel. Either like the way you did on or like the following
gulp.parallel(done => {
clean_server()
clean_client();
done();
})
I'm using Puppeteer and Jest to run some front end tests.
My tests look as follows:
describe("Profile Tab Exists and Clickable: /settings/user", () => {
test(`Assert that you can click the profile tab`, async () => {
await page.waitForSelector(PROFILE.TAB);
await page.click(PROFILE.TAB);
}, 30000);
});
Sometimes, when I run the tests, everything works as expectedly. Other times, I get an error:
Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.
at node_modules/jest-jasmine2/build/queue_runner.js:68:21 <br/>
at Timeout.callback [as _onTimeout] (node_modules/jsdom/lib/jsdom/browser/Window.js:633:19)
This is strange because:
I specified the timeout to be 30000
Whether or not I get this error is seemingly very random
Why is this happening?
The timeout you specify here needs to be shorter than the default timeout.
The default timeout is 5000 and the framework by default is jasmine in case of jest. You can specify the timeout inside the test by adding
jest.setTimeout(30000);
But this would be specific to the test. Or you can set up the configuration file for the framework.
Configuring Jest
// jest.config.js
module.exports = {
// setupTestFrameworkScriptFile has been deprecated in
// favor of setupFilesAfterEnv in jest 24
setupFilesAfterEnv: ['./jest.setup.js']
}
// jest.setup.js
jest.setTimeout(30000)
See also these threads:
setTimeout per test #5055
Make jasmine.DEFAULT_TIMEOUT_INTERVAL configurable #652
P.S.: The misspelling setupFilesAfterEnv (i.e. setupFileAfterEnv) will also throw the same error.
It should call the async/await when it is async from test.
describe("Profile Tab Exists and Clickable: /settings/user", () => {
test(`Assert that you can click the profile tab`, async (done) => {
await page.waitForSelector(PROFILE.TAB);
await page.click(PROFILE.TAB);
done();
}, 30000);
});
The answer to this question has changed as Jest has evolved. Current answer (March 2019):
You can override the timeout of any individual test by adding a third parameter to the it. I.e., it('runs slow', () => {...}, 9999)
You can change the default using jest.setTimeout. To do this:
// Configuration
"setupFilesAfterEnv": [ // NOT setupFiles
"./src/jest/defaultTimeout.js"
],
and
// File: src/jest/defaultTimeout.js
/* Global jest */
jest.setTimeout(1000)
Like others have noted, and not directly related to this, done is not necessary with the async/await approach.
This is a relatively new update, but it is much more straight forward. If you are using Jest 24.9.0 or higher you can just add testTimeout to your config:
// in jest.config.js
module.exports = {
testTimeout: 30000
}
I would like to add (this is a bit long for a comment) that even with a timeout of 3000 my tests would still sometimes (randomly) fail with
Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.
Thanks to Tarun's great answer, I think the shortest way to fix a lot of tests is:
describe('Puppeteer tests', () => {
beforeEach(() => {
jest.setTimeout(10000);
});
test('Best Jest test fest', async () => {
// Blah
});
});
For Jest 24.9+, we just need to add --testTimeout on the command line:
--testTimeout= 10000 // Timeout of 10 seconds
The default timeout value is 5000 (5000 ms - 5 seconds). This will be applicable for all test cases.
Or if you want to give timeout to particular function only then you can use this syntax while declaring the test case.
test(name, fn, timeout)
Example
test('example', async () => {
}, 10000); // Timeout of 10 seconds (default is 5000 ms)
For Jest 24.9+, you can also set the timeout from the command line by adding --testTimeout.
Here's an excerpt from its documentation:
--testTimeout=<number>
Default timeout of a test in milliseconds. Default value: 5000.
Make sure to invoke done(); on callbacks or it simply won't pass the test.
beforeAll((done /* Call it or remove it */ ) => {
done(); // Calling it
});
It applies to all other functions that have a done() callback.
Yet another solution: set the timeout in the Jest configuration file, e.g.:
{ // ... other stuff here
"testTimeout": 90000
}
You can also get timeout errors based on silly typos. For example, this seemingly innocuous mistake:
describe('Something', () => {
it('Should do something', () => {
expect(1).toEqual(1)
})
it('Should do nothing', something_that_does_not_exist => {
expect(1).toEqual(1)
})
})
Produces the following error:
FAIL src/TestNothing.spec.js (5.427s)
● Something › Should do nothing
Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.
at node_modules/jest-jasmine2/build/queue_runner.js:68:21
at Timeout.callback [as _onTimeout] (node_modules/jsdom/lib/jsdom/browser/Window.js:678:19)
While the code sample posted doesn't suffer from this, it might be a cause of failures elsewhere. Also note that I'm not setting a timeout for anything anywhere - either here or in the configuration. The 5000 ms is just the default setting.
I recently ran into this issue for a different reason: I was running some tests synchronously using jest -i, and it would just timeout. For whatever reasoning, running the same tests using jest --runInBand (even though -i is meant to be an alias) doesn't time out.
test accepts a timeout argument. See https://jestjs.io/docs/api#testname-fn-timeout. Here is a sample:
async function wait(millis) {
console.log(`sleeping for ${millis} milliseconds`);
await new Promise(r => setTimeout(r, millis));
console.log("woke up");
}
test('function', async () => {
await wait(5000);
}, 70000);
Mar 14, 2022, Jest 27.5 documentation indicates a new process:
https://jestjs.io/docs/api#beforeallfn-timeout
Pass a second parameter to test with the number of msec before timeout. Works!
test('adds 1 + 2 to equal 3', () => {
expect(3).toBe(3);
},30000);
// In jest.setup.js
jest.setTimeout(30000)
If on Jest <= 23:
// In jest.config.js
module.exports = {
setupTestFrameworkScriptFile: './jest.setup.js'
}
If on Jest > 23:
// In jest.config.js
module.exports = {
setupFilesAfterEnv: ['./jest.setup.js']
}
The timeout problem occurs when either the network is slow or many network calls are made using await. These scenarios exceed the default timeout, i.e., 5000 ms. To avoid the timeout error, simply increase the timeout of globals that support a timeout. A list of globals and their signature can be found here.
For Jest 24.9
Turns out if your expect assertions are wrong, it can sometimes spit out the exceeded timeout error message.
I was able to figure this out by putting console.log() statements in my promise callback and saw the console.log() statements were getting ran in the jest output. Once I fixed my expect assertions, the timeout error went away and tests worked.
I spent way too long to figure this out and hope this helps whoever needs to read this.
In case someone doesn't fix the problem use methods above. I fixed mine by surrounding the async func by an arrow function. As in:
describe("Profile Tab Exists and Clickable: /settings/user", () => {
test(`Assert that you can click the profile tab`, (() => {
async () => {
await page.waitForSelector(PROFILE.TAB)
await page.click(PROFILE.TAB)
}
})(), 30000);
});
For the jest versions greater than 27, you can add useRealTimers on the top of your spec file.
Here is the snippet
import { shortProcess, longProcess } from '../../src/index';
jest.useRealTimers();
describe(`something`, function () {
it('should finish fine', async function () {
await shortProcess();
expect(1).toBe(1);
});
it('should fail with a timeout', async function () {
await longProcess();
expect(1).toBe(1);
});
it('should finish fine again', async function () {
jest.setTimeout(10 * 1000);
await longProcess();
expect(1).toBe(1);
}, 10000);
});
Find the github issue here on jest repository.
This probably won't be terribly helpful to most people visiting this page, but when I was getting this error it had nothing to do with Jest. One of my method calls was getting an empty object and a null exception while running locally. Once I added a null check, the failing tests and console log in question disappeared.
if(response !== null){
this.searchSubj.next(resp);
}
else {
return;
}
For those who are looking for an explanation about
jest --runInBand, you can go to the documentation.
Running Puppeteer in CI environments
GitHub - smooth-code/jest-puppeteer: Run your tests using Jest & Puppeteer
In my case, this error started appearing randomly and wouldn't go away even after setting a timeout of 30000. Simply ending the process in the terminal and rerunning the tests resolved the issue for me. I have also removed the timeout and tests are still passing again.
Dropping my 2 cents here, I had the same issue on dosen of jest unit test (not all of them) and I notice that all started after I added to jestSetup this polyfill for MutuationObservers:
if (!global.MutationObserver) {
global.MutationObserver = function MutationObserverFun(callback) {
this.observe = function(){};
this.disconnect = function(){};
this.trigger = (mockedMutationsList) => {
callback(mockedMutationsList, this);
};
};
}
Once I removed it test start working again correctly. Hope helps someone .
add this in your test, not much to explain
beforeEach(() => {
jest.useFakeTimers()
jest.setTimeout(100000)
})
afterEach(() => {
jest.clearAllTimers()
})
I am writing a mini-framework for executing unit tests for a product I work on. I want test data to be published and managed as seamlessly as possible. With Mocha, it is easy to schedule test data cleanup using the After() hook.
You could wrap an individual test in a describe() block and use that block's Before/After method, but that I'd rather avoid that if possible.
You could pass a cleanup function to afterEach which specifically targets data populated inside a test. Though that would only be necessary for one cleanup and it seems clunky to do that.
Is it possible to generate test data within one test, just for the sake of that test, and also schedule a cleanup for it with Mocha?
Sure, just run your generation and cleanup in the test itself. If it's asynchronous, you can use the done callback to make it wait until it's called.
mocha.setup('bdd');
describe('suite', function() {
function getData() {
// Simulate asynchronous data generation
console.log('grabbing data');
return new Promise((resolve, reject) => {
setTimeout(() => resolve(100), 500);
});
}
function cleanup() {
// Simulate asynchronous cleanup
console.log('cleaning up...');
return new Promise((resolve, reject) => {
setTimeout(resolve, 500);
});
}
it('should do generation and clean up', function(done) {
// Generate some data
getData()
.then(data => {
// Test the data
if (data !== 100) {
throw new Error('How?!');
}
console.log('test passed');
// Cleanup
return cleanup();
})
.then(_ => {
// Use done() after all asynchronous work completes
console.log('done cleaning');
done();
})
.catch(err => {
// Make sure it cleans up no matter what
cleanup().then(_ => console.error(err));
});
});
});
mocha.run();
<script src="https://cdn.rawgit.com/mochajs/mocha/2.2.5/mocha.js"></script>
<div id="mocha"></div>
I think cleanup after test is generally problematic because cleanup consistency guarantees aren't very strong, ie is cleanup function guaranteed to run? probably not. If it's not assured that cleanup will take place then it could very well leave the next tests in an inconsistent state. I think it's good to make an attempt but you can guard against failure by:
cleaning up/establishing db state BEFORE each test
nuking the world so each test has a consistent state (can be accomplished by executing your test in the context of a transaction and rolling back the transaction after each test, and at least not ever committing the transaction)
having test create unique data. By leveraging unique data you can also run your tests in parallel, since it allows for multiple different tests to have an isolated view of the db. If each test writes its own data you only have to worry about provisioning the whole db at the beginning of each test run
Of the above if you're able to wrap your test in a transaction it is lightning fast, (web frameworks like django and rails do this and it is quite fast and makes tests/db state very easy to reason about)
Say I have a Component that renders a simple div containing an integer that starts at 0 and ticks up 1 every second. So after 5 seconds, the component should render "5" and after 30 seconds, it should render "30" and so on. If I wanted to test this component and make sure its rendering what it should after 5 seconds, I might write something like this.
it('should render <5> after 5 seconds', () => {
const time = mount(<Timer/>)
setTimeout(() => {
expect(time.text()).toEqual('5')
}, 5000)
})
However, this doesn't work as the test never actually runs the expect and returns a pass regardless of anything. And even if it did work, using a timeout like this would be extremely inefficient as the test would have to wait 5 seconds. And what if I wanted to simulate an even larger amount of time? After doing some searching, I found jest actually has a timer mock but I can't seem to figure out how to implement it for this case. Any help would be greatly appreciated. Thanks!
in order for Jest to know that your test is async, you need to either: 1) return a Promise, or 2) declare an argument in the test callback, like so:
it('should render <5> after 5 seconds', done => {
// test stuff
});
source: https://facebook.github.io/jest/docs/en/asynchronous.html
To get your test working:
it('should render <5> after 5 seconds', done => {
jest.useFakeTimers(); // this must be called before any async things happen
const time = mount(<Timer/>);
setTimeout(() => {
// Note the placement of this try/catch is important.
// You'd think it could be placed most anywhere, but nope...
try {
// If this assertion fails, an err is thrown.
// If we do not catch()...done.fail(e) it,
// then this test will take jasmine.DEFAULT_TIMEOUT_INTERVAL (5s unless you override)
// and then fail with an unhelpful message.
expect(time.text()).toEqual('5');
done();
} catch(e) {
done.fail(e);
}
}, 5000);
jest.runTimersToTime(5000); // This basically fast forwards 5s.
});
You must call jest.useFakeTimers(); before `require´ your component to mock setTimeout.
Here we enable fake timers by calling jest.useFakeTimers();. This
mocks out setTimeout and other timer functions with mock functions.
And to test async workflows Jest documentation says you have to return a Promise for each 'it' function callback.
To test an asynchronous function, just return a promise from it. When
running tests, Jest will wait for the promise to resolve before
letting the test complete. You can also return a promise from
beforeEach, afterEach, beforeAll or afterAll functions. For example,
let's say fetchBeverageList() returns a promise that is supposed to
resolve to a list that has lemon in it. You can test this with:
So you can write something like:
it('should render <5> after 5 seconds', () => {
jest.useFakeTimers();
//only after that, require your componenet
let Timer = require('./timer');
const time = mount(<Timer/>);
jest.runAllTimers();
expect(setTimeout.mock.calls.length).toBe(1);
expect(setTimeout.mock.calls[0][1]).toBe(5000);
expect(time.text()).toEqual('5');
})