What is a good and clean way to cleanup after a test case failed?
For a lot of test cases, I first create a clean database environment, which needs to be cleaned up after a test case finishes.
test('some test', async () => {
const { sheetService, cleanup } = await makeUniqueTestSheetService();
// do tests with expect()
await cleanup();
});
Problem is: If one of the expects fails, cleanup() is not invoked and thus the database environment is not cleaned up and jest complains Jest did not exit one second after the test run has completed. because the connection is not closed.
My current workaround looks like this, but it doesn't feel good and clean to push the cleanup hooks to an array which than is handled in the afterAll event.
const cleanUpHooks: (() => Promise<void>)[] = [];
afterAll(async () => {
await Promise.all(cleanUpHooks.map(hook => hook()));
});
test('some test', async () => {
const { sheetService, cleanup } = await makeUniqueTestSheetService();
// do tests with expect()
await cleanup();
});
Use try/finally block for such cases.
For example:
it("some test case", async done => {
try {
expect(false).toEqual(true);
console.log("abc");
done();
}finally{
console.log("finally");
// cleanup code below...
}
});
Above code would just execute the finally block (hence "finally" gets printed in console instead of "abc".
Note that the catch block gives a timeout so just use finally.
This answer must be useful.
What if you use afterEach()? it will execute after each test
test('some test', async () => {
const { sheetService, cleanup } = await makeUniqueTestSheetService();
// do tests with expect()
});
afterEach(async()=>{
await cleanup();
});
Related
As suggested in this post , I tried the steps to create dynamic tests , but I see the actual test(test.getMochaTest()in my below implementation) not getting executed. What's that I'm missing here, the call on test.getMochaTest() does not get executed in the before block.
describe('Dynamic Tests for Mocha', async () => {
let outcome ;
before(async() => {
await init().catch(() => console.error('Puppeteer environment initialization failed'));
return collectTests().then(async(collectTests) => {
console.info('4.Executing tests :');
describe('Dynamic test execution', async() => {
collectTests.forEach(async(test) => {
console.info(`\tModule under test : ${test.name}`);
// impl. of test.getMochaTest() DOES NOT get executed.
it(test.name, async() => outcome = await test.getMochaTest().catch(async () => {
console.error(`error while executing test:\t${test.name}`);
}));
});
}) ;
});
});
after(async () => {
console.info('5. Exiting tests...');
await HelperUtils.delay(10000).then(async () => { await browser.close(); });
console.log('executing after block');
});
it('placeholder', async() => {
await
console.log('place holder hack - skip it');
});
});
Array of tests is returned here :
async function collectTests():Promise<Array<ITest>> {
console.info('3.Collecting tests to execute ...');
testArray = new Array<ITest>();
const login:ITest = new SLogin('Login Check');
testArray.push(login);
return testArray;
}
The below implementation of getMochaTest in SLogin -> does not get executed .
export default class SLogin extends BTest implements ITest {
constructor(name: string) {
super(name);
}
async getMochaTest():Promise<Mocha.Func> {
return async () => {
console.log('Running Login check');
expect(true).to.equal(true);
};
}
}
It doesn't look like you're actually invoking the test.
Calling test.getMochaTest() only returns the async test function in a Promise, it doesn't execute it. So your catch block is catching errors while obtaining the function, not while executing it.
Breaking it out across multiple lines will hopefully make things clearer.
Here's what your code sample does. Notice it never executes the returned test function:
it(test.name, async () => {
const testFn = await test.getMochaTest().catch(() =>
console.error(`error while ***obtaining*** test: \t${test.name}`));
// oops - testFn never gets called!
});
And here's a corrected version where the test actually gets called:
it(test.name, async () => {
const testFn = await test.getMochaTest().catch(() =>
console.error(`error while ***obtaining*** test: \t${test.name}`));
const outcome = await testFn().catch(() =>
console.error(`error while ***executing*** test: \t${test.name}`));
});
Note: I wrote it that way with await and catch() to better match the format of your code sample. However, it's worth pointing out that it mixes async/await and Promise syntax. More idiomatic would be to catch errors with a try/catch block when using async/await.
I want to dynamically run multiple tests in Jest inside a for-loop but I am not sure where to place the asynchronous code that my test environment requires.
One option that works is to place all asynchronous code inside a test function and execute an assert statement for each iteration of the for-loop.
describe('Database Testing', () => {
test(`Testing ${items}`, async () => {
const items = await getItems(); //<-- asynchronous code
for (const item of items) {
expect('hi').toEqual('hi');
}
});
});
However, if the test fails, I will not be able to pinpoint which loop the assert statement failed in. Instead, I would like a structure similar to below where I dynamically run a test for each iteration of the for-loop.
describe('Database Testing', () => {
const items = await getItems(); //<-- asynchronous code
for (const item of items) {
test(`Testing ${item}`, async () => {
expect('hi').toEqual('hi');
});
};
});
I am unable to run the asynchronous code due to the synchronous nature of the describe function. Yet, if I use the async keyword in the describe function, I get the error 'Returning a Promise from "describe" is not supported'.
Where should I run the asynchronous code?
beforeAll can be async. You can also have it functions within describe that are async.
Here is the jest documentation on testing async: https://jestjs.io/docs/en/tutorial-async
beforeAll(async () => {
// do async things
})
describe('whatever', () => {
it('will do something async', async () => {
expect.assertions(1);
// do something async
//... expect something
})
})
Just make sure that your expect.assertions matches the appropriate number of assertions.
will something like this work:
describe('Database Testing', () => {
getItems().then(items -> {
for (const item of items) {
expect('hi').toEqual('hi');
};
});
});
How would I test this code in jest? I'd like to make sure that the error and success of the passed promise is being called as needed. I'm sure it's something sorta simple, but it's driving me crazy. Thanks very much.
handleStatusChangeRequest (changeEntryStatus) {
return changeEntryStatus().then(() => {
this.handleStatusChangeSuccess()
}).catch(err => {
this.handleErrorDisplay(err)
})
}
If your code uses promises, there is a nice way to handle asynchronous tests. Just return a promise from your test, and Jest will wait for that promise to resolve.
If the promise is rejected, the test will automatically fail.
For example, let's say that changeData, instead of using a callback, returns a promise that is supposed to resolve to the string "status has been successfully modified".
Be sure to return the promise - if you omit this return statement, your test will complete before your changeData() -[async function] completes.
Here's a convenient and easy to follow pattern
test('if the data is changed', () => {
return changeData().then((data) => {
expect(data).toBe('status has been successfully modified');
});
})
Happy testing :)
This could be refactored, but for the sake of demonstration, I left the repeating bits in.
In example.spec.js, the callback, changeEntryStatus, is stubbed to return a promise. In order to check if other instance methods (this.method) were called, they are first mocked, then assertions are called on the mock after running the method being tested. Learn more in the Jest docs. (See my thoughts on mocking methods of the unit being tested at the bottom.)
Run the example on repl.it.
example.js:
class Example {
handleStatusChangeRequest(changeEntryStatus) {
return changeEntryStatus().then(() => {
this.handleStatusChangeSuccess()
}).catch(err => {
this.handleErrorDisplay(err)
})
}
handleStatusChangeSuccess() {
console.log('stubbed handleStatusChangeSuccess')
}
handleErrorDisplay(error) {
console.log('stubbed handleErrorDisplay:', error)
}
}
module.exports = Example;
example.spec.js:
const Example = require('./entryStatus')
describe('handleStatusChangeRequest', () => {
it('should run the changeEntryStatus callback', () => {
const {handleStatusChangeRequest} = new Example()
const stub = jest.fn().mockResolvedValue()
handleStatusChangeRequest(stub)
// must return because handleStatusChangeRequest is asynchronous
return expect(stub).toHaveBeenCalled()
});
it('should call example.handleStatusChangeSuccess', async () => {
const example = new Example()
const stub = jest.fn().mockResolvedValue()
example.handleStatusChangeSuccess = jest.fn()
await example.handleStatusChangeRequest(stub)
expect(example.handleStatusChangeSuccess).toHaveBeenCalled();
})
it('should call example.handleErrorDisplay', async () => {
const example = new Example()
const fakeError = { code: 'fake_error_code' }
const stub = jest.fn().mockRejectedValue(fakeError)
example.handleErrorDisplay = jest.fn()
await example.handleStatusChangeRequest(stub)
expect(example.handleErrorDisplay).toHaveBeenCalled()
expect(example.handleErrorDisplay).toHaveBeenCalledWith(fakeError)
});
});
Opinionated Disclaimer: Mocking methods of the unit under test is a smell. Consider checking for the expected effects of calling handleStatusChangeSuccess and handleErrorDisplay instead of checking to see if they were called. Then don't even expose those methods publicly unless consumers of the class need access.
Opinionated Disclaimer: Mocking methods of the unit under test is a
smell. Consider checking for the expected effects of calling
handleStatusChangeSuccess and handleErrorDisplay instead of checking
to see if they were called. Then don't even expose those methods
publicly unless consumers of the class need access.
I wholeheartedly agree with webprojohn's disclaimer. Mocks are a smell as tests should assert the behavior of the code, not its implementation. Testing the latter makes the code brittle to change.
Stepping off my soapbox... :) We're looking for a way to test an asynchronous method. I'm not sure what assertions your tests should make to verify the behavior inside handleStatusChangeSuccess() and handleErrorDisplay(err) so the example below leaves a comment where those assertions would go. The following uses Promise.resolve() and Promise.reject() to trigger the outcomes to test. I've used async/await, Jest has other async examples in their docs.
const Example = require('./example')
describe('handleStatusChangeRequest', () => {
it('should resolve successfully', async () => {
const {handleStatusChangeRequest} = new Example();
const resolvePromise = () => Promise.resolve();
await handleStatusChangeRequest(resolvePromise);
// resolution assertions here
});
it('should resolve errors', async () => {
const {handleStatusChangeRequest} = new Example();
const fakeError = new Error('eep');
const rejectPromise = () => Promise.reject(fakeError);
// if your method doesn't throw, we can remove this try/catch
// block and the fail() polyfill
try {
await example.handleStatusChangeRequest(rejectPromise);
// if we don't throw our test shouldn't get here, so we
// polyfill a fail() method since Jest doesn't give us one.
// See https://github.com/facebook/jest/issues/2129
expect(true).toBe(false);
}
catch (e) {
// rejection assertions here
}
});
});
The answer I have looks so:
**Success tests
const instance = el.find(EntryToolBar).instance()
const spy = jest.spyOn(instance, 'handleStatusChangeSuccess')
await instance.handleStatusChangeRequest(() => Promise.resolve('cool man'))
expect(spy).toHaveBeenCalledTimes(1)
**Error tests
const instance = el.find(EntryToolBar).instance()
const spy = jest.spyOn(instance, 'handleErrorDisplay')
await instance.handleStatusChangeRequest(() => Promise.reject(Error('shit')))
expect(spy).toHaveBeenCalledTimes(1)
As I stated above, the handleStatusChangeSuccess and handleError methods are test else where with some snapshots (they just set state and render out some different jsx). I feel pretty good about this. I'm using spys/mocks, but I'm testing the implementation functions elsewhere. Sufficient?
I have following:
it('invalid use', () => {
Matcher(1).case(1, () => {});
});
The case method is supposed to throw after some delay, how can I describe it for Mocha/Chai that's what I want - the test should pass (and must not pass when exception is not thrown)?
Consider case method off limits, it cannot be changed.
For testing purposes it should be equivalent to:
it('setTimeout throw', _ => {
setTimeout(() => { throw new Error(); }, 1); // this is given, cannot be modified
});
I tried:
it('invalid use', done => {
Matcher(1).case(1, () => {});
// calls done callback after 'case' may throw
setTimeout(() => done(), MatcherConfig.execCheckTimeout + 10);
});
But that's not really helping me, because the test behavior is exactly reverted - when an exception from case (setTimeout) is not thrown, it passes (should fail) and when an exception is thrown the test fails (should succeed).
I read somewhere someone mentioning global error handler, but I would like to solve this cleanly using Mocha and/or Chai, if it is possible (I guess Mocha is already using it in some way).
You cannot handle exceptions from within a asynchronous callback, e.g. see Handle error from setTimeout. This has to do with the execution model ECMAScript uses. I suppose the only way to catch it is in fact to employ some environment-specific global error handling, e.g. process.on('uncaughtException', ...) in Node.js.
If you convert your function to Promises, however, you can easily test it using the Chai plugin chai-as-promsied:
import * as chai from 'chai';
import chaiAsPromised = require('chai-as-promised');
chai.use(chaiAsPromised);
const expect = chai.expect;
it('invalid use', async () => {
await expect(Matcher(1).case(1, () => {})).to.eventually.be.rejected;
});
Any Mocha statements like before, after or it will work asynchronously if you return a promise. I generally use something like the below for async tests.
Also don't forget to set timeout this.timeout(...) if you expect the async function to take more than 2 seconds.
it('some test', () => {
return new Promise(function(resolve,reject){
SomeAsyncFunction(function(error,vals) {
if(error) {
return reject(error);
} else {
try {
//do some chai tests here
} catch(e) {
return reject(e);
}
return resolve();
}
});
});
});
Specifically for your case, since we expect some error to be thrown after a period of time (assuming the empty callback you have provided to .case should not run due to the exception being thrown) then you can write it something like:
it('invalid use', () => {
//define the promise to run the async function
let prom = new Promise(function(resolve,reject){
//reject the promise if the function does not throw an error
//note I am assuming that the callback won't run if the error is thrown
//also note this error will be passed to prom.catch so need to do some test to make sure it's not the error you are looking for.
Matcher(1).case(1, () => {return reject(new Error('did not throw'))});
});
prom.catch(function(err){
try {
expect(err).to.be.an('error');
expect(err.message).to.not.equal('did not throw');
//more checks to see if err is the error you are looking for
} catch(e) {
//err was not the error you were looking for
return Promise.reject(e);
}
//tests passed
return Promise.resolve();
});
//since it() receives a promise as a return value it will pass or fail the test based on the promise.
return prom;
});
From Chai documentation :
When no arguments are provided, .throw invokes the target function and asserts that an error is thrown.
So you could something like
expect(Matcher(1).case(1, () => {})).to.throw
If your tested code calls setTimeout with a callback that throws and no-one is catching this is exception then:
1) this code is broken
2) the only way to see that problem is platform global exception handler like process.on('uncaughtException' mentioned by user ComFreek
The last resort chance is to stub setTimeout for duration of test (for example using sinon.stub) or just manually.
In such stubbed setTimeout you can decorate timeout handler, detect exception and call appropriate asserts.
NOTE, this is last resort solution - your app code is broken and should be fixed to properly propagate errors, not only for testing but ... well, to be good code.
Pseudocode example:
it('test', (done) => {
const originalSetTimeout = setTimeout;
setTimeout = (callback, timeout) => {
originalSetTimeout(() => {
try {
callback();
} catch(error) {
// CONGRATS, you've intercepted exception
// in _SOME_ setTimeout handler
}
}, timeout)
}
yourTestCodeThatTriggersErrorInSomeSetTimeoutCallback(done);
})
NOTE2: I intentionally didn't wrote proper async cleanup code, it's a homework. Again, see sinon.js and its sandbox
NOTE3: It will catch all setTimeout calls during test duration. Beware, there are dragons.
For example, I have something basic like this:
it.only('tests something', (done) => {
const result = store.dispatch(fetchSomething());
result.then((data) => {
const shouldBe = 'hello';
const current = store.something;
expect(current).to.equal(shouldBe);
done();
}
});
When current does not match shouldBe, instead of a message saying that they don't match, I get the generic timeout message:
Error: timeout of 2000ms exceeded. Ensure the done() callback is being
called in this test.
It's as if the expectation is pausing the script or something. How do I fix this? This is making debugging nearly impossible.
The expectation is not pausing the script, it is throwing an exception before you hit the done callback, but since it is no longer inside of the test method's context it won't get picked up by the test suite either, so you are never completing the test. Then your test just spins until the timeout is reached.
You need to capture the exception at some point, either in the callback or in the Promise's error handler.
it.only('tests something', (done) => {
const result = store.dispatch(fetchSomething());
result.then((data) => {
const shouldBe = 'hello';
const current = store.getState().get('something');
try {
expect(current).to.equal(shouldBe);
done();
} catch (e) {
done(e);
}
});
});
OR
it.only('tests something', (done) => {
const result = store.dispatch(fetchSomething());
result.then((data) => {
const shouldBe = 'hello';
const current = store.getState().get('something');
expect(current).to.equal(shouldBe);
})
.catch(done);
});
Edit
If you are not opposed to bringing in another library, there is a fairly nice library call chai-as-promised. That gives you some nice utilities for this kind of testing.