I am writing a mini-framework for executing unit tests for a product I work on. I want test data to be published and managed as seamlessly as possible. With Mocha, it is easy to schedule test data cleanup using the After() hook.
You could wrap an individual test in a describe() block and use that block's Before/After method, but that I'd rather avoid that if possible.
You could pass a cleanup function to afterEach which specifically targets data populated inside a test. Though that would only be necessary for one cleanup and it seems clunky to do that.
Is it possible to generate test data within one test, just for the sake of that test, and also schedule a cleanup for it with Mocha?
Sure, just run your generation and cleanup in the test itself. If it's asynchronous, you can use the done callback to make it wait until it's called.
mocha.setup('bdd');
describe('suite', function() {
function getData() {
// Simulate asynchronous data generation
console.log('grabbing data');
return new Promise((resolve, reject) => {
setTimeout(() => resolve(100), 500);
});
}
function cleanup() {
// Simulate asynchronous cleanup
console.log('cleaning up...');
return new Promise((resolve, reject) => {
setTimeout(resolve, 500);
});
}
it('should do generation and clean up', function(done) {
// Generate some data
getData()
.then(data => {
// Test the data
if (data !== 100) {
throw new Error('How?!');
}
console.log('test passed');
// Cleanup
return cleanup();
})
.then(_ => {
// Use done() after all asynchronous work completes
console.log('done cleaning');
done();
})
.catch(err => {
// Make sure it cleans up no matter what
cleanup().then(_ => console.error(err));
});
});
});
mocha.run();
<script src="https://cdn.rawgit.com/mochajs/mocha/2.2.5/mocha.js"></script>
<div id="mocha"></div>
I think cleanup after test is generally problematic because cleanup consistency guarantees aren't very strong, ie is cleanup function guaranteed to run? probably not. If it's not assured that cleanup will take place then it could very well leave the next tests in an inconsistent state. I think it's good to make an attempt but you can guard against failure by:
cleaning up/establishing db state BEFORE each test
nuking the world so each test has a consistent state (can be accomplished by executing your test in the context of a transaction and rolling back the transaction after each test, and at least not ever committing the transaction)
having test create unique data. By leveraging unique data you can also run your tests in parallel, since it allows for multiple different tests to have an isolated view of the db. If each test writes its own data you only have to worry about provisioning the whole db at the beginning of each test run
Of the above if you're able to wrap your test in a transaction it is lightning fast, (web frameworks like django and rails do this and it is quite fast and makes tests/db state very easy to reason about)
Related
I've written a lot of asynchronous unit tests lately, using a combination of Angular's fakeAsync, returning Promises from async test body functions, the Jasmine done callback, etc. Generally I've been able to make everything work in a totally deterministic way.
A few parts of my code interact in deeply-tangled ways with 3rd party libraries that are very complex and difficult to mock out. I can't figure out a way to hook an event or generate a Promise that's guaranteed to resolve after this library is finished doing background work, so at the moment my test is stuck using setTimeout:
class MyService {
public async init() {
// Assume library interaction is a lot more complicated to replace with a mock than this would be
this.libraryObject.onError.addEventListener(err => {
this.bannerService.open("Load failed!" + err);
});
// Makes some network calls, etc, that I have no control over
this.libraryObject.loadData();
}
}
it("shows a banner on network error", async done => {
setupLibraryForFailure();
await instance.init();
setTimeout(() => {
expect(banner.open).toHaveBeenCalled();
done();
}, 500); // 500ms is generally enough... on my machine, probably
});
This makes me nervous, especially the magic number in the setTimeout. It also scales poorly, as I'm sure 500ms is far longer than any of my other tests take to complete.
What I think I'd like to do, is be able to tell Jasmine to poll the banner.open spy until it's called, or until the test timeout elapses and the test fails. Then, the test should notice as soon as the error handler is triggered and complete. Is there a better approach, or is this a good idea? Is it a built-in pattern somewhere that I'm not seeing?
I think you can take advantage of callFake, basically calling another function once this function is called.
Something like this:
it("shows a banner on network error", async done => {
setupLibraryForFailure();
// maybe you have already spied on banner open so you have to assign the previous
// spy to a variable and use that variable for the callFake
spyOn(banner, 'open').and.callFake((arg: string) => {
expect(banner.open).toHaveBeenCalled(); // maybe not needed because we are already doing callFake
done(); // call done to let Jasmine know you're done
});
await instance.init();
});
We are setting a spy on banner.open and when it is called, it will call the callback function using the callFake and we call done inside of this callback letting Jasmine know we are done with our assertions.
I'm new to cypress and am trying to figure out how things work.
I have my own function (which calls a test controller server to reset the database). It returns a promise which completes when the DB have been successfully resetted.
function resetDatabase(){
// returns a promise for my REST api call.
}
My goal is to be able to execute it before all tests.
describe('Account test suite', function () {
// how can I call resetDb here and wait for the result
// before the tests below are invoked?
it('can log in', function () {
cy.visit(Cypress.config().testServerUrl + '/Account/Login/')
cy.get('[name="UserName"]').type("admin");
cy.get('[name="Password"]').type("123456");
cy.get('#login-button').click();
});
// .. and more test
})
How can I do that in cypress?
Update
I've tried
before(() => {
return resetDb(Cypress.config().apiServerUrl);
});
But then I get an warning saying:
Cypress detected that you returned a promise in a test, but also invoked one or more cy commands inside of that promise
I'm not invoking cy in resetDb().
Cypress have promises (Cypress.Promise), but they are not real promises, more like duck typing. In fact, Cypress isn't 100% compatible with real promises, they might, or might not, work.
Think of Cypress.Promise as a Task or an Action. They are executed sequentially with all other cypress commands.
To get your function into the Cypress pipeline you can use custom commands. The documentation doesn't state it, but you can return a Cypress.Promise from them.
Cypress.Commands.add('resetDb', function () {
var apiServerUrl = Cypress.config().apiServerUrl;
return new Cypress.Promise((resolve, reject) => {
httpRequest('PUT', apiServerUrl + "/api/test/reset/")
.then(function (data) {
resolve();
})
.catch(function (err) {
reject(err);
});
});
});
That command can then be executed from the test itself, or as in my case from before().
describe('Account', function () {
before(() => {
cy.resetDb();
});
it('can login', function () {
// test code
});
})
You can use cy.wrap( promise ), although there might still be a bug where it never times out (haven't tested).
Otherwise, you can use cy.then() (which is undocumented, can break in the future, and I'm def not doing any favors by promoting internal APIs):
cy.then(() => {
return myAsyncFunction();
});
You can use both of these commands at the top-level of spec like you'd use any command and it'll be enqueued into cypress command queue and executed in order.
But unlike cy.wrap (IIRC), cy.then() supports passing a callback, which means you can execute your async function at the time of the cy command being executed, not at the start of the spec (because expressions passed to cy commands evaluate immediately) --- that's what I'm doing in the example above.
I'm trying to get the results from a mock backend in Angular 2 for unit testing. Currently, we are using fakeAsync with a timeout to simulate the passing of time.
current working unit test
it('timeout (fakeAsync/tick)', fakeAsync(() => {
counter.getTimeout();
tick(3000); //manually specify the waiting time
}));
But, this means that we are limited to a manually defined timeout. Not when the async task is completed. What I'm trying to do is getting tick() to wait until the task is completed before continuing with the test.
This does not seem to work as intended.
Reading up on the fakeAsync and tick the answer here explains that:
tick() simulates the asynchronous passage of time.
I set up a plnkr example simulating this scenario.
Here, we call the getTimeout() method which calls an internal async task that has a timeout. In the test, we try wrapping it and calling tick() after calling the getTimeout() method.
counter.ts
getTimeout() {
setTimeout(() => {
console.log('timeout')
},3000)
}
counter.specs.ts
it('timeout (fakeAsync/tick)', fakeAsync(() => {
counter.getTimeout();
tick();
}));
But, the unit test fails with the error "Error: 1 timer(s) still in the queue."
Does the issue here in the angular repo have anything to do with this?
Is it possible to use tick() this way to wait for a timeout function? Or is there another approach that I can use?
The purpose of fakeAsync is to control time within your spec. tick will not wait for any time as it is a synchronous function used to simulate the passage of time. If you want to wait until the asynchronous function is complete, you are going to need to use async and whenStable, however, in your example, the spec will take 3 seconds to pass so I wouldn't advise this.
The reason why the counter.spec.ts is failing is that you have only simulated the passage of 0 seconds (typically used to represent the next tick of the event loop). So when the spec completes, there are still mocked timers active and that fails the whole spec. It is actually working properly by informing you that a timeout has been mocked an is unhandled.
Basically, I think you are attempting to use fakeAsync and tick in ways for which they were not intended to be used. If you need to test a timeout the way that you have proposed, the simplest way would be to mock the setTimeout function yourself so that, regardless of the time used, you can just call the method.
EDITED
I ran into a related issue where I wanted to clear the timers, and since it was not the part under test, I didn't care how long it took. I tried:
tick(Infinity);
Which worked, but was super hacky. I ended up going with
discardPeriodicTasks();
And all of my timers were cleared.
Try to add one or a combination of the following function calls to the end of your test:
flush();
flushMicrotasks();
discardPeriodicTasks();
or try to "kill" the pending tasks like this:
Zone.current.get('FakeAsyncTestZoneSpec').pendingTimers = [];
Zone.current.get('FakeAsyncTestZoneSpec').pendingPeriodicTimers = [];
flush (with optional maxTurns parameter) also flushes macrotasks. (This function is not mentionned in the Angular testing tutorial.)
flushMicrotasks flushes the microtask queue.
discardPeriodicTasks cancels "periodic timer(s) still in the queue".
Clearing the pending (periodic) timers array in the current fake async zone is a way to avoid the error if nothing else helps.
Timers in the queue do not necessarily mean that there's a problem with your code. For example, components that observe the current time may introduce such timers. If you use such components from a foreign library, you might also consider to stub them instead of "chasing timers".
For further understanding you may look at the javascript code of the fakeAsync function in zone-testing.js.
At the end of each test add:
fixture.destroy();
flush();
Try this:
// I had to do this:
it('timeout (fakeAsync/tick)', (done) => {
fixture.whenStable().then(() => {
counter.getTimeout();
tick();
done();
});
});
Source
Async
test.service.ts
export class TestService {
getTimeout() {
setTimeout(() => { console.log("test") }, 3000);
}
}
test.service.spec.ts
import { TestBed, async } from '#angular/core/testing';
describe("TestService", () => {
let service: TestService;
beforeEach(() => {
TestBed.configureTestingModule({
providers: [TestService],
});
service = TestBed.get(TestService);
});
it("timeout test", async(() => {
service.getTimeout();
});
});
Fake Async
test.service.ts
export class TestService {
readonly WAIT_TIME = 3000;
getTimeout() {
setTimeout(() => { console.log("test") }, this.WAIT_TIME);
}
}
test.service.spec.ts
import { TestBed, fakeAsync } from '#angular/core/testing';
describe("TestService", () => {
let service: TestService;
beforeEach(() => {
TestBed.configureTestingModule({
providers: [TestService],
});
service = TestBed.get(TestService);
});
it("timeout test", fakeAsync(() => {
service.getTimeout();
tick(service.WAIT_TIME + 10);
});
});
I normally use the flushMicrotasks method in my unit tests for use with my services. I had read that tick() is very similar to flushMicrotasks but also calls the jasmine tick() method.
For me all above didnt helped, but a double call of tick(<async_time>) in my test code.
My explanation so far: for every async call you a need a single/own tick()-call.
I have a .pipe(debounceTime(500)) and a timer(500).subscribe(..) afterwards and this helped:
tick(500);
tick(500);
I run mocha with mocha --slow 0 ./test/test.js to show the time it takes for calls to complete. I'm writing a library that does very async things with another service and response time matters. However some of the calls in my library cause another service to do slow operations like start, stop, remove more time to complete then exist between two consecutive tests. The result is that the printed time for the remove test is shown to be longer then the actual time it takes for the call to complete.
describe(`stopContainer ${test_name} t:0`, () => {
it(`should stop the container named ${test_name}`, () => {
return engine.stopContainer(test_name, {t:0}).should.be.fulfilled
})
})
// this shows to be much longer than the actual call takes
describe(`removeContainer ${test_name} v:1 `, () => {
it(`should remove the container named ${test_name} & volumes`, function(done) {
// needs to wait a second because of engine latency
this.timeout(5000)
setTimeout(() => {
engine.removeContainer(test_name, {v:1}).should.be.fulfilled.and.notify(done);
}, 1000)
})
})
Prints the following
stopContainer dap_test_container t:0
✓ should stop the container named dap_test_container (285ms)
removeContainer dap_test_container v:1
✓ should remove the container named dap_test_container & volumes (1405ms)
Obviously the last test took 1000ms less then is reported. But I have to do this all over the place with hundreds of tests and so the reported values become more meaningless as I cannot keep track of which ones are delayed and which are not.
note I don't mean to use this as a method of profiling my code, this is just to make my test results more meaningful.
I'd like to reduce the printed time, is there a way with mocha to reduce the printed time manually? Or does mocha provide a better construct for doing this?
It's very simple, delay in the before or after.
I made a global called wait
global.wait = wait = function(ms) {
return new Promise(resolve => {
setTimeout(resolve, ms)
})
}
And use it in the tests like so
describe(`removeContainer ${test_name} v:1 `, () => {
before('wait for latency', () => wait(1000))
it(`should remove the container named ${test_name} & volumes`, () => {
return engine.removeContainer(test_name, {v:1}).should.be.fulfilled;
})
})
Would I need to use a setTimeout? (And it works when I do). The problem I'm having is that the FeedObject does an asynchronous call, so when a new instance of it is created, it takes sometime before it can inject itself to the DOM.
describe('Default Case', function () {
before(function () {
$divHolder = $('#divHolder');
$divHolder.html('');
var myComponent = new FeedObject({
div: $divHolder[0],
prop: {
id: 'myFeed',
},
client: {}
});
myComponent.createDOMElements();
});
it('should exist', function () {
console.log($divHolder.find('.feed-item')); // This does not exist unless I use a timeout
});
EDIT: Just to clarify, the new FeedObject does an AJAX call.
What you need to determine when new FeedObject is done. Depending on what FeedObject provides, it could mean:
Providing a callback that it called when new FeedObject is done. Call done in the callback.
Listening to an event (most likely a custom jQuery event). Call done in the event handler.
Make the before hook return a promise that exists on the object returned by new FeedObject. For instance, the ready field could contain this promise, and you'd do return myComponent.ready. Mocha will wait for the promise to be resolved. (Remove the done argument from your before hook.)
I strongly suggest that you add one of the facilities above rather than use setTimeout. The problem with setTimeout is that it is always suboptimal: either you wait longer than you need for the DOM to be ready, or you set a timeout which is super close to the real time it take to be ready but in some cases the timeout will be too short (e.g. if your machine happens to have a spike of activity that does not leave enough CPU for the browser running the test code) and the test will fail for bogus reasons.
Well setTimeOut will work (until the server response takes longer than the time you specified and then it won’t work) but it’s definitely not the best or fastest way to do it. When you’re using an async call, you need to define a callback method that runs after the code is finished. The idea of the code should be something like this:
describe('Default Case', function () {
// Your code goes here
$.ajax = function(ajaxOpts)
{
var doneCallback = ajaxOpts.done;
doneCallback(simulatedAjaxResponse);
};
function fetchCallback(user)
{
expect(user.fullName).to.equal("Tomas Jakobsen");
done();
};
fetchCurrentUser(fetchCallback);
});
If you need even more details you could check this very helpful link. Hope that helps!
I would use this async chai plugin. If you are using a promise api (probably the most straight-forward way to handle async testing), this makes testing async ops very simple. You just have to remember to return any invocation that is async (or returns a promise) so chai knows to 'wait' before continuing.
describe(() => {
let sut
beforeEach(() => {
sut = new FeedObject()
return sut.createDomElements() // <- a promise
})
it('should exist', () => {
$('#target').find('.feed-item').should.exist
})
})
Also consider the goals of this testing: why are you doing it? I find that a lot of DOM insert/remove/exists testing is wasted effort. This is particularly true if you are testing a 3rd party library/framework as part of your application code. The burden of proof is on the library/framework to prove it is correct (and most well-written libraries already have a testing suite), not your app. If you are concerned about testing 'did I correctly invoke this 3rd party code' there are better ways to do that without touching the DOM.