I'm using Sinon for stubbing some data retrieval methods during unit testing. Most of these data methods are async, so the resolves syntax has been handy so far for this. What I'm trying to achieve is to dynamically generate different test data based on Math.random() to cover different branches on my code automatically, without having actually to provide hardcoded sample input data for each case. Still, I've realized that the stub just is actually called once upon initialization and, not the return value of it gets fixed/constant during the execution of the testing process (Mocha based). Is there any way to actually provide different outcomes for a single stub using? I've checked the onCall syntax, but it also provides fixed output, just selectable based on current iteration index, but not actual dynamic output, which could even be args/params based, perhaps.
All ideas are welcome!
Current stubbing using Sinon:
sinon.stub(dynamodb, 'get').resolves(stubGet())
The stub itself:
function stubGet () {
// Choose random repo
const i = Math.round(Math.random() * sampleData.length)
const repo = sampleData[i]
// Should it have "new code/push date"?
const isNew = Math.round(Math.random()) === 1
if (isNew) {
repo.pushed_at = { S: '1970-01-01T00:00:00Z' }
}
console.log('repo', repo)
const item = { Item: repo }
console.log(item)
return item
}
The goal would be to hopefully get the random repo or the isNew value.
Randomness is unpredictable. Test code should be predictable, including test data. Otherwise, your tests could be failed with some random data in someday
We should write multiple test cases, each test case uses fixed, as simple as possible test data to test each branch, scene, etc. of the code. Assert whether the returned value meets your expectations.
You should make the test code, test data predictable. For more info, see Unpredictable Test Data
Related
This has been bugging me from a long time. I am not well versed with javascript. Here it goes:
How do i store return value of a function into a variable:
lenValue = cy.get(selector).children().length
Above line of code returns undefined But when i try following in cypress test runner console then I get a valid output:
cy.$$(selector).children().length --> gives me correct number
How to return value from inside a then function and catch it to reuse later:
file1.js
function a(selector, attrName){
cy.get(selector).then(function ($el){
return $el.attr(attrName));
}
file2.js
state = file1Obj.a('#name','name')
What you're doing makes complete sense, but simply put, you cannot. (per the docs).
https://docs.cypress.io/guides/core-concepts/variables-and-aliases/#Return-Values
You can, however, use aliases to accomplish what (I think) you're after.
https://docs.cypress.io/guides/core-concepts/variables-and-aliases/#Aliases
#aeischeid shows you the wrong way to do it.
His code works only for a static site, but web pages are rarely static. As soon as API fetches are involved, lucky timing goes out the window and the test bombs.
This is why Cypress commands have automatic retry. Otherwise we could just build tests with jQuery.
Since cy.$$(selector).children().length --> gives me correct number, use that inside the helper function.
function a(selector, attrName) {
return cy.$$(selector).attr(attrName); // jQuery methods used
}
Or
function a(selector, attrName) {
return Cypress.$(selector).attr(attrName); // jQuery methods used
}
But be aware that jQuery only handles static pages, it does not retry if the attribute that you want to query arrives slowly.
For that use a command
cy.get('#name')
.should('have.attr', 'name') // retries until name exists
.then(name => { // guaranteed to have a value
// use name here
})
Here is a example from a cypress test I have that seems pretty relevant
let oldDescription;
cy.get('input#description').should(($input) => {
oldDescription = $input.val();
});
let randomDescription = Math.random().toString(36).substring(7);
cy.get('input#description').clear().type(randomDescription);
cy.get('input#description')
.parents('.ant-table-row')
.contains('Save').click();
cy.get('input#description').should('not.exist');
cy.contains(`${randomDescription}`);
cy.contains(`${oldDescription}`).should('not.exist');
because oldDescription is set inside of an asyncronous callback it isn't safe to expect it to be set, farther down the code outside of that callback, but in a lot of cases with cypress you end up having some other .get call or thing that waits, effectively pausing the code long enough that you can get away with not worrying about it.
I tried to understand difference between sinon library's fake, spy, stub and mock but not able to understand it clearly.
Can anybody help me to understand about it?
Just for understanding purpose call:
FuncInfoCollector = is a Function that records arguments, return value, the value of this(context) and exception thrown (if any) for all of its calls.
(this FuncInfoCollector is dummy name given by me, it is not present in SINON lib)
Fake = FuncInfoCollector + can only create a fake function. To replace a function that already exists in the system under test you call sinon.replace(target, fieldname, fake). You can wrap an existing function like this:
const org = foo.someMethod;
sinon.fake((...args) => org(...args));
A fake is immutable: once created, the behavior can't be changed.
var fakeFunc = sinon.fake.returns('foo');
fakeFunc();
// have call count of fakeFunc ( It will show 1 here)
fakeFunc.callCount;
Spy = FuncInfoCollector + can create new function + It can wrap a function that already exists in the system under test.
Spy is a good choice whenever the goal of a test is to verify something happened.
// Can be passed as a callback to async func to verify whether callback is called or not?
const spyFunc = sinon.spy();
// Creates spy for ajax method of jQuery lib
sinon.spy(jQuery, "ajax");
// will tell whether jQuery.ajax method called exactly once or not
jQuery.ajax.calledOnce
Stub = spy + it stubs the original function ( can be used to change behaviour of original function).
var err = new Error('Ajax Error');
// So whenever jQuery.ajax method is called in a code it throws this Error
sinon.stub(jQuery, "ajax").throws(err)
// Here we are writing assert to check where jQuery.ajax is throwing an Error or not
sinon.assert.threw(jQuery.ajax(), err);
Mock = Stub + pre-programmed expectations.
var mk = sinon.mock(jQuery)
// Should be called atleast 2 time and almost 5 times
mk.expects("ajax").atLeast(2).atMost(5);
// It throws the following exception when called ( assert used above is not needed now )
mk.expects("ajax").throws(new Error('Ajax Error'))
// will check whether all above expectations are met or not, hence assertions aren't needed
mk.verify();
Please have a look at this link also sinon.replace vs sinon.stub just to replace return value?
Just to add some more info to the otherwise good answer, we added the Fake API to Sinon because of shortcomings of the other original APIs (Stub and Spy). The fact that these APIs are chainable led to constant design issues and recurring user-problems and they were bloated to cater to quite unimportant use cases, which is why we opted for creating a new immutable API that would be simpler to use, less ambiguous and cheaper to maintain. It was built on top of the Spy and Stub Apis to let Fakes be somewhat recognizable and have explicit method for replacing props on objects (sinon.replace(obj,'prop',fake)).
Fakes can essentially be used anywhere a stub or spy can be used and so I have not used the old APIs myself in 3-4 years, as code using the more limited fakes is simpler to understand for other people.
I am using Istanbul for code coverage, but i m getting very low coverage percentage particularly in Models file.
Consider the following is the model file:
ModelA.js
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
var app = require('../server')
var db = require('../db/dbConnection');
var config = require('../configs/config')
const Schema1 = new Schema({ 'configurations': [] });
exports.save = function (aa, data, callback) {
var logMeta = {
file: 'models/modelA',
function: 'save',
data: {},
error: {}
}
if (!aa) {
return callback('aa is required')
}
global.logs[aa].log('info', 'AA: ' + aa, logMeta);
db.connectDatabase(aa, function(error, mongoDB){
if(error){
logMeta.data['error'] = error
global.logs[aa].log('error', 'error', logMeta);
return callback(error)
}
const ModelA = mongoDB.model('bbb', cccc);
ModelA.findOneAndUpdate({}, data, {upsert: true, new: true, runValidators: true}, function(error ,result){
if (error) {
logMeta.data['error'] = error
global.logs[aa].log('error', 'error', logMeta);
}
else {
logMeta.data = {}
logMeta.data['result'] = JSON.parse(JSON.stringify(result))
global.logs[aa].log('info', 'result', logMeta);
}
callback(error, result);
});
})
}
TestA.js:
var should = require('should'),
sinon = require('sinon'),
ModelA= require("../models/ModelA");
describe('Model test', function () {
it('Should save Model', function (done) {
var todoMock = sinon.mock(new ModelA({'configurations': []}));
var todo = todoMock.object;
todoMock
.expects('save')
.yields(null, 'SAVED');
todo.save(function(err, result) {
todoMock.verify();
todoMock.restore();
should.equal('SAVED', result, "Test fails due to unexpected result")
done();
});
});
});
But i am getting codecoverage percentage 20. SO how can i increase the percentage:
ALso:
1.Whether i have to mock the db.connectDatabase if yews how can i acheive that?
Whether i have to use TestDb to run all my UnitTest? Or i have to assert??
Code Coverage will work for Unit Test or integration test???
Please share your ideas. Thanks
I have been using Istanbul to 100% code cover most of my client/server projects so I might have the answers you are looking for.
How does it work
Whenever you require some local file, this gets wrapped all over the place to understand if every of its parts is reached by your code.
Not only the required file is tainted, your running test is too.
However, while it's easy to code cover the running test file, mocked classes and their code might never be executed.
todoMock.expects('save')
Accordingly to Sinon documentation:
Overrides todo. save with a mock function and returns it.
If Istanbul tainted the real save method, anything within that scope won't ever be reached so that you are actually testing that mock works, not that your real code does.
This should answer your question: Code Coverage will work for Unit Test or integration test ???
The answer is that it covers the code, which is the only thing you're interested from a code cover perspective. Covering Sinon JS is nobody goal.
No need to assert ... but
Once you've understood how Istanbul works, it follows up naturally to understand that it doesn't matter if you assert or not, all it matters is that you reach the code for real and execute it.
Asserting is just your guard against failures, not a mechanism interesting per se in any Istanbul test. When your assertion fails, your test does too, so it's good for you to know that things didn't work and there's no need to keep testing the rest of the code (early failure, faster fixes).
Whether you have to mock the db.connectDatabase
Yes, at least for the code you posted. You can assign db as generic object mock to the global context and expect methods to be called but also you can simplify your life writing this:
function createDB(err1, err2) {
return {
connectDatabase(aa, callback) {
callback(err1, {
model(name, value) {
return {
findOneAndUpdate($0, $1, $3, fn) {
fn(err2, {any: 'object'});
}
};
}
});
}
};
}
global.db = createDB(null, null);
This code in your test file can be used to create a global db that behaves differently accordingly with the amount of errors you pass along, giving you the ability to run the same test file various times with different expectations.
How to run the same test more than once
Once your test is completed, delete require.cache[require.resolve('../test/file')] and then require('../test/file') again.
Do this as many times as you need.
When there are conditional features detection
I usually run the test various times deleting global constructors in case these are patched with a fallback. I also usually store them to be able to put 'em back later on.
When the code is obvious but shouldn't be reached
In case you have if (err) process.exit(1); you rarely want to reach that part of the code. There are various comments understood by Istanbul that would help you skip parts of the test like /* istanbul ignore if */ or ignore else, or even the generic ignore next.
Please consider thinking twice if it's just you being lazy, or that part can really, safely, be skipped ... I got bitten a couple of times with a badly handled error, which is a disaster since when it happens is when you need the most your code to keep running and/or giving you all the info you need.
What is being covered?
Maybe you know this already but the coverage/lcov-report/index.html file, that you can open right away with any browser, will show you all the parts that aren't covered by your tests.
Update
I've completely rewritten this question based on subsequent investigation. Hopefully this will generate some answers.
I'm new to Postman, and trying to figure out how to most efficiently build a collection of tests for a REST application. There are a bunch of utility functions that I'd like to have accessible in each of my test scripts, but cut-and-paste-ing them in to each test script seems like a horrible solution.
In looking at the various "scopes" that Postman allows you to squirrel away data (e.g. globals, environment, collection), it seems that all of these are merely string/number stores. In other words, it properly stores them if you can/do stringify the results. But it doesn't actually allow you to store proper objects or functions. This makes sense, since each script seems to be run as a separate execution, so the idea of sharing pointers to things between different scripts doesn't make sense.
It seems like the accepted way to share utility functions is to toString() the function in the defining script (e.g. the Collection Pre-Req script), and then eval() that stringified version in the test script. For instance:
Collection Pre-Req Script
const utilFunc = () => { console.log("I am a utility function"); };
pm.environment.set("utilFunc",utilFunc.toString() );
Test Script
const utilFunc = eval(pm.environment.get("utilFunc"));
utilFunc();
The test script will successfully print to console "I am a utility function".
I've seen people do more complicated things where, if they have more than one utility function, put them in to an object like utils.func1 and utils.func2, and have the overall function return the utils object, so the test script still only has to have a single line at the top importing the whole thing.
The problem I'm running in to is scoping - since the literal text of the function is executed in the Test Script, everything thing that the utility function has to have must be in that code, or otherwise exist at eval() time in the Test Script. For instance, if I do:
Collection Pre-Req Script
const baseUtilFunc = (foo) => { console.log(foo); };
const utilFunc1 = (param) => { baseUtilFunc("One: " + param); };
const utilFunc2 = (param) => { baseUtilFunc("Two: " + param); };
pm.environment.set("utilFunc1",utilFunc1.toString() );
pm.environment.set("utilFunc2",utilFunc2.toString() );
Test Script
const utilFunc1 = eval(pm.environment.get("utilFunc1"));
const utilFunc2 = eval(pm.environment.get("utilFunc2"));
utilFunc1("Test");
This fails because, in the Test Script, baseUtilFunc does not exist. Obviously, in this example, it'd be easy to fix. But in a more complicated world where the utility functions I expect to use in my Test Scripts are themselves built on top of underlying helper functions, it gets more difficult.
So what is the right way to handle this issue? Do people just cram all the relevant logic in to one big function that they then call toString() on? Do they embed an extraction-from-environment-and-then-eval in each util function within its definition, so that it works in the Test Script context? Do they export each individual method?
There are different ways to do it. The way I did recently for one of the projects is creating a project in Git and then using raw url to fetch the data. I have a sample created at below repo
https://github.com/tarunlalwani/postman-utils
To load the file you will need to associate the below code at collection level
if (typeof pmutil == "undefined") {
var url = "https://raw.githubusercontent.com/tarunlalwani/postman-utils/master/pmutils.js";
if (pm.globals.has("pmutiljs"))
eval(pm.globals.get("pmutiljs"))
else {
console.log("pmutil not found. loading from " + url);
pm.sendRequest(url, function (err, res) {
eval(res.text());
pm.globals.set('pmutiljs', res.text())
});
}
}
As shown in below screenshot
And the later in the tests or Pre-Requests you will run the below line of code to load it
eval(pm.globals.get("pmutiljs"))
And then you can use the functions easily in test.
On the Jasmine website I see that we can disable suites by xdescribe or individual specs by xit. Is there a way to disable only an expectation (like xexpect)?
The reason why I'm asking this is because I'm writing e2e tests with Protractor and in our continuous integration we don't yet (if ever) have access to the database, though locally we can run real end to end tests with access to the database, for example.
I would like to mark individual expectations as optional, depending on a configuration or environment variable. It would be nice to make a switch once, and then create a wrapper around expect, that only fails if we are running the tests locally (with access to the database).
So for example I can create a new spec family:
if (process.env.DB_AVAILABLE) {
dbit = it;
} else {
dbit = xit;
}
and write specs that depend on database connection as following:
dbit('creates new user', function () {});
Is there a way to do the same with expect (e.g. dbexpect)?
If there is something fundamentally wrong with my approach, don't hold it back and let me know.
You could create your own xexpect by implementing all the methods/properties with an empty function:
var xexpect = function() {
return xexpect;
};
Object.getOwnPropertyNames(jasmine.Expectation.prototype).forEach(function(name){
xexpect[name] = xexpect;
});
Object.defineProperty(xexpect, 'not', {get: xexpect});
Usage :
xexpect(1).toBeGreaterThan(2);
xexpect(true).not.toEqual(true);