I'm writing tests for my Node.js/Express/Mongoose project using Mocha and Should.js, and I'm testing out my functions that access my MongoDB. I'm want these tests to be completely independent from the actual records in my database, so I want to create an entry and then load it, and do all my tests on it, then delete it. I have my actual functions written (I'm writing tests after the entire project is complete) such that the create function does not have a callback; it simply just renders a page when it's done. In my tests script, I call my load_entry function after I call create, but sometimes create takes longer than usual and thus load_entry throws an error when it cannot actually load the article since it has yet to be created. Is there any way to make sure an asynchronous function is finished without using callbacks?
Please let me know if there is any more info I can provide. I looked all over Google and couldn't find anything that really answered my question, since most solutions just say "use a callback!"
Use what is known as a promise
You can read more about it here.
There are lots of great libraries that can do this for you.
Q.js is one I personally like and it's widely used nowadays. Promises also exist in jQuery among many others.
Here's an example of using a q promise with an asynchronous json-p call: DEMO
var time;
$.ajax({
dataType: 'jsonp',
type: 'GET',
url: "http://www.timeapi.org/utc/now.json",
success: function (data) {
time = data;
},
error: function (data) {
console.log("failed");
}
})
.then(function(){ // use a promise library to make sure we synchronize off the jsonp
console.log(time);
});
This is definitely the kind of thing you want a callback for. Barring that, you're going to have to write some kind of callback wrapper that polls the database to determine when it has finished creating the relevant records, and then emits an event or does some other async thing to allow the test to continue.
Since the only native way to do asynchronous things are: setTimeout, setInterval and addEventListener, and they all take a callback you will eventually have to use a callback somewhere.
However, you can hide that by using Promises/A, also known as Deferreds.
Your code could look like this:
db.create_entry("foo", data).done(function (entry) {
db.delete_entry(entry).done(function () {
console.log("entry is deleted");
});
});
Using then-chaining:
db.create_entry("foo", data).then(function (entry) {
return db.delete_entry(entry);
}).done(function () {
console.log("entry is deleted");
});;
I found a solution that works. What I did was to add a callback to my function (next) and only call it if it's specified (i.e., for the tests):
//more stuff above
article.save(function(err){
if (!err) {
console.log(req.user.username + ' wrote ' + article.slug)
return next() || res.redirect('/admin')
}
return next(err) || res.render('articles/entry_form', {
title: 'New Entry',
article: article,
})
})
This way, when I run the actual server and no callback is specified, it won't throw an error, because it will simply return the res.render statement.
Related
I have the following:
if(typeof searchDOM === "undefined"){
dojo.xhrPut({
url: addrPath + "/ContServlet?mod=1&act=23",
handleAs: "xml",
timeout: xhrTimeout(TIMEOUT_LRG),
load: function(dom, ioArgs){
if(dom instanceof Error){
console.error(dom);
} else{
cacheDOM = dom;
}
},
error: function(response, ioArgs){
xhrError(ioArgs, methodName);
}
});
}
The variable cacheDOM is a global variable declared(but not initialised) elsewhere in another script. It is an xml document containing the entire dom, and it is passed into:
the problem is, cacheDOM is undefined when it gets to fetchXml, and this is causing problems for methods like selectNode further down the function.
I haven't had much exposure to xhr calls, or things such as deferreds or promises, but I think that they may be able to help with this. How do i code this so that the rest of the method that this block is in will only execute if cacheDOM has been assigned the value of dom? Or if deferreds are the answer, how would i incorporate them into this code? The version of dojo i am using is 1.7.8
Well, the problem is indeed that you're using an XHR request which is asynchronous. So, the fetchXml function has to wait until that request is completed.
There are several ways to do this, you could call the fetchXml function from within the load function of dojo.xhrPut, but this is not really a good solution when your project grows because it creates a lot of dependencies on each other.
So, some smart people created an API for resolving asynchronous requests, called promises/deferreds.
So, what you have to do is assigning a new deferred to cacheDOM, for example:
require(["dojo/_base/Deferred"], function(Deferred) {
cacheDOM = new Defered();
});
Then, in the fetchXml() code you have to change your code a bit to do this:
function fetchXml() {
cacheDOM.then(function(realCache) {
console.log(realCache);
});
}
So in stead of directly using cacheDOM you have to wait for it using cacheDOM.then(). It will fire a callback when it's resolved, and the data will be available in realCache.
An alternative would be to call the entire fetchXml function when the XHR request has fired:
cacheDOM.then(fetchXml);
function fetchXml(cacheDOM) {
// Work with cacheDOM
}
This might take less work and less alteration to the fetchXml function depending on how much it relies on cacheDOM.
Then finally, inside your dojo.xhrPut you will have to do the following:
cacheDOM.resolve("My data");
Where "My data" would be the actual data which you would put inside cacheDOM.
DEMO: http://jsfiddle.net/rf20s9hb/1/
I concede that, despite hours of reading and attempting, I am fundamentally unable to grasp something about Deferred promises and asynchrony in general.
The goal on my end is real, real simple: send some data to the server, and react to the contents of the response conditionally.
The response will always be a JSON object with save and error keys:
{ "save": true, "error":false}
// or
{ "save" : false,
"error" : "The server has run off again; authorities have been notifed."}
I have tried dozens and dozens of variations from the jQuery API, from other stackexchange answers, from tutorials, etc.. The examples all seem concerned with local asynchronous activity. When I need is some ability to be made aware when the AJAX request has either finished and returned a response I can inspect and make decisions about, or else to know that it's failed. Below, I've used comments to explain what I think is happening so someone can show me where I'm failing.
I know this is a repost; I am, apprently, worse than on average at grasping this.
var postData = {"id":7, "answer":"Ever since I went to Disneyland..."};
/* when(), as I understand it, should fire an event to be
responded to by then() when it's contents have run their course */
var result = $.when(
/* here I believe I'm supposed to assert what must complete
before the when() event has fired and before any chained
functions are subsequently called */
/* this should return a jqXHR object to then(), which is,
I'd thought, a queue of functions to call, in order,
UPON COMPLETION of the asynchronous bit */
$.post("my/restful/url", postData))
.then( function() {
/* since "this" is the jqXHR object generated in the $.post()
call above, and since it's supposed to be completed by now,
it's data key should be populated by the server's response—right? */
return this.data;
});
// alas, it isn't
console.log(result.data);
// >> undefined
Most examples I can find discuss a timeout function; but this seems, as I understand, to be a failsafe put in place to arbitrarily decide when the asynchronous part is said to have failed, rather than a means of stalling for time so the request can complete. Indeed, if all we can do is just wait it out, how's that any different from a synchronous request?
I'll even take links to a new read-mes, tutorials, etc. if they cover the material in a different way, use something other than modified examples from the jQuery API, or otherwise help this drooling idiot through the asynchronous mirk; here's where I've been reading to date:
jQuery API: Deferred
JQuery Fundamentals
jQuery Deferreds promises asynchronous bliss (blog)
StackOverflow: timeout for function (jQuery)
Update
This is in response to #Kevin B below:
I tried this:
var moduleA = {
var moduleB = {
postData: {"id":7, "answer":"Ever since I went to Disneyland..."};
save: function() {
return $.post("path/to/service", postData, null, "JSON");
}
};
var result = this.moduleB.save();
result.done(function(resp) {
if (resp.saved == true) {
// never reached before completion
console.log("yahoo");
} else {
console.log("Error: " + resp.error);
// >> undefined
}
});
}
You are over-complicating your code. You cannot get the data to outside of the callback, no matter how many deferred/promises you create/use (your sample creates 3 different deferred objects!)
Use the done callback.
var postData = {"id":7, "answer":"Ever since I went to Disneyland..."};
$.post("my/restful/url", postData).done(function (result) {
console.log(result.save, result.error);
});
You seem to have a misunderstanding of both asynchronous requests, the Promise pattern, and Javascripts mechanism of passing functions as an argument.
To understand what's really happening in your code I suggest you use a debugger and set some breakpoints in the code. Or, alternatively, add some console.logs in your code. This way you can see the flow of the program and might understand it better. Also be sure to log the arguments of the function you pass as an argument in the then()-method, so you understand what is passed.
ok you got it half right. the problem is that when you execute the console.log the promised is not yet fulfilled the asynchronous nature of the promises allows the code to execute before that ajax operation is done. also result is a deferred not a value, you need to handle your promised with .done instead of .then if you wish to return a value otherwise you'll continue passing promises.
so that said
var result={};
$.when(
$.post("my/restful/url", postData))
.done( function(data) {
result.data=data;
});
// here result is an object and data is a undefined since the promised has no yet been resolve.
console.log(result.data);
I am quite new (just started this week) to Node.js and there is a fundamental piece that I am having trouble understanding. I have a helper function which makes a MySQL database call to get a bit of information. I then use a callback function to get that data back to the caller which works fine but when I want to use that data outside of that callback I run into trouble. Here is the code:
/** Helper Function **/
function getCompanyId(token, callback) {
var query = db.query('SELECT * FROM companies WHERE token = ?', token, function(err, result) {
var count = Object.keys(result).length;
if(count == 0) {
return;
} else {
callback(null, result[0].api_id);
}
});
}
/*** Function which uses the data from the helper function ***/
api.post('/alert', function(request, response) {
var data = JSON.parse(request.body.data);
var token = data.token;
getCompanyId(token, function(err, result) {
// this works
console.log(result);
});
// the problem is that I need result here so that I can use it else where in this function.
});
As you can see I have access to the return value from getCompanyId() so long as I stay within the scope of the callback but I need to use that value outside of the callback. I was able to get around this in another function by just sticking all the logic inside of that callback but that will not work in this case. Any insight on how to better structure this would be most appreciated. I am really enjoying Node.js thus far but obviously I have a lot of learning to do.
Short answer - you can't do that without violating the asynchronous nature of Node.js.
Think about the consequences of trying to access result outside of your callback - if you need to use that value, and the callback hasn't run yet, what will you do? You can't sleep and wait for the value to be set - that is incompatible with Node's single threaded, event-driven design. Your entire program would have to stop executing whilst waiting for the callback to run.
Any code that depends on result should be inside the getCompanyId callback:
api.post('/alert', function(request, response) {
var data = JSON.parse(request.body.data);
var token = data.token;
getCompanyId(token, function(err, result) {
//Any logic that depends on result has to be nested in here
});
});
One of the hardest parts about learning Node.js (and async programming is general) is learning to think asynchronously. It can be difficult at first but it is worth persisting. You can try to fight and code procedurally, but it will inevitably result in unmaintainable, convoluted code.
If you don't like the idea of multiple nested callbacks, you can look into promises, which let you chain methods together instead of nesting them. This article is a good introduction to Q, one implementation of promises.
If you are concerned about having everything crammed inside the callback function, you can always name the function, move it out, and then pass the function as the callback:
getCompanyId(token, doSomethingAfter); // Pass the function in
function doSomethingAfter(err, result) {
// Code here
}
My "aha" moment came when I began thinking of these as "fire and forget" methods. Don't look for return values coming back from the methods, because they don't come back. The calling code should move on, or just end. Yes, it feels weird.
As #joews says, you have to put everything depending on that value inside the callback(s).
This often requires you passing down an extra parameter(s). For example, if you are doing a typical HTTP request/response, plan on sending the response down every step along the callback chain. The final callback will (hopefully) set data in the response, or set an error code, and then send it back to the user.
If you want to avoid callback smells you need to use Node's Event Emitter Class like so:
at top of file require event module -
var emitter = require('events').EventEmitter();
then in your callback:
api.post('/alert', function(request, response) {
var data = JSON.parse(request.body.data);
var token = data.token;
getCompanyId(token, function(err, result) {
// this works
console.log(result);
emitter.emit('company:id:returned', result);
});
// the problem is that I need result here so that I can use it else where in this function.
});
then after your function you can use the on method anywhere like so:
getCompanyId(token, function(err, result) {
// this works
console.log(result);
emitter.emit('company:id:returned', result);
});
// the problem is that I need result here so that I can use it else where in this function.
emitter.on('company:id:returned', function(results) {
// do what you need with results
});
just be careful to set up good namespacing conventions for your events so you don't get a mess of on events and also you should watch the number of listeners you attach, here is a good link for reference:
http://www.sitepoint.com/nodejs-events-and-eventemitter/
Callbacks are more and more a requirement in coding, especially when you think about Node.JS non-blocking style of working. But writing a lot of coroutine callbacks quickly becomes difficult to read back.
For example, imagine something like this Pyramid Of Doom:
// This asynchronous coding style is really annoying. Anyone invented a better way yet?
// Count, remove, re-count (verify) and log.
col.count(quertFilter, function(err, countFiltered) {
col.count(queryCached, function(err, countCached) {
col.remove(query, function(err) {
col.count(queryAll, function(err, countTotal) {
util.log(util.format('MongoDB cleanup: %d filtered and %d cached records removed. %d last-minute records left.', countFiltered, countCached, countTotal));
});
});
});
});
is something we see often and can easily become more complex.
When every function is at least a couple of lines longer, it starts to become feasible to separate the functions:
// Imagine something more complex
function mary(data, pictures) {
// Do something drastic
}
// I want to do mary(), but I need to write how before actually starting.
function nana(callback, cbFinal) {
// Get stuff from database or something
callback(nene, cbFinal, data);
}
function nene(callback, cbFinal, data) {
// Do stuff with data
callback(nini, cbFinal, data);
}
function nini(callback, data) {
// Look up pictures of Jeff Atwood
callback(data, pictures);
}
// I start here, so this story doesn't read like a book even if it's quite straightforward.
nana(nene, mary);
But there is a lot of passing vars around happening all the time. With other functions written in between, this becomes hard to read. The functions itself might be too insignificant on their own to justify giving them their own file.
Use an async flow control library like async. It provides a clean way to structure code that requires multiple async calls while maintaining whatever dependency is present between them (if any).
In your example, you'd do something like this:
async.series([
function(callback) { col.count(queryFilter, callback); },
function(callback) { col.count(queryCached, callback); },
function(callback) { col.remove(query, callback); },
function(callback) { col.count(queryAll, callback); }
], function (err, results) {
if (!err) {
util.log(util.format('MongoDB cleanup: %d filtered and %d cached records removed. %d last-minute records left.',
results[0], results[1], results[3]));
}
});
This would execute each of the functions in series; once the first one calls its callback the second one is invoked, and so on. But you can also use parallel or waterfall or whatever flow matches the flow you're looking for. I find it's much cleaner than using promises.
A different approach to callbacks are promises.
Example: jQuery Ajax. this one might look pretty familiar.
$.ajax({
url: '/foo',
success: function() {
alert('bar');
}
});
But $.ajax also returns a promise.
var request = $.ajax({
url: '/foo'
});
request.done(function() {
alert('bar');
});
A benefit is, that you simulate synchronous behavior, because you can use the returned promise instead of providing a callback to $.ajax.success and a callback to the callback and a callback.... Another advantage is, that you can chain / aggregate promises, and have error handlers for one promise-aggregate if you like.
I found this article to be pretty useful.
It describes the pro and cons of callbacks, promises and other techniques.
A popular implementation (used by e.g. AngularJS iirc) is Q.
Combined answers and articles. Please edit this answer and add libraries/examples/doc-urls in a straightforward fasion for everyone's benefit.
Documentation on Promises
Asynchronous Control Flow with Promises
jQuery deferreds
Asynchronous Libraries
async.js
async.waterfall([
function(){ // ... },
function(){ // ... }
], callback);
node fibers
step
Step(
function func1() {
// ...
return value
},
function func2(err, value) {
// ...
return value
},
function funcFinal(err, value) {
if (err) throw err;
// ...
}
);
Q
Q.fcall(func1)
.then(func2)
.then(func3)
.then(funcSucces, funcError)
API reference
Mode examples
More documentation
Because of the complexity of this application, I have a need to wrap Facebook API calls, like so.
//In main file, read is always undefined
var read = fb_connect.readStream();
// In fb_wrapper.js
function readStream () {
var stream;
FB.api('/me/feed', {limit:10000}, function (response) {
stream = response.data;
});
return stream;
}
I know that due to the asynchronous nature of the call, the rest of the readStream() function will return stream (which has no value). I am having trouble finding a way of getting the data out of the callback function scope and back up to a higher scope. The FB API call is returning fine (I have debugged it a hundred times), but getting that response data has been the battle thus far.
If anyone has any suggestions, it would be much appreciated. I searched for Facebook jQuery plug-ins (as a pre-made wrapper, perhaps) with little luck.
Judging from your question, it seems that you are looking for a synchronous call. Which means that you'd want to use the data returned from the api call right after calling it. In that case, you'll need to check whether FB.api supports synchronous calls (mostly doesn't).
Otherwise, you'll need to understand that you are making an async call here. Which means that you should put your handling code INSIDE the callback function that you pass to FB.api. This is called the "continuation" style of writing code and is the standard way to use async calls.
FB.api('/me/feed', {limit:10000}, function (response) {
var stream = response.data;
// Do your processing here, not outside!!!
});
Or:
function handlerFunction(response) {
// Do your processing here
}
FB.api('/me/feed', {limit:10000}, handlerFunction);