As I'm currently learning coding with express/Node in order to evolved (used to C and PHP/MySQL...) I have completed the MDN tutorial on express which very well done and every thing is pretty much straight forward; my personnel project is going almost done, thanks to the Mozilla teaching team.
However, here is a point I still can't figure out as I'm still not confortable with the use of Callbacks function.
The point of dealing with asynchronous timing of execution I get, but using MongoDB and mongoose in this tutorial, I've got that queries can be executed either in two steps or in one go by using directly a callback function like creating an instance of a Schema:
// Create an instance of model SomeModel
var awesome_instance = new SomeModel({ name: 'awesome' });
// Save the new model instance, passing a callback
awesome_instance.save(function (err) {
if (err) return handleError(err);
// saved!
});
or
SomeModel.create({ name: 'also_awesome' }, function (err, awesome_instance) {
if (err) return handleError(err);
// saved!
});
However, in the JS script given to fill the DB with data, it seems that both syntaxes are used, here is an example:
function authorCreate(first_name, family_name, d_birth, d_death, cb) {
authordetail = {first_name:first_name , family_name: family_name }
if (d_birth != false) authordetail.date_of_birth = d_birth
if (d_death != false) authordetail.date_of_death = d_death
var author = new Author(authordetail);
author.save(function (err) {
if (err) {
cb(err, null)
return
}
console.log('New Author: ' + author);
authors.push(author)
cb(null, author)
} );
}
What troubles me is that
"cb" is never defined,
the script works the same if I delete all callbacks from arguments "defined" in functions and their respective calling
What is the point of cb(null, author): no data need to be return, they are push to the declared array and saved to the DB at the same time.
The full script can be found here: https://raw.githubusercontent.com/hamishwillee/express-locallibrary-tutorial/master/populatedb.js
Thanks anyone who takes time to read and answer me,
Tiago
Welcome to stack
in Js world , the function can be treated as a variable , the function is a type of variable like String , Number …etc , which may confuse you .. the callback is just a function that has a variable ( argument ) name … now swift language has implemented that pattern as well .
The functions in JS could be sent to other functions as an argument and could be returned as well.
The callback argument sent by the function caller like any other variable sent to the called function but sent as a function …
You could name it anything like cb or whatever … The cb argument is a function which means when you try to use it inside the called function it may need arguments as well like cb(null, author) .. Now imagine the cb is just a variable of type function sent from the caller to the called function.. and the cb itself as a function has to receive arguments .
I know it’s confusing especially if cb is returned as well .
Sometimes callbacks don’t receive any arguments , Again you may call them anything like callback or cb as a convention .. Usually they are the last parameter, as a convention … It just needs some imagination to understand how this confusing parameter goes in and out …
Inside your called function if you didn’t use your callback so removing it will not affect the code as you will not examine errors . BUT in node world callbacks are usually important because it does something after you finish what you’re doing .
i hope that simple illustration clarifies what you’re asked because it has confued me a lot like you .
Related
I have a question about asynchronous/synchronous when we write code in JS and I have done google search but I am still a bit confused.
I understand that we use callback functions when we want to make sure that the callback function is executed only after the asynchronous task (such as accessing database) in the outer function is complete. I understand how deserialize and serialize user works.
I am confused why when we do serialize user or deserialize user with passport.js we need a callback function like this ?
passport.serializeUser((user, done) => {
done(null, user.id);
});
If all we want is that the inner arrow function that is passed as an argument to serializeUser() to be executed only after serializeUser() is finished. Or why do we need to pass it as a callback function instead of calling that arrow function below serializeUser() ? I thought JS is synchronous so it will execute the arrow function after serializeUser() is completed anyway ?
I only found serializeUser() documentation in passport documentation on how to use it, but not its implementation so I am also confused whether serializeUser() or deserializeUser()( or any other passport functions) are asynchronous functions ?
Thank you !
It is a fragment of this function on github (https://github.com/jaredhanson/passport/blob/08f57c2e3086955f06f42d9ac7ad466d1f10019c/lib/authenticator.js).
As you can see this function takes a fn parameter. Then it checks if it is a function. If yes it pushes it to this._serializers. Later it does 'some magic' on it, I think it's not important now.
As you can read in jsdoc, serializeUser purpose is to register a function used to serialize user objects. So you pass some function which is invoked later in a code with 2 arguments which labels on that particular function level are user and done.
They left you some space which you can fill with your own code. You can tell how this will behave. They gave you an opportunity to implement your own logic.
Simple example:
function doSomeMagic(fn) {
// I do some my staff
const a = 5;
const b = 10;
// and now I let you decide what to do
// you can implement your own logic in my function
const result = fn(a, b);
console.log(result);
}
doSomeMagic((a, b) => {
return a * b;
});
doSomeMagic((a, b) => {
return a + b;
});
//output:
// 50
// 15
That's exactly what they've done. You can take my function, I give you two arguments do whatever you want. You can multiply, add, substract, whatever you want. I don't have to write separated functions to do math, I can write one, pass values as arguments and let you decided what operation you want to do. And it doesn't mean my code is going to run in an async way.
I'm working on a pre-existing Twilio function and am trying to understand how the following code works. I've tested it a few times, and as long as I send the 'from' phone number any message, I get the response described in the 'body'. However, I'm trying to understand how the 'msg' and 'err' parameters can exist, given they aren't declared elsewhere in the function. Can someone help me understand how these parameters are valid? I'm new to JavaScript and only understand the basics of callbacks and promises.
EDIT: I'm starting to understand the idea more. So the 'then' and 'catch' statements are callbacks that occur after the create() function finishes. Thank you for the answers that helped me understand.
exports.handler = function(context, event, callback) {
context.getTwilioClient().messages.create({
to: '+14806484732 ',
from: '+12133151685',
body: 'Remember the special today is Shrimp Scampi'
}).then(msg => {
callback(null, msg.sid);
}).catch(err => callback(err));
}
err => callback(err) - this is a declaration of function. single parameter err, executing function callback
similar situation with
.then(msg => { // `msg` is a single parameter of declared function
callback(null, msg.sid);
}
Example
const callback = argument => console.log(argument)
// this line is similar to
// const callback = (argument) => { console.log(argument) }
// also similar to
// function callback(argument) { console.log(argument) }
const func = err => callback(err)
// this line is similar to
// const func = (err) => { callback(err) }
// also similar to
// function func(err) { callback(err) }
func('test')
The functions given in then and catch are callbacks that both take a single parameter. Each of these parameters have a specific meaning based on the function documentation, which you can look up. You are, however, free to call them whatever you like if you were to write the callbacks yourself. The names are simply chosen to convey the semantics as clearly as possible.
Thus, they are essentially just parameters of an anonymous function.
I strongly recommend looking into callbacks in Javascript, as they are used everywhere.
I am struggling with getting my head around how to overcome and handle the async nature of Node.JS. I have done quite a bit of reading on it and tried to make Node do what I want by either using a message passing solution or callback functions.
My problem is I have a object where I want to constructor to load a file and populate an array. Then I want all calls to this function use that loaded data. So I need the original call to wait for the file to be loaded and all subsequent calls to use the already loaded private member.
My issue is that the function to load load the data and get the data is being executed async even if it return a function with a callback.
Anyways, is there something simple I am missing? Or is there an easier pattern I could use here? This function should return part of the loaded file but returns undefined. I have checked that the file is actually being loaded, and works correctly.
function Song() {
this.verses = undefined;
this.loadVerses = function(verseNum, callback) {
if (this.verses === undefined) {
var fs = require('fs'),
filename = 'README.md';
fs.readFile(filename, 'utf8', function(err, data) {
if (err) {
console.log('error throw opening file: %s, err: %s', filename, err);
throw err;
} else {
this.verses = data;
return callback(verseNum);
}
});
} else {
return callback(verseNum);
}
}
this.getVerse = function(verseNum) {
return this.verses[verseNum + 1];
}
}
Song.prototype = {
verse: function(input) {
return this.loadVerses(input, this.getVerse);
}
}
module.exports = new Song();
Update:
This is how I am using the song module from another module
var song = require('./song');
return song.verse(1);
"My issue is that the function to load the data and get the data is being executed async even if it return a function with a callback."
#AlbertoZaccagni what I mean by that scentence is that this line return this.loadVerses(input, this.getVerse); returns before the file is loaded when I expect it to wait for the callback.
That is how node works, I will try to clarify it with an example.
function readFile(path, callback) {
console.log('about to read...');
fs.readFile(path, 'utf8', function(err, data) {
callback();
});
}
console.log('start');
readFile('/path/to/the/file', function() {
console.log('...read!');
});
console.log('end');
You are reading a file and in the console you will likely have
start
about to read...
end
...read!
You can try that separately to see it in action and tweak it to understand the point. What's important to notice here is that your code will keep on running skipping the execution of the callback, until the file is read.
Just because you declared a callback does not mean that the execution will halt until the callback is called and then resumed.
So this is how I would change that code:
function Song() {
this.verses = undefined;
this.loadVerses = function(verseNum, callback) {
if (this.verses === undefined) {
var fs = require('fs'),
filename = 'README.md';
fs.readFile(filename, 'utf8', function(err, data) {
if (err) {
console.log('error throw opening file: %s, err: %s', filename, err);
throw err;
} else {
this.verses = data;
return callback(verseNum);
}
});
} else {
return callback(verseNum);
}
}
}
Song.prototype = {
verse: function(input, callback) {
// I've removed returns here
// I think they were confusing you, feel free to add them back in
// but they are not actually returning your value, which is instead an
// argument of the callback function
this.loadVerses(input, function(verseNum) {
callback(this.verses[verseNum + 1]);
});
}
}
module.exports = new Song();
To use it:
var song = require('./song');
song.verse(1, function(verse) {
console.log(verse);
});
I've ignored
the fact that we're not treating the error as first argument of the callback
the fact that calling this fast enough will create racing conditions, but I believe this is another question
[Collected into an answer and expanded from my previous comments]
TL;DR You need to structure your code such that the result of any operation is only used inside that operation's callback, since you do not have access to it anywhere else.
And while assigning it to an external global variable will certainly work as expected, do so will only occur after the callback has fired, which happens at a time you cannot predict.
Commentary
Callbacks do not return values because by their very nature, they are executed sometime in the future.
Once you pass a callback function into a controlling asynchronous function, it will be executed when the surrounding function decides to call it. You do not control this, and so waiting for a returned result won't work.
Your example code, song.verse(1); cannot be expected to return anything useful because it is called immediately and since the callback hasn't yet fired, will simply return the only value it can: null.
I'm afraid this reliance on asynchronous functions with passed callbacks is an irremovable feature of how NodeJS operates; it is at the very core of it.
Don't be disheartened though. A quick survey of all the NodeJS questions here shows quite clearly that this idea that one must work with the results of async operations only in their callbacks is the single greatest impediment to anyone understanding how to program in NodeJS.
For a truly excellent explanation/tutorial on the various ways to correctly structure NodeJS code, see Managing Node.js Callback Hell with Promises, Generators and Other Approaches.
I believe it clearly and succinctly describes the problem you face and provides several ways to refactor your code correctly.
Two of the features mentioned there, Promises and Generators, are programming features/concepts, the understanding of which would I believe be of great use to you.
Promises (or as some call them, Futures) is/are a programming abstraction that allows one to write code a little more linearly in a if this then that style, like
fs.readFileAsync(path).then(function(data){
/* do something with data here */
return result;
}).catch(function(err){
/* deal with errors from readFileAsync here */
}).then(function(result_of_last_operation){
/* do something with result_of_last_operation here */
if(there_is_a_problem) throw new Error('there is a problem');
return final_result;
})
.catch(function(err){
/* deal with errors when there_is_a_problem here */
}).done(function(final_result){
/* do something with the final result */
});
In reality, Promises are simply a means of marshaling the standard callback pyramid in a more linear fashion. (Personally I believe they need a new name, since the idea of "a promise of some value that might appear in the future" is not an easy one to wrap one's head around, at least it wasn't for me.)
Promises do this by (behind the scenes) restructuring "callback hell" such that:
asyncFunc(args,function callback(err,result){
if(err) throw err;
/* do something with the result here*/
});
becomes something more akin to:
var p=function(){
return new Promise(function(resolve,reject){
asyncFunc(args,function callback(err,result){
if(err) reject(err)
resolve(result);
});
});
});
p();
where any value you provide to resolve() becomes the only argument to the next "then-able" callback and any error is passed via rejected(), so it can be caught by any .catch(function(err){ ... }) handlers you define.
Promises also do all the things you'd expect from the (somewhat standard) async module, like running callbacks in series or in parallel and operating over the elements of an array, returning their collected results to a callback once all the results have been gathered.
But you will note that Promises don't quite do what you want, because everything is still in callbacks.
(See bluebird for what I believe is the simplest and thus, best Promises package to learn first.)
(And note that fs.readFileAsync is not a typo. One useful feature of bluebird is that it can be made to add this and other Promises-based versions of fs's existing functions to the standard fs object. It also understands how to "promisify" other modules such as request and mkdirp).
Generators are the other feature described in the tutorial above, but are only available in the new, updated but not yet officially released version of JavaScript (codenamed "Harmony").
Using generators would also allow you to write code in a more linear manner, since one of the features it provides is the ability of waiting on the results of an asynchronous operation in a way that doesn't wreak havoc with the JavaScript event-loop. (But as I said, it's not a feature in general use yet.)
You can however use generators in the current release of node if you'd like, simply add "--harmony" to the node command line to tell it to turn on the newest features of the next version of JavaScript.
Someone please help me. I have been reading a lot of javascript documentation and fiddling with javascript.
I am able to use the functions but I don't quite get this essential syntactic jazz going on here
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(1337, 'myhost');
console.log('Server running at http://myhost:1337/');
I cant figure out why its okay to use req and res in the anonymous function above. It's like they live somewhere inside of http. They are not declared anywhere! They are made up variable names in an anonymous function that reference inner objects or something. It's crazy.
How does a callback function like this?
or like
stream.on('data', function(data){
// use data... i dont know where its coming from
// or how it got there, but use it
});
If you could post a small example that mimics this process and syntax or explain how these callback functions can work like this or how I can pass these undeclared variables into functions like this I would greatly appreciate it.
A similar example to the answer posted below.
var b = {
somefunction : function( data ){
var x = 100;
x = x+1; // 101
return data(x); // returns the callback function data w/ value x
}
};
b.somefunction(function(foo){
foo++; // 101 + 1
console.log(foo); // prints 102
});
The main issue is that Javascript is a functional language so you can pass functions as parameters to other functions. In other languages you may have experienced passing a pointer or handle to a function, for example. Consider a simple cases in which you have some math functions:
function add(a,b) {return (a+b)};
function subtract(a,b) {return (a-b)}:
Now I could create a new function:
function longWayAroundTheBarn(num1,num2,theFunc)
{
// invoke the passed function with the passed parameters
return theFunc(num1,num2);
}
And call it like this:
console.log(longWayAroundTheBarn(1,2,add));
> 3
Or even like this:
console.log(longWayAroundTheBarn(longWayAroundTheBarn(2,2,subtract),4,add);
> 4
Obviously this would be a silly use callbacks, but you can imagine generally that the ability to 'plug-in' a function this way can be pretty powerful.
Consider if you couldn't pass functions. This might be one way you would implement this:
function reallyLongWayAround(num1,num2,funcName)
{
if(funcName==='add')
return add(num1 ,num2);
else if (funcName === 'subtract')
return subtract(num1, num2);
}
You can imagine that besides being really tedious code to have to write and maintain, it's not nearly so powerful because the reallyLongWayAround function could only ever call code it knew about. By passing functions, my longWayAroundTheBarn doesn't care if I create new functions and pass it to it. Note that because of weak typing, it doesn't even need to care about the parameters it is passing. Maybe you want to implement something like
function businessDaysBetween(d1,d2)
{
// implementation left as a reader exercise
};
It would work just fine to call:
longWayAroundTheBarn(new Date(2014,1,15), new Date(2014,1,22),businessDaysBetween)
Returning to the specific case you raised, req and res are not 'local variables' as one answer indicates - they are called parameters or arguments. You aren't passing anything into them. They are being passed to you by the calling function. You could actually call them fred and barney if you wanted, although it would be a terrible idea. The main point is that they will be called populated with request and response objects.
You actually don't even have to have the parameters in your function signature, you could just have a callback as below, and make use of the second parameter passed to your function by reading the arguments array (Note, it's not actually an array but behaves similarly in many respects). This would be a terrible, terrible idea, but again, trying to illustrate the point.
var http = require('http');
http.createServer(function () {
arguments[1].writeHead(200, {'Content-Type': 'text/plain'});
arguments[1].end('Hello World\n');
}).listen(1337, 'myhost');
The req and res are actually local variables of the anonymous function.
I have an example below similar to the codes posted in your question.
// static class
var HelperClass = {
"doSomething" : function ( callback ) {
// do something here, for example ajax call to a server
var data = ajaxCallFromServer();
// return the data via callback
callback ( data );
};
};
// you codes here calling the HelperClass
// calling .doSomething method with an anonymous function callback
// that expects 1 parameter. This parameter is returned by the above
// code via callback ( data ). And in the anonymous function
// i called it as retData, you can call it whatever you like
HelperClass.doSomething( function ( retData ) {
// do soemthing with your retData here
});
You should use the documentation.
So for example for createServer, follow this page - http://nodejs.org/api/http.html#http_http_createserver_requestlistener
http.createServer([requestListener])# Returns a new web server object.
The requestListener is a function which is automatically added to the
'request' event.
Then, you check the 'request' event -
Event: 'request'# function (request, response) { }
Emitted each time there is a request. Note that there may be multiple
requests per connection (in the case of keep-alive connections).
request is an instance of http.IncomingMessage and response is an
instance of http.ServerResponse.
Regarding the stream, exactly same. The docs here - http://nodejs.org/api/stream.html#stream_writable_stream
Look for
Event: 'data'
The point is that http.createServer function takes an argument that is not a variable, but a function, if that makes sense. In javascript you can do that. And that function expects arguments that are specified in it's API. You can make it anonymous, like in your example, or declared like below:
function serverFunction(req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}
var http = require('http');
http.createServer(serverFunction).listen(1337, 'myhost');
but in the end it does not matter, it just behaves accordingly to what is specified in its the API.
I am quite new (just started this week) to Node.js and there is a fundamental piece that I am having trouble understanding. I have a helper function which makes a MySQL database call to get a bit of information. I then use a callback function to get that data back to the caller which works fine but when I want to use that data outside of that callback I run into trouble. Here is the code:
/** Helper Function **/
function getCompanyId(token, callback) {
var query = db.query('SELECT * FROM companies WHERE token = ?', token, function(err, result) {
var count = Object.keys(result).length;
if(count == 0) {
return;
} else {
callback(null, result[0].api_id);
}
});
}
/*** Function which uses the data from the helper function ***/
api.post('/alert', function(request, response) {
var data = JSON.parse(request.body.data);
var token = data.token;
getCompanyId(token, function(err, result) {
// this works
console.log(result);
});
// the problem is that I need result here so that I can use it else where in this function.
});
As you can see I have access to the return value from getCompanyId() so long as I stay within the scope of the callback but I need to use that value outside of the callback. I was able to get around this in another function by just sticking all the logic inside of that callback but that will not work in this case. Any insight on how to better structure this would be most appreciated. I am really enjoying Node.js thus far but obviously I have a lot of learning to do.
Short answer - you can't do that without violating the asynchronous nature of Node.js.
Think about the consequences of trying to access result outside of your callback - if you need to use that value, and the callback hasn't run yet, what will you do? You can't sleep and wait for the value to be set - that is incompatible with Node's single threaded, event-driven design. Your entire program would have to stop executing whilst waiting for the callback to run.
Any code that depends on result should be inside the getCompanyId callback:
api.post('/alert', function(request, response) {
var data = JSON.parse(request.body.data);
var token = data.token;
getCompanyId(token, function(err, result) {
//Any logic that depends on result has to be nested in here
});
});
One of the hardest parts about learning Node.js (and async programming is general) is learning to think asynchronously. It can be difficult at first but it is worth persisting. You can try to fight and code procedurally, but it will inevitably result in unmaintainable, convoluted code.
If you don't like the idea of multiple nested callbacks, you can look into promises, which let you chain methods together instead of nesting them. This article is a good introduction to Q, one implementation of promises.
If you are concerned about having everything crammed inside the callback function, you can always name the function, move it out, and then pass the function as the callback:
getCompanyId(token, doSomethingAfter); // Pass the function in
function doSomethingAfter(err, result) {
// Code here
}
My "aha" moment came when I began thinking of these as "fire and forget" methods. Don't look for return values coming back from the methods, because they don't come back. The calling code should move on, or just end. Yes, it feels weird.
As #joews says, you have to put everything depending on that value inside the callback(s).
This often requires you passing down an extra parameter(s). For example, if you are doing a typical HTTP request/response, plan on sending the response down every step along the callback chain. The final callback will (hopefully) set data in the response, or set an error code, and then send it back to the user.
If you want to avoid callback smells you need to use Node's Event Emitter Class like so:
at top of file require event module -
var emitter = require('events').EventEmitter();
then in your callback:
api.post('/alert', function(request, response) {
var data = JSON.parse(request.body.data);
var token = data.token;
getCompanyId(token, function(err, result) {
// this works
console.log(result);
emitter.emit('company:id:returned', result);
});
// the problem is that I need result here so that I can use it else where in this function.
});
then after your function you can use the on method anywhere like so:
getCompanyId(token, function(err, result) {
// this works
console.log(result);
emitter.emit('company:id:returned', result);
});
// the problem is that I need result here so that I can use it else where in this function.
emitter.on('company:id:returned', function(results) {
// do what you need with results
});
just be careful to set up good namespacing conventions for your events so you don't get a mess of on events and also you should watch the number of listeners you attach, here is a good link for reference:
http://www.sitepoint.com/nodejs-events-and-eventemitter/