Develop Reference

When to use promises over callback? [duplicate]

I've been developing JavaScript for a few years and I don't understand the fuss about promises at all.
It seems like all I do is change:
api(function(result){
api2(function(result2){
api3(function(result3){
// do work
});
});
});
Which I could use a library like async for anyway, with something like:
api().then(function(result){
api2().then(function(result2){
api3().then(function(result3){
// do work
});
});
});
Which is more code and less readable. I didn't gain anything here, it's not suddenly magically 'flat' either. Not to mention having to convert things to promises.
So, what's the big fuss about promises here?

Promises are not callbacks. A promise represents the future result of an asynchronous operation. Of course, writing them the way you do, you get little benefit. But if you write them the way they are meant to be used, you can write asynchronous code in a way that resembles synchronous code and is much more easy to follow:
api().then(function(result){
return api2();
}).then(function(result2){
return api3();
}).then(function(result3){
// do work
});
Certainly, not much less code, but much more readable.
But this is not the end. Let's discover the true benefits: What if you wanted to check for any error in any of the steps? It would be hell to do it with callbacks, but with promises, is a piece of cake:
api().then(function(result){
return api2();
}).then(function(result2){
return api3();
}).then(function(result3){
// do work
}).catch(function(error) {
//handle any error that may occur before this point
});
Pretty much the same as a try { ... } catch block.
Even better:
api().then(function(result){
return api2();
}).then(function(result2){
return api3();
}).then(function(result3){
// do work
}).catch(function(error) {
//handle any error that may occur before this point
}).then(function() {
//do something whether there was an error or not
//like hiding an spinner if you were performing an AJAX request.
});
And even better: What if those 3 calls to api, api2, api3 could run simultaneously (e.g. if they were AJAX calls) but you needed to wait for the three? Without promises, you should have to create some sort of counter. With promises, using the ES6 notation, is another piece of cake and pretty neat:
Promise.all([api(), api2(), api3()]).then(function(result) {
//do work. result is an array contains the values of the three fulfilled promises.
}).catch(function(error) {
//handle the error. At least one of the promises rejected.
});
Hope you see Promises in a new light now.

Yes, Promises are asynchronous callbacks. They can't do anything that callbacks can't do, and you face the same problems with asynchrony as with plain callbacks.
However, Promises are more than just callbacks. They are a very mighty abstraction, allow cleaner and better, functional code with less error-prone boilerplate.
So what's the main idea?
Promises are objects representing the result of a single (asynchronous) computation. They resolve to that result only once. There's a few things what this means:
Promises implement an observer pattern:
You don't need to know the callbacks that will use the value before the task completes.
Instead of expecting callbacks as arguments to your functions, you can easily return a Promise object
The promise will store the value, and you can transparently add a callback whenever you want. It will be called when the result is available. "Transparency" implies that when you have a promise and add a callback to it, it doesn't make a difference to your code whether the result has arrived yet - the API and contracts are the same, simplifying caching/memoisation a lot.
You can add multiple callbacks easily
Promises are chainable (monadic, if you want):
If you need to transform the value that a promise represents, you map a transform function over the promise and get back a new promise that represents the transformed result. You cannot synchronously get the value to use it somehow, but you can easily lift the transformation in the promise context. No boilerplate callbacks.
If you want to chain two asynchronous tasks, you can use the .then() method. It will take a callback to be called with the first result, and returns a promise for the result of the promise that the callback returns.
Sounds complicated? Time for a code example.
var p1 = api1(); // returning a promise
var p3 = p1.then(function(api1Result) {
var p2 = api2(); // returning a promise
return p2; // The result of p2 …
}); // … becomes the result of p3
// So it does not make a difference whether you write
api1().then(function(api1Result) {
return api2().then(console.log)
})
// or the flattened version
api1().then(function(api1Result) {
return api2();
}).then(console.log)
Flattening does not come magically, but you can easily do it. For your heavily nested example, the (near) equivalent would be
api1().then(api2).then(api3).then(/* do-work-callback */);
If seeing the code of these methods helps understanding, here's a most basic promise lib in a few lines.
What's the big fuss about promises?
The Promise abstraction allows much better composability of functions. For example, next to then for chaining, the all function creates a promise for the combined result of multiple parallel-waiting promises.
Last but not least Promises come with integrated error handling. The result of the computation might be that either the promise is fulfilled with a value, or it is rejected with a reason. All the composition functions handle this automatically and propagate errors in promise chains, so that you don't need to care about it explicitly everywhere - in contrast to a plain-callback implementation. In the end, you can add a dedicated error callback for all occurred exceptions.
Not to mention having to convert things to promises.
That's quite trivial actually with good promise libraries, see How do I convert an existing callback API to promises?

In addition to the already established answers, with ES6 arrow functions Promises turn from a modestly shining small blue dwarf straight into a red giant. That is about to collapse into a supernova:
api().then(result => api2()).then(result2 => api3()).then(result3 => console.log(result3))
As oligofren pointed out, without arguments between api calls you don't need the anonymous wrapper functions at all:
api().then(api2).then(api3).then(r3 => console.log(r3))
And finally, if you want to reach a supermassive black hole level, Promises can be awaited:
async function callApis() {
let api1Result = await api();
let api2Result = await api2(api1Result);
let api3Result = await api3(api2Result);
return api3Result;
}

In addition to the awesome answers above, 2 more points may be added:
1. Semantic difference:
Promises may be already resolved upon creation. This means they guarantee conditions rather than events. If they are resolved already, the resolved function passed to it is still called.
Conversely, callbacks handle events. So, if the event you are interested in has happened before the callback has been registered, the callback is not called.
2. Inversion of control
Callbacks involve inversion of control. When you register a callback function with any API, the Javascript runtime stores the callback function and calls it from the event loop once it is ready to be run.
Refer The Javascript Event loop for an explanation.
With Promises, control resides with the calling program. The .then() method may be called at any time if we store the promise object.

In addition to the other answers, the ES2015 syntax blends seamlessly with promises, reducing even more boilerplate code:
// Sequentially:
api1()
.then(r1 => api2(r1))
.then(r2 => api3(r2))
.then(r3 => {
// Done
});
// Parallel:
Promise.all([
api1(),
api2(),
api3()
]).then(([r1, r2, r3]) => {
// Done
});

Promises are not callbacks, both are programming idioms that facilitate async programming. Using an async/await-style of programming using coroutines or generators that return promises could be considered a 3rd such idiom. A comparison of these idioms across different programming languages (including Javascript) is here: https://github.com/KjellSchubert/promise-future-task

No, Not at all.
Callbacks are simply Functions In JavaScript which are to be called and then executed after the execution of another function has finished. So how it happens?
Actually, In JavaScript, functions are itself considered as objects and hence as all other objects, even functions can be sent as arguments to other functions. The most common and generic use case one can think of is setTimeout() function in JavaScript.
Promises are nothing but a much more improvised approach of handling and structuring asynchronous code in comparison to doing the same with callbacks.
The Promise receives two Callbacks in constructor function: resolve and reject. These callbacks inside promises provide us with fine-grained control over error handling and success cases. The resolve callback is used when the execution of promise performed successfully and the reject callback is used to handle the error cases.

No promises are just wrapper on callbacks
example
You can use javascript native promises with node js
my cloud 9 code link : https://ide.c9.io/adx2803/native-promises-in-node
/**
* Created by dixit-lab on 20/6/16.
*/
var express = require('express');
var request = require('request'); //Simplified HTTP request client.
var app = express();
function promisify(url) {
return new Promise(function (resolve, reject) {
request.get(url, function (error, response, body) {
if (!error && response.statusCode == 200) {
resolve(body);
}
else {
reject(error);
}
})
});
}
//get all the albums of a user who have posted post 100
app.get('/listAlbums', function (req, res) {
//get the post with post id 100
promisify('http://jsonplaceholder.typicode.com/posts/100').then(function (result) {
var obj = JSON.parse(result);
return promisify('http://jsonplaceholder.typicode.com/users/' + obj.userId + '/albums')
})
.catch(function (e) {
console.log(e);
})
.then(function (result) {
res.end(result);
}
)
})
var server = app.listen(8081, function () {
var host = server.address().address
var port = server.address().port
console.log("Example app listening at http://%s:%s", host, port)
})
//run webservice on browser : http://localhost:8081/listAlbums

JavaScript Promises actually use callback functions to determine what to do after a Promise has been resolved or rejected, therefore both are not fundamentally different. The main idea behind Promises is to take callbacks - especially nested callbacks where you want to perform a sort of actions, but it would be more readable.

Promises overview:
In JS we can wrap asynchronous operations (e.g database calls, AJAX calls) in promises. Usually we want to run some additional logic on the retrieved data. JS promises have handler functions which process the result of the asynchronous operations. The handler functions can even have other asynchronous operations within them which could rely on the value of the previous asynchronous operations.
A promise always has of the 3 following states:
pending: starting state of every promise, neither fulfilled nor rejected.
fulfilled: The operation completed successfully.
rejected: The operation failed.
A pending promise can be resolved/fullfilled or rejected with a value. Then the following handler methods which take callbacks as arguments are called:
Promise.prototype.then() : When the promise is resolved the callback argument of this function will be called.
Promise.prototype.catch() : When the promise is rejected the callback argument of this function will be called.
Although the above methods skill get callback arguments they are far superior than using
only callbacks here is an example that will clarify a lot:
Example
function createProm(resolveVal, rejectVal) {
return new Promise((resolve, reject) => {
setTimeout(() => {
if (Math.random() > 0.5) {
console.log("Resolved");
resolve(resolveVal);
} else {
console.log("Rejected");
reject(rejectVal);
}
}, 1000);
});
}
createProm(1, 2)
.then((resVal) => {
console.log(resVal);
return resVal + 1;
})
.then((resVal) => {
console.log(resVal);
return resVal + 2;
})
.catch((rejectVal) => {
console.log(rejectVal);
return rejectVal + 1;
})
.then((resVal) => {
console.log(resVal);
})
.finally(() => {
console.log("Promise done");
});
The createProm function creates a promises which is resolved or rejected based on a random Nr after 1 second
If the promise is resolved the first then method is called and the resolved value is passed in as an argument of the callback
If the promise is rejected the first catch method is called and the rejected value is passed in as an argument
The catch and then methods return promises that's why we can chain them. They wrap any returned value in Promise.resolve and any thrown value (using the throw keyword) in Promise.reject. So any value returned is transformed into a promise and on this promise we can again call a handler function.
Promise chains give us more fine tuned control and better overview than nested callbacks. For example the catch method handles all the errors which have occurred before the catch handler.

Promises allows programmers to write simpler and far more readable code than by using callbacks.
In a program, there are steps want to do in series.
function f() {
step_a();
step_b();
step_c();
...
}
There's usually information carried between each step.
function f() {
const a = step_a( );
const b = step_b( a );
const c = step_c( b );
...
}
Some of these steps can take a (relatively) long time, so sometimes you want to do them in parallel with other things. One way to do that is using threads. Another is asynchronous programming. (Both approaches has pros and cons, which won't be discussed here.) Here, we're talking about asynchronous programming.
The simple way to achieve the above when using asynchronous programming would be to provide a callback which is called once a step is complete.
// step_* calls the provided function with the returned value once complete.
function f() {
step_a(
function( a )
step_b(
function( b )
step_c(
...
)
},
)
},
)
}
That's quite hard to read. Promises offer a way to flatten the code.
// step_* returns a promise.
function f() {
step_a()
.then( step_b )
.then( step_c )
...
}
The object returned is called a promise because it represents the future result (i.e. promised result) of the function (which could be a value or an exception).
As much as promises help, it's still a bit complicated to use promises. This is where async and await come in. In a function declared as async, await can be used in lieu of then.
// step_* returns a promise.
async function f()
const a = await step_a( );
const b = await step_b( a );
const c = await step_c( b );
...
}
This is undeniably much much more readable than using callbacks.

Promise.all after a foreach loop

I'm trying to use Promise.all() with an array of Promises that is being populated inside of a foreach looop right before but it seems like the Promise.all() is not waiting the promises to be all completed before executing its callback.
What's wrong with the following code? (I tried to simplify it before posting, so parts of it might not make complete sense, but that promised and loops are all there).
class test {
constructor(sql) {
Promise.all([this.sync(sql, 0), this.sync(sql, 1)]).then((data) => {
console.log(data);
});
}
sync(sql, id = 0) {
return new Promise((resolve, reject) => {
request.get('http://localhost/test/' + id, {
json: true
}, (req, res) => {
var promises = [];
res.body['items'].forEach(item => {
promises.push(new Promise((resolve, reject) => {
this.existingRecord(sql, item['id']).then(() => {
resolve(false);
}).catch(() => {
this.add(sql, item).then(resolve(id));
})
}))
});
Promise.all(promises).then((data) => resolve(data));
});
});
}
add(sql, data) {
return new Promise((resolve, reject) => {
console.log('Inserting ' + data['id']);
var request = new sql.Request();
var query = `INSERT INTO test (col1, col2) VALUES (${utils.prepareInsertdata(data)})`;
request.query(query, (err, result) => {
if (err) {
console.log('ERROR INSERTING: ' + data['id']);
console.log(err);
}
resolve();
});
});
}
}

First off, you're making it much harder to write good, clean, error-free code when you have a mix of promises and regular callbacks in your control flow. I find that the best way to write asynchronous code using promises is to first take any asynchronous operations that are not promise based and create promise-based wrappers for them and then write my logic and control flow using only promises. This makes a consistent path for flow of control and for error handling and it removes the mess of promisifying things from the actual main logic.
Then, I see several significant issues in your code.
Asynchronous operations in the constructor
It's almost never a good idea to put asynchronous operations in the constructor. This is because the constructor HAS to return the object itself so that leaves no simple way to communicate back to the code that created your object when the asynchronous operations are actually done and if they succeeded of failed. It is not entirely clear to me what you're trying to accomplish with those async operations, but this is likely a bad design pattern. I favor a factory function that returns a promise that resolves to the new object for combining the creation of an object with asynchronous operations. This gives you everything you need, a fully formed object, knowledge of when the async operations are done and an ability to have error handling for the async operations. You can see more about this factory function option and some other design options here:
Asynchronous operations in constructor
Improve .then() handler construction
When you do this:
this.add(sql, item).then(resolve(id));
You are calling resolve(id) immediately and passing that to .then() rather than waiting for the .then() handler to be called before calling resolve(id). All of this is complicated because you're mixing regular callbacks and promises.
Creating new wrapped promises rather than just returning existing promises
This is related to your mix of regular callbacks and regular promises, but you'd much rather just return an existing promise than wrap it in a new promise that you have to manually resolve and reject. More than half the time, you will miss proper error handling when manually wrapping things in a new promise and it just results in more code than is needed.
Race Conditions
In any sort of multi-user database environment, you can't write database code such as:
if (record exists) {
do one thing
} else {
create new record
}
This is a race condition. If some other database request comes in during the processing of this, it could change the database in the middle of this and you'd be trying to create a record that just got created by another piece of code.
The usual solution varies by database (and you don't say exactly which database library you're using). Usually, you want to let the database manage the creation of unique records making it so that a duplicate record (by whatever key you're managing uniqueness in this table by) isn't allowed in the database and the concurrency of that is managed by the database itself. Some databases have an atomic operation such as findOrCreate() that will find an existing record or create a new one in an atomic fashion. Other databases have other approaches. But, it's important to make sure that adding unique records to the database is an atomic operation that can't ever create unwanted duplicates.
I'd suggest this implementation:
// use promise version of request library (already promisified for us)
const rp = require('request-promise');
class test {
constructor() {
}
init(sql) {
return Promise.all([this.sync(sql, 0), this.sync(sql, 1)]).then((data) => {
console.log(data);
// do something with the data here - probably store it in instance data
});
}
sync(sql, id = 0) {
return rp.get('http://localhost/test/' + id, {json: true}).then(res => {
// process all items
return Promise.all(res.body.items.map(item => {
return this.existingRecord(sql, item.id).then(() => {
return false;
}).catch(() => {
// it's probably bad form here to do this for all possible database errors
// probably this should be looking for a specific error of id not found
// or something like that.
// This is also likely a race condition. You would typically avoid the race
// condition by making the item key unique in the database and just doing an add and letting
// the database tell you the add failed because the item already exists
// This will allow the database to control the concurrency and avoid race conditions
return this.add(sql, item);
});
}));
});
}
}
// factory function that returns promise that resolves to a new object
// don't use new test() elsewhere
function createTestObj(sql) {
let t = new test();
return t.init(sql).then(() => {
// resolve to our new object
return t;
});
}
For your add() method, I'd switch to using the promise interface in your sql database. There should either be one built-in or a 3rd party package that will add one on top of your database interface. This will prevent the manual creation of promises and the incomplete error handling in your add() method.

JavaScript design against nesting and duplication [duplicate]

I've been developing JavaScript for a few years and I don't understand the fuss about promises at all.
It seems like all I do is change:
api(function(result){
api2(function(result2){
api3(function(result3){
// do work
});
});
});
Which I could use a library like async for anyway, with something like:
api().then(function(result){
api2().then(function(result2){
api3().then(function(result3){
// do work
});
});
});
Which is more code and less readable. I didn't gain anything here, it's not suddenly magically 'flat' either. Not to mention having to convert things to promises.
So, what's the big fuss about promises here?

Promises are not callbacks. A promise represents the future result of an asynchronous operation. Of course, writing them the way you do, you get little benefit. But if you write them the way they are meant to be used, you can write asynchronous code in a way that resembles synchronous code and is much more easy to follow:
api().then(function(result){
return api2();
}).then(function(result2){
return api3();
}).then(function(result3){
// do work
});
Certainly, not much less code, but much more readable.
But this is not the end. Let's discover the true benefits: What if you wanted to check for any error in any of the steps? It would be hell to do it with callbacks, but with promises, is a piece of cake:
api().then(function(result){
return api2();
}).then(function(result2){
return api3();
}).then(function(result3){
// do work
}).catch(function(error) {
//handle any error that may occur before this point
});
Pretty much the same as a try { ... } catch block.
Even better:
api().then(function(result){
return api2();
}).then(function(result2){
return api3();
}).then(function(result3){
// do work
}).catch(function(error) {
//handle any error that may occur before this point
}).then(function() {
//do something whether there was an error or not
//like hiding an spinner if you were performing an AJAX request.
});
And even better: What if those 3 calls to api, api2, api3 could run simultaneously (e.g. if they were AJAX calls) but you needed to wait for the three? Without promises, you should have to create some sort of counter. With promises, using the ES6 notation, is another piece of cake and pretty neat:
Promise.all([api(), api2(), api3()]).then(function(result) {
//do work. result is an array contains the values of the three fulfilled promises.
}).catch(function(error) {
//handle the error. At least one of the promises rejected.
});
Hope you see Promises in a new light now.

Yes, Promises are asynchronous callbacks. They can't do anything that callbacks can't do, and you face the same problems with asynchrony as with plain callbacks.
However, Promises are more than just callbacks. They are a very mighty abstraction, allow cleaner and better, functional code with less error-prone boilerplate.
So what's the main idea?
Promises are objects representing the result of a single (asynchronous) computation. They resolve to that result only once. There's a few things what this means:
Promises implement an observer pattern:
You don't need to know the callbacks that will use the value before the task completes.
Instead of expecting callbacks as arguments to your functions, you can easily return a Promise object
The promise will store the value, and you can transparently add a callback whenever you want. It will be called when the result is available. "Transparency" implies that when you have a promise and add a callback to it, it doesn't make a difference to your code whether the result has arrived yet - the API and contracts are the same, simplifying caching/memoisation a lot.
You can add multiple callbacks easily
Promises are chainable (monadic, if you want):
If you need to transform the value that a promise represents, you map a transform function over the promise and get back a new promise that represents the transformed result. You cannot synchronously get the value to use it somehow, but you can easily lift the transformation in the promise context. No boilerplate callbacks.
If you want to chain two asynchronous tasks, you can use the .then() method. It will take a callback to be called with the first result, and returns a promise for the result of the promise that the callback returns.
Sounds complicated? Time for a code example.
var p1 = api1(); // returning a promise
var p3 = p1.then(function(api1Result) {
var p2 = api2(); // returning a promise
return p2; // The result of p2 …
}); // … becomes the result of p3
// So it does not make a difference whether you write
api1().then(function(api1Result) {
return api2().then(console.log)
})
// or the flattened version
api1().then(function(api1Result) {
return api2();
}).then(console.log)
Flattening does not come magically, but you can easily do it. For your heavily nested example, the (near) equivalent would be
api1().then(api2).then(api3).then(/* do-work-callback */);
If seeing the code of these methods helps understanding, here's a most basic promise lib in a few lines.
What's the big fuss about promises?
The Promise abstraction allows much better composability of functions. For example, next to then for chaining, the all function creates a promise for the combined result of multiple parallel-waiting promises.
Last but not least Promises come with integrated error handling. The result of the computation might be that either the promise is fulfilled with a value, or it is rejected with a reason. All the composition functions handle this automatically and propagate errors in promise chains, so that you don't need to care about it explicitly everywhere - in contrast to a plain-callback implementation. In the end, you can add a dedicated error callback for all occurred exceptions.
Not to mention having to convert things to promises.
That's quite trivial actually with good promise libraries, see How do I convert an existing callback API to promises?

In addition to the already established answers, with ES6 arrow functions Promises turn from a modestly shining small blue dwarf straight into a red giant. That is about to collapse into a supernova:
api().then(result => api2()).then(result2 => api3()).then(result3 => console.log(result3))
As oligofren pointed out, without arguments between api calls you don't need the anonymous wrapper functions at all:
api().then(api2).then(api3).then(r3 => console.log(r3))
And finally, if you want to reach a supermassive black hole level, Promises can be awaited:
async function callApis() {
let api1Result = await api();
let api2Result = await api2(api1Result);
let api3Result = await api3(api2Result);
return api3Result;
}

In addition to the awesome answers above, 2 more points may be added:
1. Semantic difference:
Promises may be already resolved upon creation. This means they guarantee conditions rather than events. If they are resolved already, the resolved function passed to it is still called.
Conversely, callbacks handle events. So, if the event you are interested in has happened before the callback has been registered, the callback is not called.
2. Inversion of control
Callbacks involve inversion of control. When you register a callback function with any API, the Javascript runtime stores the callback function and calls it from the event loop once it is ready to be run.
Refer The Javascript Event loop for an explanation.
With Promises, control resides with the calling program. The .then() method may be called at any time if we store the promise object.

In addition to the other answers, the ES2015 syntax blends seamlessly with promises, reducing even more boilerplate code:
// Sequentially:
api1()
.then(r1 => api2(r1))
.then(r2 => api3(r2))
.then(r3 => {
// Done
});
// Parallel:
Promise.all([
api1(),
api2(),
api3()
]).then(([r1, r2, r3]) => {
// Done
});

Promises are not callbacks, both are programming idioms that facilitate async programming. Using an async/await-style of programming using coroutines or generators that return promises could be considered a 3rd such idiom. A comparison of these idioms across different programming languages (including Javascript) is here: https://github.com/KjellSchubert/promise-future-task

No, Not at all.
Callbacks are simply Functions In JavaScript which are to be called and then executed after the execution of another function has finished. So how it happens?
Actually, In JavaScript, functions are itself considered as objects and hence as all other objects, even functions can be sent as arguments to other functions. The most common and generic use case one can think of is setTimeout() function in JavaScript.
Promises are nothing but a much more improvised approach of handling and structuring asynchronous code in comparison to doing the same with callbacks.
The Promise receives two Callbacks in constructor function: resolve and reject. These callbacks inside promises provide us with fine-grained control over error handling and success cases. The resolve callback is used when the execution of promise performed successfully and the reject callback is used to handle the error cases.

No promises are just wrapper on callbacks
example
You can use javascript native promises with node js
my cloud 9 code link : https://ide.c9.io/adx2803/native-promises-in-node
/**
* Created by dixit-lab on 20/6/16.
*/
var express = require('express');
var request = require('request'); //Simplified HTTP request client.
var app = express();
function promisify(url) {
return new Promise(function (resolve, reject) {
request.get(url, function (error, response, body) {
if (!error && response.statusCode == 200) {
resolve(body);
}
else {
reject(error);
}
})
});
}
//get all the albums of a user who have posted post 100
app.get('/listAlbums', function (req, res) {
//get the post with post id 100
promisify('http://jsonplaceholder.typicode.com/posts/100').then(function (result) {
var obj = JSON.parse(result);
return promisify('http://jsonplaceholder.typicode.com/users/' + obj.userId + '/albums')
})
.catch(function (e) {
console.log(e);
})
.then(function (result) {
res.end(result);
}
)
})
var server = app.listen(8081, function () {
var host = server.address().address
var port = server.address().port
console.log("Example app listening at http://%s:%s", host, port)
})
//run webservice on browser : http://localhost:8081/listAlbums

JavaScript Promises actually use callback functions to determine what to do after a Promise has been resolved or rejected, therefore both are not fundamentally different. The main idea behind Promises is to take callbacks - especially nested callbacks where you want to perform a sort of actions, but it would be more readable.

Promises overview:
In JS we can wrap asynchronous operations (e.g database calls, AJAX calls) in promises. Usually we want to run some additional logic on the retrieved data. JS promises have handler functions which process the result of the asynchronous operations. The handler functions can even have other asynchronous operations within them which could rely on the value of the previous asynchronous operations.
A promise always has of the 3 following states:
pending: starting state of every promise, neither fulfilled nor rejected.
fulfilled: The operation completed successfully.
rejected: The operation failed.
A pending promise can be resolved/fullfilled or rejected with a value. Then the following handler methods which take callbacks as arguments are called:
Promise.prototype.then() : When the promise is resolved the callback argument of this function will be called.
Promise.prototype.catch() : When the promise is rejected the callback argument of this function will be called.
Although the above methods skill get callback arguments they are far superior than using
only callbacks here is an example that will clarify a lot:
Example
function createProm(resolveVal, rejectVal) {
return new Promise((resolve, reject) => {
setTimeout(() => {
if (Math.random() > 0.5) {
console.log("Resolved");
resolve(resolveVal);
} else {
console.log("Rejected");
reject(rejectVal);
}
}, 1000);
});
}
createProm(1, 2)
.then((resVal) => {
console.log(resVal);
return resVal + 1;
})
.then((resVal) => {
console.log(resVal);
return resVal + 2;
})
.catch((rejectVal) => {
console.log(rejectVal);
return rejectVal + 1;
})
.then((resVal) => {
console.log(resVal);
})
.finally(() => {
console.log("Promise done");
});
The createProm function creates a promises which is resolved or rejected based on a random Nr after 1 second
If the promise is resolved the first then method is called and the resolved value is passed in as an argument of the callback
If the promise is rejected the first catch method is called and the rejected value is passed in as an argument
The catch and then methods return promises that's why we can chain them. They wrap any returned value in Promise.resolve and any thrown value (using the throw keyword) in Promise.reject. So any value returned is transformed into a promise and on this promise we can again call a handler function.
Promise chains give us more fine tuned control and better overview than nested callbacks. For example the catch method handles all the errors which have occurred before the catch handler.

Promises allows programmers to write simpler and far more readable code than by using callbacks.
In a program, there are steps want to do in series.
function f() {
step_a();
step_b();
step_c();
...
}
There's usually information carried between each step.
function f() {
const a = step_a( );
const b = step_b( a );
const c = step_c( b );
...
}
Some of these steps can take a (relatively) long time, so sometimes you want to do them in parallel with other things. One way to do that is using threads. Another is asynchronous programming. (Both approaches has pros and cons, which won't be discussed here.) Here, we're talking about asynchronous programming.
The simple way to achieve the above when using asynchronous programming would be to provide a callback which is called once a step is complete.
// step_* calls the provided function with the returned value once complete.
function f() {
step_a(
function( a )
step_b(
function( b )
step_c(
...
)
},
)
},
)
}
That's quite hard to read. Promises offer a way to flatten the code.
// step_* returns a promise.
function f() {
step_a()
.then( step_b )
.then( step_c )
...
}
The object returned is called a promise because it represents the future result (i.e. promised result) of the function (which could be a value or an exception).
As much as promises help, it's still a bit complicated to use promises. This is where async and await come in. In a function declared as async, await can be used in lieu of then.
// step_* returns a promise.
async function f()
const a = await step_a( );
const b = await step_b( a );
const c = await step_c( b );
...
}
This is undeniably much much more readable than using callbacks.

Using promises/callbacks to wait for function to finish in javascript

I have a function which creates a database object out of three arrays. The arrays are filled in an each loop, one of the arrays relies on the value in the same iteration of the loop.
The dependent array uses the requests library and the cheerio library to grab a string to populate the array with.
Currently the dependent array fills with nulls which I think is because the loop is not waiting for the request to be returned.
I am still learning and would like to get this to work without direct blocking to keep things asynchronous so I'm looking into promises/callbacks.
This is being done server-side but from what I've seen in cheerios docs there is no promises capability.
Here's what I have so far. (getFile() is the function that isn't filling the 'c' array, it also depends on the current value being put into 'b'). I do know that the getFile function gets the correct value with a console log test, so the issue must be in the implementation of filling 'c'.
addToDB() is a function which saves a value into mongoDB, from testing I know that the objects are correctly being put into the db, just the c array is not correct.
function getInfo(path) {
$(path).each(function(i,e) {
a.push(...)
b.push(value)
c.push(getFile(value))
})
var entry = new DB...//(a,b,c)
addToDB(entry);
}
function getFile(...) {
request(fullUrl, function (err, resp, page) {
if (!err && resp.statusCode == 200) {
var $ = cheerio.load(page); // load the page
srcEp = $(this).attr("src");
return srcEp;
} // end error and status code
}); // end request
}
I've been reading about promises/callbacks and then() but I've yet to find anything which works.

First, you have to get your mind around the fact that any process that relies, at least in part, on an asynchronous sub-process, is itself inherently asynchronous.
At the lowest level of this question's code, request() is asynchronous, therefore its caller, getFile() is asynchronous, and its caller, getInfo() is also asynchronous.
Promises are an abstraction of the outcome of asynchronous processes and help enormously in coding the actions to be taken when such processes complete - successfully or under failure.
Typically, low-level asynchronous functions should return a promise to be acted on by their callers, which will, in turn, return a promise to their callers, and so on up the call stack. Inside each function, returned promise(s) may be acted on using promise methods, chiefly .then(), and may be aggregated using Promise.all() for example (syntax varies).
In this question, there is no evidence that request() currently returns a promise. You have three options :
discover that request() does, in fact, return a promise.
rewrite request() to return a promise.
write an adapter function (a "promisifier") that calls request(), and generates/returns the promise, which is later fulfilled or rejected depending on the outcome of request().
The first or second options would be ideal but the safe assumption for me (Roamer) is to assume that an adapter is required. Fortunately, I know enough from the question to be able to write one. Cheerio appears not to include jQuery's promise implementation, so a dedicated promise lib will be required.
Here is an adapter function, using syntax that will work with the Bluebird lib or native js promises:
//Promisifier for the low level function request()
function requestAsync(url) {
return new Promise(function(resolve, reject) {
request(url, function(err, resp, page) {
if (err) {
reject(err);
} else {
if (resp.statusCode !== 200) {
reject(new Error('request error: ' + resp.statusCode));
}
} else {
resolve(page);
}
});
});
}
Now getFile(...) and getInfo() can be written to make use of the promises returned from the lowest level's adapter.
//intermediate level function
function getFile(DOMelement) {
var fullUrl = ...;//something derived from DOMelement. Presumably either .val() or .text()
return requestAsync(fullUrl).then(function (page) {
var $ = cheerio.load(page);
srcEp = $(???).attr('src');//Not too sure what the selector should be. `$(this)` definitely won't work.
return srcEp;
});
}
//high level function
function getInfo(path) {
var a = [], b = [], c = [];//presumably
// Now map the $(path) to an array of promises by calling getFile() inside a .map() callback.
// By chaining .then() a/b/c are populated when the async data arrives.
var promises = $(path).map(function(i, e) {
return getFile(e).then(function(srcEp) {
a[i] = ...;
b[i] = e;
c[i] = srcEp;
});
});
//return an aggregated promise to getInfo's caller,
//in case it needs to take any action on settlement.
return Promise.all(promises).then(function() {
//What to do when all promises are fulfilled
var entry = new DB...//(a,b,c)
addToDB(entry);
}, function(error) {
//What to do if any of the promises fails
console.log(error);
//... you may want to do more.
});
}

How to wrangle Node.JS async

I am struggling with getting my head around how to overcome and handle the async nature of Node.JS. I have done quite a bit of reading on it and tried to make Node do what I want by either using a message passing solution or callback functions.
My problem is I have a object where I want to constructor to load a file and populate an array. Then I want all calls to this function use that loaded data. So I need the original call to wait for the file to be loaded and all subsequent calls to use the already loaded private member.
My issue is that the function to load load the data and get the data is being executed async even if it return a function with a callback.
Anyways, is there something simple I am missing? Or is there an easier pattern I could use here? This function should return part of the loaded file but returns undefined. I have checked that the file is actually being loaded, and works correctly.
function Song() {
this.verses = undefined;
this.loadVerses = function(verseNum, callback) {
if (this.verses === undefined) {
var fs = require('fs'),
filename = 'README.md';
fs.readFile(filename, 'utf8', function(err, data) {
if (err) {
console.log('error throw opening file: %s, err: %s', filename, err);
throw err;
} else {
this.verses = data;
return callback(verseNum);
}
});
} else {
return callback(verseNum);
}
}
this.getVerse = function(verseNum) {
return this.verses[verseNum + 1];
}
}
Song.prototype = {
verse: function(input) {
return this.loadVerses(input, this.getVerse);
}
}
module.exports = new Song();
Update:
This is how I am using the song module from another module
var song = require('./song');
return song.verse(1);

"My issue is that the function to load the data and get the data is being executed async even if it return a function with a callback."
#AlbertoZaccagni what I mean by that scentence is that this line return this.loadVerses(input, this.getVerse); returns before the file is loaded when I expect it to wait for the callback.
That is how node works, I will try to clarify it with an example.
function readFile(path, callback) {
console.log('about to read...');
fs.readFile(path, 'utf8', function(err, data) {
callback();
});
}
console.log('start');
readFile('/path/to/the/file', function() {
console.log('...read!');
});
console.log('end');
You are reading a file and in the console you will likely have
start
about to read...
end
...read!
You can try that separately to see it in action and tweak it to understand the point. What's important to notice here is that your code will keep on running skipping the execution of the callback, until the file is read.
Just because you declared a callback does not mean that the execution will halt until the callback is called and then resumed.
So this is how I would change that code:
function Song() {
this.verses = undefined;
this.loadVerses = function(verseNum, callback) {
if (this.verses === undefined) {
var fs = require('fs'),
filename = 'README.md';
fs.readFile(filename, 'utf8', function(err, data) {
if (err) {
console.log('error throw opening file: %s, err: %s', filename, err);
throw err;
} else {
this.verses = data;
return callback(verseNum);
}
});
} else {
return callback(verseNum);
}
}
}
Song.prototype = {
verse: function(input, callback) {
// I've removed returns here
// I think they were confusing you, feel free to add them back in
// but they are not actually returning your value, which is instead an
// argument of the callback function
this.loadVerses(input, function(verseNum) {
callback(this.verses[verseNum + 1]);
});
}
}
module.exports = new Song();
To use it:
var song = require('./song');
song.verse(1, function(verse) {
console.log(verse);
});
I've ignored
the fact that we're not treating the error as first argument of the callback
the fact that calling this fast enough will create racing conditions, but I believe this is another question

[Collected into an answer and expanded from my previous comments]
TL;DR You need to structure your code such that the result of any operation is only used inside that operation's callback, since you do not have access to it anywhere else.
And while assigning it to an external global variable will certainly work as expected, do so will only occur after the callback has fired, which happens at a time you cannot predict.
Commentary
Callbacks do not return values because by their very nature, they are executed sometime in the future.
Once you pass a callback function into a controlling asynchronous function, it will be executed when the surrounding function decides to call it. You do not control this, and so waiting for a returned result won't work.
Your example code, song.verse(1); cannot be expected to return anything useful because it is called immediately and since the callback hasn't yet fired, will simply return the only value it can: null.
I'm afraid this reliance on asynchronous functions with passed callbacks is an irremovable feature of how NodeJS operates; it is at the very core of it.
Don't be disheartened though. A quick survey of all the NodeJS questions here shows quite clearly that this idea that one must work with the results of async operations only in their callbacks is the single greatest impediment to anyone understanding how to program in NodeJS.
For a truly excellent explanation/tutorial on the various ways to correctly structure NodeJS code, see Managing Node.js Callback Hell with Promises, Generators and Other Approaches.
I believe it clearly and succinctly describes the problem you face and provides several ways to refactor your code correctly.
Two of the features mentioned there, Promises and Generators, are programming features/concepts, the understanding of which would I believe be of great use to you.
Promises (or as some call them, Futures) is/are a programming abstraction that allows one to write code a little more linearly in a if this then that style, like
fs.readFileAsync(path).then(function(data){
/* do something with data here */
return result;
}).catch(function(err){
/* deal with errors from readFileAsync here */
}).then(function(result_of_last_operation){
/* do something with result_of_last_operation here */
if(there_is_a_problem) throw new Error('there is a problem');
return final_result;
})
.catch(function(err){
/* deal with errors when there_is_a_problem here */
}).done(function(final_result){
/* do something with the final result */
});
In reality, Promises are simply a means of marshaling the standard callback pyramid in a more linear fashion. (Personally I believe they need a new name, since the idea of "a promise of some value that might appear in the future" is not an easy one to wrap one's head around, at least it wasn't for me.)
Promises do this by (behind the scenes) restructuring "callback hell" such that:
asyncFunc(args,function callback(err,result){
if(err) throw err;
/* do something with the result here*/
});
becomes something more akin to:
var p=function(){
return new Promise(function(resolve,reject){
asyncFunc(args,function callback(err,result){
if(err) reject(err)
resolve(result);
});
});
});
p();
where any value you provide to resolve() becomes the only argument to the next "then-able" callback and any error is passed via rejected(), so it can be caught by any .catch(function(err){ ... }) handlers you define.
Promises also do all the things you'd expect from the (somewhat standard) async module, like running callbacks in series or in parallel and operating over the elements of an array, returning their collected results to a callback once all the results have been gathered.
But you will note that Promises don't quite do what you want, because everything is still in callbacks.
(See bluebird for what I believe is the simplest and thus, best Promises package to learn first.)
(And note that fs.readFileAsync is not a typo. One useful feature of bluebird is that it can be made to add this and other Promises-based versions of fs's existing functions to the standard fs object. It also understands how to "promisify" other modules such as request and mkdirp).
Generators are the other feature described in the tutorial above, but are only available in the new, updated but not yet officially released version of JavaScript (codenamed "Harmony").
Using generators would also allow you to write code in a more linear manner, since one of the features it provides is the ability of waiting on the results of an asynchronous operation in a way that doesn't wreak havoc with the JavaScript event-loop. (But as I said, it's not a feature in general use yet.)
You can however use generators in the current release of node if you'd like, simply add "--harmony" to the node command line to tell it to turn on the newest features of the next version of JavaScript.