Develop Reference

Need help understanding Promise.all and async

This is the code I'm going to be talking about.
view: async (req, res, next) => {
let formato = req.query.formato || 'details';
try {
let mode = Client.getMode(formato);
let options = {
columns: mode.columns,
withRelated: mode.withRelated,
};
let client_promises = [ ]
req.query['ids'].map( id => {
let client = Client.findById(id, options);
client_promises.push(client)
})
let response = await Promise.all(cliente_promesas)
return res.json({ success: true, data: response });
} catch (error) {
return next(error);
}
},
I understand that the .map function iterates over an array of ids, which then is passed to Client.findById so it can return a promise to be fulfilled, getting the client's data when it resolves.
Now, those promises are pushed to an array, which then is passed to Promise.all, but I dont really understand where they are being resolved. Isnt Promise.all just passing a resolved promise when all promises in the array are resolved?
I also understand that await just makes the code wait for the resolution of the promise before continuing.
But where are the promises in client_promises being resolved?
I know this is basic but for the life of me I cant seem to read enough manuals and guides to be able to understand this.

All Promise.all does is wait for every Promise in the array passed to it to be done. It doesn't "fire" them. In fact every Promise in the array passed might even be in a resolved state. The Promises are "hot" from the moment they are created.
For your example, the function to get an item by id starts immediately and synchronously returns a Promise that will eventually resolve to whatever object is retrieved by id. This is key: the returning of the Promise is not the asynchronous part, it's the resolution.
Promise.all makes all the promises one big promise.
Consider the example below. I take 3 ids and do exactly what you do, I call findById and put what is returned by the function (a Promise) into an array, promises. In the findById, I also add an extra .then call to demonstrate how these Promises are "hot" and not waiting to be called by Promise.all. Also note that I still return the p.
By the time we actually get to the Promise.all, all of the Promises in promises have actually resolved, that's why you see them print to console first. They all take 100-600 milliseconds to resolve, but we actively wait a full 1000 milliseconds before we call Promise.all.
Unfortunately (kind of) there is no API to reveal the state of a Promise using the native implementation. I think there used to be ways with user land libraries like Bluebird and jQuery, but not with how the browser does it. Otherwise, we could inspect the Promise right before calling the Promise.all and see whether they were resolved or pending.
/**
* Utility function to wait a duration before resolving
* #param {number} [delay=100] How long to wait
* #returns {Promise<undfined>} Promise that will resolve after delay
*/
function timeout(delay = 100) {
return new Promise(resolve => {
setTimeout(resolve, delay);
});
}
/**
* Simulated find function that returns an object
* #param {number} id The id of the object to get
* #returns {Promise<object>} The fake DB object
*/
async function findById(id) {
await timeout((Math.random() * 500) + 100);
return {
id,
data: `foo-${id}`
};
}
(async function() {
const idsToGet = [1, 2, 3];
const promises = idsToGet.map(id => {
// Call to get the object
const p = findById(id);
// Let's add another listener here to know when they are done
p.then(r => {
console.log(`${r.id} is done`);
});
// Still return the actual Promise created by findById
return p;
});
// Just for kicks
await timeout(1000);
// Wait for EVERYTHING to be done
const res = await Promise.all(promises);
console.log(JSON.stringify(res, null, 4));
}());

Promise.all() takes list/ array of unresolved promises.
It is kind of like a big one promise that takes all the unresolved promises and until all of them are resolved the Promise.all() is unresolved.
const promise1 = Promise.resolve(3);
const promise2 = 42;
const promise3 = new Promise(function(resolve, reject) {
setTimeout(resolve, 100, 'foo');
});
Promise.all([promise1, promise2, promise3]).then(function(values) {
console.log(values);
});
In the above example, you can see that we are passing 3 promises to Promise.all()
and when all of them are fulfilled then only .then will be triggered.
in your case:
Client.findById(id, options);
must be returning a promise object which you are pushing into client_promises array.
and then feeding them to Promise.all(), until this point, your promises are just promises they are not resolved.
promise.all also returns a promise and as you have put await in front of it.
it will wait until of them re resolved and give you an array of resolved promises.

The promises are actually "fired" (that is, the code starts to run) when you do let client = Client.findById(id, options);. await Promise.all then waits until all of them are resolved or one of them is rejected in which case Promise.all also rejects with the value of the first promise rejection.

JavaScript Serial Promises with setTimeout

I am working on building Promise chain with setTimeouts in them. All of the Promises needs to be run in series not parallel. I am using Bluebird module to achieve serial flow of Promise execution.
Can someone please explain me why this code gives me output of 1,2,3,4 instead of 4,3,2,1?
var bluebirdPromise = require('bluebird');
function p1(value) {
return new Promise(function(resolve, reject) {
setTimeout(function(resolve) {
console.log(value);
resolve;
}, value * 1000);
});
}
var arr = [p1(4), p1(3), p1(2), p1(1)];
bluebirdPromise.reduce(arr,
function(item, index, length) {
}).then(function (result) {
});

There are several issues:
The console.log you have is not made dependent on a previous resolved promise. There is just the time-out that determines when the output will happen. As you create all four promises at the "same" time, and thus all four setTimeout calls are invoked simultaneously, their callbacks will be called at the determined time-out. It does not matter how you chain promises afterwards... To solve this, you need to move the console.log in a then callback, because that callback will only be executed when the previous promise in the chain has been resolved.
The resolve function is not called in your code. You need to add parentheses.
The resolve parameter of the setTimeout callback hides the real function with the same name: you need to remove that parameter.
Here is the suggested correction. For this snippet I have replaced the bluebird reduce with a standard Array#reduce, but it would work similarly with bluebird's reduce:
function p1(value) {
return new Promise(function(resolve, reject) {
setTimeout(function() { // ***
resolve(value); // ***
}, value * 1000);
});
}
var arr = [p1(4), p1(3), p1(2), p1(1)];
arr.reduce(function(promise, next) {
return promise.then(_ => next).then( value => {
console.log(value); // ***
return value;
});
}, Promise.resolve());

If you have a promise-creator function, p, and you want to run a sequence of promises in serial, there's no need for you to load an array with promises – instead, just let it be a normal array of values
Notice I'm not using value * 1000 here either – in your code, you thought you had to artificially choreograph the promises to fire in a particular order using calculated setTimeout delays; that's just not the case. Carefully observe the evaluation of the code below to see how we have a 1-second delay between each promise and .then keeps things in order
Also note that this code will start outputting immediately after the first promise resolves – as opposed to waiting for all promises to resolve before outputting all values
const p = x =>
new Promise(f =>
setTimeout(f, 1e3, x))
const arr = [4,3,2,1]
arr.reduce((acc, x) =>
acc.then(() => p(x)).then(console.log), Promise.resolve())
OK, so you have these promises running in serial order, but why? Unless later steps somehow depend on the result of earlier steps, there's no reason why you'd want to slow these down – ie, each promise's result is not dependent on the others, so just calculate them all as fast as possible. But you're worried that the order will be lost, right? Don't worry, everything will be OK – I'll even use a random delay to show you that the time each promise takes matters not
const p = x =>
new Promise(f =>
setTimeout(f, 1e3 * Math.random(), x))
const arr = [4,3,2,1]
arr.map(p).reduce((acc, x) =>
acc.then(() => x).then(console.log), Promise.resolve())
So now, all of the promises can start immediately, and output will start as soon as the first promise is resolved (unlike Promise.all which would wait for all promises to finish before any values are available to you).
I only mention this as an alternative because the example you provided shows no real need for the promises to be executed in serial. You may have naively simplified your problem's domain, but only you know whether that's the case.

Resolving promises synchronously with Q.all(promises)

I am using the Kriskowal Q library to handle promises.
I wrote the following function to wait until all promises have been resolved.
function multiplePromises(targetArray) {
var promises = {};
for (var i=0; i<targetArray.length; i++) {
var promise = singlePromise(); // any async. function
promises[i] = promise;
};
return Q.all(promises);
};
and I call it as follows:
multiplePromises(targetArray).then(
function(success){
console.log("success");
},
function(error){
console.log("error", error);
}
);
I was wondering however whether there is an order in which the promises are resolved (e.g. is it synchronous?). I.e. does the function wait to trigger the next promise i+1 until promise i is resolved? Or alternatively is it like with all other async. methods, that it actually fires all the single promises and just waits until they are all done?
If the second is the case, how would one rewrite this function to make sure that promise i+1 is ONLY triggered once promise i has been resolved?
Update: test
I did a test and put:
promises[i] = i;
to check whether it resolves sycnhronously and it seems the case. However, it could be just that my async function is fast enough to actually resolve it that quick. Does this seem right?

there are several ways to achieve what you want
Minimal change to your code would be
function multiplePromises(targetArray) {
var promises = [];
var p = Promise.resolve(); // I don't know the Q equivalent of this
for (var i=0; i<targetArray.length; i++) {
p = p.then(function() {
return singlePromise();
});
promises[i] = p;
};
return Q.all(promises);
};

The promises will be executed by Q in the order you declare them, but there's no way to ensure the return order will be the same. That is how async works.
If you want them to resolve in order, the only way I can think of is calling them one after the other instead of doing it in a batch.
This other response will provide more info and some solutions:
https://stackoverflow.com/a/36795097/7705625

Promises in theory could be resolved in order (and it could be easy to write an example in which they would) but you should never count on that.
The whole point of functions like Promise.all() (or Q.all() or Bluebird.all() etc.) is that if waits for all of the promises to get resolved, no matter what is the order of that, and as soon as they are all resolved then the promise that the Promise.all() itself returns gets resolved with an array of values that the original promises resolved to.
In that array you will get always the same order as the order of promises in the array that was an argument to Promise.all(). But the order in time in which the original promises were resolves is not known and is not relevant, as that will have no effect on the result of using Promise.all() whatsoever.

resolve a promise within another promise

In NodeJS, I'm trying to resolve an outside promise when the resolve is within another promise. The outside resolve is called, but the inside one is not.
Why is this? Shouldn't the resolve pass down into the inner promise and resolve up the parent resolve from new Promise(function(resolve...)?
function doIt() {
return new Promise(function(resolve) {
// resolve('resolve called!'); // This works
Object.keys(objects).forEach(function(key) {
return deptSettings(key)
.then(function(settings) {
... do stuff
if (no more stuff)
resolve('inside resolve called!'); // Not called
});
});
});
}

What you have would work just fine — for the first promise that resolves within your forEach. After that, all subsequent resolve calls are ignored.
It sounds like you have a bunch of things to do and want doIt to return a promise that resolves when they're all done. Assuming it's okay that they run in parallel, you'd use Promise.all for that:
function doIt() {
return Promise.all(
Object.keys(objects).map(function(key) {
return deptSettings(key)
.then(function(settings) {
// ... do stuff
// Return final result (for this deptSettings)
});
})
);
}
doIt's promise will resolve when all of the promises created by the Object.keys(...).map resolve (or reject when the first of them rejects). The resolution value will be an array of the values returned by Return final result above.
From a comment on the question:
so shouldn't that single promise be resolve and shortcut the rest of whatever will be called?
That suggests you want to resolve the promise once, based on some condition, and short-circuit the remaining work you were doing in Object.keys. If so, you could just remove that first resolve and the first deptSettings promise that resolved would trigger the single resolve of doIt — but there's an idiomatic verson of that, which is Promise.race:
function doIt() {
return Promise.race( // Only change is here
Object.keys(objects).map(function(key) {
return deptSettings(key)
.then(function(settings) {
// ... do stuff
// Return final result (for this deptSettings)
});
})
);
}
race takes an array (well, any iterable) of promises and settles (resolves or rejects) based on the first one of those that settles.

You can't mix a forEach loop with Promise. Sync and async shouldn't be mixed together.
What you can do is have an array of promises, and use Promise.all to ensure that all the promises in the array are resolved.
For example:
var myPromises = [];
Object.keys(objects).forEach(function(key) {
myPromises.push(your-function-that-returns-a-promise)
});
return Promise.all(myPromises).then(
// do stuff after
);
Here is an example

Promise.all on same array

I'm looking at promises-based code that does something like this:
var promises = [];
function eventHandler(e)
{
promises.push(getSomeData(e.opts));
Promise.all(promises)
.then(...)
.then(function()
{
promises = [];
});
}
Shouldn't the Promise.all based promise execute as many times as the eventHandler is called?
EDIT: what's happening here is the event handler gets called a few times and the previous promises are not finished yet. I need a way to be able to extend the list of promises and have only one final then run.

Update, OP has clarified they'd like to wait for all promises added before the original ones resolved as well -.all does not do this. Instead you can use thenable chaining.
var p = Promise.resolve([]); // create empty array promise, `when` in dojo
function eventHandler(e){
p = p.then(function(value){ // chain off the last value
return getSomeData(e.opts).then(function(single){
return value.push(single); // add to the array
});
});
}
This will add an item to the array each time the eventHandler is called - every time you .then it you will get the current values and not the last ones - but you can .then it as many times as you'd like. You can for example chain on when no events happened since the chaining time by:
var p2 == p;
p.then(function(results){
if(p !== p2) return p; // wait in case more events happened, but no more
return results;
});
Or wait for as long as you'd like:
function noMoreAddedInterim(){
var p2 = p;
return p.then(function(results){
if(p2 !== p) return noMoreAddedInterim(); // changed
else return results; // no more added
});
}
You have a race condition, you're calling Promise.all and only cleaning the array later so many promises are in the results twice or more.
Instead - put promises in a temporary variable, .all that and clear promises before the .then. Since you're adding promises one at a time you probably don't even need the .all here.
Also, you should attach a .catch handler to not miss any errors.