I have code that looks something like this in javascript:
forloop {
//async call, returns an array to its callback
}
After ALL of those async calls are done, I want to calculate the min over all of the arrays.
How can I wait for all of them?
My only idea right now is to have an array of booleans called done, and set done[i] to true in the ith callback function, then say while(not all are done) {}
edit: I suppose one possible, but ugly solution, would be to edit the done array in each callback, then call a method if all other done are set from each callback, thus the last callback to complete will call the continuing method.
You haven't been very specific with your code, so I'll make up a scenario. Let's say you have 10 ajax calls and you want to accumulate the results from those 10 ajax calls and then when they have all completed you want to do something. You can do it like this by accumulating the data in an array and keeping track of when the last one has finished:
Manual Counter
var ajaxCallsRemaining = 10;
var returnedData = [];
for (var i = 0; i < 10; i++) {
doAjax(whatever, function(response) {
// success handler from the ajax call
// save response
returnedData.push(response);
// see if we're done with the last ajax call
--ajaxCallsRemaining;
if (ajaxCallsRemaining <= 0) {
// all data is here now
// look through the returnedData and do whatever processing
// you want on it right here
}
});
}
Note: error handling is important here (not shown because it's specific to how you're making your ajax calls). You will want to think about how you're going to handle the case when one ajax call never completes, either with an error or gets stuck for a long time or times out after a long time.
jQuery Promises
Adding to my answer in 2014. These days, promises are often used to solve this type of problem since jQuery's $.ajax() already returns a promise and $.when() will let you know when a group of promises are all resolved and will collect the return results for you:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push($.ajax(...));
}
$.when.apply($, promises).then(function() {
// returned data is in arguments[0][0], arguments[1][0], ... arguments[9][0]
// you can process it here
}, function() {
// error occurred
});
ES6 Standard Promises
As specified in kba's answer: if you have an environment with native promises built-in (modern browser or node.js or using babeljs transpile or using a promise polyfill), then you can use ES6-specified promises. See this table for browser support. Promises are supported in pretty much all current browsers, except IE.
If doAjax() returns a promise, then you can do this:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push(doAjax(...));
}
Promise.all(promises).then(function() {
// returned data is in arguments[0], arguments[1], ... arguments[n]
// you can process it here
}, function(err) {
// error occurred
});
If you need to make a non-promise async operation into one that returns a promise, you can "promisify" it like this:
function doAjax(...) {
return new Promise(function(resolve, reject) {
someAsyncOperation(..., function(err, result) {
if (err) return reject(err);
resolve(result);
});
});
}
And, then use the pattern above:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push(doAjax(...));
}
Promise.all(promises).then(function() {
// returned data is in arguments[0], arguments[1], ... arguments[n]
// you can process it here
}, function(err) {
// error occurred
});
Bluebird Promises
If you use a more feature rich library such as the Bluebird promise library, then it has some additional functions built in to make this easier:
var doAjax = Promise.promisify(someAsync);
var someData = [...]
Promise.map(someData, doAjax).then(function(results) {
// all ajax results here
}, function(err) {
// some error here
});
Checking in from 2015: We now have native promises in most recent browser (Edge 12, Firefox 40, Chrome 43, Safari 8, Opera 32 and Android browser 4.4.4 and iOS Safari 8.4, but not Internet Explorer, Opera Mini and older versions of Android).
If we want to perform 10 async actions and get notified when they've all finished, we can use the native Promise.all, without any external libraries:
function asyncAction(i) {
return new Promise(function(resolve, reject) {
var result = calculateResult();
if (result.hasError()) {
return reject(result.error);
}
return resolve(result);
});
}
var promises = [];
for (var i=0; i < 10; i++) {
promises.push(asyncAction(i));
}
Promise.all(promises).then(function AcceptHandler(results) {
handleResults(results),
}, function ErrorHandler(error) {
handleError(error);
});
You can use jQuery's Deferred object along with the when method.
deferredArray = [];
forloop {
deferred = new $.Deferred();
ajaxCall(function() {
deferred.resolve();
}
deferredArray.push(deferred);
}
$.when(deferredArray, function() {
//this code is called after all the ajax calls are done
});
You can emulate it like this:
countDownLatch = {
count: 0,
check: function() {
this.count--;
if (this.count == 0) this.calculate();
},
calculate: function() {...}
};
then each async call does this:
countDownLatch.count++;
while in each asynch call back at the end of the method you add this line:
countDownLatch.check();
In other words, you emulate a count-down-latch functionality.
This is the most neat way in my opinion.
Promise.all
FetchAPI
(for some reason Array.map doesn't work inside .then functions for me. But you can use a .forEach and [].concat() or something similar)
Promise.all([
fetch('/user/4'),
fetch('/user/5'),
fetch('/user/6'),
fetch('/user/7'),
fetch('/user/8')
]).then(responses => {
return responses.map(response => {response.json()})
}).then((values) => {
console.log(values);
})
Use an control flow library like after
after.map(array, function (value, done) {
// do something async
setTimeout(function () {
// do something with the value
done(null, value * 2)
}, 10)
}, function (err, mappedArray) {
// all done, continue here
console.log(mappedArray)
})
In Node.js you can use async/await to control the async flow
async/await is supported in Node.js 7.6
util function to promisify callback is supported in Node.js v8
Sample Code:
const foo = async () => {
try {
const ids = [100, 101, 102];
const fetchFromExternalApi = util.promisify(fetchFromExternalApiCallback);
const promises = ids.map((id) => fetchFromExternalApi(id));
const dataList = await Promise.resolve(promises); // dataList is an array
return dataList;
} catch (err) {
// error handling
}
};
I see several response with Promise.all(), but this function stop if any promise generate an exception...
The best solution in 2022 is Promise.allSettled() (documentation here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled
A quick sample:
const allPromises = [];
for (r in records) {
const promise = update_async(r);
allPromises.push(promise);
};
await Promise.allSettled(allPromises);
At the end, you obtain with allPromises an array with the result of each promise:
when ok --> {status: “fulfilled”, value: xxx }
when error --> {status: "rejected", reason: Error: an error}
Related
Hi I need to execute promises one after the other how do I achieve this using promise.all any help would be awesome. Below is the sample of my code I am currently using but it executes parallel so the search will not work properly
public testData: any = (req, res) => {
// This method is called first via API and then promise is triggerd
var body = req.body;
// set up data eg 2 is repeated twice so insert 2, 5 only once into DB
// Assuming we cant control the data and also maybe 3 maybe inside the DB
let arrayOfData = [1,2,3,2,4,5,5];
const promises = arrayOfData.map(this.searchAndInsert.bind(this));
Promise.all(promises)
.then((results) => {
// we only get here if ALL promises fulfill
console.log('Success', results);
res.status(200).json({ "status": 1, "message": "Success data" });
})
.catch((err) => {
// Will catch failure of first failed promise
console.log('Failed:', err);
res.status(200).json({ "status": 0, "message": "Failed data" });
});
}
public searchAndInsert: any = (data) => {
// There are database operations happening here like searching for other
// entries in the JSON and inserting to DB
console.log('Searching and updating', data);
return new Promise((resolve, reject) => {
// This is not an other function its just written her to make code readable
if(dataExistsInDB(data) == true){
resolve(data);
} else {
// This is not an other function its just written her to make code readable
insertIntoDB(data).then() => resolve(data);
}
});
}
I looked up in google and saw the reduce will help I would appreciate any help on how to convert this to reduce or any method you suggest (Concurrency in .map did not work)
the Promises unfortunatelly does not allow any control of their flow. It means -> once you create new Promise, it will be doing its asynchronous parts as they like.
The Promise.all does not change it, its only purpose is that it checks all promises that you put into it and it is resolved once all of them are finished (or one of them fail).
To be able to create and control asynchronous flow, the easiest way is to wrap the creation of Promise into function and create some kind of factory method. Then instead of creating all promises upfront, you just create only one promise when you need it, wait until it is resolved and after it continue in same behaviour.
async function doAllSequentually(fnPromiseArr) {
for (let i=0; i < fnPromiseArr.length; i++) {
const val = await fnPromiseArr[i]();
console.log(val);
}
}
function createFnPromise(val) {
return () => new Promise(resolve => resolve(val));
}
const arr = [];
for (let j=0; j < 10; j++) {
arr.push(createFnPromise(Math.random()));
}
doAllSequentually(arr).then(() => console.log('finished'));
PS: It is also possible without async/await using standard promise-chains, but it requires to be implemented with recursion.
If anyone else cares about ESLint complaining about the use of "for" and the "no await in loop" here is a typescript ESLint friendly version of the above answer:
async function runPromisesSequentially<T>(promises: Array<Promise<T>>):Promise<Array<T>> {
if (promises.length === 0) return [];
const [firstElement, ...rest] = promises;
return [await firstElement, ...(await runPromisesSequentially(rest))];
}
You can then just replace Promise.all by runPromisesSequentially.
#lmX2015's answer is close but it's taking in promises that have already started executing.
A slight modification fixes it
export async function runPromisesSequentially<T>(functions: (() => Promise<T>)[]): Promise<T[]> {
if (functions.length === 0) {
return [];
}
const [first, ...rest] = functions;
return [await first(), ...(await runPromisesSequentially(rest))];
}
In a controller function, I make some operations:
Get a list of organizations with a promise
In the then of this promise, I loop through each of them to extract some data and populate some of my controller attributes.
One of this operation is to call another promise to gather all users attached to this organization, with a loop inside of it to extract name and other stuff.
When I get ALL of it, so every organization has been parsed, and within them all users too, I must call a function to update my view.
I got it working by setting some flags (orgParsed and usersParsed) but I find it to be... a code shame.
I heard about a way of maybe doing this by using $q to wait for the two promises and maybe loops inside their "then" to be resolve before calling my view function. But I struggle applying this code change since the second promise use the result of the first to gather the organization ID.
Here is my current code:
this.getOrgData = function () {
return Service.getList().then(function (result) {
var orgCount = result.Objects.length;
var orgParsed = 0;
_.forEach(result.Objects, function (org) {
org.Users = [];
// Some logic here using 'org' data
Service.getUsers(org.Id, 0, 0).then(function (userResult) {
usersParsed = 0;
_.forEach(userResult.Objects, function (user) {
// Some Logic here using 'user.Name'
usersParsed++;
});
orgParsed++;
if (orgParsed === orgCount && usersParsed === userResult.Objects.length) {
self.sortMenuList(); // My view Function
}
});
});
$scope.$broadcast("getOrgData");
});
};
Do you see any way to trigger my self.sortMenuList() function only when I can be sure I got all users of every companies parsed in more elegant/efficient/safe way?
Yes, that counting should definitely be replaced by $q.all, especially as you did not bother to handle any errors.
this.getOrgData = function () {
return Service.getList().then(function (result) {
$scope.$broadcast("getOrgData"); // not sure whether you want that here before the results from the loop
return $q.all(_.map(result.Objects, function (org) {
org.Users = [];
// Some logic here using 'org' data
return Service.getUsers(org.Id, 0, 0).then(function (userResult) {
_.forEach(userResult.Objects, function (user) {
// Some Logic here using 'user.Name'
});
});
}));
}).then(function() {
self.sortMenuList(); // My view Function;
})
};
The problem you describe sounds like you want to wait until a certain amount of promises are all resolved, and then do something with the result. That's really easy when you use Promise.all():
this.getOrgData = function () {
return Service.getList().then(function (result) {
var promises = [];
_.forEach(result.Objects, function (org) {
org.Users = [];
// Some logic here using 'org' data
// Store the promise for this user in the promises array
promises.push(Service.getUsers(org.Id, 0, 0));
});
// userResults is an array of all the results of the promises, in the same order as the getUsers was called
Promise.all(promises).then(function (userResults) {
_.forEach(userResults, function(userResult) {
_.forEach(userResult.Objects, function (user) {
// Some Logic here using 'user.Name'
});
});
self.sortMenuList();
});
$scope.$broadcast("getOrgData");
});
};
I'm working on a NodeJS backend service, and use promises to load any data. Now, I want to make a combination (array) of items coming from different sources. I have the following solution, but I don't think this is the right way to get around something like this.
var results = [];
loop(items, index, data, results).then(function() {
console.log(results);
});
function loop(items, index, data, results) {
return utils.getPromise(function(resolve, reject) {
if (index === items.length) {
// Stop
resolve();
} else {
doAction(items[index], data).then(function(result) {
if (result) {
mergeResults(results, result)
} else {
loop(items, index + 1, data, results).then(resolve, reject);
}
}, reject);
}
});
}
function doAction(item, data) {
return utils.getPromise(function(resolve, reject) {
item.doIt(data).then(resolve, reject);
});
}
I think the right way would be to return a promise immediately, and add results on the fly, but I don't know exactly how to do this. Any suggestions?
You could use Promise.all to gather all the results from the promises. The .all promise itself is then resolved with an array containing all the single results from each promise.
I think the right way would be to return a promise immediately
Like you suggested you would just return the promise and create an array from these promises. After the loop you give this array into Promise.all.
Might look like this:
var promiseArray = [];
for(var i=0;i<array.length;i++){
promiseArray.push(doSomethingAsync(array[i]));
}
Promise.all(promiseArray).then(function(responseArray){
//do something with the results
});
function randTimeout(t) {
return new Promise(function (resolve) {
setTimeout(function () {
console.log('time done:' + t);
resolve();
}, t);
});
}
var timeouts = [];
for (var l = 0; l < 20; l ++) {
timeouts.push(Math.random() * 1000 | 0);
}
console.log('start');
Promise.map(timeouts, function (t) {
return randTimeout(t);
}, {concurrency: 3});
<script src="https://cdnjs.cloudflare.com/ajax/libs/bluebird/3.4.6/bluebird.min.js"></script>
Like #Tobi says, you can use Promise.All..
But Promise.all has one gotcha!!.
If the array your processing has lots of items, sending lots of items to go off and do some async work at the same time might be sub-optimal. And in some cases impossible, eg. opening too many files at once in node will get you a too many handles error.
The Bluebird promise library has a useful function called map, and has an option called concurrency.. This allows you to process X amounts of promises at a time. Personally I think the map function with a concurrency option should be part of the Promise spec, as it's so useful. Without using the concurrency option it would have the same effect as doing promise.all, without having to make an array first.
In the above example I created 20 random timeouts, of max of 1 second. Without using map the maximum time doing this would be 1 second, as all promises would be executed at once. But using the concurrency option of 3, it's doing 3 at a time, and as you can see takes longer than 1 second to complete.
I'm trying to make loop of ajax calls from an array and saving each data result that I'll print when all calls are successfully received.
The problem is that if there is any ajax call returning error, the whole process is aborted and the callback is not executed. (Try using listWithErrorResult)
How to push something on error without breaking the loop?
JSFIDDLE
var list = ['UQ13nr6urIo', 'hxa_Z0eZ83E', 'ulHB2mNlovg'];
var listWithErrorResult = ['UQ13nr6urIo', 'hxa_Z0eZ83E', 'IWEHeIdBkc4'];
var callback = function() {
console.log("done");
console.log(tracks);
};
var requests = [], tracks = [];
for(i = 0; i < list.length; i++) {
requests.push($.ajax({
url: 'http://www.youtubeinmp3.com/fetch/?format=JSON&video=https://www.youtube.com/watch?v='+list[i],
dataType: "json",
success: function(data) {
console.log('suc');
tracks.push(data);
},error: function() {
console.log('err');
tracks.push('err');
}
}));
}
$.when.apply(undefined, requests).then(function(results){callback()});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
You can just attach a .then() handle to your ajax call and make sure every ajax call resolves rather than rejects. This will get you an array of results at the end that is guaranteed to include all your ajax calls.:
var callback = function(/* tracks are passed here as arguments */) {
console.log("done");
var tracks = Array.prototype.slice.call(arguments);
console.log(tracks);
};
var requests = [];
for(var i = 0; i < list.length; i++) {
requests.push($.ajax({
url: 'http://www.youtubeinmp3.com/fetch/?format=JSON&video=https://www.youtube.com/watch?v='+list[i],
dataType: "json",
}).then(function(data) {
return data;
}, function(err) {
// convert failure into success so $.when() doesn't stop
return $.Deferred().resolve('err');
}));
}
$.when.apply($, requests).then(callback);
You can also use a version of $.settle() that I wrote that lets all promises finish and gives you all results, even if some promises reject.
Instead of $.when() (which stops on the first rejected promise), you can use this new function $.settle() which returns you the results of all the promises. It returns an array of PromiseInspection objects which allow you to query the resolved value or rejected reason for each promise.
You use $.settle() just like $.when() except it returns all results in PromiseInspection objects and it always resolves (never rejects). Because of jQuery's non-standard means of passing multiple arguments to .then() handlers (particular Ajax calls), if .settle() detects multiple arguments, it copies them into an array so the PromiseInspection.value returned from a successful ajax call is an array [data, textStatus, jqXHR]. This is the worst part of jQuery's non-standard promises. Presumably this is fixed/changed in jQuery 3.x which is supposed to be standard's compliant. This code will work with either since it auto-detects if the .then() handler is sent more than one argument and converts them into a single array argument.
// ES6 version of settle
jQuery.settle = function(promises) {
function PromiseInspection(fulfilled, val) {
return {
isFulfilled: function() {
return fulfilled;
}, isRejected: function() {
return !fulfilled;
}, isPending: function() {
// PromiseInspection objects created here are never pending
return false;
}, value: function() {
if (!fulfilled) {
throw new Error("Can't call .value() on a promise that is not fulfilled");
}
return val;
}, reason: function() {
if (fulfilled) {
throw new Error("Can't call .reason() on a promise that is fulfilled");
}
return val;
}
};
}
return $.when.apply($, promises.map(function(p) {
return p.then(function(val) {
if (arguments.length > 1) {
val = Array.prototype.slice.call(arguments);
}
return new PromiseInspection(true, val);
}, function(err) {
if (arguments.length > 1) {
err = Array.prototype.slice.call(arguments);
}
return new PromiseInspection(false, err);
});
}));
}
You have to add error callback to $.when.apply then too as its an error condition when one of the call fails.
$.when.apply(undefined, requests).then(
function(results){callback()},
function(results){callback()}
);
Working jsBin here
I am having trouble with creating / understanding promises. I understand the advantages and understand how to use them. Creating own promise-functionality is the difficult part. Simply, how do I convert this function to work with promises:
ret.getDataByGame = function (gameID, playerID) {
var cb = new callbackHelper();
models.gameData.find( { }, function (err, found) {
if (err) {
console.log("error in getting gamedata for gameID: "+gameID);
cb.setData(void 0);
} else {
cb.setData(found);
}
});
return cb;
};
function callbackHelper() {
var self = this;
this.data = false;
this.setData = function (data) {
self.data = data;
};
It should not matter what framework or vanilla js you use to show the example to me.
ret.getGameDataByGame = lib.promisify(models.gameData.find);
might suffice. Or use a dedicated node-style callback helper function:
ret.getGameDataByGame = function(gameID, playerID) {
return lib.ninvoke(models.gameData, "find", {…});
};
For the Q library, check the Adapting Node section of its docs.
For creating a promise with the pattern you've used for your callbackHelper thing, your promise library typically offers Deferreds. You would use them like this:
ret.getDataByGame = function (gameID, playerID) {
var def = new lib.Deferred();
models.gameData.find({…}, function (err, found) {
if (err) {
def.reject("error in getting gamedata for gameID: "+gameID);
} else {
def.fulfill(found);
}
});
return def.promise;
};
See also the The Beginning section in the Q docs.
Just to give a second input, I quickly looked at the promise implementation of Q docs, but this is the implementation that I use, which is supported by default, in browsers (except IE). With respect to your posted algorithm:
//define promise structure for callback function of interest
ret.getDataByGame = function(gameID, playerID){
return new Promise(function(resolve,reject)
{
try
{
//do any callback function etc. which you want to do
models.gameData.find({},function(err, found){
if(err)
{
console.log("error in getting gamedata for gameID: "+gameID);
reject(err); //if there is error, save as reject
}
else
resolve(found); //if have solution, save as resolve
}
}
catch(exc)
{reject('Error exc gameData.find: '+exc.message);}
}); //end of Promise
}
And then where you call your class functions etc.:
//where you physically call the function you defined as a promise function
ret.getDataByGame('input1','input2').then(function(output){
alert("woohoo, you are awesome!, output = "+output);
},function(error){
alert("Output error:\r\n"+error);
});
Here is the definition and implementation of promises which I consider as the "standard" thus far, with browser support versions: Promise doc + tutorial. An the cool thing if you do it for massive amounts of data, and they are async, you really optimize your execution time!! such as:
//repeat promise function
function repeatPromise(inputDataArray)
{
for(var i = 0; i < inputDataArray.length; i++)
{
//where you physically call the function you defined as a promise function
ret.getDataByGame(inputDataArray[i].input1,inputDataArray[i].input2).then(function(resolve){
alert("Output is in async, output = "+resolve);
},function(error){
alert("Output error:\r\n"+error);
});
} //end of for loop
} //end of function
Hope this helps :)