I made a function that connects to an API which returns an array with 100 objects at a time. Without the loop the getPageData() function works when you pass an int. However when I try to iterate through it, it gives me back nothing. Set timeout function doesn't seem to help the loop.
Please help
async function updateWholeDB(){
var results = [];
for (let i = 0; i < getPages(); i++) {
setTimeout(function() {
getPageData(i).then((data) => {
console.log(data)
results.push(data) ;
})
}, 2000)
}
return Promise.all(results);
}
Promise.all() does it's work properly when you pass it an array of promises, not an array of results. It then returns a promise that resolves to an array of results. So, you're not using it properly.
In addition, there doesn't appear to be any reason for a setTimeout() here if things are coded properly.
If what you're trying to do is to end up with an array of results from calling getPageData() a bunch of times, you can do that like this:
function updateWholeDB(){
let promises = [];
for (let i = 0; i < getPages(); i++) {
promises.push(getPageData(i));
}
return Promise.all(promises);
}
// usage
updateWholeDB().then(results => {
console.log(results);
}).catch(err => {
console.log(err);
});
This assumes that getPageData() returns a promise that resolves properly with the data you are interested in. If that's not the case, then you will have to show us that code too so we can offer advice on how to fix it.
Related
I'm working with fetching information from a github repository. I want to get the list of pull requests within that repo, get the list of commits associated with each pull request, then for each commit I want to get information such as the author of the commit, the number of files associated with each commit and the number of additions and deletions made to each file. I'm using axios and the github API to accomplish this. I know how to work with the API, but the promises and async functions are keeping me from accomplishing my task. I have the following code:
const axios = require('axios');
var mapOfInformationObjects = new Map();
var listOfCommits = [];
var listOfSHAs = [];
var gitApiPrefix = link I'll use to start fetching data;
var listOfPullRequestDataObjects = [];
var listOfPullRequestNumbers = [];
var mapOfPullNumberToCommits = new Map();
function getAllPullRequests(gitPullRequestApiLink) {
return new Promise((resolve, reject) => {
axios.get(gitPullRequestApiLink).then((response) =>{
listOfPullRequestDataObjects = response['data'];
var k;
for (k = 0; k < listOfPullRequestDataObjects.length; k++){
listOfPullRequestNumbers.push(listOfPullRequestDataObjects[k]['number']);
}
resolve(listOfPullRequestNumbers);
}).catch((error) => {
reject(error);
})
})
}
function getCommitsForEachPullRequestNumber(listOfPRNumbers) {
var j;
for (j = 0; j < listOfPRNumbers.length; j++) {
currPromise = new Promise((resolve, reject) => {
currentGitApiLink = gitApiPrefix + listOfPRNumbers[j] + "/commits";
axios.get(currentGitApiLink).then((response) => {
mapOfPullNumberToCommits.set(listOfPRNumbers[j], response['data']);
resolve("Done with Pull Request Number: " + listOfPRNumbers[j]);
}).catch((error) => {
reject(error);
})
})
}
}
function getListOfCommits(gitCommitApiLink){
return new Promise((resolve, reject) => {
axios.get(gitCommitApiLink).then((response) => {
resolve(response);
}).catch((error) => {
reject(error);
})
})
}
So far, I made some functions that I would like to call sequentially.
First I'd like to call getAllPullRequestNumbers(someLink)
Then I'd like to call getCommitsForEachPullRequestNumber(listofprnumbers)
Then getListOfCommits(anotherLink)
So it would look something like
getAllPullRequestNumbers(someLink)
getCommitsForEachPullRequestNumber(listofprnumbers)
getListOfCommits(anotherlink)
But two problems arise:
1) I'm not sure if this is how you would call the functions so that the first function in the sequence completes before the other.
2) Because I'm not familiar with Javascript, I'm not sure, especially with the getCommitsForEachPullRequestNumber function since you run a loop and call axios.get() on each iteration of the loop, if this is how you work with promises within the functions.
Would this be how you would go about accomplishing these two tasks? Any help is much appreciated. Thanks!
When you a number of asynchronous operations (represented by promises) that you can run all together and you want to know when they are all done, you use Promise.all(). You collect an array of promises and pass it to Promise.all() and it will tell you when they have all completed or when one of them triggers an error. If all completed, Promise.all() will return a promise that resolves to an array of results (one for each asynchronous operation).
When you're iterating an array to do your set of asynchronous operations, it then works best to use .map() because that helps you create a parallel array of promises that you can feed to Promise.all(). Here's how you do that in getCommitsForEachPullRequestNumber():
function getCommitsForEachPullRequestNumber(listOfPRNumbers) {
let mapOfPullNumberToCommits = new Map();
return Promise.all(listOfPRNumbers.map(item => {
let currentGitApiLink = gitApiPrefix + item + "/commits";
return axios.get(currentGitApiLink).then(response => {
// put data into the map
mapOfPullNumberToCommits.set(item, response.data);
});
})).then(() => {
// make resolved value be the map we created, now that everything is done
return mapOfPullNumberToCommits;
});
}
// usage:
getCommitsForEachPullRequestNumber(list).then(results => {
console.log(results);
}).catch(err => {
console.log(err);
});
Then, in getListOfCommits(), since axios already returns a promise, there is no reason to wrap it in a manually created promise. That is, in fact, consider a promise anti-pattern. Instead, just return the promise that axios already returns. In fact, there's probably not even a reason to have this as a function since one can just use axios.get() directly to achieve the same result:
function getListOfCommits(gitCommitApiLink){
return axios.get(gitCommitApiLink);
}
Then, in getAllPullRequests() it appears you are just doing one axios.get() call and then processing the results. That can be done like this:
function getAllPullRequests(gitPullRequestApiLink) {
return axios.get(gitPullRequestApiLink).then(response => {
let listOfPullRequestDataObjects = response.data;
return listOfPullRequestDataObjects.map(item => {
return item.number;
});
});
}
Now, if you're trying to execute these three operations sequentially in this order:
getAllPullRequests(someLink)
getCommitsForEachPullRequestNumber(listofprnumbers)
getListOfCommits(anotherlink)
You can chain the promises from those three operations together to sequence them:
getAllPullRequests(someLink)
.then(getCommitsForEachPullRequestNumber)
.then(mapOfPullNumberToCommits => {
// not entirely sure what you want to do here, perhaps
// call getListOfCommits on each item in the map?
}).catch(err => {
console.log(err);
});
Or, if you put this code in an async function, then you can use async/awit:
async function getAllCommits(someLink) {
let pullRequests = await getAllPullRequests(someLink);
let mapOfPullNumberToCommits = await getCommitsForEachPullRequestNumber(pullRequests);
// then use getlistOfCommits() somehow to process mapOfPullNumberToCommits
return finalResults;
}
getAllCommits.then(finalResults => {
console.log(finalResults);
}).catch(err => {
console.log(err);
});
not as clean as jfriend00 solution,
but I played with your code and it finally worked
https://repl.it/#gui3/githubApiPromises
you get the list of commits in the variable listOfCommits
I don't understand the purpose of your last function, so I dropped it
I have a code to:
Read last three data from Firebase
Iterate each retrieved data
Push a Promise-returning function expression to an array of Promise to be processed sequentially later
Process said array sequentially
Code:
firebase.database().ref('someRef').limitToLast(3).on('value', snapshot => {
let promiseArray = [];
snapshot.forEach(e => {
promiseArray.push(() => {
firebase.storage().ref(e.key).getDownloadURL().then(url => {
//Do something with URL
//In this case, I print out the url to see the order of URL retrieved
//Unfortunately, the order was incorrect
return 'Resolved, please continue'; //Return something to resolve my Promise
});
});
});
let result = Promise.resolve([]);
promiseArray.forEach(promise => {
result = result.then(promise);
});
});
I think that it should already be correct. However, the result I want to get is wrong. What did I miss?
EDIT
I seem to have missed a point. In my Promise array, I want the first function to resolve the Promise it returns first before continuing to the second function.
You should use reduce. A very good example you will find here: https://decembersoft.com/posts/promises-in-serial-with-array-reduce/
forEach is a synchronous method. You can use map to create the array of promises and then use promise.all.
firebase.database().ref('someRef').limitToLast(3).on('value', snapshot => {
let promiseArray = [];
const promiseArray = snapshot.map(e => firebase.storage().ref(e.key).getDownloadURL());
Promise.all(promiseArray).then((resultArr) => {
// Do anything with your result array
});
}
For sequential execution of promises you can use async await.
firebase.database().ref('someRef').limitToLast(3).on('value', async (snapshot) => {
let promiseArray = [];
const promiseArray = snapshot.map(e => firebase.storage().ref(e.key).getDownloadURL());
let result;
for(let i = 0; i < promiseArray.length; i++) {
result = await promiseArray[i];
}
});
I figured it out: apparently, I forgot to my function is not returning a Promise. Because of that, when I'm chaining the thens, it's not waiting for my Promise to resolve first as it wasn't even returned in the first place. I'm basically returning a void, thus the functions continue without waiting for the previous Promise to resolve. Simply adding return fixes the problem:
firebase.database().ref('someRef').limitToLast(3).on('value', snapshot => {
let promiseArray = [];
snapshot.forEach(e => {
promiseArray.push(() => {
return firebase.storage().ref(e.key).getDownloadURL().then(url => { //Add return here
//Do something with URL
//In this case, I print out the url to see the order of URL retrieved
//Unfortunately, the order was incorrect
return 'Resolved, please continue'; //Return something to resolve my Promise
});
});
});
let result = Promise.resolve([]);
promiseArray.forEach(promise => {
result = result.then(promise);
});
});
So, without boring anyone with the backstory, I need to access data from a number of APIs in order to run my script. The data needs to all be loaded before I execute the script, which I'm normally comfortable doing: I just declare some fetch requests, write a Promise.all, then continue on with the function.
HOWEVER, I've hit something of a snafu with a certain API that limits the number of results I can pull from one request to 100 and I need to query all of the results. I didn't think this was a huge deal since I figured I can just make a couple extra requests by affixing "&page=X" to the end of the request.
The plan, then, is to request the total number of pages from the API and then feed that into a for loop to push a number of fetch requests into an array of promises (i.e., link://to/api/data&page=1, link://to/api/data&page=2, etc). When I actually attempt to create this array with a for loop, though, the array returns empty. Here's my work:
const dataUrlNoPage = 'link/to/api/data&page=';
const totalPages = 3; //usually pulled via a function, but just using a static # for now
let apiRequestLoop = function(inp) {
return new Promise(function(resolve){
let promiseArray = [];
for (let i = 1; i <= inp; i++) {
let dataUrlLoop = dataUrlNoPage + i;
fetch(dataUrlLoop).then(function(response){
promiseArray.push(response.json());
})
}
resolve(promiseArray);
})
}
let finalPromiseArray = apiRequestLoop(totalPages).then(result => {
let requestArray = [apiRequest1,apiRequest2];
//requestArray contains earlier fetch promises
result.forEach(foo => {
requestArray.push(foo);
}
);
return requestArray;
});
So, what's tripping me up is really the loop, and how it's not returning an array of promises. When I look at it in the console, it shows up as a blank array, but I can expand it and see the promises I was expecting. I see the "Value below was evaluated just now" response. No matter how many promises or .thens, I write, however, the array is never actually populated at run time.
What's going on? Can I not generate fetch promises via a for loop?
(Also, just to cut this line of questioning off a bit, yes, the API I'm trying to access is Wordpress. Looking around, most people suggest creating a custom endpoint, but let's assume for the purpose of this project I am forbidden from doing that.)
You have several problems here.
The first is that you have the function provided to new Promise itself containing promise creations. Don't do this! It's a definite anti-pattern and doesn't keep your code clean.
The second is this basic bit of code:
let promiseArray = [];
for (let i = 1; i <= inp; i++) {
let dataUrlLoop = dataUrlNoPage + i;
fetch(dataUrlLoop).then(function(response){
promiseArray.push(response.json());
})
}
resolve(promiseArray);
This says:
create an empty array
loop through another array, doing HTTP requests
resolve your promise with the empty array
when the HTTP requests are completed, add them to the array
Step four will always come after step three.
So, you need to add the promises to your array as you go along, and have the overall promise resolve when they are all complete.
let apiRequestLoop = function(inp) {
let promiseArray = [];
for (let i = 1; i <= inp; i++) {
let dataUrlLoop = dataUrlNoPage + i;
promiseArray.push(fetch(dataUrlLoop).then(function(response) {
return response.json();
}));
}
return Promise.all(promiseArray);
}
or, with an arrow function to clean things up:
let apiRequestLoop = function(inp) {
let promiseArray = [];
for (let i = 1; i <= inp; i++) {
let dataUrlLoop = dataUrlNoPage + i;
promiseArray.push(fetch(dataUrlLoop).then(response => response.json()));
}
return Promise.all(promiseArray);
}
A few points:
you want to actually put the promises themselves into the array, not push to the array in a .then() chained to the promise.
you probably want to skip creating a new Promise in your function. Just get an array of all the promises from your loop, then return a Promise.all on the array.
Like this:
let apiRequestLoop = function(inp) {
let promiseArray = [];
for (let i = 1; i <= inp; i++) {
let dataUrlLoop = dataUrlNoPage + i;
promiseArray.push(fetch(dataUrlLoop))
}
return Promise.all(promiseArray);
}
In your final .then statement, in finalPromiseArray, your result will be an array of the results from all the promises. like this [response1, response2, response3, ...]
See the Promise.all documentation for more details.
I am using Firebase Cloud Firestore, however, I think this may be more of a JavaScript asynchronous vs synchronous promise return issue.
I am doing a query to get IDs from one collection, then I am looping over the results of that query to lookup individual records from another collection based on that ID.
Then I want to store each found record into an array and then return the entire array.
results.length is always 0 because return results fires before the forEach completes. If I print results.length from inside the forEach it has data.
How can I wait until the forEach is done before returning from the outer promise and the outer function itself?
getFacultyFavoritesFirebase() {
var dbRef = db.collection("users").doc(global.user_id).collection("favorites");
var dbQuery = dbRef.where("type", "==", "faculty");
var dbPromise = dbQuery.get();
var results = [];
return dbPromise.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
var docRef = db.collection("faculty").doc(doc.id);
docRef.get().then(function(doc) {
if (doc.exists) {
results.push(doc);
}
})
});
console.log(results.length);
return results;
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
}
The trick here is to populate results with promises rather than the result. You can then call Promise.all() on that array of promises and get the results you want. Of course, you can't check if doc.exists before pushing the promise so you will need to deal with that once Promise.all() resolves. For example:
function getFacultyFavoritesFirebase() {
var dbRef = db.collection("users").doc(global.user_id).collection("favorites");
var dbQuery = dbRef.where("type", "==", "faculty");
var dbPromise = dbQuery.get();
// return the main promise
return dbPromise.then(function(querySnapshot) {
var results = [];
querySnapshot.forEach(function(doc) {
var docRef = db.collection("faculty").doc(doc.id);
// push promise from get into results
results.push(docRef.get())
});
// dbPromise.then() resolves to a single promise that resolves
// once all results have resolved
return Promise.all(results)
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
}
getFacultyFavoritesFirebase
.then(results => {
// use results array here and check for .exists
}
If you have multiple items of work to perform at the same time that come from a loop, you can collect all the promises from all the items of work, and wait for them all to finish with Promise.all(). The general form of a possible solution looks like this:
const promises = [] // collect all promises here
items.forEach(item => {
const promise = item.doWork()
promises.push(promise)
})
Promise.all(promises).then(results => {
// continue processing here
// results[0] is the result of the first promise in the promises array
})
You can adapt this to something that suits your own specific form.
Use for of instead of forEach. Like this:
for (const item of array) {
//do something
}
console.log("finished");
"finished" will be logged after finishing the loop.
Well I know, the thread is old, but the problem is still the same. And because I did run into the same issue and non of the answers did work for me, I want share my solution.
I think it will help someone out there. And maybe it will help me, if I run into the same problem again. ;-)
So the solution is super easy. firebase implements "map" but not direct on snaposhot, but on snapshot.docs.map.
In combination with Promieses.all it works just fine.
const promise = snapshot.docs.map(async (tenant) => {
return CheckTenant(tenant.id).catch(error =>
reject(error),
);
});
Promise.all(promise).then(result => {
// do somothing with the result});
I am trying to read a JSON object using a for loop to format the JSON data and send it back to the client by putting the formatted response into a model object.
Inside for loop, i am dealing with two promises based upon few conditions. There are two functions, each having a promise returned.How can I get my final data after all the promises are resolved? Thanks in advance.
for (var i = 0, i<jsonData.length; i++){
if(someCOndition){
getSomeData().then(function(data){
//some operation using data
})
}
if(someOtherCOndition){
getSomeOtherData().then(function(data){
//some operation using data
})
}
}
Promise.all([ promise1, promise2 ]) (Promise.all() on MDN) in case of standard JS Promises (ES2015+). It returns a new promise, which gets resolved once all passed promises get resolved. But be aware - it will get rejected immediately when at least one promise gets rejected (it won't wait for any other promise).
You might do as follows;
var promises = [],
JSONData_1 = ["chunk_11","chunk_12","chunk_13"],
JSONData_2 = ["chunk_21","chunk_22","chunk_23"],
getJSONData = (b,i) => new Promise((resolve,reject) => setTimeout(_ => b ? resolve(JSONData_1[i])
: resolve(JSONData_2[i]),1000));
for (var i = 0; i < JSONData_1.length; i++){
if(Math.random() < 0.5) promises.push(getJSONData(true,i));
else promises.push(getJSONData(false,i));
}
Promise.all(promises)
.then(a => console.log(a));
You can use jQuery.when().
var deferredList = [];
for (var i = 0, i<jsonData.length; i++){
if(someCOndition){
deferredList.push(getSomeData().then(function(data){
//some operation using data
}))
}
if(someOtherCOndition){
taskList.push(getSomeOtherData().then(function(data){
//some operation using data
}))
}
}
JQuery.when(taskList).done(function(){
// final to do..
}).fail(){
// even if single one fails ! be aware of this
}
jQuery.when() MDN
You can do it in multiple ways. We can also use for of loop with async..await to get the result synchronously while looping, if that is a requirement. Something like this:
function downloadPage(url) {
return Promise.resolve('some value');
}
async function () {
for(let url of urls) {
let result = await downloadPage(url);
// Process the result
console.log(result);
}
}
You could do something like this..
var arr=[],arr2=[];
for (var i = 0, i<jsonData.length; i++){
if(someCOndition){
//push onto the array inputs for getSomeData()
arr.push(jsonData[i]);
}
if(someOtherCOndition){
arr2.push(jsonData[i]);
}
}
processArr(0);
processArr2(0);
function processArr(idx){
if (idx>=arr.length) {
//done
}
else {
getSomeData().then(function(data){
// some operation using data
// optionally store in a results array
// recurse
processArr(idx+1)
})
}
}
function processArr2(idx){
if (idx>=arr2.length) {
//done
}
else {
getSomeotherData().then(function(data){
// some operation using data
// recurse
processArr2(idx+1)
})
}
}