Develop Reference

Empty Array returned and not being populated when pushing objects

I am using Node JS and Express JS, here is my controller code:
const UserComment = require("../model/UserComment");
router.post("/get/comments", async (request, response) =>{
try{
let currentUserID = request.body.userID;
let myUserComment = await UserComment.find({userID: currentUserID});
let friendsCommentsArray = [ ...myUserComment];
let friendsComments = await axios.post(`http://localhost:5000/router/accounts/account/following/list`, {userID: currentUserID})
.then((resp) => {
resp.data.message.map((parentArrayOfArray) =>{
parentArrayOfArray.map((friendID) =>{
let friendsCommentsToLookUp = UserComment.find({userID: friendID})
friendsCommentsToLookUp.then((commentsArray) =>{
commentsArray.map((comment) =>{
if(String(comment.userID) === friendID){
friendsCommentsArray.push(comment);
}else{
console.log("no")
}
})
});
});
});
}).catch((err) =>{
console.log("err: ", err);
throw err;
});
return response.status(200).json({message: friendsPostsArray});
}catch(err){
return response.status(400).json({message: `${err}`});
}
});
The friendsCommentsArray, when I console.log it I can see the data, but when I return it, it’s empty. What is the problem, why is it empty, even though i'm pushing every comment iterated over to the friendsCommentsArray.
However, the returned friendsCommentsArray is empty. how to solve this issue ?
Thanks.

To make await Promise.all() work you need to return the promise
return axios.get(`http://localhost:5000/comments/by/post/${post._id}`)

Generally when you use await, you don't need to use .then(). Your problem is that your inner .map() is using friendsCommentsToLookUp.then(), but nothing is waiting for these promises to resolve before you move on in your code. One might think that you can await the friendsCommentsToLookUp promise, but this won't work, as the calls to the map callback are not awaited.
Removing the .then()'s makes this easier to work with:
const resp = await axios.post(`http://localhost:5000/router/accounts/account/following/list`, {userID: currentUserID});
const message = resp.data.message;
for(const parentArrayOfArray of message) {
for(const friendID of parentArrayOfArray) {
const commentsArray = await UserComment.find({userID: friendID});
for(const comment of commentsArray) {
if(String(comment.userID) === friendID){
friendsCommentsArray.push(comment);
}
}
}
}
Above the for..of allows us to pause moving to the next iteration of the for loop until the Promises within the current iteration of the for loop have resolved. ie: it's sequential (note: if you tried to do this with .forEach() or .map(), your code would proceed directly to the portion after the loop before your Promises have resolved). Although, what you're after doesn't need to be sequential. We can create an array of Promises that we pass to Promise.all() which we can wait to resolve in parallel. Below I've shown a different approach of using .flatMap() to create an array of Promises that we can await in parallel with Promise.all():
const resp = await axios.post(`http://localhost:5000/router/accounts/account/following/list`, {userID: currentUserID});
const message = resp.data.message;
const promises = message.flatMap(parentArr => parentArr.map(async friendID => {
const commentsArray = await UserComment.find({userID: friendID});
return commentsArray.filter(comment => String(comment.userID) === friendID);
}));
const nestedComments = await Promise.all(promises);
const friendsCommentsArray = [...myUserComment, ...nestedComments.flat()];

instead of push try concatenation array and let me know if its work.
friendsCommentsArray = [...friendsCommentsArray , {...comment}];
// insted of
friendsCommentsArray.push(comment);
also try to use forEach instead of map() while you don't want to return a new array from your map statement.
The map method is very similar to the forEach method—it allows you to execute a function for each element of an array. But the difference is that the map method creates a new array using the return values of this function. map creates a new array by applying the callback function on each element of the source array. Since map doesn't change the source array, we can say that it’s an immutable method.

Array shown empty but inside data available

I am getting url from firebase using ".getDownloadUrl" and then push into an array but array shown empty but inside data available in console
I am using the following code:
const getPostUrl = async (item) => {
const url = await storage().ref(item.data?.postName).getDownloadUrl();
return { ...item._data, URL: url };
};
const getUrls = async () => {
let response=[]
storage().list((data)=>console.log(data, "knok knok tera baaap aaya"))
PostCollection.doc(UserInfo.uid)
.collection("post")
.onSnapshot(documentSnapshot => {
documentSnapshot.docs.forEach ( async (item, i) => {
const getdata = await getPostUrl(item)
response.push(getdata)
// const url = await storage().ref(item._data?.postName).getDownloadUrl()
// const obj = { ...item._data, URL:res }
});
// setPosts(postArr);
// setUrls(newArr);
});
let xyz = await Promise.resolve(response)
console.log(response, xyz, "RESPONSE========>");
};
Note: Originally code provided as image: here

Diagnosis
Your code is executed asyncronously. In simple terms that means that conceptually there is no guarantee of execution order among code portions flagged as async and wrt to the (synchronous) rest of the code.
Specifically that applies to the handler passed as argument to .forEach where the array is filled and the console output statement.
What likely happens is that the array is written to the console before the code to fill it with downloaded information has executed.
Remedy
Add additional synchronization to guarantee that the array is filled before using/outputting it.
Implementation
Instead of await-ing the result of async execution in various places of the code, pass the abstraction of the future results as Promises. In particular, collect the responses as an array of Promises instead of an array of results. If all promises are resolved (in production code you would prefer to have them all 'settled', as some may fail), continue processing, eg. output to console.
Pitfalls
Promise.all (For the in-browser implementation, see here (MDN), available in most Promise libraries) processes the argument array as is without regard to whether it contains the proper number of arguments. This is no problem here as the array-filling forEach loop is executed synchronously but must be looked after when modifying the code.
Beware: The following code is untested !
const getPostUrl = (item) => {
const p_url =
storage().ref(item.data?.postName).getDownloadUrl()
.then ( (url) => { return Promise.resolve({ ...item._data, URL: url }, (e) => { /* flag error */ } )
;
return p_url;
};
const getUrls = async () => {
let p_response=[]
storage().list((data)=>console.log(data, "knok knok tera baaap aaya"))
PostCollection.doc(UserInfo.uid)
.collection("post")
.onSnapshot(documentSnapshot => {
documentSnapshot.docs.forEach ( (item, i) => {
const getdata = getPostUrl(item)
p_response.push(getdata)
// const url = await storage().ref(item._data?.postName).getDownloadUrl()
// const obj = { ...item._data, URL:res }
});
Promise.all ( p_response )
.then ( (a_results) => {
console.log ( `Response ========> ${JSON.stringify(a_results)}.`)
}, (e) => { /* flag error */ }
);
// setPosts(postArr);
// setUrls(newArr);
});
};
Recommendation
Encapsulate the pattern of collecting an array of a predetermined number of promises and firing when they are all settled into a class of their own (or use a Promise/sync library that comes with it).

Best way to to use async/promise/callbacks in for loop [duplicate]

This question already has answers here:
How do I convert an existing callback API to promises?
(24 answers)
Closed 2 years ago.
I'm building a trading bot that needs to get stock names from separate files. But even I have used async function and await in my code, that doesn't work.
My index file init method.
const init = async () => {
const symbols = await getDownTrendingStock();
console.log("All stocks that are down: " + symbols);
const doOrder = async () => {
//do stuff
}
doOrder();
}
my getDownTrendeingStock file
const downStocks = []
function getDownTrendingStock () {
for(i = 0; i < data.USDTPairs.length; i++){
const USDTPair = data.USDTPairs[i] + "USDT";
binance.prevDay(USDTPair, (error, prevDay, symbol) => {
if(prevDay.priceChangePercent < -2){
downStocks.push(symbol)
}
});
}
return downStocks;
}
I have tried to also use async in for loop because the getDownTrendinStock function returns an empty array before for loop is finished. I didn't find the right way to do that because I was confused with all async, promise and callback stuff. What is the right statement to use in this situation?
Output:
All stocks that are down:
Wanted output:
All stocks that are down: [BTCUSDT, TRXUSDT, ATOMUSDT...]

I think the main issue in the code you posted is that you are mixing callbacks and promises.
This is what's happening in your getDownTrendingStock function:
You start iterating over the data.USDTPairs array, picking the first element
You call binance.prevDay. This does nothing yet, because its an asynchronous function that takes a bit of time. Notably, no data is added to downStocks yet.
You continue doing 1-2, still, no data is added
You return downStocks, which is still empty.
Your function is complete, you print the empty array
Now, at some point, the nodejs event loop continues and starts working on those asynchronous tasks you created earlier by calling binance.prevDay. Internally, it probably calls an API, which takes time; once that call is completed, it calls the function you provided, which pushes data to the downStocks array.
In summary, you didn't wait for the async code to complete. You can achieve this in multiple ways.
One is to wrap this in a promise and then await that promise:
const result= await new Promise((resolve, reject) => {
binance.prevDay(USDTPair, (error, prevDay, symbol) => {
if (error) {
reject(error);
} else {
resolve({prevDay, symbol});
}
});
});
if(result.prevDay.priceChangePercent < -2){
downStocks.push(result.symbol)
}
Note that you can probably also use promisify for this. Also, this means that you will wait for one request to finish before starting the next, which may slow down your code considerably, depending on how many calls you need; you may want to look into Promise.all as well.

Generally speaking, I use two technics:
const asyncFunc = () => {smthAsync};
const arrayToProcess = [];
// 1
const result = await arrayToProcess.reduce((acc, value) => acc.then(() => asyncFunc(value)), Promise.resolve(someInitialValue));
// 2
// here will be eslint errors
for(let i = 0 i < arrayToProcess.length; i+=1) {
const processResult = await asyncFunc(value);
// do with processResult what you want
};

Resolving Promises sequentially not working

I have a code to:
Read last three data from Firebase
Iterate each retrieved data
Push a Promise-returning function expression to an array of Promise to be processed sequentially later
Process said array sequentially
Code:
firebase.database().ref('someRef').limitToLast(3).on('value', snapshot => {
let promiseArray = [];
snapshot.forEach(e => {
promiseArray.push(() => {
firebase.storage().ref(e.key).getDownloadURL().then(url => {
//Do something with URL
//In this case, I print out the url to see the order of URL retrieved
//Unfortunately, the order was incorrect
return 'Resolved, please continue'; //Return something to resolve my Promise
});
});
});
let result = Promise.resolve([]);
promiseArray.forEach(promise => {
result = result.then(promise);
});
});
I think that it should already be correct. However, the result I want to get is wrong. What did I miss?
EDIT
I seem to have missed a point. In my Promise array, I want the first function to resolve the Promise it returns first before continuing to the second function.

You should use reduce. A very good example you will find here: https://decembersoft.com/posts/promises-in-serial-with-array-reduce/

forEach is a synchronous method. You can use map to create the array of promises and then use promise.all.
firebase.database().ref('someRef').limitToLast(3).on('value', snapshot => {
let promiseArray = [];
const promiseArray = snapshot.map(e => firebase.storage().ref(e.key).getDownloadURL());
Promise.all(promiseArray).then((resultArr) => {
// Do anything with your result array
});
}
For sequential execution of promises you can use async await.
firebase.database().ref('someRef').limitToLast(3).on('value', async (snapshot) => {
let promiseArray = [];
const promiseArray = snapshot.map(e => firebase.storage().ref(e.key).getDownloadURL());
let result;
for(let i = 0; i < promiseArray.length; i++) {
result = await promiseArray[i];
}
});

I figured it out: apparently, I forgot to my function is not returning a Promise. Because of that, when I'm chaining the thens, it's not waiting for my Promise to resolve first as it wasn't even returned in the first place. I'm basically returning a void, thus the functions continue without waiting for the previous Promise to resolve. Simply adding return fixes the problem:
firebase.database().ref('someRef').limitToLast(3).on('value', snapshot => {
let promiseArray = [];
snapshot.forEach(e => {
promiseArray.push(() => {
return firebase.storage().ref(e.key).getDownloadURL().then(url => { //Add return here
//Do something with URL
//In this case, I print out the url to see the order of URL retrieved
//Unfortunately, the order was incorrect
return 'Resolved, please continue'; //Return something to resolve my Promise
});
});
});
let result = Promise.resolve([]);
promiseArray.forEach(promise => {
result = result.then(promise);
});
});

Sequential execution of Promise.all

Hi I need to execute promises one after the other how do I achieve this using promise.all any help would be awesome. Below is the sample of my code I am currently using but it executes parallel so the search will not work properly
public testData: any = (req, res) => {
// This method is called first via API and then promise is triggerd
var body = req.body;
// set up data eg 2 is repeated twice so insert 2, 5 only once into DB
// Assuming we cant control the data and also maybe 3 maybe inside the DB
let arrayOfData = [1,2,3,2,4,5,5];
const promises = arrayOfData.map(this.searchAndInsert.bind(this));
Promise.all(promises)
.then((results) => {
// we only get here if ALL promises fulfill
console.log('Success', results);
res.status(200).json({ "status": 1, "message": "Success data" });
})
.catch((err) => {
// Will catch failure of first failed promise
console.log('Failed:', err);
res.status(200).json({ "status": 0, "message": "Failed data" });
});
}
public searchAndInsert: any = (data) => {
// There are database operations happening here like searching for other
// entries in the JSON and inserting to DB
console.log('Searching and updating', data);
return new Promise((resolve, reject) => {
// This is not an other function its just written her to make code readable
if(dataExistsInDB(data) == true){
resolve(data);
} else {
// This is not an other function its just written her to make code readable
insertIntoDB(data).then() => resolve(data);
}
});
}
I looked up in google and saw the reduce will help I would appreciate any help on how to convert this to reduce or any method you suggest (Concurrency in .map did not work)

the Promises unfortunatelly does not allow any control of their flow. It means -> once you create new Promise, it will be doing its asynchronous parts as they like.
The Promise.all does not change it, its only purpose is that it checks all promises that you put into it and it is resolved once all of them are finished (or one of them fail).
To be able to create and control asynchronous flow, the easiest way is to wrap the creation of Promise into function and create some kind of factory method. Then instead of creating all promises upfront, you just create only one promise when you need it, wait until it is resolved and after it continue in same behaviour.
async function doAllSequentually(fnPromiseArr) {
for (let i=0; i < fnPromiseArr.length; i++) {
const val = await fnPromiseArr[i]();
console.log(val);
}
}
function createFnPromise(val) {
return () => new Promise(resolve => resolve(val));
}
const arr = [];
for (let j=0; j < 10; j++) {
arr.push(createFnPromise(Math.random()));
}
doAllSequentually(arr).then(() => console.log('finished'));
PS: It is also possible without async/await using standard promise-chains, but it requires to be implemented with recursion.

If anyone else cares about ESLint complaining about the use of "for" and the "no await in loop" here is a typescript ESLint friendly version of the above answer:
async function runPromisesSequentially<T>(promises: Array<Promise<T>>):Promise<Array<T>> {
if (promises.length === 0) return [];
const [firstElement, ...rest] = promises;
return [await firstElement, ...(await runPromisesSequentially(rest))];
}
You can then just replace Promise.all by runPromisesSequentially.

#lmX2015's answer is close but it's taking in promises that have already started executing.
A slight modification fixes it
export async function runPromisesSequentially<T>(functions: (() => Promise<T>)[]): Promise<T[]> {
if (functions.length === 0) {
return [];
}
const [first, ...rest] = functions;
return [await first(), ...(await runPromisesSequentially(rest))];
}