I have a situation where I'm looping over some data and I need to perform an asynchronous operation per each array element. I don't want to await the result inside the loop. Instead I want to simply start off the promises execution from the loop. After the loop is done I want to wait until all promises resolved and then return some result.
The code is below:
const t1 = delay => new Promise(resolve => {
setTimeout(() => {
console.log('from t1')
resolve()
}, delay)
})
const promises = []
const results = []
const array = [1 ,2]
array.forEach((element, index) => {
const temp = t1((index + 1) * 1000).then(() => results.push(5))
promises.push(temp)
})
const mainFunc = async () => {
const bigPromise = () => new Promise(resolve => {
if (results.length === 2) {
resolve()
}
})
await bigPromise()
console.log(results)
}
mainFunc()
I thought I could create another promise bigPromise which will check if the array length is 2. This way I will know that all promises resolved. However this doesn't work. How can I wait for some notification in Javascript? I wouldn't want to use setInterval for this problem.
Related
I have several db mutations that I would like to execute all at once, instead of synchronously. The problem that I'm running into, is that when I try to push these promises into an array, they execute.
What am I doing wrong here? I've also tried pushing anonymous functions, like this,
promises.push(
async () => await someDbMutation1({ someForeignKey: "10" }),
)
but they aren't execute during Promise.all.
import * as React from "react";
import "./styles.css";
const someDbMutation1 = async ({ someForeignKey }) => {
return await new Promise((resolve) => {
console.log("should not enter");
return setTimeout(() => {
resolve("aa");
}, 2000);
});
};
const someDbMutation2 = async ({ someParameter }) =>
await new Promise((resolve) =>
setTimeout(() => {
resolve();
}, 2000)
);
export default function App() {
const [loaded, setLoaded] = React.useState(false);
React.useEffect(() => {
init();
}, []);
const init = React.useCallback(async () => {
const promises = [
someDbMutation1({ someForeignKey: "10" }),
someDbMutation2({ someParameter: "abc" })
];
// await Promise.all(promises);
setLoaded(true);
}, []);
return <div className="App">{loaded && <div>done</div>}</div>;
}
I would expect these promises in the promises array to be executed during a call to Promise.all, but clearly that's not the case here. I've noticed this only recently, when I passed null as a value to a foreign key, at which point the key constraint in my db picked it up and threw an error.
Now I'm worried, because I frequently use a promises array and loop over db objects and push mutation queries into promises -- this means, that each request is executed twice! I'm not sure what I'm missing here.
For the first part of your question where you say that it's not executing:
promises.push(
async () => await someDbMutation1({ someForeignKey: "10" }),
)
it's because you are pushing an anonymous function not a promise - They are two different things. Based on your array name, I think expected behavior would be for you to do this instead:
promises.push(
someDbMutation1({ someForeignKey: "10" })
)
If you want all promises to be executed at a single point in time then you could do this instead:
queries.push(
async () => await someDbMutation1({ someForeignKey: "10" }),
)
/ ** -- some point later -- ** /
const promises = queries.map(q => q()) // Execute queries
const results = await Promise.all(promises) // Wait for queries to finish
In addition, you have a misunderstanding on how Promise.all works here:
I would expect these promises in the promises array to be executed during a call to Promise.all
Promise.all doesn't execute the promises, it waits for the promises to resolve. There is a reference here.
So in this part:
const promises = [
someDbMutation1({ someForeignKey: "10" }),
someDbMutation2({ someParameter: "abc" })
];
You are actually executing the functions so that if you were to console.log the promises array it would look something like this:
[
Promise (unresolved),
Promise (unresolved)
];
And then after await Promise.all(), the promises array would look like this:
[
Promise (resolved: value),
Promise (resolved: value)
];
Issue 1: Promises must be awaited in the block that actually awaits for them.
const someDbMutation1 = async ({ someForeignKey }) => {
return new Promise((resolve) => {
console.log("should not enter");
return setTimeout(() => {
resolve("aa");
}, 2000);
});
};
const someDbMutation2 = async ({ someParameter }) =>
await new Promise((resolve) =>
setTimeout(() => {
resolve();
}, 2000)
);
The problem is you are executing the promises. You should instead add them into an array as anonymous functions that call your function with the parameters you want. So this should look like this. :
const init = React.useCallback(async () => {
const promises = [
async () => someDbMutation1({ someForeignKey: "10" }),
async () => someDbMutation2({ someParameter: "abc" })
];
await Promise.all(promises);
setLoaded(true);
}, []);
I hope this is the answer you are looking for.
I have an async piece of javascript code that need to execute and then the next statements can be executed
function getEventData() {
return new Promise(resolve => {
eventList.forEach((event) => {
db.collection('Events').doc(event)?.collection('registeredStudents')?.get()
.then((querySnapshot) => {
eventData.push({"id": event, "number": querySnapshot.size})
})
})
resolve();
})
}
getEventData().then(console.log(eventData))
eventList is an array with 17 elements and it list through the database around 17 times, that is inefficient but I had no choice so had to do like that. Anyways in the console I get an empty array logged. How do I solve that. PS: I am new to async javascript and promises.
You can use Promise.all():
function getEventData() {
return new Promise(async (resolve) => {
await Promise.all(
eventList.map(async (event) => {
let querySnapshot = await db.collection('Events').doc(event)?.collection('registeredStudents')?.get()
eventData.push({"id": event, "number": querySnapshot.size})
})
);
resolve();
})
}
getEventData().then(console.log(eventData))
This is a simple demo of how to use promises in a loop. The async operation I am performing is "sleeping" (aka nothing happens until everything has "slept")..
const sleepTimes = [1, 2, 3];
const promises = [];
sleepTimes.forEach(time => {
const promise = new Promise((resolve, reject) => {
return setTimeout(resolve, time * 1000, time * 1000);
});
promises.push(promise);
});
console.log(promises);
Promise.all(promises).then((values) => {
document.body.innerHTML = values;
});
Which would mean your function should look something like:
function getEventData() {
const promises = [];
eventList.forEach((event) => {
// I assume this already returns a promise, so there's no need to wrap it in one
const promise = db.collection("Events").doc(event)?.collection("registeredStudents")?.get();
promises.push(promise);
});
return Promise.all(promises);
}
getEventData().then((values) => console.log(values));
The correct way to deal with your problem is not using new Promise with resolve, yet by using async await.
When you await a promise, it waits for it to resolve and returns the result (not a promise!) before continuing to the next line.
A function awaiting a promise returns a promise when called.
Promise.all combines multiple promises to one, so you can wait for multiple promises to resolve all at once, and have them run at the same time (async).
async function getEventData(eventList) {
try {
const eventData = [];
await Promise.all(eventList.map(async event => {
const querySnapshot = await db.collection('Events').doc(event)?.collection('registeredStudents')?.get();
eventData.push({"id": event, "number": querySnapshot.size});
}));
console.log(eventData)
} catch (exception) {
// error handling
}
}
I'm looping an array and reading from a CSV file so I can reduce its data, I've implemented something like this. I want to have a Map variable populated from a csv file in memory and get data from it. But I want to populate it once (or when I explicitly do so, but that's other thing).
when running I would like to print like this:
before
after
But I'm getting this:
before
before
before
after
after
after
Apparently when reading from the CSV is not awaiting. Is there any way I can read from CSV asynchronous?
This is my code:
let myMap = new Map();
const parseMap = async(stream) => {
const map = new Map();
return new Promise((resolve, reject) => {
stream
.on('data', elems => {
const key = elems[0];
const value = elems.slice(-1)[0];
map.set(key, value);
})
.on('error', () => reject)
.on('end', () => {
console.log('promise');
resolve(map);
});
});
}
const getMap = async(path_string) => {
const stream = fs.createReadStream(path_string).pipe(parse({delimiter: DELIMITER}));
return await parseMap(stream);
}
const translate = async (key) => {
if (!myMap.size) {
console.log('before');
myMap = await getMap('my-csv.csv');
console.log('after');
}
return myMap.get(key);
};
const guidRankings = await ['a', 'b', 'c'].reduce(async (accumulator, value, idx) => {
const data = await translate(value);
if (data) {
(await accumulator).push({
data,
order: idx
});
}
return accumulator;
}, Promise.resolve([]));
The reduce method can't handle an ascynrous callback.
You need to pass it a function that returns the actual value you care about, not one that returns a promise.
map your array values through translate to get an array of promises
Use await Promise.all to wait for them all to settle and get an array of the translated values
reduce that array (without doing anything asynchronous inside the reducer).
Assuming that the 3 calls can be made at same time it just using map to store the promises, using Promise All to wait for them to complete, and then a map to get your array.
const promises = ['a', 'b', 'c'].map(value) => translate(value));
const results = await Promise.all(promises);
const guidRankings = results.map((data, order) => ({ data, order }));
If for some reason they have to be made one at a time you want to use for await of
I am making multiple calls with Promise.
My API endpoints to fetch are:
https://www.api-football.com/demo/v2/statistics/357/5/2019-08-30
https://www.api-football.com/demo/v2/statistics/357/5/2019-09-30
https://www.api-football.com/demo/v2/statistics/357/5/2019-10-30
See the code
export function getTeamsStats(league, team, type) {
return function(dispatch) {
const url = "https://www.api-football.com/demo/v2/statistics";
let dates = ["2019-08-30", "2019-09-30", "2019-10-30"];
const getAllData = (dates, i) => {
return Promise.allSettled(dates.map(x => url + '/' + 357 + '/' + 5 + '/' + x).map(fetchData));
}
const fetchData = (URL) => {
return axios
.get(URL)
.then(res => {
const {
matchsPlayed: { total: teamsTotalMatchsPlayed},
} = res.data.api.statistics.matchs;
const matchsPlayed = teamsTotalMatchsPlayed;
dispatch(receivedTeamsStat(matchsPlayed, type));
})
.catch(e => {
console.log(e);
});
}
getAllData(dates).then(resp=>{console.log(resp)}).catch(e=>{console.log(e)})
}
}
Then in my component, I put in an array the matches played from this specific team ( Sao Paulo in this example ) from the initial date 30-8-2019 to 30-10-2019
const [dataHomeTeam, setDataHomeTeam] = useState([]);
useEffect(() => {
if (!team.matchsPlayed) {
return ;
}
setDataHomeTeam(prev =>
prev.concat([
{
matches: team.matchsPlayed,
}
])
);
},[team.matchsPlayed]);
console.log('Data Array', dataHomeTeam);
The problem is that normally the in the first render of the page I have the right order of the matches made from 30-8-2019 to 30-10-2019
See the console log image
But sometimes not, see here
So the question is, how can I make sure that the Promise is returning me the right order of the requests?
I am using Promise.allSettled, the multiple asynchronous tasks that are not dependent on one another to complete successfully, but the order is not always right.
This should be resolved via Promise.all because as described in documentation
Returned values will be in order of the Promises passed, regardless of
completion order.
The problem you have is that you are setting your state by calling dispatch for each promise based on it's completion and since promises can finish at any given time (e.g. 3rd request could finish first because they are sent at the same time), dispatch will be called and your state management will set that as first response (that's why you "sometimes" get such behaviour, because it's up to the network which one will finish first).
This would mean that either you should change your dispatch to receive array of already finished promises, or call dispatch one-by-one once they are finished which is shown in code below - resolve all and dispatch in order:
export function getTeamsStats(league, team, type) {
return function (dispatch) {
const url = "https://www.api-football.com/demo/v2/statistics";
let dates = ["2019-08-30", "2019-09-30", "2019-10-30"];
const getAllData = (dates, i) => {
return Promise.all(dates.map(x => url + '/' + 357 + '/' + 5 + '/' + x).map(fetchData));
}
const fetchData = (URL) => {
return axios
.get(URL)
.then(res => {
const {
matchsPlayed: { total: teamsTotalMatchsPlayed },
} = res.data.api.statistics.matchs;
return teamsTotalMatchsPlayed;
})
.catch(e => {
console.log(e);
});
}
getAllData(dates).then(resp => {
console.log(resp)
// 'resp' here are all 'teamsTotalMatchsPlayed' in correct order (here I mean order of call, not promise completion)
// so just dispatch them in order
resp.map(matchsPlayed => receivedTeamsStat(matchsPlayed, type));
}).catch(e => {
console.log(e)
})
}
}
Please note that I've maybe made some syntax error, but you get the idea.
The ideal way of achieving your requirment is by using Promise.all(). It is because of two main reasons,
To maintain the return value in the order of the Promises passed, regardless of completion order.
Returned values will be in order of the Promises passed, regardless of
completion order.
Refer to the Return value section in the link
To reject the returned promise (short circuit) if any of the promises in the iterable are rejected.
This is also important as well. We don't need to wait for all the asynchronous iterable promises to be resolved/rejected, if the first iterable promise of fetchData rejects we can short circuit and reject the returned promise.
On the other hand Promise.allSettled(),
Promise.allSettled() method returns a promise that resolves after all of the given promises have either fulfilled or rejected, with an
array of objects that each describes the outcome of each promise.
It also doesn't maintain the order in the returned promise.
Promise.allSettled() method never short-circuits. Always Promise fulfilled, never rejected.
Refer to the following comparison table between Promise.all() and Promise.allSettled(),
<script src="https://gist.github.com/Seralahthan/9934ba2bd185a8ccfbdd8e4b3523ea23.js"></script>
How, you have done,
function updateUI(value) {
console.log(value);
// Do something to update the UI.
}
// Note that order of resolution of Promises is 2, 1, 3
const promise1 = new Promise((resolve) => setTimeout(resolve, 200, 1)).then(updateUI);
const promise2 = new Promise((resolve) => setTimeout(resolve, 100, 2)).then(updateUI);
const promise3 = new Promise((resolve) => setTimeout(resolve, 300, 3)).then(updateUI);
const promises = [promise1, promise2, promise3];
Promise.allSettled(promises).
then((value) => console.log('Nothing to do here', value));
// Output: 2, 1, 3
// Here we update the UI as soon as the result is obtained. As a result, the UI is also updated in the
// order in which the promise was resolved.
In other words, we wait not only for the network call but we wait for both the network call and the UI update to complete for each id which is not you want.
How you should have done instead,
// Note that order of resolution of Promises is 2, 1, 3 (Same as previous)
const promise1 = new Promise((resolve) => setTimeout(resolve, 200, 1));
const promise2 = new Promise((resolve) => setTimeout(resolve, 100, 2));
const promise3 = new Promise((resolve) => setTimeout(resolve, 300, 3));
const promises = [promise1, promise2, promise3];
Promise.allSettled(promises).
then((results) => results.forEach((result) => updateUI(result.value)));
// Output: 1, 2, 3
// Here, we wait for all the network requests to complete and then loop through the results and update the UI.
// This ensures that the result is in order.
If you don't want to wait for all Promises to resolve and want to update the UI as soon as one resolves and still maintain order, then you need to pass the position of the element in the array and then use that position to update the element in place at the given position in the array.
Hope this helps.
export function getTeamsStats(league, team, type) {
return function (dispatch) {
const URL = "https://www.api-football.com/demo/v2/statistics";
let DATES = ["2019-08-30", "2019-09-30", "2019-10-30"];
return Promise.all(
DATES.map(date => axios.get(`${URL}/357/5/${date}`))
.then(responseList => {
responseList.map((res) => {
const {
matchsPlayed: { total: teamsTotalMatchsPlayed },
} = res.data.api.statistics.matchs;
const matchsPlayed = teamsTotalMatchsPlayed;
dispatch(receivedTeamsStat(matchsPlayed, type));
});
})
.catch((e) => console.log(e))
);
};
}
Promise.all preserves's the order. You should wait for all the api promise's to resolve first, before action dipatching.
Helpful Article: Promise.all: Order of resolved values
// The Below code already contains the suggestions from the answers and hence works :)
within the below script I tried to fully execute the 'createDatabase' function before the .then call at the end starts to run. Unfortunately I couldn't figure out a solution to achieve just that.
In generell the flow should be as followed:
GetFiles - Fully Execute it
CreateDatabase - Fully Execute it (while awaiting each .map call to finish before starting the next)
Exit the script within the .then call
Thanks a lot for any advise :)
const db = require("../database")
const fsp = require("fs").promises
const root = "./database/migrations/"
const getFiles = async () => {
let fileNames = await fsp.readdir(root)
return fileNames.map(fileName => Number(fileName.split(".")[0]))
}
const createDatabase = async fileNumbers => {
fileNumbers.sort((a, b) => a - b)
for (let fileNumber of fileNumbers) {
const length = fileNumber.toString().length
const x = require(`.${root}${fileNumber.toString()}.js`)
await x.create()
}
}
const run = async () => {
let fileNumbers = await getFiles()
await createDatabase(fileNumbers)
}
run()
.then(() => {
console.log("Database setup successfully!")
db.end()
process.exitCode = 0
})
.catch(err => {
console.log("Error creating Database!", err)
})
The x.create code looks as follows:
const dbQ = (query, message) => {
return new Promise((resolve, reject) => {
db.query(query, (err, result) => {
if (err) {
console.log(`Error: ${err.sqlMessage}`)
return reject()
}
console.log(`Success: ${message}!`)
return resolve()
})
})
}
x.create = async () => {
const query = `
CREATE TABLE IF NOT EXISTS Country (
Code CHAR(2) UNIQUE NOT NULL,
Flag VARCHAR(1024),
Name_de VARCHAR(64) NOT NULL,
Name_en VARCHAR(64) NOT NULL,
Primary Key (Code)
)`
const result = await dbQ(query, "Created Table COUNTRY")
return result
}
If you want each x.create to fully execute before the next one starts, i.e. this is what I interpret where you say while awaiting each .map call to finish before starting the next - then you could use async/await with a for loop as follows:
const createDatabase = async fileNumbers => {
fileNumbers.sort((a, b) => a - b);
for (let fileNumber of fileNumbers) {
const x = require(`.${root}${fileNumber.toString()}.js`);
await x.create();
})
}
However, this also assumes that x.create() returns a Promise - as you've not shown what is the typical content of .${root}${fileNumber.toString()}.js file is, then I'm only speculating
The other interpretation of your question would simply require you to change createDatabase so that promises is actually an array of promises (at the moment, it's an array of undefined
const createDatabase = async fileNumbers => {
fileNumbers.sort((a, b) => a - b);
const promises = fileNumbers.map(fileNumber => {
const x = require(`.${root}${fileNumber.toString()}.js`);
return x.create();
})
await Promise.all(promises);
}
Now all .create() run in "parallel", but createDatabase only resolves onces all promises returned by x.create() resolve - again, assumes that x.create() returns a promise