I am making multiple calls with Promise.
My API endpoints to fetch are:
https://www.api-football.com/demo/v2/statistics/357/5/2019-08-30
https://www.api-football.com/demo/v2/statistics/357/5/2019-09-30
https://www.api-football.com/demo/v2/statistics/357/5/2019-10-30
See the code
export function getTeamsStats(league, team, type) {
return function(dispatch) {
const url = "https://www.api-football.com/demo/v2/statistics";
let dates = ["2019-08-30", "2019-09-30", "2019-10-30"];
const getAllData = (dates, i) => {
return Promise.allSettled(dates.map(x => url + '/' + 357 + '/' + 5 + '/' + x).map(fetchData));
}
const fetchData = (URL) => {
return axios
.get(URL)
.then(res => {
const {
matchsPlayed: { total: teamsTotalMatchsPlayed},
} = res.data.api.statistics.matchs;
const matchsPlayed = teamsTotalMatchsPlayed;
dispatch(receivedTeamsStat(matchsPlayed, type));
})
.catch(e => {
console.log(e);
});
}
getAllData(dates).then(resp=>{console.log(resp)}).catch(e=>{console.log(e)})
}
}
Then in my component, I put in an array the matches played from this specific team ( Sao Paulo in this example ) from the initial date 30-8-2019 to 30-10-2019
const [dataHomeTeam, setDataHomeTeam] = useState([]);
useEffect(() => {
if (!team.matchsPlayed) {
return ;
}
setDataHomeTeam(prev =>
prev.concat([
{
matches: team.matchsPlayed,
}
])
);
},[team.matchsPlayed]);
console.log('Data Array', dataHomeTeam);
The problem is that normally the in the first render of the page I have the right order of the matches made from 30-8-2019 to 30-10-2019
See the console log image
But sometimes not, see here
So the question is, how can I make sure that the Promise is returning me the right order of the requests?
I am using Promise.allSettled, the multiple asynchronous tasks that are not dependent on one another to complete successfully, but the order is not always right.
This should be resolved via Promise.all because as described in documentation
Returned values will be in order of the Promises passed, regardless of
completion order.
The problem you have is that you are setting your state by calling dispatch for each promise based on it's completion and since promises can finish at any given time (e.g. 3rd request could finish first because they are sent at the same time), dispatch will be called and your state management will set that as first response (that's why you "sometimes" get such behaviour, because it's up to the network which one will finish first).
This would mean that either you should change your dispatch to receive array of already finished promises, or call dispatch one-by-one once they are finished which is shown in code below - resolve all and dispatch in order:
export function getTeamsStats(league, team, type) {
return function (dispatch) {
const url = "https://www.api-football.com/demo/v2/statistics";
let dates = ["2019-08-30", "2019-09-30", "2019-10-30"];
const getAllData = (dates, i) => {
return Promise.all(dates.map(x => url + '/' + 357 + '/' + 5 + '/' + x).map(fetchData));
}
const fetchData = (URL) => {
return axios
.get(URL)
.then(res => {
const {
matchsPlayed: { total: teamsTotalMatchsPlayed },
} = res.data.api.statistics.matchs;
return teamsTotalMatchsPlayed;
})
.catch(e => {
console.log(e);
});
}
getAllData(dates).then(resp => {
console.log(resp)
// 'resp' here are all 'teamsTotalMatchsPlayed' in correct order (here I mean order of call, not promise completion)
// so just dispatch them in order
resp.map(matchsPlayed => receivedTeamsStat(matchsPlayed, type));
}).catch(e => {
console.log(e)
})
}
}
Please note that I've maybe made some syntax error, but you get the idea.
The ideal way of achieving your requirment is by using Promise.all(). It is because of two main reasons,
To maintain the return value in the order of the Promises passed, regardless of completion order.
Returned values will be in order of the Promises passed, regardless of
completion order.
Refer to the Return value section in the link
To reject the returned promise (short circuit) if any of the promises in the iterable are rejected.
This is also important as well. We don't need to wait for all the asynchronous iterable promises to be resolved/rejected, if the first iterable promise of fetchData rejects we can short circuit and reject the returned promise.
On the other hand Promise.allSettled(),
Promise.allSettled() method returns a promise that resolves after all of the given promises have either fulfilled or rejected, with an
array of objects that each describes the outcome of each promise.
It also doesn't maintain the order in the returned promise.
Promise.allSettled() method never short-circuits. Always Promise fulfilled, never rejected.
Refer to the following comparison table between Promise.all() and Promise.allSettled(),
<script src="https://gist.github.com/Seralahthan/9934ba2bd185a8ccfbdd8e4b3523ea23.js"></script>
How, you have done,
function updateUI(value) {
console.log(value);
// Do something to update the UI.
}
// Note that order of resolution of Promises is 2, 1, 3
const promise1 = new Promise((resolve) => setTimeout(resolve, 200, 1)).then(updateUI);
const promise2 = new Promise((resolve) => setTimeout(resolve, 100, 2)).then(updateUI);
const promise3 = new Promise((resolve) => setTimeout(resolve, 300, 3)).then(updateUI);
const promises = [promise1, promise2, promise3];
Promise.allSettled(promises).
then((value) => console.log('Nothing to do here', value));
// Output: 2, 1, 3
// Here we update the UI as soon as the result is obtained. As a result, the UI is also updated in the
// order in which the promise was resolved.
In other words, we wait not only for the network call but we wait for both the network call and the UI update to complete for each id which is not you want.
How you should have done instead,
// Note that order of resolution of Promises is 2, 1, 3 (Same as previous)
const promise1 = new Promise((resolve) => setTimeout(resolve, 200, 1));
const promise2 = new Promise((resolve) => setTimeout(resolve, 100, 2));
const promise3 = new Promise((resolve) => setTimeout(resolve, 300, 3));
const promises = [promise1, promise2, promise3];
Promise.allSettled(promises).
then((results) => results.forEach((result) => updateUI(result.value)));
// Output: 1, 2, 3
// Here, we wait for all the network requests to complete and then loop through the results and update the UI.
// This ensures that the result is in order.
If you don't want to wait for all Promises to resolve and want to update the UI as soon as one resolves and still maintain order, then you need to pass the position of the element in the array and then use that position to update the element in place at the given position in the array.
Hope this helps.
export function getTeamsStats(league, team, type) {
return function (dispatch) {
const URL = "https://www.api-football.com/demo/v2/statistics";
let DATES = ["2019-08-30", "2019-09-30", "2019-10-30"];
return Promise.all(
DATES.map(date => axios.get(`${URL}/357/5/${date}`))
.then(responseList => {
responseList.map((res) => {
const {
matchsPlayed: { total: teamsTotalMatchsPlayed },
} = res.data.api.statistics.matchs;
const matchsPlayed = teamsTotalMatchsPlayed;
dispatch(receivedTeamsStat(matchsPlayed, type));
});
})
.catch((e) => console.log(e))
);
};
}
Promise.all preserves's the order. You should wait for all the api promise's to resolve first, before action dipatching.
Helpful Article: Promise.all: Order of resolved values
I have multiple(hundreds) of files in a folder i need to read. Currently I am using a method that reads all the files and then returns the content as an array. But i need to change it so that i read a file and i send it to the client and then i read the second one and so one until the end... to make it more efficient. Any suggestions?
Currently code:
const fse = require("fs-extra");
const path = require("path");
return (
fse
.readdir(direc)
.then((filenames: any) => {
return filenames.map((filename: any) => path.join(direc, filename));
})
.then((filepaths: any) => {
return filepaths.map((filepath: any) =>
fse.readFile(filepath).then((filecontents: any) => {
return JSON.parse(filecontents);
})
);
})
// Promise.all consumes an array of promises, and returns a
// new promise that will resolve to an array of concrete "answers"
.then((filecontents: any) => Promise.all(filecontents))
.then((realcontents: any) => {
return realcontents;
})
);
Instead of putting all the file reading promises in an array and then calling Promise.all, chain them together. I'm not very familiar with Typescript syntax, so I'll provide an example code in plain JS and I hope you can convert whatever's needed:
// ...
.then(filepaths => {
let promise = Promise.resolve();
filepaths.forEach(filepath => {
promise = promise
.then(() => fs.readFile(filePath))
.then(handleFile)
.catch(handleError);
});
return promise;
})
// ...
Then you just need to create a handleFile function that takes in the file contents of a single file and does whatever you want with it (parses it, sends data back over a response, etc.). This way each file will be handled in order. You're going to want your handleError function to return a resolved promise, too, so one failed file read doesn't break the entire chain.
I'm running into the following problem.
I have the following promise (simplified):
module.exports.checkVotes = (groupId) =>{
return new Promise((resolve, reject)=>{
// some db stuff onoing
...
.then((votes)=>{return resolve(votes}})
.catch((err)=>{return reject(err}})
})
}
At some point I'm looping through an object. For each entry i have to call promise above. But before the 2. Promise starts, the first one have to be finished...and so on.
I tried this, but as soon as I call the promise it gets executed.
.then(()=>{
for (let i=0;i<groups.length;i++){
// for each group I want to call the promise later
//but it executes as soon as I push it.
tasklist.push(this.checkVotes(groups[i]))
}
return tasklist.reduce((promiseChain, currentTask) => {
return promiseChain.then(chainResults =>
currentTask.then(currentResult =>
[ ...chainResults, currentResult ]
)
);
}, Promise.resolve([])).then(arrayOfResults => {
console.log(arrayOfResults)
}).catch((err) =>{console.log(err)});
})
})
I can not run this with Promise.all() because for some database things, I need this to run sequentially. Moreover I can not hardcode this because the number of groups is variable.
How can I solve this?
Thanks for helping
Your problem was with putting promises in the taskList, not tasks (functions that return promises). The idea behind making them run in sequence is to call them in the then callback in the reduce:
return groups.reduce((promiseChain, currentGroup) => {
return promiseChain.then(chainResults =>
this.checkVotes(currentGroup).then(currentResult => [...chainResults, currentResult])
// ^^^^^^^^^^^^^^^
);
}, Promise.resolve([]))
But if you can use async/await syntax, you don't need to do all of this anyway and can write the much simpler and more efficient
const results = [];
for (const group of groups)
results.push(await this.checkVotes(group));
seems to work like this:
groupIds = [12, 45, 34];
return groupIds .reduce((promise, groupId) => {
return promise.then(() => {
return this.checkVotes(groupId)
.then((votes)=>{
votesList[groupId] = votes
})
.catch((err)=>{
throw err
})
})
}, Promise.resolve())
I know that promise.all() fails when even 1 of the promise is failed. I only want to try for failed promises and don't want to run promise.all() again.
Any recommendations on how I can achieve this in minimal way?
Promises are eager construct and model a value obtained asynchronously,
a Promise is produced using some kind of producer, like fetch for instance.
If you retain a reference to this producer then you can replay the nmechanism
that produced the Promise in the first place.
// producer function
function getData (arg) {
const result = new Promise();
return result.then(value => {
return { ok:true, value };
}, error => {
return {
ok: false,
value: error,
// retry is a function which calls the producer with the same arguments
retry: () => getData(arg)
};
})
}
Then if you have something like:
const data = [];
// Promise<{ok: boolean, value: any, retry?: function}>
// No promises will fail in this array
const asyncResults = data.map(getResults);
Promise.all(asyncResults)
.then((results) => {
const successes = results.filter(res => res.ok);
const retrys = results.filter(res => !res.ok).map(res => res.retry()); // retry all failed promises
})
Memory leaks, stack overflow: because I retain a reference to original arguments in order to retry and the algorithm is recursive there could be a memory leak. However the algorithm cannot "stack overflow":
getData calls do not get "deeper" over time (see retry definition)
the asyncrhonicity of the algorithm prevent this behaviour if a promise was never resolved
old data is properly discarded when accessing the results as const resultData = results.filter(res => res.ok).map(res => res.value);
However the algorithm could take a long time to settle if a promise keep on not getting resolved and prevent access to the rest of the values.
In an alternative I suggest you take a look at another async primitive, not yet part of the language (maybe some day) : Observables which are designed for this kind of tasks: lazy, retry-able async operations.
You may use async package and wrap all promise calls with closure with done argument.
Then simply resolve results.
const async = require('async');
const promiseAll = promises => {
return new Promise((resolve) => {
// preparing array of functions which has done method as callback
const parallelCalls = promises.map(promise => {
return done => {
promise
.then(result => done(null, result)
.catch(error => {
console.error(error.message);
done();
});
}
});
// calling array of functions in parallel
async.parallel(
parallelCalls,
(_, results) => resolve(results.filter(Boolean))
);
});
};
router.get('/something', async (req, res) => {
...
const results = await promiseAll(promises);
...
});
Or we can simply do Promise.all without using async package:
router.get('/something', async (req, res) => {
...
const results = (
await Promise.all(
promises.map(promise => {
return new Promise(resolve => {
promise.then(resolve).catch(e => resolve());
});
});
)
).filter(Boolean);
...
});
I'm currently a little stuck with a race condition, and I'm pulling my hair out. Essentially, I'm querying an API, adding the results to the DB, then doing stuff with the returned/saved data.
I'm less asking about this specific problem, and more the design pattern of how to fix this sort of issue. The line p.push(formatter.createAccounts(account, id)); might run upto 1000 times, and might take 20 seconds or so.
Any ideas on what I'm doing wrong will be super helpful
Thanks,
Ollie
// get all users
Promise.all(updatedUsers)
.then(() => {
// create an array of promises
const p = [];
users.then((user) => {
...
ids.forEach((id) => {
const accounts = foo.getAccounts(id, true); // returns a promise
accounts.then((account) => {
// add the return data from here to the promise array at the bottom
p.push(formatter.createAccounts(account, id));
});
});
});
// this would ideally be a promise array of the data from above - but instead it just evaluates to [] (what it was set to).
Promise.all(p).then(() => {
// do my stuff that relies on the data here
})
});
The problem is that you are not including the foo.getAccounts promise into your promises array. Modified version:
Promise.all(updatedUsers)
.then(() => {
users.then((user) => {
return Promise.all(ids.map(id => {
//for each ID we return a promise
return foo.getAccounts(id, true).then(account => {
//keep returning a promise
return formatter.createAccounts(account, id)
})
}).then((results) => {
//results is an array of the resolved values from formatter.createAccounts
// do my stuff that relies on the data here
})
})
//rest of closing brackets