I need to make several async calls that depend on each other. I initially wrote the code and used Promise.all to make async in steps. I looped through my data and created a async method in order to put all of the needed actions into an array to pass into Promise.all(). This works fine, but how could I do the same using Observables. I've read that forkJoin is the equivalent of Promise.all, but how could i loop through the data and wrap my async function, and then execute it before moving to the next flatMap?
public getMonthly(){
return this.http.get(url)
.flatMap(response => {
// need to convert this?
let actions = response.json().map(this.asyncMonthlyRecord);
return Promise.all(actions);
})
.flatMap(()=> this.queryMonthly())
.map(this.convertFromSQl)
.catch((error:any)=> Observable.throw(error || 'Server Error'));
}
private asyncMonthlyRecord = (record):Promise<any> => {
return this.setUsage(record,'HILowMonthly');
}
private queryMonthly(){
return this.storage.query('SELECT * FROM HILowMonthly')
}
getMonthly().subscribe(x => console.info(x)); // logs data from SQLite perfectly...
I think what you want is something like this
Rx.Observable.of({ itemIds: [1, 2, 3, 4, 5 ]})
.mergeMap(response => {
const promises = response.itemIds
.map(id => {
return new Promise((resolve, reject) => {
// Wait for some random time, then resolve the promise.
const timeout = Math.floor(Math.random() * 5000);
setTimeout(() => {
console.log(`Finished promise ${id}`); // debug only
resolve({id, resolved: true})
}, timeout);
});
});
// Wait until all Promises have been resolved, then return all
// of them in a single and ordered array.
return Rx.Observable.forkJoin(promises);
})
.subscribe(x => console.log(x));
Working code on jsbin
Notice that the promises resolve in an arbitrary order but are returned in the correct order.
The commented code in the jsbin example also shows each promise could be resolved individually and merged back into the original stream in case the order of the promises is not important.
Related
So getAstronautsData make request to API then return array of promises. This promises mast make request to Wiki API and parse response in object. Then exampleAsyncFunc must wait all promises and return one big object with all info about Astronauts.
But if I use Promise.all function ending and console is clear.
function getAstronautsData() {
return new Promise((resolve, reject) => {
getData('http://api.open-notify.org/astros.json', "http", (data) => {
resolve(data) // get Astronauts list from API
})
}).then((astronautsList) => {
return astronautsList.people.map((person => // return array of promises
new Promise(resolve => {
getWikiData(person.name, (data) => { // request on Wiki API
resolve({info: data.extract, img: data.thumbnail.source})
})
})
))
})
}
async function exampleAsyncFunc (){
let promisesList = await getAstronautsData()
// next code just few variant was i try
let data = await Promise.all(promisesList)// it's not working.
console.log(data)
Promise.all(promisesList).then(data => console.log(data)) //it's not working. Function display nothing
promisesList.forEach((promise) => { //it's working but not so elegant
promise.then(data => console.log(data))
})
}
exampleAsyncFunc ()
function getWikiData(searhTerm, callback) {
getData(getUrlString(searhTerm), "https", (data) => {
const regex = new RegExp(searhTerm.replaceAll(" ", ".*"));
for (let page in data.query.pages) {
if (data.query.pages[page].title === searhTerm || regex.test(data.query.pages[page].title)) {
callback(data.query.pages[page])
return
}else{
callback(null)
}
}
})
}
You appear to be using Promise.all correctly, but if any of the Promises in Promise.all rejects, then overall Promise.all promise will reject too and nothing will happen, where in your forEach version it'll simply skip those promises silently and move on to the next entries.
Likewise if any of the promises in the list stays pending: if so then the Promise.all promise will never resolve. This could be because you have a long list of return values and the whole list takes a longer-than-expected time to resolve, or because your getWikiData call encounters an error and you don't pass that out to reject that particular promise in your array.
You can debug this behavior by ensuring that each of your calls to then is followed by .catch(console.error) (or some more robust error handler).
Let me first disclose that I am a big promise partisan and frankly deplore callbacks. The implication here is that I would not have written your getData and getWikiData with callbacks.
I will also point out that I second what #t.niese said in the comments: Because it does not make sense having both let data = await Promise.all(promisesList) and promisesList.forEach((promise) => {.
Anyway, your code is unnecessarily complex and can be simplified like so:
function getAstronautsData(callback) {
getData('http://api.open-notify.org/astros.json', "http", data => {
callback(data.people.map(person =>
new Promise(resolve => {
getWikiData(person.name, data => {
resolve(data);
})
}))
)
})
}
function exampleAsyncFunc (){
getAstronautsData(promises => {
Promise.all(promises)
.then(result => {
//result will contain those resolved promises
console.log(result);
})
});
}
exampleAsyncFunc ()
Notice that I am passing a callback to getAstronautsData and call it from inside that function with the array of promises you ultimately want to resolve. No need for async here either as you can see.
Ok, problem was in API (in API one of astronauts have name "Tom Marshburn" but on Wiki his page have title "Thomas Marshburn") and function getWikiData not return any data on error. So i fixed this problem.
Thanks you all for you help!!!
I am making multiple calls with Promise.
My API endpoints to fetch are:
https://www.api-football.com/demo/v2/statistics/357/5/2019-08-30
https://www.api-football.com/demo/v2/statistics/357/5/2019-09-30
https://www.api-football.com/demo/v2/statistics/357/5/2019-10-30
See the code
export function getTeamsStats(league, team, type) {
return function(dispatch) {
const url = "https://www.api-football.com/demo/v2/statistics";
let dates = ["2019-08-30", "2019-09-30", "2019-10-30"];
const getAllData = (dates, i) => {
return Promise.allSettled(dates.map(x => url + '/' + 357 + '/' + 5 + '/' + x).map(fetchData));
}
const fetchData = (URL) => {
return axios
.get(URL)
.then(res => {
const {
matchsPlayed: { total: teamsTotalMatchsPlayed},
} = res.data.api.statistics.matchs;
const matchsPlayed = teamsTotalMatchsPlayed;
dispatch(receivedTeamsStat(matchsPlayed, type));
})
.catch(e => {
console.log(e);
});
}
getAllData(dates).then(resp=>{console.log(resp)}).catch(e=>{console.log(e)})
}
}
Then in my component, I put in an array the matches played from this specific team ( Sao Paulo in this example ) from the initial date 30-8-2019 to 30-10-2019
const [dataHomeTeam, setDataHomeTeam] = useState([]);
useEffect(() => {
if (!team.matchsPlayed) {
return ;
}
setDataHomeTeam(prev =>
prev.concat([
{
matches: team.matchsPlayed,
}
])
);
},[team.matchsPlayed]);
console.log('Data Array', dataHomeTeam);
The problem is that normally the in the first render of the page I have the right order of the matches made from 30-8-2019 to 30-10-2019
See the console log image
But sometimes not, see here
So the question is, how can I make sure that the Promise is returning me the right order of the requests?
I am using Promise.allSettled, the multiple asynchronous tasks that are not dependent on one another to complete successfully, but the order is not always right.
This should be resolved via Promise.all because as described in documentation
Returned values will be in order of the Promises passed, regardless of
completion order.
The problem you have is that you are setting your state by calling dispatch for each promise based on it's completion and since promises can finish at any given time (e.g. 3rd request could finish first because they are sent at the same time), dispatch will be called and your state management will set that as first response (that's why you "sometimes" get such behaviour, because it's up to the network which one will finish first).
This would mean that either you should change your dispatch to receive array of already finished promises, or call dispatch one-by-one once they are finished which is shown in code below - resolve all and dispatch in order:
export function getTeamsStats(league, team, type) {
return function (dispatch) {
const url = "https://www.api-football.com/demo/v2/statistics";
let dates = ["2019-08-30", "2019-09-30", "2019-10-30"];
const getAllData = (dates, i) => {
return Promise.all(dates.map(x => url + '/' + 357 + '/' + 5 + '/' + x).map(fetchData));
}
const fetchData = (URL) => {
return axios
.get(URL)
.then(res => {
const {
matchsPlayed: { total: teamsTotalMatchsPlayed },
} = res.data.api.statistics.matchs;
return teamsTotalMatchsPlayed;
})
.catch(e => {
console.log(e);
});
}
getAllData(dates).then(resp => {
console.log(resp)
// 'resp' here are all 'teamsTotalMatchsPlayed' in correct order (here I mean order of call, not promise completion)
// so just dispatch them in order
resp.map(matchsPlayed => receivedTeamsStat(matchsPlayed, type));
}).catch(e => {
console.log(e)
})
}
}
Please note that I've maybe made some syntax error, but you get the idea.
The ideal way of achieving your requirment is by using Promise.all(). It is because of two main reasons,
To maintain the return value in the order of the Promises passed, regardless of completion order.
Returned values will be in order of the Promises passed, regardless of
completion order.
Refer to the Return value section in the link
To reject the returned promise (short circuit) if any of the promises in the iterable are rejected.
This is also important as well. We don't need to wait for all the asynchronous iterable promises to be resolved/rejected, if the first iterable promise of fetchData rejects we can short circuit and reject the returned promise.
On the other hand Promise.allSettled(),
Promise.allSettled() method returns a promise that resolves after all of the given promises have either fulfilled or rejected, with an
array of objects that each describes the outcome of each promise.
It also doesn't maintain the order in the returned promise.
Promise.allSettled() method never short-circuits. Always Promise fulfilled, never rejected.
Refer to the following comparison table between Promise.all() and Promise.allSettled(),
<script src="https://gist.github.com/Seralahthan/9934ba2bd185a8ccfbdd8e4b3523ea23.js"></script>
How, you have done,
function updateUI(value) {
console.log(value);
// Do something to update the UI.
}
// Note that order of resolution of Promises is 2, 1, 3
const promise1 = new Promise((resolve) => setTimeout(resolve, 200, 1)).then(updateUI);
const promise2 = new Promise((resolve) => setTimeout(resolve, 100, 2)).then(updateUI);
const promise3 = new Promise((resolve) => setTimeout(resolve, 300, 3)).then(updateUI);
const promises = [promise1, promise2, promise3];
Promise.allSettled(promises).
then((value) => console.log('Nothing to do here', value));
// Output: 2, 1, 3
// Here we update the UI as soon as the result is obtained. As a result, the UI is also updated in the
// order in which the promise was resolved.
In other words, we wait not only for the network call but we wait for both the network call and the UI update to complete for each id which is not you want.
How you should have done instead,
// Note that order of resolution of Promises is 2, 1, 3 (Same as previous)
const promise1 = new Promise((resolve) => setTimeout(resolve, 200, 1));
const promise2 = new Promise((resolve) => setTimeout(resolve, 100, 2));
const promise3 = new Promise((resolve) => setTimeout(resolve, 300, 3));
const promises = [promise1, promise2, promise3];
Promise.allSettled(promises).
then((results) => results.forEach((result) => updateUI(result.value)));
// Output: 1, 2, 3
// Here, we wait for all the network requests to complete and then loop through the results and update the UI.
// This ensures that the result is in order.
If you don't want to wait for all Promises to resolve and want to update the UI as soon as one resolves and still maintain order, then you need to pass the position of the element in the array and then use that position to update the element in place at the given position in the array.
Hope this helps.
export function getTeamsStats(league, team, type) {
return function (dispatch) {
const URL = "https://www.api-football.com/demo/v2/statistics";
let DATES = ["2019-08-30", "2019-09-30", "2019-10-30"];
return Promise.all(
DATES.map(date => axios.get(`${URL}/357/5/${date}`))
.then(responseList => {
responseList.map((res) => {
const {
matchsPlayed: { total: teamsTotalMatchsPlayed },
} = res.data.api.statistics.matchs;
const matchsPlayed = teamsTotalMatchsPlayed;
dispatch(receivedTeamsStat(matchsPlayed, type));
});
})
.catch((e) => console.log(e))
);
};
}
Promise.all preserves's the order. You should wait for all the api promise's to resolve first, before action dipatching.
Helpful Article: Promise.all: Order of resolved values
I want to create a queue where I always execute two httpRequests in parallel. When one request completes, the next should start so that always two run in parallel.
I want to merge the results and return it as a whole, when all requests have finished.
My first attempt was to create a Subject (queue) where I pass the urls with queue.next(url). Immediately after theat I complete the subject. I use mergeMap to limit the execution to two requests at a time (queueObservable).
I Subscribe to the queueObservable and add the response to an array (list).
Then I await the promise of the queueObservable, which should resolve when the observable completes.
But what is happening is, that the promise gets resolved, even before the first next callback is executed and before the observable is completed.
In my second attempt, I created a new Promise and resolve it in the completed callback of the queueObservable. This approach is working fine, but I don't understand why the first approach is not working.
Can you explain it to me?
And is there a smarter solution to solve this problem?
Kind regards,
David
import { of, Subject } from 'rxjs';
import { delay, mergeMap } from 'rxjs/operators';
const testFunction1 = async () => {
const queue = new Subject();
const list = [];
const queueObservable = queue.pipe(mergeMap(val => of(val).pipe(delay(500)), 2));
queueObservable.subscribe((next) => {
console.log(next);
list.push(next);
}, () => { },
(complete) => {
console.log('Observable1 completed');
});
queue.next('HTTP1: 1');
queue.next('HTTP1: 2');
queue.next('HTTP1: 3');
queue.next('HTTP1: 4');
queue.complete();
await queueObservable.toPromise(); // I expect this to resolve when all values passed the mergeMap pipe.
return list;
}
const testFunction2 = async () => {
const list = [];
await new Promise(resolve => {
const queue = new Subject();
const queueObservable = queue.pipe(mergeMap(val => of(val).pipe(delay(500)), 2));
queueObservable.subscribe((next) => {
console.log(next);
list.push(next);
}, () => { },
() => {
console.log('Observable2 completed'); // This gets called when all values passed mergeMap pipe.
resolve();
});
queue.next('HTTP2: 1');
queue.next('HTTP2: 2');
queue.next('HTTP2: 3');
queue.next('HTTP2: 4');
queue.complete();
});
return list;
}
testFunction1().then(res => {
console.log('Over1: ' + JSON.stringify(res));
});
testFunction2().then(res => {
console.log('Over2: ' + JSON.stringify(res));
});
Edit:
This diagram shows basically what I want to archieve. The piped observable (queueObservable) completes, when all other observables are completed. When I call toPromise on the piped observable it should only resolve when the piped Observable is completed (At least thats my understanding of the toPromise functionality).
Could you do something like this (fiddle)
const urlsSink = new Subject();
const responses = urlsSink.pipe(bufferCount(2), mergeMap((urls) => {
return combineLatest(urls.map(url => makeHttpRequest(url)));
}));
Then you can subscribe to responses and push your URLs into urlsSink. bufferCount will group URLs into groups of two. combineLatest will combine the two HTTP observables into one combined HTTP observable.
This is because you are delaying the values, the values are not yet emitted but you completed the subject. Once the subject complete, the promise should be resolved.
Try this
setTimeout(() => queue.complete(), 2000);
This will force case 1 to work but this is not suitable for making request.
seems queueObservable's promise follow queue's completion.
you could use toArray operator to merge to result.
const queueObservable = from(listOfURL).pipe(
mergeMap(val => of(val).pipe(delay(500)), 2),
mergeMap(val => fetch(url + val)),
toArray(),
);
I'm trying to figure out why promises seem to work differently inside of for...of vs. map().
data is an array of objects of the form {app: MY_APP, link: MY_LINK}. I'm trying to convert it to the form {app: MY_APP, status: true OR false}}. The function checkLink() is async because it uses node-fetch and basically returns whether the given link is 200. Using for...of, I get the desired object:
let objToSend = [];
for (obj of data) {
objToSend.push({ app: obj.app, status: await checkLink(obj.link) });
}
// returns [{app: MY_APP, status: true}, ...]
But using map, as follows, I get an array of pending promises instead. Any help would be appreciated:
let objToSend = data.map(async obj => {
return {
app: obj.app,
status: await checkLink(obj.link)
};
});
// returns [ Promise { <pending> }, ...]
I also tried doing Promise.all(objToSend) after the map code, but it instead returns Promise { <pending> }
I'll leave the explanation up to the other answers, but just want to point out that there is also an performance difference.
You first solution waits on every iteration for the promise to resolve. Only calling checkLink if the previous one has resolved. This is an sequential solution.
check link1 => wait => check link2 => wait => check link3 => wait
The second solution iterates over every elements and sends out requests, without waiting for promises to resolve (for this reason an array of promises is returned). If you wait for all promises to be resolved you find this solution is a lot faster, since the requests are send out in parallel.
check link1 => check link2 => check link3 => wait for all
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
async function checkLink(link) {
await sleep(Math.random() * 1000 + 500); // sleep between 500 and 1500 ms
console.log(link);
return `status #${link}`;
}
(async function () {
const data = new Array(5).fill().map((_, index) => ({ app: "app", link: index }));
let objToSend;
console.log("solution #1");
objToSend = [];
for (let obj of data) {
objToSend.push({ app: obj.app, status: await checkLink(obj.link) });
}
console.log(objToSend);
console.log("==============================================");
console.log("solution #2");
objToSend = await Promise.all(data.map(async obj => {
return {
app: obj.app,
status: await checkLink(obj.link)
};
}));
console.log(objToSend);
})();
In the snippet the first solution takes 500/1500 * 5 = 2500/7500 between 2500 and 7500 ms. While the second solution takes between 500 and 1500 ms (depending on the slowest value to resolve).
Async functions always returns Promises. In your map function, as the callback returns promises, an array of promises is being created.
To convert it to your format, use it with Promise.all():
(async()=>{
let objToSend = await Promise.all(data.map(async obj => {
return {
app: obj.app,
status: await checkLink(obj.link)
};
}));
console.log(objToSend) //[{app: MY_APP, status: true}, ...]
})()
The await always halts the execution of the async function it is in, it doesn't work differently in both cases. In the later example however, you do call some async functions, and those calls will evaluate to promises, whereas in your first example, all those awaits are in the same async function.
I also tried doing Promise.all(objToSend) after the map code, but it instead returns Promise { <pending> }
Yup, await that and you get an array of results.
I'm running into the following problem.
I have the following promise (simplified):
module.exports.checkVotes = (groupId) =>{
return new Promise((resolve, reject)=>{
// some db stuff onoing
...
.then((votes)=>{return resolve(votes}})
.catch((err)=>{return reject(err}})
})
}
At some point I'm looping through an object. For each entry i have to call promise above. But before the 2. Promise starts, the first one have to be finished...and so on.
I tried this, but as soon as I call the promise it gets executed.
.then(()=>{
for (let i=0;i<groups.length;i++){
// for each group I want to call the promise later
//but it executes as soon as I push it.
tasklist.push(this.checkVotes(groups[i]))
}
return tasklist.reduce((promiseChain, currentTask) => {
return promiseChain.then(chainResults =>
currentTask.then(currentResult =>
[ ...chainResults, currentResult ]
)
);
}, Promise.resolve([])).then(arrayOfResults => {
console.log(arrayOfResults)
}).catch((err) =>{console.log(err)});
})
})
I can not run this with Promise.all() because for some database things, I need this to run sequentially. Moreover I can not hardcode this because the number of groups is variable.
How can I solve this?
Thanks for helping
Your problem was with putting promises in the taskList, not tasks (functions that return promises). The idea behind making them run in sequence is to call them in the then callback in the reduce:
return groups.reduce((promiseChain, currentGroup) => {
return promiseChain.then(chainResults =>
this.checkVotes(currentGroup).then(currentResult => [...chainResults, currentResult])
// ^^^^^^^^^^^^^^^
);
}, Promise.resolve([]))
But if you can use async/await syntax, you don't need to do all of this anyway and can write the much simpler and more efficient
const results = [];
for (const group of groups)
results.push(await this.checkVotes(group));
seems to work like this:
groupIds = [12, 45, 34];
return groupIds .reduce((promise, groupId) => {
return promise.then(() => {
return this.checkVotes(groupId)
.then((votes)=>{
votesList[groupId] = votes
})
.catch((err)=>{
throw err
})
})
}, Promise.resolve())