Calling $http.post in batches and chaining promises - javascript

I have a use case where I have to call $http.post(request) on batches of the input data.
For this, I created an array of requests. For each of them, I need to get the response of $http.post(), append it to an existing array and pass it to a rendering function. I have to make the next call only when the previous one completes and since $http.post() returns a promise (according to this), I am trying to do this using the reduce function.
function callToHttpPost(request) {
return $http.post('someurl', request);
}
function outerFunc($scope, someId) {
let completeArray = [];
let arrayOfRequests = getRequestInBatches();
arrayOfRequests.reduce((promiseChain, currentRequest) => {
console.log(promiseChain);
return promiseChain.then((previousResponse) => {
completeArray.push.apply(completeArray, previousResponse.data);
render($scope, completeArray, someId);
return callToHttpPost(currentRequest);
});
}, Promise.resolve()).catch(e => errorHandler($scope, e, someId));
}
(I have referred MDN and this answer)
But this gives me TypeError: previousResponse is undefined. The log statement shows the first promise as resolved (since it is the initial value passed to the reduce function), but the other promises show up as rejected due to this error. How can I resolve this?

Using vanilla Javascript
If the outerFunc function can be used in an async context (which it looks like it can, given that it returns nothing and the results are passed to the render function as they are built up), you could clean that right up, paring it down to:
async function outerFunc ($scope, someId) {
const completeArray = [];
try {
for (const request of getRequestInBatches()) {
const { data } = await callToHttpPost(request);
completeArray.push(...data);
render($scope, completeArray, someId);
}
} catch (e) {
errorHandler($scope, e, someId);
}
}
The sequential nature will be enforced by the async/await keywords.
Using RxJS
If you're able to add a dependency on RxJS, your can change the function to:
import { from } from 'rxjs';
import { concatMap, scan } from 'rxjs/operators';
function outerFunc ($scope, someId) {
from(getRequestInBatches()).pipe(
concatMap(callToHttpPost),
scan((completeArray, { data }) => completeArray.concat(...data), [])
).subscribe(
completeArray => render($scope, completeArray, someId),
e => errorHandler($scope, e, someId)
);
}
which revolves around the use of Observable instead of Promise. In this version, the sequential nature is enforced by the concatMap operator, and the complete array of results is reduced and emitted while being built up by the scan operator.

The error was in passing the initial value. In the first iteration of the reduce function, Promise.resolve() returns undefined. This is what is passed as previousResponse. Passing Promise.resolve({ data: [] }) as the initialValue to the reduce function solved the issue.
arrayOfRequests.reduce((promiseChain, currentRequest) => {
console.log(promiseChain);
return promiseChain.then((previousResponse) => {
completeArray.push.apply(completeArray, previousResponse.data);
render($scope, completeArray, someId);
return callToHttpPost(currentRequest);
});
}, Promise.resolve({ data: [] }))
.then(response => {
completeArray.push.apply(completeArray, previousResponse.data);
render($scope, completeArray, someId);
displaySuccessNotification();
})
.catch(e => errorHandler($scope, e, someId));
(edited to handle the final response)

Related

Synchronously populate/modify an array and return the modified array inside a promise

I'm pretty new to ReactJS and redux, so I've never really had to work with this before. I'm retrieving data from an API call in my project. I want to modify the data by adding a new property to the object. However, because the code is not ran synchronously, the unmodified array is being returned (I assume) instead of the modified array.
export function loadThings() {
return dispatch => {
return dispatch({
type: 'LOAD_THINGS',
payload: {
request: {
url: API_GET_THINGS_ENDPOINT,
method: 'GET'
}
}
}).then(response => {
let things = response.payload.data;
// Retrieve all items for the loaded "things"
if(things) {
things.forEach((thing, thingIndex) => {
things[thingIndex].items = []
if (thing.hasOwnProperty('channels') && thing.channels) {
thing.channels.forEach(channel => {
if (channel.hasOwnProperty('linkedItems') && channel.linkedItems) {
channel.linkedItems.forEach(itemName => {
dispatch(loadItems(itemName)).then(
item => things[thingIndex].items.push(item) // push the items inside the "things"
)
})
}
})
}
})
}
things.forEach(data => console.log(data.items.length, data.items)) // data.items.length returns 0, data.items returns a populated array
return things // return the modified array
}).catch(error => {
//todo: handle error
return false
})
}
}
As you can see, I perform an API call which returns data named response. The array is populated with all "things". If things exists, I want to load extra information named "items". Based on the information in the things array, I will perform another API call (which is done by dispatching the loadItems function) which returns another promise. Based on the data in the results of that API call, I will push into the items property (which is an array) of the things object.
As you can see in the comments, if I loop through the things array and log the items property which I just created, it's basically returning 0 as length, which means the things array is being returned before the things array is being modified.
I would like to know two things:
What is causing my code to run async. Is it the
dispatch(loadItems(itemName)) function since it returns a promise?
How am I able to synchronously execute my code?
Please note: this function loadThings() also returns a promise (if you're not familair with redux).
You might be interested in knowing what I tried myself to fix the code
Since I fail to understand the logic why the code is ran async, I've been trying hopeless stuff. Such as wrapping the code in another Promise.all and return the modified array in that promise. I used the then method of that promise to modify the things array, which had the exact same result. Probably because return things is being executed outside of that promise.
I'd really love to know what is going on
Edit
I have added the loadItems() code to the question, as requested:
export function loadItems(itemName) {
return dispatch => {
const url = itemName ? API_GET_ITEMS_ENDPOINT + `/${itemName}` : API_GET_ITEMS_ENDPOINT;
return dispatch({
type: 'LOAD_ITEMS',
payload: {
request: {
url: url,
method: 'GET'
}
}
}).then(response => {
return response.payload.data
})
}
}
My approach would be to map over things, creating arrays of promises for all of their items wrapped in a Promise.all, which gives you an array of Promise.all's.
Then you return things and this array of promises in another Promise.all and in the next then block, you can just assign the arrays to each thing with a simple for loop:
export function loadThings() {
return dispatch => {
return dispatch({
type: 'LOAD_THINGS',
payload: {
request: {
url: API_GET_THINGS_ENDPOINT,
method: 'GET'
}
}
}).then(response => {
let things = response.payload.data;
// Retrieve all items for the loaded "things"
const items = things.map((thing) => {
const thingItems = []
if (thing.hasOwnProperty('channels') && thing.channels) {
thing.channels.forEach(channel => {
if (channel.hasOwnProperty('linkedItems') && channel.linkedItems) {
channel.linkedItems.forEach(itemName => {
thingItems.push(dispatch(loadItems(itemName)));
});
}
});
}
return Promise.all(thingItems);
});
return Promise.all([things, Promise.all(items)])
})
.then(([things, thingItems]) => {
things.forEach((thing, index) => {
thing.items = thingItems[index];
})
return things;
})
.catch(error => {
//todo: handle error
return false
})
}
}
Edit:
You need to push the dispatch(loadItmes(itemName)) calls directly into thingItems.
I guess you could refactor it like the following:
export function loadThings() {
return dispatch => {
return dispatch({
type: 'LOAD_THINGS',
payload: {
request: {
url: API_GET_THINGS_ENDPOINT,
method: 'GET'
}
}
}).then(response => {
let things = response.payload.data;
// Retrieve all items for the loaded "things"
if( things ) {
return Promise.all( things.reduce( (promises, thing) => {
if (thing.channels) {
thing.items = [];
promises.push( ...thing.channels.map( channel =>
channel.linkedItems &&
channel.linkedItems.map( item =>
loadItems(item).then( result => thing.items.push( result ) )
) ).flat().filter( i => !!i ) );
}
return promises;
}, []) );
}
return things;
}).catch(error => {
//todo: handle error
return false
})
}
}
In case you would have things, it would check for the channels and the linkedItems for that channel, and create a promise that will push the result back to the thing.items array.
By returning the Promise.all, the continuation of the loadThings would only complete when the Promise.all was resolved. In case there are no things, just things gets returned (which would be a falsy value, so I am wondering how valid that statement could be)
I haven't actually tested the refactoring so there might be some brackets in need of adjusting to your situation, but I guess it gives you an idea?

trigger a synchronous call inside loop wait till two api calls get success then next iteration need to starts in angular 6

Tried with below code not wait for post call success jumping to next iteration before response comes.
Requirement:Need to have next iteration after the success of two api(POST/PATCH) calls
for (item of data) {
A(item)
}
A(value) {
const resp = this.post(url, {
'rationale': value['rationale']
})
.mergeMap(tempObj => {
value['detail'] = tempObj['id']
return this.patch(url, value['extid'], value)
})
.subscribe()
}
Recently I have used the toPromise function with angular http to turn an observable into a promise. If you have the outer loop inside an async function, this may work:
// This must be executed inside an async function
for (item of data) {
await A(item)
}
async A(value) {
const resp = await this.post(url, {
'rationale': value['rationale']
})
.mergeMap(tempObj => {
value['detail'] = tempObj['id']
return this.patch(url, value['extid'], value)
}).toPromise();
}
Use from to emit the items of an array. Use concatMap to map to an Observable and only map to the next one when the previous completed.
const resp$ = from(data).pipe(
concatMap(value => this.post(url, { 'rationale': value['rationale'] }).pipe(
switchMap(tempObj => {
value['detail'] = tempObj['id']
return this.patch(url, value['extid'], value)
})
))
)
I used switchMap instead of mergeMap to indicate that the feature of mergeMap to run multiple Observables simultaneously isn't used.
You could use:
<form (ngSubmit)="submit$.next(form.value)" ...
In your component:
submit$= new Subject();
ngOnInit {
submit$.pipe(
exhaustMap(value =>
this.post(url, {'rationale': value.rationale}))
.pipe(concatMap( response => {
value.detail = response.id;
return this.patch(url, value.extid, value);
}))).subscribe(); // rember to handle unsubcribe
}
The reason I use exhaustMap generally post and path are mutating calls, so that operator ensures first submit is process ignore the rest while processing AKA avoid double submit
An even better way is using ngrx effects, which if you dont know it yet I recomend to learn it
submit$ = createEffect(
() => this.actions$.pipe(
ofType(FeatureActions.submit),
exhaustMap( ({value}) => // if action has value property
this.post(url, { rationale : value.rationale}))
.pipe(concatMap( response => {
value.detail = response.id;
return this.patch(url, value.extid, value);
})),
map(result => FeatureActions.submitSuccess(result))
)
);

Chaining different API calls in a Vue component including one with a for loop

I'm trying to understand how to chain two different API calls including one with a for loop in a 'notes' Vue component. I have a really basic experience of promises and I'm looking to improve.
I'm making a first API call to get all the notes and pushing them into an array using a Vuex mutation. During that first API call I'm also mapping the different users emails into an Object.
Using this mapped object, I'm making a second API call inside a for loop to get all the users avatars.
Here's what the first API call looks like :
getAllNotesAPI(entity) {
noteService.getNotes(entity)
.then((response) => {
if (response.data.length === '0') {
// Set hasData to false if the response is 0
this.hasData = false;
} else {
// Push data into the note array using a store mutation
this.setAllNotes(response.data);
}
// Mapping all users emails into 'userEmails'
this.userEmails = [...new Set(response.data.map(x => x.userEmail))];
// Calling my second API call here to get all the avatars associated with these emails
for (let i = 0; i < this.userEmails.length; i++) {
this.getAvatarAPI(this.userEmails[i])
}
})
.catch((error) => {
console.log(error);
})
.finally(() => {
this.endLoader('notes');
});
},
this.getAvatarAPI is the second API call which looks like this :
getAvatarAPI(login) {
userService.getAvatar(login)
.then((response) => {
let newAvatar = {
userEmail: login,
picture: response.data.picture
};
// Push the response into a userAvatar Object using a store mutation
this.setUserAvatar(newAvatar);
}).catch((error) => {
console.log(error)
})
},
I've tried using async / await but couldn't figure out how to bind this inside of an async function (this.getAvatarAPI(this.userEmails)) was undefined, I've tried chaining using multiples then but couldn't figure out how to : get all my notes then all my avatars then end the 'note' loader once both those API calls are done.
If any of you could give me some pointers or the beginning of an answer that would be truly appreciated !
First whilst not related to your problem, avoid for loop when non necessary:
Do you need the i index?
for (let i = 0; i < this.userEmails.length; i++) {
this.getAvatarAPI(this.userEmails[i])
}
no. You need the userMail. Then
this.userEmails.forEach(userMail => {
this.getAvatarAPI(userEmail)
})
Now, to synchronize promises, you need to return a promise (let's not talk about async yet)
make getAvatarAPI return a promise
getAvatarAPI(login) {
return userService.getAvatar(login).then(blabla) // notice the return here
retrieve the promises of getAvatar API
let promises = this.userEmails.map(userMail => {
return getAvatarAPI(userEmail)
})
return after all promises have fulfilled
let promises = this.userEmails.map(userMail => {
return getAvatarAPI(userEmail)
})
return Promise.all(promises)
On a side note with async/await
If you use it you are not forced anymore to write return, you need to write async/await though
The underlying idea stay the same. Specifying the async keywords says that your function will return a promise-like.
e.g
async function p () {
return 5
}
p.then(x => console.log(x)) // does print 5 even though we didn't explicitely write return Promise.resolve(5)
Now you have to ensure you await the async function when you call it:
getAvatarAPI: async login => {
return userService.getAvatar(login).then(blabla)
}
// DO NOT do it
this.userEmails.forEach(userMail => {
return await this.getAvatarAPI(userEmail)
})
In forEach loop above, you will do your getAvatarAPI call in sequence because await "stops" iterating as long as getAvatarAPI has not resolved.
The proper way would be
getAllNotesAPI: async entity => {
try { // notice the necesary try-catch block
const response = await noteService.getNotes(entity)
blabla
let promises = this.userEmails.map(userMail => {
return this.getA...
})
let result = await Promise.all(promises)
// eventually return result, or simply await Promise... without lefthand-side assignment
} catch (error) {
console.log(error);
}
console.log(this.end('loader'))
}

Promise All retry

I know that promise.all() fails when even 1 of the promise is failed. I only want to try for failed promises and don't want to run promise.all() again.
Any recommendations on how I can achieve this in minimal way?
Promises are eager construct and model a value obtained asynchronously,
a Promise is produced using some kind of producer, like fetch for instance.
If you retain a reference to this producer then you can replay the nmechanism
that produced the Promise in the first place.
// producer function
function getData (arg) {
const result = new Promise();
return result.then(value => {
return { ok:true, value };
}, error => {
return {
ok: false,
value: error,
// retry is a function which calls the producer with the same arguments
retry: () => getData(arg)
};
})
}
Then if you have something like:
const data = [];
// Promise<{ok: boolean, value: any, retry?: function}>
// No promises will fail in this array
const asyncResults = data.map(getResults);
Promise.all(asyncResults)
.then((results) => {
const successes = results.filter(res => res.ok);
const retrys = results.filter(res => !res.ok).map(res => res.retry()); // retry all failed promises
})
Memory leaks, stack overflow: because I retain a reference to original arguments in order to retry and the algorithm is recursive there could be a memory leak. However the algorithm cannot "stack overflow":
getData calls do not get "deeper" over time (see retry definition)
the asyncrhonicity of the algorithm prevent this behaviour if a promise was never resolved
old data is properly discarded when accessing the results as const resultData = results.filter(res => res.ok).map(res => res.value);
However the algorithm could take a long time to settle if a promise keep on not getting resolved and prevent access to the rest of the values.
In an alternative I suggest you take a look at another async primitive, not yet part of the language (maybe some day) : Observables which are designed for this kind of tasks: lazy, retry-able async operations.
You may use async package and wrap all promise calls with closure with done argument.
Then simply resolve results.
const async = require('async');
const promiseAll = promises => {
return new Promise((resolve) => {
// preparing array of functions which has done method as callback
const parallelCalls = promises.map(promise => {
return done => {
promise
.then(result => done(null, result)
.catch(error => {
console.error(error.message);
done();
});
}
});
// calling array of functions in parallel
async.parallel(
parallelCalls,
(_, results) => resolve(results.filter(Boolean))
);
});
};
router.get('/something', async (req, res) => {
...
const results = await promiseAll(promises);
...
});
Or we can simply do Promise.all without using async package:
router.get('/something', async (req, res) => {
...
const results = (
await Promise.all(
promises.map(promise => {
return new Promise(resolve => {
promise.then(resolve).catch(e => resolve());
});
});
)
).filter(Boolean);
...
});

use forEach() with promises while access previous promise results in a .then() chain?

I have the following functions with promises:
const ajaxRequest = (url) => {
return new Promise(function(resolve, reject) {
axios.get(url)
.then((response) => {
//console.log(response);
resolve(response);
})
.catch((error) => {
//console.log(error);
reject();
});
});
}
const xmlParser = (xml) => {
let { data } = xml;
return new Promise(function(resolve, reject) {
let parser = new DOMParser();
let xmlDoc = parser.parseFromString(data,"text/xml");
if (xmlDoc.getElementsByTagName("AdTitle").length > 0) {
let string = xmlDoc.getElementsByTagName("AdTitle")[0].childNodes[0].nodeValue;
resolve(string);
} else {
reject();
}
});
}
I'm trying to apply those functions for each object in array of JSON:
const array = [{"id": 1, "url": "www.link1.com"}, {"id": 1, "url": "www.link2.com"}]
I came up with the following solution:
function example() {
_.forEach(array, function(value) {
ajaxRequest(value.url)
.then(response => {
xmlParser(response)
.catch(err => {
console.log(err);
});
});
}
}
I was wondering if this solution is acceptable regarding 2 things:
Is it a good practice to apply forEach() on promises in the following matter.
Are there any better ways to pass previous promise results as parameter in then() chain? (I'm passing response param).
You can use .reduce() to access previous Promise.
function example() {
return array.reduce((promise, value) =>
// `prev` is either initial `Promise` value or previous `Promise` value
promise.then(prev =>
ajaxRequest(value.url).then(response => xmlParser(response))
)
, Promise.resolve())
}
// though note, no value is passed to `reject()` at `Promise` constructor calls
example().catch(err => console.log(err));
Note, Promise constructor is not necessary at ajaxRequest function.
const ajaxRequest = (url) =>
axios.get(url)
.then((response) => {
//console.log(response);
return response;
})
.catch((error) => {
//console.log(error);
});
The only issue with the code you provided is that result from xmlParser is lost, forEach loop just iterates but does not store results. To keep results you will need to use Array.map which will get Promise as a result, and then Promise.all to wait and get all results into array.
I suggest to use async/await from ES2017 which simplifies dealing with promises. Since provided code already using arrow functions, which would require transpiling for older browsers compatibility, you can add transpiling plugin to support ES2017.
In this case your code would be like:
function example() {
return Promise.all([
array.map(async (value) => {
try {
const response = await ajaxRequest(value.url);
return xmlParser(response);
} catch(err) {
console.error(err);
}
})
])
}
Above code will run all requests in parallel and return results when all requests finish. You may also want to fire and process requests one by one, this will also provide access to previous promise result if that was your question:
async function example(processResult) {
for(value of array) {
let result;
try {
// here result has value from previous parsed ajaxRequest.
const response = await ajaxRequest(value.url);
result = await xmlParser(response);
await processResult(result);
} catch(err) {
console.error(err);
}
}
}
Another solution is using Promise.all for doing this, i think is a better solution than looping arround the ajax requests.
const array = [{"id": 1, "url": "www.link1.com"}, {"id": 1, "url": "www.link2.com"}]
function example() {
return Promise.all(array.map(x => ajaxRequest(x.url)))
.then(results => {
return Promise.all(results.map(data => xmlParser(data)));
});
}
example().then(parsed => {
console.log(parsed); // will be an array of xmlParsed elements
});
Are there any better ways to pass previous promise results as
parameter in then() chain?
In fact, you can chain and resolve promises in any order and any place of code. One general rule - any chained promise with then or catch branch is just new promise, which should be chained later.
But there are no limitations. With using loops, most common solution is reduce left-side foldl, but you also can use simple let-variable with reassign with new promise.
For example, you can even design delayed promises chain:
function delayedChain() {
let resolver = null
let flow = new Promise(resolve => (resolver = resolve));
for(let i=0; i<100500; i++) {
flow = flow.then(() => {
// some loop action
})
}
return () => {
resolver();
return flow;
}
}
(delayedChain())().then((result) => {
console.log(result)
})

Categories

Resources