Develop Reference

RxJs MergeMap and finalize not working as expected (as I thought)

I am having problems with finalize or complete using the code below. I want to be able to call loading = false after all is done. I tried to use finalize and complete but it is being called too early and multiple times. Any help? Thanks
displayPost(postDetail) {
//some rendering stuff
renderHTML(postDetail);
}
// returns a post query object
getPostQuery(username, postId) {
query: `{
post (id:"${username}", postId:"${postId}") {
id
title
content }`
return query;
}
let loading = true
//getUserHTTP() is an http request
getUserHTTP(userId).pipe(
switchMap(user => {
const ids = [];
// let's say oldest 100 posts
for (let i = 0; i < 100; i++) {
ids.push(getPostQuery(user.username, i))
}
return from(ids);
}),
mergeMap(query => this.httpClient.post(url, query)),
finalize(() => {
//this is being called too early and multiple times
loading = false;
})
).subscribe({
next: result => { /* doing staff */ displayPost(result) },
error: err => { /* handle errors */ },
complete: () => { /* not working */ }
})

mergeMap would trigger multiple parallel requests with each having it's own stream path. So the finalize would be triggered for each stream. Instead you could use forkJoin to trigger multiple requests in parallel that will only emit after all the individual streams complete.
If you're using toPromise() to convert the HTTP observable to a promise, avoid doing it. It isn't required here. You could also skip the from when using forkJoin.
Try the following
import { forkJoin } from 'rxjs';
import { switchMap, finalize } from 'rxjs/operators';
let loading = true;
getUserHTTP(userId).pipe(
switchMap(user => {
const reqs$ = [];
for (let i = 0; i < 100; i++) { // let's say oldest 100 posts
ids.push(
this.httpClient.post( // <-- do the HTTP request here
url,
getPostQuery(user.username, i)
)
);
}
return forkJoin(req$); // <-- return `forkJoin` here
}),
finalize(() => loading = false)
).subscribe({
next: result => { /* doing stuff */ displayPost(result) },
error: err => { /* handle errors */ },
complete: () => { }
});

Synchronously populate/modify an array and return the modified array inside a promise

I'm pretty new to ReactJS and redux, so I've never really had to work with this before. I'm retrieving data from an API call in my project. I want to modify the data by adding a new property to the object. However, because the code is not ran synchronously, the unmodified array is being returned (I assume) instead of the modified array.
export function loadThings() {
return dispatch => {
return dispatch({
type: 'LOAD_THINGS',
payload: {
request: {
url: API_GET_THINGS_ENDPOINT,
method: 'GET'
}
}
}).then(response => {
let things = response.payload.data;
// Retrieve all items for the loaded "things"
if(things) {
things.forEach((thing, thingIndex) => {
things[thingIndex].items = []
if (thing.hasOwnProperty('channels') && thing.channels) {
thing.channels.forEach(channel => {
if (channel.hasOwnProperty('linkedItems') && channel.linkedItems) {
channel.linkedItems.forEach(itemName => {
dispatch(loadItems(itemName)).then(
item => things[thingIndex].items.push(item) // push the items inside the "things"
)
})
}
})
}
})
}
things.forEach(data => console.log(data.items.length, data.items)) // data.items.length returns 0, data.items returns a populated array
return things // return the modified array
}).catch(error => {
//todo: handle error
return false
})
}
}
As you can see, I perform an API call which returns data named response. The array is populated with all "things". If things exists, I want to load extra information named "items". Based on the information in the things array, I will perform another API call (which is done by dispatching the loadItems function) which returns another promise. Based on the data in the results of that API call, I will push into the items property (which is an array) of the things object.
As you can see in the comments, if I loop through the things array and log the items property which I just created, it's basically returning 0 as length, which means the things array is being returned before the things array is being modified.
I would like to know two things:
What is causing my code to run async. Is it the
dispatch(loadItems(itemName)) function since it returns a promise?
How am I able to synchronously execute my code?
Please note: this function loadThings() also returns a promise (if you're not familair with redux).
You might be interested in knowing what I tried myself to fix the code
Since I fail to understand the logic why the code is ran async, I've been trying hopeless stuff. Such as wrapping the code in another Promise.all and return the modified array in that promise. I used the then method of that promise to modify the things array, which had the exact same result. Probably because return things is being executed outside of that promise.
I'd really love to know what is going on
Edit
I have added the loadItems() code to the question, as requested:
export function loadItems(itemName) {
return dispatch => {
const url = itemName ? API_GET_ITEMS_ENDPOINT + `/${itemName}` : API_GET_ITEMS_ENDPOINT;
return dispatch({
type: 'LOAD_ITEMS',
payload: {
request: {
url: url,
method: 'GET'
}
}
}).then(response => {
return response.payload.data
})
}
}

My approach would be to map over things, creating arrays of promises for all of their items wrapped in a Promise.all, which gives you an array of Promise.all's.
Then you return things and this array of promises in another Promise.all and in the next then block, you can just assign the arrays to each thing with a simple for loop:
export function loadThings() {
return dispatch => {
return dispatch({
type: 'LOAD_THINGS',
payload: {
request: {
url: API_GET_THINGS_ENDPOINT,
method: 'GET'
}
}
}).then(response => {
let things = response.payload.data;
// Retrieve all items for the loaded "things"
const items = things.map((thing) => {
const thingItems = []
if (thing.hasOwnProperty('channels') && thing.channels) {
thing.channels.forEach(channel => {
if (channel.hasOwnProperty('linkedItems') && channel.linkedItems) {
channel.linkedItems.forEach(itemName => {
thingItems.push(dispatch(loadItems(itemName)));
});
}
});
}
return Promise.all(thingItems);
});
return Promise.all([things, Promise.all(items)])
})
.then(([things, thingItems]) => {
things.forEach((thing, index) => {
thing.items = thingItems[index];
})
return things;
})
.catch(error => {
//todo: handle error
return false
})
}
}
Edit:
You need to push the dispatch(loadItmes(itemName)) calls directly into thingItems.

I guess you could refactor it like the following:
export function loadThings() {
return dispatch => {
return dispatch({
type: 'LOAD_THINGS',
payload: {
request: {
url: API_GET_THINGS_ENDPOINT,
method: 'GET'
}
}
}).then(response => {
let things = response.payload.data;
// Retrieve all items for the loaded "things"
if( things ) {
return Promise.all( things.reduce( (promises, thing) => {
if (thing.channels) {
thing.items = [];
promises.push( ...thing.channels.map( channel =>
channel.linkedItems &&
channel.linkedItems.map( item =>
loadItems(item).then( result => thing.items.push( result ) )
) ).flat().filter( i => !!i ) );
}
return promises;
}, []) );
}
return things;
}).catch(error => {
//todo: handle error
return false
})
}
}
In case you would have things, it would check for the channels and the linkedItems for that channel, and create a promise that will push the result back to the thing.items array.
By returning the Promise.all, the continuation of the loadThings would only complete when the Promise.all was resolved. In case there are no things, just things gets returned (which would be a falsy value, so I am wondering how valid that statement could be)
I haven't actually tested the refactoring so there might be some brackets in need of adjusting to your situation, but I guess it gives you an idea?

Vuex action not waiting to finish axios promise

I encounter a strange situation developing an application in Laravel + VueJS/Vuex stack.
I understand that if a promise is not returned the parent function calling it will not wait for it to resolve so things will go asynchronous. Axios returns a promise by default when calling a resourse through http.
So i have the parent function which looks like this:
fetchInvoiceSeries() {
var arr = []
let invsrs = this.$store.getters['getInvoiceSeries']
if (invsrs == null) {
return this.$store
.dispatch('get_invoice_series')
.then(() => {
invsrs = this.$store.getters['getInvoiceSeries']
if (invsrs != null) {
invsrs.forEach(function(s) {
arr.push({
value: s.id,
text: s.series + ' / ' + s.increment
})
})
this.series = arr
} else {
console.log('Error while fetching invoice series!')
}
})
.catch(e => {
console.log(e)
})
} else {
invsrs.forEach(function(s) {
arr.push({
value: s.id,
text: s.series + ' / ' + s.increment
})
})
this.series = arr
}
}
And here is the function defined in action part of the vuex module:
get_invoice_series({ commit }) {
return get('/api/series/0')
.then(response => {
if (response.data && typeof response.data !== undefined) {
let payload = response.data
commit('SET_INVOICE_SERIES', payload)
} else {
console.log('error', error)
}
})
.catch(error => {
console.log('error', error)
})
},
So as you can see i am returning the get request from axios inside the action. In the parent i am calling the action and the "then" keyword in order to do some processing after the action it's done. Also i am using arrow function because i need the context in the parent function in order to call this.$store ...
The problem is that even after checking the getter to see if the state have the invoice series and getting them using the get_invoice_series action i still don't have the invoice series in memory judging by the code i wrote. The console keeps loggin 'Error while fetching invoice series!' the first time i execute the code and the second time (after the information exists in state), the code skips fetching the invoice series (as expected).
Can you tell me what i am doing wrong ? Thank you!

Your error comes from invsrs being null the first time, and not null the second time.
This means that your function get_invoice_series({ commit }) is asynchronous, and that it returns a promise.
For more readability, maybe you should make your call independently from your return statement, using async/await expressions :
async get_invoice_series({ commit }) {
const response = await get('/api/series/0')
if (response.data === undefined) return null
const payload = response.data
commit('SET_INVOICE_SERIES', payload)
return payload
},
And then make your calls wait for this fetch to process :
async fetchInvoiceSeries() {
let arr = []
const invsrs = await this.$store.getters['getInvoiceSeries']
// ...
It's pure conjecture here, let me know if it helps or not.

trigger a synchronous call inside loop wait till two api calls get success then next iteration need to starts in angular 6

Tried with below code not wait for post call success jumping to next iteration before response comes.
Requirement:Need to have next iteration after the success of two api(POST/PATCH) calls
for (item of data) {
A(item)
}
A(value) {
const resp = this.post(url, {
'rationale': value['rationale']
})
.mergeMap(tempObj => {
value['detail'] = tempObj['id']
return this.patch(url, value['extid'], value)
})
.subscribe()
}

Recently I have used the toPromise function with angular http to turn an observable into a promise. If you have the outer loop inside an async function, this may work:
// This must be executed inside an async function
for (item of data) {
await A(item)
}
async A(value) {
const resp = await this.post(url, {
'rationale': value['rationale']
})
.mergeMap(tempObj => {
value['detail'] = tempObj['id']
return this.patch(url, value['extid'], value)
}).toPromise();
}

Use from to emit the items of an array. Use concatMap to map to an Observable and only map to the next one when the previous completed.
const resp$ = from(data).pipe(
concatMap(value => this.post(url, { 'rationale': value['rationale'] }).pipe(
switchMap(tempObj => {
value['detail'] = tempObj['id']
return this.patch(url, value['extid'], value)
})
))
)
I used switchMap instead of mergeMap to indicate that the feature of mergeMap to run multiple Observables simultaneously isn't used.

You could use:
<form (ngSubmit)="submit$.next(form.value)" ...
In your component:
submit$= new Subject();
ngOnInit {
submit$.pipe(
exhaustMap(value =>
this.post(url, {'rationale': value.rationale}))
.pipe(concatMap( response => {
value.detail = response.id;
return this.patch(url, value.extid, value);
}))).subscribe(); // rember to handle unsubcribe
}
The reason I use exhaustMap generally post and path are mutating calls, so that operator ensures first submit is process ignore the rest while processing AKA avoid double submit
An even better way is using ngrx effects, which if you dont know it yet I recomend to learn it
submit$ = createEffect(
() => this.actions$.pipe(
ofType(FeatureActions.submit),
exhaustMap( ({value}) => // if action has value property
this.post(url, { rationale : value.rationale}))
.pipe(concatMap( response => {
value.detail = response.id;
return this.patch(url, value.extid, value);
})),
map(result => FeatureActions.submitSuccess(result))
)
);

Reactjs and redux - How to prevent excessive api calls from a live-search component?

I have created this live-search component:
class SearchEngine extends Component {
constructor (props) {
super(props);
this.handleChange = this.handleChange.bind(this);
this.handleSearch = this.handleSearch.bind(this);
}
handleChange (e) {
this.props.handleInput(e.target.value); //Redux
}
handleSearch (input, token) {
this.props.handleSearch(input, token) //Redux
};
componentWillUpdate(nextProps) {
if(this.props.input !== nextProps.input){
this.handleSearch(nextProps.input, this.props.loginToken);
}
}
render () {
let data= this.props.result;
let searchResults = data.map(item=> {
return (
<div key={item.id}>
<h3>{item.title}</h3>
<hr />
<h4>by {item.artist}</h4>
<img alt={item.id} src={item.front_picture} />
</div>
)
});
}
return (
<div>
<input name='input'
type='text'
placeholder="Search..."
value={this.props.input}
onChange={this.handleChange} />
<button onClick={() => this.handleSearch(this.props.input, this.props.loginToken)}>
Go
</button>
<div className='search_results'>
{searchResults}
</div>
</div>
)
}
It is part of a React & Redux app I'm working on and is connected to the Redux store.
The thing is that when a user types in a search query, it fires an API call for each of the characters in the input, and creating an excessive API calling, resulting in bugs like showing results of previous queries, not following up with the current search input.
My api call (this.props.handleSearch):
export const handleSearch = (input, loginToken) => {
const API= `https://.../api/search/vector?query=${input}`;
}
return dispatch => {
fetch(API, {
headers: {
'Content-Type': 'application/json',
'Authorization': loginToken
}
}).then(res => {
if (!res.ok) {
throw Error(res.statusText);
}
return res;
}).then(res => res.json()).then(data => {
if(data.length === 0){
dispatch(handleResult('No items found.'));
}else{
dispatch(handleResult(data));
}
}).catch((error) => {
console.log(error);
});
}
};
My intention is that it would be a live-search, and update itself based on user input. but I am trying to find a way to wait for the user to finish his input and then apply the changes to prevent the excessive API calling and bugs.
Suggestions?
EDIT:
Here's what worked for me.
Thanks to Hammerbot's amazing answer I managed to create my own class of QueueHandler.
export default class QueueHandler {
constructor () { // not passing any "queryFunction" parameter
this.requesting = false;
this.stack = [];
}
//instead of an "options" object I pass the api and the token for the "add" function.
//Using the options object caused errors.
add (api, token) {
if (this.stack.length < 2) {
return new Promise ((resolve, reject) => {
this.stack.push({
api,
token,
resolve,
reject
});
this.makeQuery()
})
}
return new Promise ((resolve, reject) => {
this.stack[1] = {
api,
token,
resolve,
reject
};
this.makeQuery()
})
}
makeQuery () {
if (! this.stack.length || this.requesting) {
return null
}
this.requesting = true;
// here I call fetch as a default with my api and token
fetch(this.stack[0].api, {
headers: {
'Content-Type': 'application/json',
'Authorization': this.stack[0].token
}
}).then(response => {
this.stack[0].resolve(response);
this.requesting = false;
this.stack.splice(0, 1);
this.makeQuery()
}).catch(error => {
this.stack[0].reject(error);
this.requesting = false;
this.stack.splice(0, 1);
this.makeQuery()
})
}
}
I made a few changes in order for this to work for me (see comments).
I imported it and assigned a variable:
//searchActions.js file which contains my search related Redux actions
import QueueHandler from '../utils/QueueHandler';
let queue = new QueueHandler();
Then in my original handleSearch function:
export const handleSearch = (input, loginToken) => {
const API= `https://.../api/search/vector?query=${input}`;
}
return dispatch => {
queue.add(API, loginToken).then... //queue.add instead of fetch.
Hope this helps anyone!

I think that they are several strategies to handle the problem. I'm going to talk about 3 ways here.
The two first ways are "throttling" and "debouncing" your input. There is a very good article here that explains the different techniques: https://css-tricks.com/debouncing-throttling-explained-examples/
Debounce waits a given time to actually execute the function you want to execute. And if in this given time you make the same call, it will wait again this given time to see if you call it again. If you don't, it will execute the function. This is explicated with this image (taken from the article mentioned above):
Throttle executes the function directly, waits a given time for a new call and executes the last call made in this given time. The following schema explains it (taken from this article http://artemdemo.me/blog/throttling-vs-debouncing/):
I was using those first techniques at first but I found some downside to it. The main one was that I could not really control the rendering of my component.
Let's imagine the following function:
function makeApiCall () {
api.request({
url: '/api/foo',
method: 'get'
}).then(response => {
// Assign response data to some vars here
})
}
As you can see, the request uses an asynchronous process that will assign response data later. Now let's imagine two requests, and we always want to use the result of the last request that have been done. (That's what you want in a search input). But the result of the second request comes first before the result of the first request. That will result in your data containing the wrong response:
1. 0ms -> makeApiCall() -> 100ms -> assigns response to data
2. 10ms -> makeApiCall() -> 50ms -> assigns response to data
The solution for that to me was to create some sort of "queue". The behaviour of this queue is:
1 - If we add a task to the queue, the task goes in front of the queue.
2 - If we add a second task to the queue, the task goes in the second position.
3 - If we add a third task to the queue, the task replaces the second.
So there is a maximum of two tasks in the queue. As soon as the first task has ended, the second task is executed etc...
So you always have the same result, and you limit your api calls in function of many parameters. If the user has a slow internet connexion, the first request will take some time to execute, so there won't be a lot of requests.
Here is the code I used for this queue:
export default class HttpQueue {
constructor (queryFunction) {
this.requesting = false
this.stack = []
this.queryFunction = queryFunction
}
add (options) {
if (this.stack.length < 2) {
return new Promise ((resolve, reject) => {
this.stack.push({
options,
resolve,
reject
})
this.makeQuery()
})
}
return new Promise ((resolve, reject) => {
this.stack[1] = {
options,
resolve,
reject
}
this.makeQuery()
})
}
makeQuery () {
if (! this.stack.length || this.requesting) {
return null
}
this.requesting = true
this.queryFunction(this.stack[0].options).then(response => {
this.stack[0].resolve(response)
this.requesting = false
this.stack.splice(0, 1)
this.makeQuery()
}).catch(error => {
this.stack[0].reject(error)
this.requesting = false
this.stack.splice(0, 1)
this.makeQuery()
})
}
}
You can use it like this:
// First, you create a new HttpQueue and tell it what to use to make your api calls. In your case, that would be your "fetch()" function:
let queue = new HttpQueue(fetch)
// Then, you can add calls to the queue, and handle the response as you would have done it before:
queue.add(API, {
headers: {
'Content-Type': 'application/json',
'Authorization': loginToken
}
}).then(res => {
if (!res.ok) {
throw Error(res.statusText);
}
return res;
}).then(res => res.json()).then(data => {
if(data.length === 0){
dispatch(handleResult('No vinyls found.'));
}else{
dispatch(handleResult(data));
}
}).catch((error) => {
console.log(error);
});
}