Develop Reference

How to make recursive HTTP calls using RxJs operators?

I use the following method in odder to retrieve data by passing pageIndex (1) and pageSize (500) for each HTTP call.
this.demoService.geList(1, 500).subscribe(data => {
this.data = data.items;
});
The response has a property called isMore and I want to modify my method in odder to continue HTTP calls if isMore is true. I also need to merge the returned values and finally return the accumulated values.
For example, assuming that there are 5000 records and until 10th HTTP call, the service returns true for isMore value. After 10th HTTP call, it returns false and then this method sets this.data value with the merged 5000 records. For this problem, should I use mergeMap or expand or another RxJs operator? What is the proper way to solve this problem?
Update: I use the following approach, but it does not merge the returned values and not increase the pageIndex. For this reason it does not work (I tried to make some changes, but could not make it work).
let pageIndex = 0;
this.demoService.geList(pageIndex+1, 500).pipe(
expand((data) => {
if(data.isComplete) {
return of(EMPTY);
} else {
return this.demoService.geList(pageIndex+1, 500);
}
})
).subscribe((data) => {
//your logic here
});
Update II:
of({
isMore : true,
pageIndex: 0,
items: []
}).pipe(
expand(data => demoService.geList(data.pageIndex+1, 100)
.pipe(
map(newData => ({...newData, pageIndex: data.pageIndex+1}))
)),
// takeWhile(data => data.isMore), //when using this, it does not work if the total record is less than 100
takeWhile(data => (data.isMore || data.pageIndex === 1)), // when using this, it causing +1 extra HTTP call unnecessarily
map(data => data.items),
reduce((acc, items) => ([...acc, ...items]))
)
.subscribe(data => {
this.data = data;
});
Update III:
Finally I made it work by modifying Elisseo's approach as shown below. Howeveri **I need to make it void and set this.data parameter in this getData() method. How can I do this?
getData(pageIndex, pageSize) {
return this.demoService.geList(pageIndex, pageSize).pipe(
switchMap((data: any) => {
if (data.isMore) {
return this.getData(pageIndex+1, pageSize).pipe(
map((res: any) => ({ items: [...data.items, ...res.items] }))
);
}
return of(data);
})
);
}
I want to merge the following subscribe part to this approach but I cannot due to some errors e.g. "Property 'pipe' does not exist on type 'void'."
.subscribe((res: any) => {
this.data = res;
});

getData(pageIndex, pageSize) {
return this.demoService.getList(pageIndex, pageSize).pipe(
switchMap((data: any) => {
if (!data.isCompleted) {
return this.getData(pageIndex+1, pageSize).pipe(
map((res: any) => ({ data: [...data.data, ...res.data] }))
);
}
return of(data);
})
);
}
stackblitz
NOTE: I updated pasing as argument pageIndex+1 as #mbojko suggest -before I wrote pageIndex++
UPDATE 2
Using expand operator we need take account that we need feed the "recursive function" with an object with pageIndex -it's necesarry in our call- for this, when we make this.demoService.getList(data.pageIndex+1,10) we need "transform the result" adding a new property "pageIndex". for this we use "map"
getData() {
//see that initial we create "on fly" an object with properties: pageIndex,data and isCompleted
return of({
pageIndex:1,
data:[],
isCompleted:false
}).pipe(
expand((data: any) => {
return this.demoService.getList(data.pageIndex,10).pipe(
//here we use map to create "on fly" and object
map((x:any)=>({
pageIndex:data.pageIndex+1, //<--pageIndex the pageIndex +1
data:[...data.data,...x.data], //<--we concatenate the data using spread operator
isCompleted:x.isCompleted})) //<--isCompleted the value
)
}),
takeWhile((data: any) => !data.isCompleted,true), //<--a take while
//IMPORTANT, use "true" to take account the last call also
map(res=>res.data) //finally is we only want the "data"
//we use map to return only this property
)
}
Well we can do a function like this:
getData() {
of({pageIndex:1,data:[],isCompleted:false}).pipe(
expand((data: any) => {
return this.demoService.getList(data.pageIndex,10).pipe(
tap(x=>{console.log(x)}),
map((x:any)=>({
pageIndex:data.pageIndex+1,
data:[...data.data,...x.data],
isComplete:x.isComplete}))
)
}),
takeWhile((data: any) => !data.isComplete,true), //<--don't forget the ",true"
).subscribe(res=>{
this.data=res.data
})
}
See that in this case we don't return else simple subscribe to the function and equal a variable this.data to res.data -it's the reason we don't need the last map
Update 3 by Mrk Sef
Finally, if you don't want your stream to emit intermittent values and you just want the final concatenated data, you can remove the data concatenation from expand, and use reduce afterward instead.
getData() {
of({
pageIndex: 1,
data: [],
isCompleted: false
})
.pipe(
expand((prevResponse: any) => this.demoService.getList(prevResponse.pageIndex, 10).pipe(
map((nextResponse: any) => ({
...nextResponse,
pageIndex: prevResponse.pageIndex + 1
}))
)
),
takeWhile((response: any) => !response.isCompleted, true),
// Keep concatenting each new array (data.items) until the stream
// completes, then emit them all at once
reduce((acc: any, data: any) => {
return [...acc, ...data.data];
}, [])
)
.subscribe(items => {
this.data=items;
});
}

It doesn't matter if you're total record change as long as api response give you the isMore flag.
I'm skipping the part how to implement reducer action event i'm assuming you've already done that part. So i will just try to explain with pseudo codes.
You have a table or something like that with pagination data. on intial state you can just create an loadModule effect or using this fn:
getPaginationDataWithPageIndex(pageIndex = 1){
this.store.dispatch(new GetPaginationData({ pageIndex: pageIndex, dataSize: 500}));
}
in your GetPaginationData effect
... map(action => {
return apicall.pipe(map((response)=> {
if(response.isMore){
return new updateState({data:response.data, isMore: responseisMore})
} else {
return new updateState({isMore: response.isMore}),
}
}})
`
all you have to left is subscribing store in your .ts if isMore is false you will not display the next page button. and on your nextButton or prevButton's click method you should have to just dispatch the action with pageIndex

I do not think recursion is the correct approach here:
interval(0).pipe(
map(count => this.demoService.getList(count + 1, 500)),
takeWhile(reponse => response.isMore, true),
reduce((acc, curr) => //reduce any way you like),
).subscribe();
This should make calls to your endpoint until the endpoint returns isMore === false. The beautiful thing about interval is that we get the count variable for free.
But if you are set on using recrsion, here is the rxjs-way to do that using the expand-operator (see the docs). I find it slightly less readable, as it requires an if-else-construct which increases code complexity. Also the outer 'counter' variable just isn't optimal.
let index = 1;
this.demoService.geList(index, 500).pipe(
expand(response => response.isMore ? this.demoService.geList(++index, 500) : empty()),
reduce((acc, curr) => //reduce here)
).subscribe();

double combineLatest doesn't emit update

In my project there are activities that people have created, joined, bookmarked or organized. I've read a lot of these question already. But most of the code was less complex or people forgot to subscribe...
I would like to get all the activities in a certain time period and then add creator information (name, image, etc) and add booleans if the user retrieving these activities has joined/bookmarked/organized this activity. The code I used before would provide live updates (ex. I join an activity, by adding my userId to the participants array and the activity.joined would update to true).
Previous code:
public getActivities(user: UserProfileModel): Observable<Array<ActivityModel>> {
const now: number = moment().startOf('day').unix();
const later: number = moment().startOf('day').add(30, 'day').unix();
return this.afs.collection<ActivityModel>(`cities/${user.city.id}/activities`, ref => ref
.where('datetime', '>=', now)
.where('datetime', '<=', later))
.valueChanges({ idField: 'id' })
.pipe(
map(activities => activities.map(activity => {
const bookmarked = activity.bookmarkers ? activity.bookmarkers.includes(user.uid) : false;
const joined = activity.participants ? activity.participants.includes(user.uid) : false;
const organized = activity.organizers ? activity.organizers.includes(user.uid) : false;
return { bookmarked, joined, organized, ...activity } as ActivityModel;
}))
);
}
The I wanted to add the creator as an observable object, so their latest changes in name or profile picture would be shown. But with this code change, my getActivities doesn't emit any updates anymore...
My new code:
public getActivities(user: UserProfileModel): Observable<Array<CombinedActivityCreatorModel>> {
const now: number = moment().startOf('day').unix();
const later: number = moment().startOf('day').add(30, 'day').unix();
return this.afs.collection<ActivityModel>(`cities/${user.city.id}/activities`, ref => ref
.where('datetime', '>=', now)
.where('datetime', '<=', later))
.valueChanges({ idField: 'id' })
.pipe(
concatMap(activities => {
const completeActivityData = activities.map(activity => {
const activityCreator: Observable<UserProfileModel> = this.getCreator(activity.creator);
const bookmarked = activity.bookmarkers ? activity.bookmarkers.includes(user.uid) : false;
const joined = activity.participants ? activity.participants.includes(user.uid) : false;
const organized = activity.organizers ? activity.organizers.includes(user.uid) : false;
return combineLatest([
of({ bookmarked, joined, organized, ...activity }),
activityCreator
]).pipe(
map(([activityData, creatorObject]: [ActivityModel, UserProfileModel]) => {
return {
...activityData,
creatorObject: creatorObject
} as CombinedActivityCreatorModel;
})
);
});
return combineLatest(completeActivityData);
})
);
}
The code has become a bit complex, that I don't see the solution myself. Anybody that can offer some assistance?

Looks like one of activityCreator doesn't emit a value, combineLatest requires all observables to emit at least once.
I would recommend you to debug how activityCreator behaves.
If it's fine that it doesn't emit you have 2 options: startWith to set a value for an initial emit, or defaultIfEmpty, it emits in case if stream is going to be closed without any emit.
activityCreator = this.getCreator(activity.creator).pipe(
// startWith(null), // for example if you want to trigger combineLatest.
// defaultIfEmpty(null), // in case of an empty stream.
);
another thing is concatMap it requires an observable to complete, only then it switches to the next one, parhaps mergeMap or switchMap fits here better.
Try the code below and add its output to the comments. Thanks.
const activityCreator: Observable<UserProfileModel> = this.getCreator(activity.creator).pipe(
tap(
() => console.log('getCreator:emit'),
() => console.log('getCreator:error'),
() => console.log('getCreator:completed'),
),
);

Why cant I pass an objects string value into a API sorting function in React?

I am trying to inject the value of {clickedYear} into my filter function so that I can use the information dynamically.
How can I pull the value from the {clickedYear} so that the response from the API is sorted properly within an array? On the console.log the information is there in a string but it's not working within the function. The sorting doesn't work and returns an empty array...
here is my code:
const InformationBoxLayout = ({ clickedYear }) => {
const [activeYear, setActiveYear] = useState([]);
useEffect(() => {
axios
.get('http://www.mocky.io/v2/5ea446a43000005900ce2ca3')
.then((response) =>
setActiveYear(
response.data.timelineInfo.filter(
(item) => item.year === { clickedYear }
)
)
);
}, []);
return (...some code...)

You don't need {} around clickedYear when you're not in a JSX expression, which you aren't in the filter callback. So:
setActiveYear(
response.data.timelineInfo.filter(
(item) => item.year === clickedYear
// −−−−−−−−−−−−−−−−−−−−−−−−−−−−^^^^^^^^^^^^
)
)
The reason it wasn't working is that { clickedYear } is an object literal creating an object with a clickedYear property (it's shorthand property notation for { clickedYear: clickedYear }). An object will never be === a string (I see in the JSON that year is a string). Also ensure that clickedYear is a string, not a number, since again you're using === which will never be true for different types.
A second thing you need to do is pass clickedYear as a dependency to your useEffect hook, since you use its value within it:
useEffect(() => {
axios
.get('http://www.mocky.io/v2/5ea446a43000005900ce2ca3')
.then((response) =>
setActiveYear(
response.data.timelineInfo.filter(
(item) => item.year === clickedYear
// −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−^^^^^^^^^^^
)
)
);
}, [clickedYear]);
// ^^^^^^^^^^^^^
Without that dependency, your effect callback will only be triggered once. With it, it will be triggered each time clickedYear changes.
Side note: If you like, you can use destructuring in the filter callback:
setActiveYear(
response.data.timelineInfo.filter(
({year}) => year === clickedYear
// −−−−−−^^^^^^−−−−−^^^^
)
)

Different Pipe Based on Filter RxJS

I have the following problem. I want to subscribe to an observable using the async pipe in Angular. But the data in the pipe can originate from different data sources and the pipe can look different depending on the data source.
The way it is implemented now does not work, because I override the first value of this.data$ and thus source A never gets subscribed to.
Is it possible to 'split' the pipe depending on a filter?
problem.component.ts:
// Data source A and logic A
this.data$ = this.service.context.pipe(
filter(context => context.flag === true),
switchMap(context => this.dataSourceA(context.id))
map(data => this.processDataA(data))
);
// Data source B and logic B
this.data$ = this.service.context.pipe(
filter(context => context.flag === false),
switchMap(context => this.dataSourceB(context.id))
map(data => this.processDataB(data))
);
problem.component.html
<pro-table [data]="data$ | async"></pro-table>

Move the logic to your switchMap()
this.data$ = this.service.context.pipe(
switchMap(context => context.flag
? this.dataSourceA(context.id).pipe(map(d => this.processDataA(d)))
: this.dataSourceB(context.id).pipe(map(d => this.processDataB(d)))
)
);

That's just wrong way to achieve the result you desire. You have to take over to something called "higher order observable": Observable<Observable<T>>. Like a "higher order function" (that able to produce another function based on it's arguments), this guy is able to produce another Observable based on whatever parametrization you rely on.
this.service.context.pipe(
switchMap(context => context.flag ? this.dataSourceA(context.id).pipe(...) : this.dataSourceB(context.id).pipe(...))
);
...it might seem a littble bit esoteric at the beginging, but actually it isn't. \
P.S. Important point to remember: higher order observables let you parametrize your streams if a fancy way: by dealing with streams of streams.

You can add a Subject which will catch many data sources and subscribe to that:
this.data$ = new Subject();
observer = {
next(data) { this.data$.next(data) },
error(msg) { /*handle error*/ }
};
// Data source A and logic A
this.service.context.pipe(
filter(context => context.flag === true),
switchMap(context => this.dataSourceA(context.id))
map(data => this.processDataA(data))
).subscribe(observer);
this.service.context.pipe(
filter(context => context.flag === false),
switchMap(context => this.dataSourceB(context.id))
map(data => this.processDataB(data))
).subscribe(observer);
In this situation though it seems like you can use the flag to keep it in one chunk:
this.service.context.pipe(
switchMap(context => context.flag ?
this.dataSourceA(context.id) : this.dataSourceB(context.id)
)
map(data => this.processData(data)) // handle both cases
).subscribe(observer);

How can I access original observable value in tap after map?

I am building an Angular 6 guard. After the boolean is calculated by the map operator, I would like the original value of user available in the tap operator to calculate the redirect.
return this.auth.user$.pipe(
take(1),
// Existing Users cannot register
map(user => user ? false : true),
tap(canAccess => {
if (!canAccess) {
// Calculate route based on user value
}
})
);
I am very new to RxJS. Is there a way to access the original user value in the tap operator without creating a duplicate user observable?
(I guess I could technically tap before the map, but it seems odd to redirect before transforming the value.)

You could use map, tap and then map again. The last map allows you to end up with an Observable<boolean>, which I suppose you require in the end.
map(user => [user, !user]) maps the original value to an array of two values: the user and their access flag (boolean)
tap([user, canAccess]) => { ... } takes both values and does the computations you need on the user depending on the flag
map(([user, canAccess]) => canAccess) takes both values and return the one you need in the end (the boolean flag)
This way you don't have to duplicate your canAccess condition.
return this.auth.user$.pipe(
take(1),
map(user => [user, !user]), // !user should be equivalent to user ? false : true
tap(([user, canAccess]) => {
if (!canAccess) {
// Calculate route based on user value
}
}),
map(([user, canAccess]) => canAccess)
);

Doing what you're doing right now will transform the user to a boolean. So in that next tap, you'll eventually get a boolean.
Why don't you use the user directly in the tap and get rid of that map?
If user is defined, if(!user) will resolve to false anyway.
And if user is undefined or null, then if(!user) will resolve to true.
And yeah, as you said, since you need to return an Observable<boolean>, you'll have to map it to a boolean
So you can use this:
return this.auth.user$.pipe(
take(1),
tap(user => {
if (!user) {
// Calculate route based on user value
}
}),
map(user => user ? false : true)
);