I want to achieve the following behavior in RxJS but could not find a way using the available operators:
Stream A: Generated by a continuous stream of events (e.g. browser scroll)
Stream B: Generated by another arbitrary event (e.g. some kind of user input)
When B emits a value, I want to pause the processing of A, until a specified amount of time has passed. All values emitted by A in this timeframe are thrown away.
When B emits another value during this interval, the interval is reset.
After the interval has passed, the emitted values of A are no longer filtered.
// Example usage.
streamA$
.pipe(
unknownOperator(streamB$, 800),
tap(val => doSomething(val))
)
// Output: E.g. [event1, event2, <skips processing because streamB$ emitted>, event10, ...]
// Operator API.
const unknownOperator = (pauseProcessingWhenEmits: Observable<any>, pauseIntervalInMs: number) => ...
I thought that throttle could be used for this use case, however it will not let any emission through, until B has emitted for the first time (which might be never!).
streamA$
.pipe(
// If B does not emit, this never lets any emission of A pass through!
throttle(() => streamB$.pipe(delay(800)), {leading: false}),
tap(val => doSomething(val))
)
An easy hack would be to e.g. subscribe manually to B, store the timestamp when a value was emitted in the Angular component and then filter until the specified time has passed:
(obviously goes against the side-effect avoidance of a reactive framework)
streamB$
.pipe(
tap(() => this.timestamp = Date.now())
).subscribe()
streamA$
.pipe(
filter(() => Date.now() - this.timestamp > 800),
tap(val => doSomething(val))
)
I wanted to check with the experts here if somebody knows an operator (combination) that does this without introducing side-effects, before I build my own custom operator :)
I think this would be an approach:
bModified$ = b$.pipe(
switchMap(
() => of(null).pipe(
delay(ms),
switchMapTo(subject),
ignoreElements(),
startWith(null).
)
)
)
a$.pipe(
multicast(
new Subject(),
subject => merge(
subject.pipe(
takeUntil(bModified$)
),
NEVER,
)
),
refCount(),
)
It may seem that this is not a problem whose solution would necessarily involve multicasting, but in the above approach I used a sort of local multicasting.
It's not that expected multicasting behavior because if you subscribe to a$ multiple times(let's say N times), the source will be reached N times, so the multicasting does not occur at that level.
So, let's examine each relevant part:
multicast(
new Subject(),
subject => merge(
subject.pipe(
takeUntil(bModified$)
),
NEVER,
)
),
The first argument will indicate the type of Subject to be used in order to achieve that local multicasting. The second argument is a function, more accurately called a selector. Its single argument is the argument specified before(the Subject instance). This selector function will be called every time a$ is being subscribed to.
As we can see from the source code:
selector(subject).subscribe(subscriber).add(source.subscribe(subject));
the source is subscribed, with source.subscribe(subject). What's achieved through selector(subject).subscribe(subscriber) is a new subscriber that will be part of the Subject's observers list(it's always the same Subject instance), because merge internally subscribes to the provided observables.
We used merge(..., NEVER) because, if the subscriber that subscribed to the selector completes, then, next time the a$ stream becomes active again, the source would have to be resubscribed. By appending NEVER, the observable resulted form calling select(subject) will never complete, because, in order for merge to complete, all of its observables have to complete.
subscribe(subscriber).add(source.subscribe(subject)) creates a connection between subscribed and the Subject, such that when subscriber completes, the Subject instance will have its unsubscribe method called.
So, let's assume we have subscribed to a$: a$.pipe(...).subscribe(mySubscriber). The Subject instance in use will have one subscriber and if a$ emits something, mySubscriber will receive it(through the subject).
Now let's cover the case when bModified$ emits
bModified$ = b$.pipe(
switchMap(
() => of(null).pipe(
delay(ms),
switchMapTo(subject),
ignoreElements(),
startWith(null).
)
)
)
First of all, we're using switchMap because one requirement is that when b$ emits, the timer should reset. But, the way I see this problem, 2 things have to happen when b$ emits:
start a timer (1)
pause a$'s emissions (2)
(1) is achieved by using takeUntil in the Subject's subscribers. By using startWith, b$ will emit right away, so that a$'s emissions are ignored. In the switchMap's inner observable we're using delay(ms) to specify how long the timer should take. After it elapses, with the help of switchMapTo(subject), the Subject will now get a new subscriber, meaning that a$'s emissions will be received by mySubscriber(without having to resubscribe to the source). Lastly, ignoreElements is used because otherwise when a$ emits, it would mean that b$ also emit, which will cause a$ to be stopped again. What comes after switchMapTo(subject) are a$'s notifications.
Basically, we're able to achieve the pausable behavior this way: when the Subject instance as one subscriber(it will have at most one in this solution), it is not paused. When it has none, it means it is paused.
EDIT: alternatively, you could have a look at the pause operator from rxjs-etc.
Related
I have an observable (onAuthStateChanged) from the Firebase client that:
emits null immediately if the user is not signed in, and
emits null and then a user object a few moments later if the user is signed in.
const observable = new Observable((obs) => {
return app.auth().onAuthStateChanged(
obs.next,
obs.error,
obs.complete
)
})
What I want is to:
ignore any emitted null values for the first 1000ms of the app lifecycle (null coming after 1000ms is accepted)
always emit user object regardless of what time it comes
if no user object comes in the first 1000ms, then emit null at the 1000ms mark
Here is what I've done (and it seems to work). However, I'm reluctant to use this code as it doesn't seem that concise:
const o1 = observable.pipe(skipUntil(timer(1000)))
const o2 = observable.pipe(
takeUntil(o1),
filter((user) => user !== null)
)
const o3 = timer(1000).pipe(takeUntil(o2), mapTo(null))
merge(o1, o2, o3).subscribe({
next: setUser,
error: console.log,
complete: () => console.log("error: obs completed, this shouldn't happen"),
})
Is there a way to do this without merge? I tried going through the docs but I'm quite lost.
Thanks for your help!
You could use concat instead of merge. Think of it as using the first source until it completes, then use the second source.
const nonNullUser = firebaseUser.pipe(
filter(user => user !== null),
takeUntil(timer(1000))
);
const user = concat(nonNullUser, firebaseUser);
user.subscribe(...);
I just realized that this solution will not explicitly perform step #3 "emit null at the 1000ms mark". I was thinking subscribing to firebaseUser would emit the latest value. But, I'm not sure if that's true for your scenario.
If not, we could easily achieve this by adding shareReplay like this:
const firebaseUser = observable.pipe(shareReplay(1));
While I liked the answer from #BizzyBob I was genuinely intrigued by these requirements that I wanted to see what other options were available. Here's what I produced:
const auth$ = observable.pipe(
startWith(null)
)
const null$ = timer(1000).pipe(
switchMap(_=>auth$)
)
const valid$ = auth$.pipe(
filter(user=>!!user)
)
const user$ = race(null$, valid$);
We have our source auth$ observable which gets your Firebase data. However, startWith() will immediately emit null before any values coming from Firebase.
I declared two observables for null and non-null cases, null$ and valid$.
The null$ observable will subscribe to auth$ after 1000ms. When this happens it immediately emits null thanks to the startWith() operator.
The valid$ observable subscribes to auth$ immediately but only emits valid user data thanks to filter(). It won't emit startWith(null) because it is caught by the filter.
Last, we declare user$ by using the race() operator. This operator accepts a list of observables as its parameters. The first observable to emit a value wins and is the resulting subscription.
So in our race, valid$ has 1000ms to emit a valid user. If it doesn't, race() will subscribe to null$ resulting in the immediate null, and all future values coming from Firebase.
I am struggling to understand the difference between the flatMap and concatMap in rxJs.
The most clear answer that I could understand was that here difference-between-concatmap-and-flatmap
So I went and tried things out by my self.
import "./styles.css";
import { switchMap, flatMap, concatMap } from "rxjs/operators";
import { fromFetch } from "rxjs/fetch";
import { Observable } from "rxjs";
function createObs1() {
return new Observable<number>((subscriber) => {
setTimeout(() => {
subscriber.next(1);
subscriber.complete();
}, 900);
});
}
function createObs2() {
return new Observable<number>((subscriber) => {
setTimeout(() => {
subscriber.next(2);
//subscriber.next(22);
//subscriber.next(222);
subscriber.complete();
}, 800);
});
}
function createObs3() {
return new Observable<number>((subscriber) => {
setTimeout(() => {
subscriber.next(3);
//subscriber.next(33);
//subscriber.next(333);
subscriber.complete();
}, 700);
});
}
function createObs4() {
return new Observable<number>((subscriber) => {
setTimeout(() => {
subscriber.next(4);
subscriber.complete();
}, 600);
});
}
function createObs5() {
return new Observable<number>((subscriber) => {
setTimeout(() => {
subscriber.next(5);
subscriber.complete();
}, 500);
});
}
createObs1()
.pipe(
flatMap((resp) => {
console.log(resp);
return createObs2();
}),
flatMap((resp) => {
console.log(resp);
return createObs3();
}),
flatMap((resp) => {
console.log(resp);
return createObs4();
}),
flatMap((resp) => {
console.log(resp);
return createObs5();
})
)
.subscribe((resp) => console.log(resp));
console.log("hellooo");
I have used that playground here playground example
Questions
1)
From my understanding the use of flatMap should mix the outputs so that the console logs are like (1,3,2,4,5). I have tried more than 30 times and always come on the same row (1, 2, 3, 4, 5)
What am I doing wrong or have undestood wrong?
2)
If on createObs2() and createObs3() you remove the comments and include the code with multiple emitted events then things get messy. Even if you change to concatMap it messes things and results come mixed. Multiple numbers that I expect only once come multiple times. The result can be (1, 2, 33, 3, 2, 22, 3, 33, 4, 5, 4, 3, 4, 5) Why this happens?
How I test the example on playground. I just remove only 1 letter from the last console.log("hello"). Only one change for example console.log("heloo") and is then observed and project is compiled again and output printed in console.
Edit: The reason I have gone to flatMap and concatMap was to find a replacement for nested subscriptions in angular using the http library.
createObs1().subscribe( (resp1) => {
console.log(resp1);
createObs2().subscribe( (resp2) => {
console.log(resp2);
createObs3().subscribe( (resp3) => {
console.log(resp3);
createObs4().subscribe( (resp4) => {
console.log(resp4);
createObs5().subscribe( (resp5) => {
console.log(resp5);
})
})
})
})
})
Your test scenario is not really sufficient to see the differences between these two operators. In your test case, each observable only emits 1 time. If an observable only emits a single value, there is really no different between concatMap and flatMap (aka mergeMap). The differences can only be seen when there are multiple emissions.
So, let's use a different scenario. Let's have a source$ observable that simply emits an incrementing integer every 1 second. Then, within our "Higher Order Mapping Operator" (concatMap & mergeMap), we will return an observable that emits a variable number of times every 1 second, then completes.
// emit number every second
const source$ = interval(1000).pipe(map(n => n+1));
// helper to return observable that emits the provided number of times
function inner$(max: number, description: string): Observable<string> {
return interval(1000).pipe(
map(n => `[${description}: inner source ${max}] ${n+1}/${max}`),
take(max),
);
}
Then let's define two separate observables based on the source$ and the inner$; one using concatMap and one using flatMap and observe the output.
const flatMap$ = source$.pipe(
flatMap(n => inner$(n, 'flatMap$'))
);
const concatMap$ = source$.pipe(
concatMap(n => inner$(n, 'concatMap$'))
);
Before looking the differences in the output, let's talk about what these operators have in common. They both:
subscribe to the observable returned by the passed in function
emit emissions from this "inner observable"
unsubscribe from the inner observable(s)
What's different, is how they create and manage inner subscriptions:
concatMap - only allows a single inner subscription at a time. As it receives emissions, it will only subscribe to one inner observable at a time. So it will initially subscribe to the observable created by "emission 1", and only after it completes, will it subscribe to the observable created by "emission 2". This is consistent with how the concat static method behaves.
flatMap (aka mergeMap) - allows many inner subscriptions. So, it will subscribe to the inner observables as new emissions are received. This means that emissions will not be in any particular order as it will emit whenever any of its inner observables emit. This is consistent with how the merge static method behaves (which is why I personally prefer the name "mergeMap").
Here's a StackBlitz that shows the output for the above observables concatMap$ and mergeMap$:
Hopefully, the above explanation helps to clear up your questions!
#1 - "use of flatMap should mix the outputs"
The reason this wasn't working as you expected was because only one emission was going through the flatMap, which means you only ever had a single "inner observable" emitting values. As demonstrated in the above example, once flatMap receives multiple emissions, it can have multiple inner observables that emit independently.
#2 - "...and include the code with multiple emitted events then things get messy."
The "things get messy" is due to having multiple inner subscription that emit values.
For the part you mention about using concatMap and still getting "mixed" output, I would not expect that. I have seen weird behavior in StackBlitz with observable emissions when "auto save" is enabled (seems like sometimes it doesn't completely refresh and old subscriptions seem to survive the auto refresh, which gives very messy console output). Maybe code sandbox has a similar problem.
#3 - "The reason I have gone to flatMap and concatMap was to find a replacement for nested subscriptions in angular using the http library"
This makes sense. You don't want to mess around with nested subscriptions, because there isn't a great way to guarantee the inner subscriptions will be cleaned up.
In most cases with http calls, I find that switchMap is the ideal choice because it will drop emissions from inner observables you no longer care about. Imagine you have a component that reads an id from a route param. It uses this id to make an http call to fetch data.
itemId$ = this.activeRoute.params.pipe(
map(params => params['id']),
distinctUntilChanged()
);
item$ = this.itemId$.pipe(
switchMap(id => http.get(`${serverUrl}/items/${id}`)),
map(response => response.data)
);
We want item$ to emit only the "current item" (corresponds to the id in the url). Say our UI has a button the user can click to navigate to the next item by id and your app finds itself with a click-happy user who keeps smashing that button, which changes the url param even faster than the http call can return the data.
If we chose mergeMap, we would end up with many inner observables that would emit the results of all of those http calls. At best, the screen will flicker as all those different calls come back. At worst (if the calls came back out of order) the UI would be left displaying data that isn't in sync with the id in the url :-(
If we chose concatMap, the user would be forced to wait for all the http calls to be completed in series, even though we only care about that most recent one.
But, with switchMap, whenever a new emission (itemId) is received, it will unsubscribe from the previous inner observable and subscribe to the new one. This means it will not ever emit the results from the old http calls that are no longer relevant. :-)
One thing to note is that since http observables only emit once, the choice between the various operators (switchMap, mergeMap, concatMap) may not seem to make a difference, since they all perform the "inner observable handling" for us. However, it's best to future-proof your code and choose the one that truly gives you the behavior you would want, should you start receiving more than a single emission.
Every time the first observable emits, a second observable is created in the flatMap and starts emitting. However, the value from the first observable is not passed along any further.
Every time that second observable emits, the next flatMap creates a third observable, and so on. Again, the original value coming into the flatMap is not passed along any further.
createObs1()
.pipe(
flatMap(() => createObs2()), // Merge this stream every time prev observable emits
flatMap(() => createObs3()), // Merge this stream every time prev observable emits
flatMap(() => createObs4()), // Merge this stream every time prev observable emits
flatMap(() => createObs5()), // Merge this stream every time prev observable emits
)
.subscribe((resp) => console.log(resp));
// OUTPUT:
// 5
So, it's only the values emitted from createObs5() that actually get emitted to the observer. The values emitted from the previous observables have just been triggering the creation of new observables.
If you were to use merge, then you would get what you may have been expecting:
createObs1()
.pipe(
merge(createObs2()),
merge(createObs3()),
merge(createObs4()),
merge(createObs5()),
)
.subscribe((resp) => console.log(resp));
// OUTPUT:
// 5
// 4
// 3
// 2
// 1
I am working through a Udemy course on RxJs 6 and need to ask this as it was not crystal clear to me.
Note: This is a type ahead tutorial I am currently in at the moment. So on the keyup event this method is firing off.
ngAfterViewInit() {
const searchLessons$ = fromEvent<any>(this.input.nativeElement, 'keyup')
.pipe(
map(event => event.target.value),
debounceTime(400),
distinctUntilChanged(),
// switchMap cancels prior calls.
switchMap(search => this.loadLessons(search))
);
const initialLessons$ = this.loadLessons();
this.lessons$ = concat(initialLessons$, searchLessons$);
}
Does the code mean,
for all events that fire the code will collect responses from completed calls to the loadLessons
the value of the event is referenced as search
then the => will trigger a call to the loadLessons(search)
Continue of 3: If the value of the event were lets just say an array of values, would that mean that for the => call, a separate call to the loadLessons(search) would be made passing for each individual array value
Continue of 3: or would it just pass in the entire array?
Here is line per line explanation:
ngAfterViewInit() {
const searchLessons$ = fromEvent<any>(this.input.nativeElement, 'keyup') // whenever keyup is triggered on this.input
.pipe(
map(event => event.target.value), // we extract input value from event target
debounceTime(400), // we wait for last event in 400ms span
distinctUntilChanged(), // we check that the input value did change
switchMap(search => this.loadLessons(search)) // and with that input value changed we call this.LoadLessons and then wait for its return
);
const initialLessons$ = this.loadLessons(); // this will call initial loadLeason
this.lessons$ = concat(initialLessons$, searchLessons$); // this will connect return of initial call and changes triggered by key up this is not secure for race conditions
}
Ad1. all key up events on input
Ad2. the value of input is referenced as search
Ad3. yes it would just push array as argument
Without seeing the loadLessons func, i can only assume. i will also assume you are using the concat Rxjs method.
So basically what the code does, get the "initial load of lessons" , and subscribe to it on the concatMethod, after that call completes, it goes to subscribe to the second observable searchLessons.
The searchLessons will be called again every input search, and add the new values to the lessons subscription, on the search observable .
If the params given to the loadSeassion is an array, it will depend how that method (loadSessions) works. not with rxjs although it can be done in this case i cant really tell you :)
Warning: RxJS newb here.
Here is my challenge:
When an onUnlink$ observable emits...
Immediately start capturing values from an onAdd$ observable, for a maximum of 1 second (I'll call this partition onAddBuffer$).
Query a database (creating a doc$ observable) to fetch a model we'll use to match against one of the onAdd$ values
If one of the values from the onAddBuffer$ observable matches the doc$ value, do not emit
If none of the values from the onAddBuffer$ observable matches the doc$ value, or if the onAddBuffer$ observable never emits, emit the doc$ value
This was my best guess:
// for starters, concatMap doesn't seem right -- I want a whole new stream
const docsToRemove$ = onUnlink$.concatMap( unlinkValue => {
const doc$ = Rx.Observable.fromPromise( db.File.findOne({ unlinkValue }) )
const onAddBuffer$ = onAdd$
.buffer( doc$ ) // capture events while fetching from db -- not sure about this
.takeUntil( Rx.Observable.timer(1000) );
// if there is a match, emit nothing. otherwise wait 1 second and emit doc
return doc$.switchMap( doc =>
Rx.Observable.race(
onAddBuffer$.single( added => doc.attr === added.attr ).mapTo( Rx.Observable.empty() ),
Rx.Observable.timer( 1000 ).mapTo( doc )
)
);
});
docsToRemove$.subscribe( doc => {
// should only ever be invoked (with doc -- the doc$ value) 1 second
// after `onUnlink$` emits, when there are no matching `onAdd$`
// values within that 1 second window.
})
This always emits EmptyObservable. Maybe it's because single appears to emit undefined when there is no match, and I'm expecting it not to emit at all when there is no match? The same thing happens with find.
If I change single to filter, nothing ever emits.
FYI: This is a rename scenario with file system events -- if an add event follows within 1 second of an unlink event and the emitted file hashes match, do nothing because it's a rename. Otherwise it's a true unlink and it should emit the database doc to be removed.
This is my guess how you could do this:
onUnlink$.concatMap(unlinkValue => {
const doc$ = Rx.Observable.fromPromise(db.File.findOne({ unlinkValue })).share();
const bufferDuration$ = Rx.Observable.race(Rx.Observable.timer(1000), doc$);
const onAddBuffer$ = onAdd$.buffer(bufferDuration$);
return Observable.forkJoin(onAddBuffer$, doc$)
.map(([buffer, docResponse]) => { /* whatever logic you need here */ });
});
The single() operator is a little tricky because it emits the item that matches the predicate function only after the source Observable completes (or emits an error when there're two items or no matching items).
The race() is tricky as well. If one of the source Observables completes and doesn't emit any value race() will just complete and not emit anything. I reported this some time ago and this is the correct behavior, see https://github.com/ReactiveX/rxjs/issues/2641.
I guess this is what went wrong in your code.
Also note that .mapTo(Rx.Observable.empty()) will map each value into an instance of Observable. If you wanted to ignore all values you can use filter(() => false) or the ignoreElements() operator.
I'm trying to use an rxjs observable to delegate, but share, a piece of expensive work across the lifetime of an application.
Essentially, something like:
var work$ = Observable.create((o) => {
const expensive = doSomethingExpensive();
o.next(expensive);
observer.complete();
})
.publishReplay(1)
.refCount();
Now, this works fine and does exactly what I want, except for one thing: if all subscribers unsubscribe, then when the next one subscribes, my expensive work happens again. I want to keep it.
now, I could use a subject, or I could remove the refCount() and use connect manually (and never disconnect). But that would make the expensive work happen the moment I connect, not the first time a subscriber tries to consume work$.
Essentially, I want something akin to refCount that only looks at the first subscription to connect, and never disconnect. A "lazy connect".
Is such a thing possible at all?
How does publishReplay() actually work
It internally creates a ReplaySubject and makes it multicast compatible. The minimal replay value of ReplaySubject is 1 emission. This results in the following:
First subscription will trigger the publishReplay(1) to internally subscribe to the source stream and pipe all emissions through the ReplaySubject, effectively caching the last n(=1) emissions
If a second subscription is started while the source is still active the multicast() will connect us to the same replaySubject and we will receive all next emissions until the source stream completes.
If a subscription is started after the source is already completed the replaySubject has cached the last n emissions and it will only receive those before completing.
const source = Rx.Observable.from([1,2])
.mergeMap(i => Rx.Observable.of('emission:'+i).delay(i * 100))
.do(null,null,() => console.log('source stream completed'))
.publishReplay(1)
.refCount();
// two subscriptions which are both in time before the stream completes
source.subscribe(val => console.log(`sub1:${val}`), null, () => console.log('sub1 completed'));
source.subscribe(val => console.log(`sub2:${val}`), null, () => console.log('sub2 completed'));
// new subscription after the stream has completed already
setTimeout(() => {
source.subscribe(val => console.log(`sub_late-to-the-party:${val}`), null, () => console.log('sub_late-to-the-party completed'));
}, 500);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.0.3/Rx.js"></script>