Let's say I have a stream of actions. They're either Prompts, Responses (to prompts) or Effects. They come at irregular intervals, but assume 1 second delay between each one.
On every PROMPT action I want to emit that action and a BEGIN action (let's say we want to show the message to user for N seconds). All other items should be delayed by N seconds, after which the END action fires (hiding the message) and everything continues.
This is my code for it (for https://rxviz.com/):
const { interval, from, zip, timer } = Rx;
const { concatMap, delayWhen } = RxOperators;
const PROMPT = 'P';
const RESPONSE = 'R';
const EFFECT = 'E';
const BEGIN = '^';
const END = '&';
const convertAction = action => (action === PROMPT) ? [PROMPT, BEGIN, END] : [action];
// Just actions coming at regular intervals
const action$ = zip(
from([PROMPT, RESPONSE, EFFECT, PROMPT, RESPONSE, EFFECT, EFFECT, EFFECT]),
interval(1000),
(a, b) => a,
);
action$.pipe(
concatMap(action =>
from(convertAction(action)).pipe(
delayWhen(action => (action == END) ? timer(5000) : timer(0)),
),
),
);
What I really want to do is for first RESPONSE action after PROMPT to not be affected by the delay. If it comes before END action, it should be shown right away. So, instead of
P^ &REP^ &REEE
I want to receive
P^ R &EP^R &EEE
How can I achieve it while keeping each RESPONSE after their corresponding PROMPT? Assume no events can come between PROMPT and RESPONSE.
If I understand it right, this is a very interesting problem to address with Observables streams. This is the way I would attack it.
First I would store in a constant actionDelayed$ the result of your original logic, i.e. a stream where we have introduced, after each PROMPT, BEGIN and END actions divided by a delay.
const actionDelayed$ = action$.pipe(
concatMap(action =>
from(convertAction(action)).pipe(
delayWhen(action => (action == END) ? timer(5000) : timer(0)),
),
),
);
Then I would create 2 separate streams, response$ and promptDelayed$, containing only the RESPONSE actions before the delay was introduced and the PROMPT actions after the delayed was introduced, like this
const response$ = action$.pipe(
filter(a => a == RESPONSE)
)
const promptDelayed$ = actionDelayed$.pipe(
filter(a => a == PROMPT)
)
With these 2 streams, I can create another stream of RESPONSE actions emitted just after the PROMPT delayed actions are emitted, like this
const responseN1AfterPromptN$ = zip(response$, promptDelayed$).pipe(
map(([r, p]) => r)
)
At this point I have just to remove all RESPONSE actions from actionDelayed$ like this
const actionNoResponseDelayed$ = actionDelayed$.pipe(
filter(a => a != RESPONSE)
)
and merge actionNoResponseDelayed$ with responseN1AfterPromptN$ to get the final stream.
The entirety of the code, to be tried with rxviz is this
const { interval, from, zip, timer, merge } = Rx;
const { concatMap, delayWhen, share, filter, map } = RxOperators;
const PROMPT = 'P';
const RESPONSE = 'R';
const EFFECT = 'E';
const BEGIN = '^';
const END = '&';
const convertAction = action => (action === PROMPT) ? [PROMPT, BEGIN, END] : [action];
// Just actions coming at regular intervals
const action$ = zip(
from([PROMPT, RESPONSE, EFFECT, PROMPT, RESPONSE, EFFECT, EFFECT, EFFECT]),
interval(1000),
(a, b) => a,
).pipe(share());
const actionDelayed$ = action$.pipe(
concatMap(action =>
from(convertAction(action)).pipe(
delayWhen(action => (action == END) ? timer(5000) : timer(0)),
),
),
share()
);
const response$ = action$.pipe(
filter(a => a == RESPONSE)
)
const promptDelayed$ = actionDelayed$.pipe(
filter(a => a == PROMPT)
)
const responseN1AfterPromptN$ = zip(response$, promptDelayed$).pipe(
map(([r, p]) => r)
)
const actionNoResponseDelayed$ = actionDelayed$.pipe(
filter(a => a != RESPONSE)
)
merge(actionNoResponseDelayed$, responseN1AfterPromptN$)
The use of the share operator while creating action$ and actionDelayed$ streams allows to avoid repeated subscriptions to these streams when creating the subsequent streams used in the solution.
It may not work this way because you're using concatMap. As you know, it waits for the inner observable to complete before starting to process(to subscribe) the pending ones. It internally uses a buffer, such that if an inner observable is still active(did not complete), the emitted value will be added to that buffer. When the inner observable becomes inactive, the oldest value from the buffer is selected and a new inner observable will be created, based on the provided callback function.
There is also delayWhen, which emits a complete notification after all of its pending observables complete:
// called when an inner observable sends a `next`/`complete` notification
const notify = () => {
// Notify the consumer.
subscriber.next(value);
// Ensure our inner subscription is cleaned up
// as soon as possible. Once the first `next` fires,
// we have no more use for this subscription.
durationSubscriber?.unsubscribe();
if (!closed) {
active--;
closed = true;
checkComplete();
}
};
checkComplete() will check if there is a need to send a complete notification to the main stream:
const checkComplete = () => isComplete && !active && subscriber.complete();
We've seen that active decreases in notify(). isComplete becomes true when the main source completes:
// this is the `complete` callback
() => {
isComplete = true;
checkComplete();
}
So, this is why it does not work this way:
the PROMPT action is used to create the concatMap's first inner observable
the observable emits 3 consecutive actions [PROMPT, BEGIN, END]
the first 2 will get timer(0), whereas the third one, END, will get (timer(5000)); notice that in this time, before the PROMPT action got emitted, the isComplete variable is set to true, because from() completes synchronously in this case
so there is a timer(5000) that keeps the inner obs. active; then a RESPONSE is emitted from the actions$ stream, but since there is no place for it yet, it will be added to the buffer and an inner obs. will be created when timer(5000) finally expires
A way to solve this might be to replace concatMap with mergeMap.
Related
My function (lets call it myFunction) is getting an array of streams (myFunction(streams: Observable<number>[])). Each of those streams produces values from 1 to 100, which acts as a progress indicator. When it hits 100 it is done and completed. Now, when all of those observables are done I want to emit a value. I could do it this way:
public myFunction(streams: Observable<number>[]) {
forkJoin(streams).subscribe(_values => this.done$.emit());
}
This works fine, but imagine following case:
myFunction gets called with 2 streams
one of those streams is done, second one is still progressing
myFunction gets called (again) with 3 more streams (2nd one from previous call is still progressing)
I'd like to somehow add those new streams from 3rd bullet to the "queue", which would result in having 5 streams in forkJoin (1 completed, 4 progressing).
I've tried multiple approaches but can't get it working anyhow... My latest approach was this:
private currentProgressObs: Observable<any> | null = null;
private currentProgressSub: Subscription | null = null;
public myFunction(progressStreams: Observable<number>[]) {
const isUploading = this.cumulativeUploadProgressSub && !this.cumulativeUploadProgressSub.closed;
const currentConcatObs = this.currentProgressObs?.pipe(concatAll());
const currentStream = isUploading && this.currentProgressObs ? this.currentProgressObs : of([100]);
if (this.currentProgressSub) {
this.currentProgressSub.unsubscribe();
this.currentProgressSub = null;
}
this.currentProgressObs = forkJoin([currentStream, ...progressStreams]);
this.currentProgressSub = this.currentProgressObs.subscribe(
_lastProgresses => {
this._isUploading$.next(false); // <----- this is the event I want to emit when all progress is completed
this.currentProgressSub?.unsubscribe();
this.currentProgressSub = null;
this.currentProgressObs = null;
},
);
}
Above code only works for the first time. Second call to the myFunction will never emit the event.
I also tried other ways. I've tried recursion with one global stream array, in which I can add streams while the subscription is still avctive but... I failed. How can I achieve this? Which operator and in what oreder should I use? Why it will or won't work?
Here is my suggestion for your issue.
We will have two subjects, one to count the number of request being processed (requestsInProgress) and one more to mange the requests that are being processed (requestMerger)
So the thing that will do is whenever we want to add new request we will pass it to the requestMerger Subject.
Whenever we receive new request for processing in the requestMerger stream we will first increment the requestInProgress counter and after that we will merge the request itself in the source observable. While merging the new request/observable to the source we will also add the finalize operator in order to track when the request has been completed (reached 100), and when we hit the completion criteria we will decrement the request counter with the decrementCounter function.
In order to emit result e.g. to notify someone else in the app for the state of the pending requests we can subscribe to the requestsInProgress Subject.
You can test it out either here or in this stackBlitz
let {
interval,
Subject,
BehaviorSubject
} = rxjs
let {
mergeMap,
map,
takeWhile,
finalize,
first,
distinctUntilChanged
} = rxjs.operators
// Imagine next lines as a service
// Subject responsible for managing strems
let requestMerger = new Subject();
// Subject responsible for tracking streams in progress
let requestsInProgress = new BehaviorSubject(0);
function incrementCounter() {
requestsInProgress.pipe(first()).subscribe(x => {
requestsInProgress.next(x + 1);
});
}
function decrementCounter() {
requestsInProgress.pipe(first()).subscribe(x => {
requestsInProgress.next(x - 1);
});
}
// Adds request to the request being processed
function addRequest(req) {
// The take while is used to complete the request when we have `value === 100` , if you are dealing with http-request `takeWhile` might be redudant, because http request complete by themseves (e.g. the finalize method of the stream will be called even without the `takeWhile` which will decrement the requestInProgress counter)
requestMerger.next(req.pipe(takeWhile(x => x < 100)));
}
// By subscribing to this stream you can determine if all request are processed or if there are any still pending
requestsInProgress
.pipe(
map(x => (x === 0 ? "Loaded" : "Loading")),
distinctUntilChanged()
)
.subscribe(x => {
console.log(x);
document.getElementById("loadingState").innerHTML = x;
});
// This Subject is taking care to store or request that are in progress
requestMerger
.pipe(
mergeMap(x => {
// when new request is added (recieved from the requestMerger Subject) increment the requrest being processed counter
incrementCounter();
return x.pipe(
finalize(() => {
// when new request has been completed decrement the requrest being processed counter
decrementCounter();
})
);
})
)
.subscribe(x => {
console.log(x);
});
// End of fictional service
// Button that adds request to be processed
document.getElementById("add-stream").addEventListener("click", () => {
addRequest(interval(1000).pipe(map(x => x * 25)));
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.6.6/rxjs.umd.min.js"></script>
<div style="display:flex">
<button id="add-stream">Add stream</button>
<h5>Loading State: <span id="loadingState">false</span> </h5>
</div>
Your problem is that each time your call your function, you are creating a new observable. Your life would be much easier if all calls of your function pushed all upload jobs through the same stream.
You can achieve this using a Subject.
I would suggest you push single "Upload Jobs" though a simple subject and design an observable that emits the state of all upload jobs whenever anything changes: A simple class that offers a createJob() method to submit jobs, and a jobs$ observable to reference the state:
class UploadService {
private jobs = new Subject<UploadJob>();
public jobs$ = this.jobs.pipe(
mergeMap(job => this.processJob(job)),
scan((collection, job) => collection.set(job.id, job), new Map<string, UploadJob>()),
map(jobsMap => Array.from(jobsMap.values()))
);
constructor() {
this.jobs$.subscribe();
}
public createJob(id: string) {
this.jobs.next({ id, progress: 0 });
}
private processJob(job: UploadJob) {
// do work and return observable that
// emits updated status of UploadJob
}
}
Let's break it down:
jobs is a simple subject, that we can push "jobs" through
createJob simply calls jobs.next() to push the new job through the stream
jobs$ is where all the magic happens. It receives each UploadJob and uses:
mergeMap to execute whatever function actually does the work (I called it processJob() for this example) and emits its values into the stream
scan is used to accumulate these UploadJob emissions into a Map (for ease of inserting or updating)
map is used to convert the map into an array (Map<string, UploadJob> => UploadJob[])
this.jobs$.subscribe() is called in the constructor of the class so that jobs will be processed
Now, we can easily derive your isUploading and cumulativeProgress from this jobs$ observable like so:
public isUploading$ = this.jobs$.pipe(
map(jobs => jobs.some(j => j.progress !== 100)),
distinctUntilChanged()
);
public progress$ = this.jobs$.pipe(
map(jobs => {
const current = jobs.reduce((sum, j) => sum + j.progress, 0) / 100;
const total = jobs.length ?? current;
return current / total;
})
);
Here's a working StackBlitz demo.
I have an Angular code where in i am trying to subscribe to my 1st api and implementing a while loop inside this subscription. Further i need to subscribe to another api inside the while loop. Reason --> I need to subscribe to the inner api multiple times and the while loop should end based on a flag returned by inner api. I tried implementing the below but its not working. Need some help.
CallBanyanToFetchQuotes() {
const url1 = 'http://ws.integration.banyantechnology.com/services/api/rest/ImportForQuote';
this.http.post(url1, payload)
.subscribe(importForQuoteResponse => {
this.importForQuoteResponse = importForQuoteResponse;
console.log('LoadID = ' + this.importForQuoteResponse.Load.Loadinfo.LoadID);
this.loadId = this.importForQuoteResponse.Load.Loadinfo.LoadID;
while (!this.ratingCompleted) {
const url2 = 'http://ws.integration.banyantechnology.com/services/api/rest/GetQuotes';
this.http.post(url2, payload)
.subscribe(getQuoteResponse => {
this.getQuoteResponse = getQuoteResponse;
if (this.getQuoteResponse.RatingCompleted === true) {
this.ratingCompleted = true;
}
});
}
});
}
this.http.post(url1, payload).pipe(
switchMap(importForQuoteResponse=>{
this.importForQuoteResponse = importForQuoteResponse;
this.loadId = this.importForQuoteResponse.Load.Loadinfo.LoadID;
return timer(0,1000).pipe(
switchMap(()=>this.http.post(url2, payload)),
tap(res=>this.getQuoteResponse=res),
takeWhile(res=>!res.RatingCompleted,true),
filter(res=>res.RatingCompleted === true)
)
})).subscribe(()=>{
this.ratingCompleted = true;
})
a "fool example" in stackblitz
the before code can be explained like: we make the first post, but, we don't want this subscribtion, so we change this subscription to a timer (switchMap). But we don't want the timer, else a second post (another switchMap). Each time timer is executed, is executed the second post and we get the response using tap. We make the call while the response was false (takeWhile) -it's important make the takewhile(...,true), the "true" makes return the last value- and filter the response (filter) so only get the "subscribe" when the response is true.
NOTE: I use timer(0,1000) to make a call each 1000 miliseconds, feel free to change the interval
You can use expand to simulate a while loop. expand passes the input through to the destination immediately, maps to an Observable and receives its output as the next input. Map to EMPTY to end this recursion.
// move the urls out of the function if they are static
const url1 = 'http://ws.integration.banyantechnology.com/services/api/rest/ImportForQuote';
const url2 = 'http://ws.integration.banyantechnology.com/services/api/rest/GetQuotes';
callBanyanToFetchQuotes() {
this.http.post(url1, payload).pipe(
// process response from url1 http request
tap(importForQuoteResponse => {
this.importForQuoteResponse = importForQuoteResponse;
console.log('LoadID = ' + this.importForQuoteResponse.Load.Loadinfo.LoadID);
this.loadId = this.importForQuoteResponse.Load.Loadinfo.LoadID;
}),
// switch to url2 http request
switchMap(_ => this.http.post(url2, payload))
// execute url2 request again if the rating is incomplete or end execution with EMTPY
expand(quoteResponse => quoteResponse.RatingCompleted ? EMPTY : this.http.post(url2, payload))
// process responses from url2 requests
).subscribe(quoteResponse => {
this.getQuoteResponse = quoteResponse;
if (quoteResponse.RatingCompleted === true) {
this.ratingCompleted = true;
}
});
}
The expand approach guarantees that the next http call will be made directly and only after you received a response from the previous one.
I am trying to figure out, how mergeAll works and created examples:
const clicks = Rx.Observable.interval(4000).map(()=> "first");
const higherOrder = clicks.map((ev) => Rx.Observable.interval(1000).map(() => "inner").take(10));
const firstOrder = higherOrder.mergeAll();
firstOrder.subscribe(x => console.log(x));
the output it always inner and first never outputted. After calling mergeAll() the clicks observable is no more relevant?
On more example:
const input = document.getElementById("window");
const clicks = Rx.Observable.fromEvent(input, 'keyup').map(() => "Hello");
const interval = Rx.Observable.interval(4000);
const result = clicks.window(interval)
.map(win => {
return win.take(1);
})
.mergeAll(); // flatten the Observable-of-Observables
result.subscribe(x => console.log("Result " + x));
on subscribe, I've got the result from outer observable "Result Hello" not the inner observable. What kind of role plays mergeAll in this case?
Why the win variable is an instance observable not Hello?
After calling mergeAll() the clicks observable is no more relevant?
Correct. You map each individual click to a stream of "inner" events. mergeAll simply merges those streams together. The click event lives in this resulting stream only very faintly as the point in time where a specific merged stream starts. It becomes a bit more clear this way:
const clicks$ = Rx.Observable.interval(1000);
const higherOrder$ = clicks$.map(click => Rx.Observable.interval(500)
.map(counter => `${click}–${counter}`)
);
higherOrder$.mergeAll().subscribe(console.log);
The documentation and its marble diagram might also help you understand:
you should use switchMap.
firstOrder.switchMap(x => console.log(x)).subscribe( value => console.log(value));
or
result.switchMap(x => console.log("Result " + x)).subscribe( value => console.log(value));
switch expects a stream of Observables, when it get an Observable pushed onto it’s input stream it unsubscribes from any previous Observables and subscribes to the new one and then emits any values from that Observable onto it’s output stream.
I have a Rx.Observable.webSocket Subject. My server endpoint can not handle messages receiving the same time (<25ms). Now I need a way to stretch the next() calls of my websocket subject.
I have created another Subject requestSubject and subscribe to this.
Then calling next of the websocket inside the subscription.
requestSubject.delay(1000).subscribe((request) => {
console.log(`SENDING: ${JSON.stringify(request)}`);
socketServer.next(JSON.stringify(request));
});
Using delay shifts each next call the same delay time, then all next calls emit the same time later ... thats not what I want.
I tried delay, throttle, debounce but it does not fit.
The following should illustrate my problem
Stream 1 | ---1-------2-3-4-5---------6----
after some operation ...
Stream 2 | ---1-------2----3----4----5----6-
Had to tinker a bit, its not as easy as it looks:
//example source stream
const source = Rx.Observable.from([100,500,1500,1501,1502,1503])
.mergeMap(i => Rx.Observable.of(i).delay(i))
.share();
stretchEmissions(source, 1000)
.subscribe(val => console.log(val));
function stretchEmissions(source, spacingDelayMs) {
return source
.timestamp()
.scan((acc, curr) => {
// calculate delay needed to offset next emission
let delay = 0;
if (acc !== null) {
const timeDelta = curr.timestamp - acc.timestamp;
delay = timeDelta > spacingDelayMs ? 0 : (spacingDelayMs - timeDelta);
}
return {
timestamp: curr.timestamp,
delay: delay,
value: curr.value
};
}, null)
.mergeMap(i => Rx.Observable.of(i.value).delay(i.delay), undefined, 1);
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.4.2/Rx.js"></script>
Basically we need to calculate the needed delay between emissions so we can space them. We do this using timestamp() of original emissions and the mergeMap overload with a concurrency of 1 to only subscribe to the next delayed value when the previous is emitted. This is a pure Rx solution without further side effects.
Here are two solutions using a custom stream and using only rxjs-operators - since it looks quite complicated I would not advice you to use this solution, but to use a custom stream (see 1st example below):
Custom stream (MUCH easier to read and maintain, probably with better performance as well):
const click$ = Rx.Observable
.fromEvent(document.getElementById("btn"), "click")
.map((click, i) => i);
const spreadDelay = 1000;
let prevEmitTime = 0;
click$
.concatMap(i => { // in this case you could also use "flatMap" or "mergeMap" instead of "concatMap"
const now = Date.now();
if (now - prevEmitTime > spreadDelay) {
prevEmitTime = now;
return Rx.Observable.of(i); // emit immediately
} else {
prevEmitTime += spreadDelay;
return Rx.Observable.of(i).delay(prevEmitTime - now); // emit somewhere in the future
}
})
.subscribe((request) => {
console.log(`SENDING: ${request}`);
});
<script src="https://unpkg.com/rxjs/bundles/Rx.min.js"></script>
<button id="btn">Click me!</button>
Using only RxJS Operators (contains issues, probably shouldn't use):
const click$ = Rx.Observable
.fromEvent(document.getElementById("btn"), "click")
.map((click, i) => i);
click$
// window will create a new substream whenever no click happened for 1001ms (with the spread out
.window(click$
.concatMap(i => Rx.Observable.of(i).delay(1000))
.debounceTime(1001)
)
.mergeMap(win$ => Rx.Observable.merge(
win$.take(1).merge(), // emitting the "first" click immediately
win$.skip(1)
.merge()
.concatMap(i => Rx.Observable.of(i).delay(1000)) // each emission after the "first" one will be spread out to 1 seconds
))
.subscribe((request) => {
console.log(`SENDING: ${request}`);
});
<script src="https://unpkg.com/rxjs/bundles/Rx.min.js"></script>
<button id="btn">Click me!</button>
Mark van Straten's solution didn't work completely accurately for me. I found a much more simple and accurate solution based from here.
const source = from([100,500,1500,1501,1502,1503]).pipe(
mergeMap(i => of(i).pipe(delay(i)))
);
const delayMs = 500;
const stretchedSource = source.pipe(
concatMap(e => concat(of(e), EMPTY.pipe(delay(delayMs))))
);
We have this basic algorithm, which is written in imperative style:
const db = new DB()
const markAsCompleted = ( taskId ) => {
// Mark the task as completed
db.markTaskAsCompleted( taskId )
// Update parent status
const parentId = db.getTaskParentId( taskId )
if ( parentId ) {
// Get parent's incomplete children
const isIncomplete = ( task ) => task.status != 'completed'
const parentChildren = db.getTaskChildren( parentId )
const incompleteTasks = parentChildren.filter( isIncomplete )
// If all children have completed, mark the parent as completed
if ( incompleteTasks.length === 0 ) {
db.markTaskAsCompleted( taskId ) // Simple case - no recursion
// markAsCompleted( parentId ) // Complex case - with recursion
}
}
}
The db operations all involve side-effects (obviously).
How can this be converted to functional programming style? That is, a point-free style based on function composition, whilst using the IO monad and having the client 'pulling the trigger' on side-effects. As in this piece of code:
loudCat = argsIO.chain(R.traverse(IO.of, readFile))
.map(R.join('\n'))
.map(R.toUpper)
.chain(stdoutWrite);
loudCat.runIO();
Preferably, the solution will use either ramda-fantasy or folktale.
Also note the two cases:
simple with no recursion (uncommented)
complex with recursion (commented)
Usually db lookups are async and therefore the async adt is used instead of the io adt, but as you asked for the io...
Create a wrapper around your db that can return ios of your computations, then chain away:
// replace return values of these functions with your actual db lookups
const db = {
getTaskChildren: (taskId) => IO.of([]),
getTaskParentId: (taskId) => IO.of('123'),
markTaskAsCompleted: (taskId) => IO.of(1),
};
const checkParent = (parentId) =>
db.getTaskChildren(parentId)
.map((children) =>
children.filter(({ status }) => status !== 'completed'))
.chain((incomplete) =>
incomplete.length === 0
? db.markTaskAsCompleted(parentId)
: IO.of(null)); // do nothing
const markAsCompleted = (taskId) =>
db.markTaskAsCompleted(taskId)
.chain(() => db.getTaskParentId(taskId))
.chain(checkParent);
markAsCompleted('abc').run();
The flow shouldnt change if the io is swapped for async.