Develop Reference

Add streams dynamically to combined stream (eg forkJoin)

My function (lets call it myFunction) is getting an array of streams (myFunction(streams: Observable<number>[])). Each of those streams produces values from 1 to 100, which acts as a progress indicator. When it hits 100 it is done and completed. Now, when all of those observables are done I want to emit a value. I could do it this way:
public myFunction(streams: Observable<number>[]) {
forkJoin(streams).subscribe(_values => this.done$.emit());
}
This works fine, but imagine following case:
myFunction gets called with 2 streams
one of those streams is done, second one is still progressing
myFunction gets called (again) with 3 more streams (2nd one from previous call is still progressing)
I'd like to somehow add those new streams from 3rd bullet to the "queue", which would result in having 5 streams in forkJoin (1 completed, 4 progressing).
I've tried multiple approaches but can't get it working anyhow... My latest approach was this:
private currentProgressObs: Observable<any> | null = null;
private currentProgressSub: Subscription | null = null;
public myFunction(progressStreams: Observable<number>[]) {
const isUploading = this.cumulativeUploadProgressSub && !this.cumulativeUploadProgressSub.closed;
const currentConcatObs = this.currentProgressObs?.pipe(concatAll());
const currentStream = isUploading && this.currentProgressObs ? this.currentProgressObs : of([100]);
if (this.currentProgressSub) {
this.currentProgressSub.unsubscribe();
this.currentProgressSub = null;
}
this.currentProgressObs = forkJoin([currentStream, ...progressStreams]);
this.currentProgressSub = this.currentProgressObs.subscribe(
_lastProgresses => {
this._isUploading$.next(false); // <----- this is the event I want to emit when all progress is completed
this.currentProgressSub?.unsubscribe();
this.currentProgressSub = null;
this.currentProgressObs = null;
},
);
}
Above code only works for the first time. Second call to the myFunction will never emit the event.
I also tried other ways. I've tried recursion with one global stream array, in which I can add streams while the subscription is still avctive but... I failed. How can I achieve this? Which operator and in what oreder should I use? Why it will or won't work?

Here is my suggestion for your issue.
We will have two subjects, one to count the number of request being processed (requestsInProgress) and one more to mange the requests that are being processed (requestMerger)
So the thing that will do is whenever we want to add new request we will pass it to the requestMerger Subject.
Whenever we receive new request for processing in the requestMerger stream we will first increment the requestInProgress counter and after that we will merge the request itself in the source observable. While merging the new request/observable to the source we will also add the finalize operator in order to track when the request has been completed (reached 100), and when we hit the completion criteria we will decrement the request counter with the decrementCounter function.
In order to emit result e.g. to notify someone else in the app for the state of the pending requests we can subscribe to the requestsInProgress Subject.
You can test it out either here or in this stackBlitz
let {
interval,
Subject,
BehaviorSubject
} = rxjs
let {
mergeMap,
map,
takeWhile,
finalize,
first,
distinctUntilChanged
} = rxjs.operators
// Imagine next lines as a service
// Subject responsible for managing strems
let requestMerger = new Subject();
// Subject responsible for tracking streams in progress
let requestsInProgress = new BehaviorSubject(0);
function incrementCounter() {
requestsInProgress.pipe(first()).subscribe(x => {
requestsInProgress.next(x + 1);
});
}
function decrementCounter() {
requestsInProgress.pipe(first()).subscribe(x => {
requestsInProgress.next(x - 1);
});
}
// Adds request to the request being processed
function addRequest(req) {
// The take while is used to complete the request when we have `value === 100` , if you are dealing with http-request `takeWhile` might be redudant, because http request complete by themseves (e.g. the finalize method of the stream will be called even without the `takeWhile` which will decrement the requestInProgress counter)
requestMerger.next(req.pipe(takeWhile(x => x < 100)));
}
// By subscribing to this stream you can determine if all request are processed or if there are any still pending
requestsInProgress
.pipe(
map(x => (x === 0 ? "Loaded" : "Loading")),
distinctUntilChanged()
)
.subscribe(x => {
console.log(x);
document.getElementById("loadingState").innerHTML = x;
});
// This Subject is taking care to store or request that are in progress
requestMerger
.pipe(
mergeMap(x => {
// when new request is added (recieved from the requestMerger Subject) increment the requrest being processed counter
incrementCounter();
return x.pipe(
finalize(() => {
// when new request has been completed decrement the requrest being processed counter
decrementCounter();
})
);
})
)
.subscribe(x => {
console.log(x);
});
// End of fictional service
// Button that adds request to be processed
document.getElementById("add-stream").addEventListener("click", () => {
addRequest(interval(1000).pipe(map(x => x * 25)));
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.6.6/rxjs.umd.min.js"></script>
<div style="display:flex">
<button id="add-stream">Add stream</button>
<h5>Loading State: <span id="loadingState">false</span> </h5>
</div>

Your problem is that each time your call your function, you are creating a new observable. Your life would be much easier if all calls of your function pushed all upload jobs through the same stream.
You can achieve this using a Subject.
I would suggest you push single "Upload Jobs" though a simple subject and design an observable that emits the state of all upload jobs whenever anything changes: A simple class that offers a createJob() method to submit jobs, and a jobs$ observable to reference the state:
class UploadService {
private jobs = new Subject<UploadJob>();
public jobs$ = this.jobs.pipe(
mergeMap(job => this.processJob(job)),
scan((collection, job) => collection.set(job.id, job), new Map<string, UploadJob>()),
map(jobsMap => Array.from(jobsMap.values()))
);
constructor() {
this.jobs$.subscribe();
}
public createJob(id: string) {
this.jobs.next({ id, progress: 0 });
}
private processJob(job: UploadJob) {
// do work and return observable that
// emits updated status of UploadJob
}
}
Let's break it down:
jobs is a simple subject, that we can push "jobs" through
createJob simply calls jobs.next() to push the new job through the stream
jobs$ is where all the magic happens. It receives each UploadJob and uses:
mergeMap to execute whatever function actually does the work (I called it processJob() for this example) and emits its values into the stream
scan is used to accumulate these UploadJob emissions into a Map (for ease of inserting or updating)
map is used to convert the map into an array (Map<string, UploadJob> => UploadJob[])
this.jobs$.subscribe() is called in the constructor of the class so that jobs will be processed
Now, we can easily derive your isUploading and cumulativeProgress from this jobs$ observable like so:
public isUploading$ = this.jobs$.pipe(
map(jobs => jobs.some(j => j.progress !== 100)),
distinctUntilChanged()
);
public progress$ = this.jobs$.pipe(
map(jobs => {
const current = jobs.reduce((sum, j) => sum + j.progress, 0) / 100;
const total = jobs.length ?? current;
return current / total;
})
);
Here's a working StackBlitz demo.

Reactive Programming: How to subscribe and sample an event emitter?

I'm suscribing to an event emitter in a React Native application that is consuming react-native-ble-manager.
handleUpdateValueForCharacteristic(data) {
console.log('Received data from ' + data.peripheral + ' characteristic ' + data.characteristic, data.value);
}
bleManagerEmitter.addListener('BleManagerDidUpdateValueForCharacteristic', this.handleUpdateValueForCharacteristic );
I'm dealing with a Bluetooth event Stream which frequency is either 50, 100 or 200 events per second (Hz).
I'm interrested in all events at 50 Hz, half of them at 100 Hz and a quarter of them at 200 Hz.
What is the correct way to subscribe to this event Stream with RxJS and which operator should I use to sample the data?
I may be wrong but I can't seem to find a helper method to create an observable from an event emitter.

fromEventPattern should be what you're looking for.
It lets you create an observable based on custom event emissions (like what you've got with this BLE manager).
I've provided a snippet below outlining how you might use it.
Notice the scan() and filter() combination. By using the former operator to keep track of the nth event, it effectively changes the rate at which events are sampled and thus handled by any subscribers.
In your scenario, you'll want the scan() to also keep track of the event emitted, so you can eventually map() it after the filter() call, so subscribers will receive it. The key point here is to keep track of event state within the scan() (i.e. tick and event data properties, t and data in the snippet respectively) as you accumulate events.
const { fromEventPattern } = rxjs;
const { filter, map, scan } = rxjs.operators;
// Tweak parameters to vary demo
const hz = 200;
const sample = 4;
function addEmitterHandler(handler) {
// bleManagerEmitter.addListener('event', handler)
const intervalId = setInterval(() => {
handler({ timestamp: Date.now() });
}, 1000 / hz);
return intervalId;
}
function removeEmitterHandler(handler, intervalId) {
// bleManagerEmitter.removeListener(...)
clearInterval(intervalId);
}
// Simulate emissions using the `setInterval()` call
const emitter = fromEventPattern(
addEmitterHandler,
removeEmitterHandler
);
emitter.pipe(
// Use `scan()` and `filter()` combination to adjust sampling
scan((state, data) => {
const t = (state.t % (sample + 1)) + 1;
return { t, data };
}, { t: 0, data: null }),
filter(state => state.t % sample === 0),
// Use `map()` to forward event data only
map(state => state.data),
).subscribe(data => console.log(data));
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.4.0/rxjs.umd.min.js"></script>

Implementing Stream in JavaScript

I want to implement a stream object that can do this:
// a -------1------2----3
// map -----\------\----\
// b --------2------4----6
const a = new Stream();
const b = a.map(value => value * 2);
b.subscribe(console.log);
a.push(1);
// 2
a.push(2);
// 4
a.push(3);
// 6
The idea here is that the object b can subscribe new callbacks to stream a. The map function should listen when push is called and apply the mapped out function as well as the originally subscribed one. This is the implementation I have so far:
class Stream {
constructor(queue = []) {
this.queue = queue;
}
subscribe(action) {
if (typeof action === 'function') {
this.queue.push(action);
}
}
map(callback) {
this.queue = this.queue.map(
actionFn => arg => action(callback(arg))
);
return this;
}
push(value) {
this.queue.forEach(actionFn => {
actionFn.call(this, value);
});
}
}
The problem with current implementation is that originally the queue in class Stream is empty so it doesn't go through it. Would appreciate any suggestions or help. I would like to not use any library for this.

Your map needs to create a new Transform stream and return it. Instead of the subscribe you could simply use the standard on('data') event or, better use the read method.
Lastly - you could simply use my work and have your map method already efficiently implemented by using scramjet, which does exactly what you shown above - and moreover it supports async functions. :)
Here's how you'd use it (in some getStream function):
const {DataStream} = require('scramjet');
const stream = new DataStream();
stream.write(1); // you can also use await stream.whenWrote(1);
stream.write(2);
stream.write(3);
return stream.map(x => x * 2);
and then read it somewhere else:
stream.on('data', x => console.log(`x: ${x}`));
// x: 2
// x: 4
// x: 6
Take a look at the scramjet docs here

After such a long time of asking this question I was able to go back to the problem and come up with a simple solution. Since one stream should listen to the one it is subscribed to, we should return the instance of the original in order to preserve the values from the previous stream. Here is the code that I found to be working well:
class Stream {
constructor() {
this.subscriptions = [];
this.mappedActions = [];
}
subscribe(callback) {
this.subscriptions.push(callback);
}
map(actionFunc) {
this.mappedActions.push(actionFunc);
return this;
}
push(opValue) {
this.subscriptions.forEach(cb => {
if (this.mappedActions.length) {
this.mappedActions.forEach(action => {
cb(action.call(this, opValue));
});
} else {
cb(opValue);
}
});
}
}
const a = new Stream();
const b = a.map(value => value * 1 / 2);
const c = b.map(value => value * 3);
c.subscribe(console.log);
c.push(1);
c.push(2);
c.push(3);
// expected output in the console:
// 0.5
// 3
// 1
// 6
// 1.5
// 9
Hope anyone who stumbles upon this interesting problem will find my solution useful. If there is any changes you would like to make, feel free to do so or ping me!

RxJS does not play nice with Math.random()

I am trying to learn Rxjs and I am seeing some behaviour that I did not expect. The javascript code in question is listed below
function updateText(css_link, observable){
observable.subscribe(x => {
const container = document.querySelector(css_link);
container.textContent = `${x}`;
});
}
function log(observable) {
observable.subscribe(i => {
console.log(i);
});
}
let source = Rx.Observable.timer(0, 1000)
.map(() => {return {value: Math.random()}});
let double = source
.map(x => {return {value: x.value * 2}});
let diff = source
.pairwise()
.map(a => JSON.stringify(a));
updateText("#source", source.map(x => x.value));
updateText("#double", source.map(x => x.value));
updateText("#diff", diff);
It turns out that the output of the double stream are double values of new random numbers, not the random numbers that came from source. When looking at the output of diff I again get the impression that the random numbers are generated independantly in source, double and diff.
I am learning Rxjs and I may be missing a point. I thought that these streams are immutable but that they do depend on one another.
You can find a version of this code on jsbin with some html that is getting updated.

This is because every time you subscribe you're creating a new chain with a new source Observable. This means source, double and diff each one of them has its own timer.
You can see that this is true by printing a message to console every time you're creating a new timer:
let source = Rx.Observable.defer(() => {
console.log('new source');
return Rx.Observable.timer(0, 1000)
.map(() => {return {value: Math.random()}});
});
Now you'll see three messages "new source" in console.
If you want to share a single source Observable you can use multicasting and in particular the share() operator.
let source = Rx.Observable.defer(() => {
console.log('new source');
return Rx.Observable.timer(0, 1000)
.map(() => {return {value: Math.random()}});
}).share();
Now you'll see only one "new source" in console and it should work as you expect.
So your source can look like this:
let source = Rx.Observable.timer(0, 1000)
.map(() => {return {value: Math.random()}})
.share();
Your updated demo: https://jsbin.com/guyigox/3/edit?js,console,output

Filter Observable by Observable

I want to make sure that Observable.subscribe() doesn't get executed if a different Observable yields true.
An example use case would be making sure that user can trigger a download only if the previous one has finished (or failed) and there's only one download request executed at a time.
In order to control the execution flow, I had to rely on a state variable which seems a bit odd to me - is this a good pattern? In a v. likely case that it isn't - what would be a better approach?
I ended up with two subscriptions: Actions.sync (using a Subject, public API, initialises a sync request) and isActive (resolves to true or `false, the name should be pretty self-explanatory.
let canDownload = true; // this one feels really, really naughty
const startedSyncRequests = new Rx.Subject();
const isActiveSync = startedSyncRequests.map(true)
.merge(completeSyncRequests.map(false))
.merge(failedSyncRequests.map(false))
.startWith(false)
.subscribe(isActive => canDownload = !isActive)
syncResources = ()=>{
startedSyncRequests.onNext();
// Mocked async job
setTimeout(()=> {
completeSyncRequests.onNext();
}, 1000);
};
Actions.sync
.filter( ()=> canDownload ) // so does this
.subscribe( syncResources );

You want exclusive().
Actions.sync
.map(() => {
//Return a promise or observable
return Rx.Observable.defer(() => makeAsyncRequest());
})
.exclusive()
.subscribe(processResults);
The above will generate an observable every time the user makes a request. However, exclusive will drop any observables that come in before the previous one has completed, and then flattens the resulting messages into a single observable.
Working example (using interval and delay):
var interval = Rx.Observable.interval(1000).take(20);
interval
.map(function(i) {
return Rx.Observable.return(i).delay(1500);
})
.exclusive()
//Only prints every other item because of the overlap
.subscribe(function(i) {
var item = $('<li>' + i + '</li>');
$('#thelist').append(item);
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/2.5.3/rx.all.js"></script>
<div>
<ul id="thelist">
</ul>
</div>
Reference: here