Develop Reference

React state disappearing

I have a react app that uses the MS Graph API (so it's a bit difficult to post a minimal reproducible example). It has a state variable called chats that is designed to hold the result of fetching a list of chats from the graph API. I have to poll the API frequently to get new chats.
I query the chats endpoint, build an array of newChats and then setChats. I then set a timeout that refreshes the data every 10 seconds (it checks for premature invocation through the timestamp property stored in the state). If the component is unmounted, a flag is set, live (useRef), which stops the refresh process. Each chat object is then rendered by the Chat component (not shown).
Here's the code (I've edited by hand here to remove some irrelevant bits around styles and event propagation so it's possible that typo's have crept in -- it compiles and runs in reality).
const Chats = () => {
const [chats, setChats] = useState({ chats: [], timestamp: 0 });
const live = useRef(true);
const fetchChats = () => {
if (live.current && Date.now() - chats.timestamp < 9000) return;
fetchData(`${baseBeta}/me/chats`).then(res => {
if (res.value.length === chats.chats.length) return;
const chatIds = chats.chats.map(chat => chat.id);
const newChats = res.value.filter(chat => !chatIds.includes(chat.id));
if (newChats.length > 0) {
setChats(c => ({ chats: [...c.chats, ...newChats], timestamp: Date.now() }));
}
setTimeout(fetchChats, 10000);
});
};
useEffect(() => {
fetchChats();
return () => (live.current = false);
}, [chats]);
return (
<div>
{chats.chats.map(chat => (
<Chat chat={chat} />
))}
</div>
);
};
The Chat component must also make some async calls for data before it is rendered.
This code works, for a second or two. I see the Chat component rendered on the screen with the correct details (chat member names, avatars, etc.), but almost before it has completed rendering I see the list elements being removed, apparently one at a time, though that could just be the way its rendered -- it could be all at once. The list collapses on the screen, showing that the chat state has been cleared out. I don't know why this is happening.
I've stepped through the code in the debugger and I can see the newChats array being populated. I can see the setChats call happen. If I put a breakpoint on that line then it is only invoked once and that's the only line that sets that particular state.
So, what's going on? I'm pretty sure React isn't broken. I've used it before without much trouble. What's changed recently is the inclusion of the refresh code. I'm suspicious that the reset is taking away the state. My understanding is that the fetchChats method will be rendered every time the chats state changes and so should see the current value of the chats state. Just in case this wasn't happening, I passed the chats state from the useEffect like this:
useEffect(() => {
fetchChats(chats);
return () => (live.current = false);
}, [chats]);
With the necessary changes in fetchChats to make this work as expected. I get the same result, the chats state is lost after a few seconds.
Edit
Still Broken:
After #Aleks answer my useEffect now looks like this:
useEffect(() => {
let cancel = null;
let live = true;
const fetchChats = () => {
if (Date.now() - chats.timestamp < 9000) return;
fetchData(`${baseBeta}/me/chats`).then(res => {
if (res.value.length === chats.chats.length) return;
const chatIds = chats.chats.map(chat => chat.id);
const newChats = res.value.filter(chat => chat.chatType === "oneOnOne" && !chatIds.includes(chat.id));
if (newChats.length > 0 && live) {
setChats(c => ({ chats: [...c.chats, ...newChats], timestamp: Date.now() }));
}
cancel = setTimeout(fetchChats, 10000);
});
};
fetchChats();
return () => {
live = false;
cancel?.();
};
}, []);
The result of this is that the chats are loaded, cleared, and loaded again, repeatedly. This is better, at least they're reloading now, whereas previously they would disappear forever. They are reloaded every 10 seconds, and cleared out almost immediately still.
Eventually, probably due to random timings in the async calls, the entries in the list are duplicated and the 2nd copy starts being removed immediately instead of the first copy.

There are multiple problems. First this
setTimeout(fetchChats, 10000); will trigger
useEffect(() => {
fetchChats(chats);
return () => (live.current = false);
}, [chats])
You will get 2 fetches one after another.
But the bug you're seeing is because of this
return () => (live.current = false);
On second useEffect trigger, clean up function above with run and live.current will be forever false from now on.
And as Nikki9696 said you you need to clear Timeout in clean up function
The easiest fix to this is, probably
useEffect(() => {
let cancel = null;
let live = true;
const fetchChats = () => {
// not needed
//if ( Date.now() - chats.timestamp < 9000) return;
fetchData(`${baseBeta}/me/chats`).then(res => {
//this line is not needed
//if (res.value.length === chats.chats.length) return;
// remove all the filtering, it can be done elsewhere where
// you can access fresh chat state
//const chatIds = chats.chats.map(chat => chat.id);
//const newChats = res.value.filter(chat =>
//!chatIds.includes(chat.id));
if (res.value?.length > 0&&live) {
setChats(c => ({ chats: [...c.chats, ...res.value], timestamp: Date.now() }));
cancel = setTimeout(fetchChats, 10000);
}
});
};
fetchChats()
return () => { live=false; if(cancel)window.clearTimeout(cancel) };
}, []);
Edit: typo cancel?.() to window.clearTimeout(cancel);

Ok, I have an idea what's happening and how to fix it. I am still not sure why it is behaving like this, so please comment if you understand it better than me.
Basically, for some reason I don't understand, the function fetchChats only ever sees the initial state of chats. I am making the mistake of filtering my newly fetched list against this state, in which the array is empty.
If I change my useEffect code to do this instead:
setChats(c => {
return {
chats: [
...c.chats,
...res.value.filter(cc => {
const a = c.chats.map(chat => chat.id);
return !a.includes(cc.id);
})
],
timestamp: Date.now()
};
});
Then my filter is passed the current value of the state for chats rather than the initial state.
I thought that because the function containing this code is in the function that declares the chat state, whenever that state changed the whole function would be rendered with the new value of chats making it available to its nested functions. This isn't the case here and I don't understand why.
The solution, to only trust the values of the state that is handed to me during the setState (setChats) call, works fine and I'll go with it, but I'd love to know what is wrong with reading the state directly.

Add streams dynamically to combined stream (eg forkJoin)

My function (lets call it myFunction) is getting an array of streams (myFunction(streams: Observable<number>[])). Each of those streams produces values from 1 to 100, which acts as a progress indicator. When it hits 100 it is done and completed. Now, when all of those observables are done I want to emit a value. I could do it this way:
public myFunction(streams: Observable<number>[]) {
forkJoin(streams).subscribe(_values => this.done$.emit());
}
This works fine, but imagine following case:
myFunction gets called with 2 streams
one of those streams is done, second one is still progressing
myFunction gets called (again) with 3 more streams (2nd one from previous call is still progressing)
I'd like to somehow add those new streams from 3rd bullet to the "queue", which would result in having 5 streams in forkJoin (1 completed, 4 progressing).
I've tried multiple approaches but can't get it working anyhow... My latest approach was this:
private currentProgressObs: Observable<any> | null = null;
private currentProgressSub: Subscription | null = null;
public myFunction(progressStreams: Observable<number>[]) {
const isUploading = this.cumulativeUploadProgressSub && !this.cumulativeUploadProgressSub.closed;
const currentConcatObs = this.currentProgressObs?.pipe(concatAll());
const currentStream = isUploading && this.currentProgressObs ? this.currentProgressObs : of([100]);
if (this.currentProgressSub) {
this.currentProgressSub.unsubscribe();
this.currentProgressSub = null;
}
this.currentProgressObs = forkJoin([currentStream, ...progressStreams]);
this.currentProgressSub = this.currentProgressObs.subscribe(
_lastProgresses => {
this._isUploading$.next(false); // <----- this is the event I want to emit when all progress is completed
this.currentProgressSub?.unsubscribe();
this.currentProgressSub = null;
this.currentProgressObs = null;
},
);
}
Above code only works for the first time. Second call to the myFunction will never emit the event.
I also tried other ways. I've tried recursion with one global stream array, in which I can add streams while the subscription is still avctive but... I failed. How can I achieve this? Which operator and in what oreder should I use? Why it will or won't work?

Here is my suggestion for your issue.
We will have two subjects, one to count the number of request being processed (requestsInProgress) and one more to mange the requests that are being processed (requestMerger)
So the thing that will do is whenever we want to add new request we will pass it to the requestMerger Subject.
Whenever we receive new request for processing in the requestMerger stream we will first increment the requestInProgress counter and after that we will merge the request itself in the source observable. While merging the new request/observable to the source we will also add the finalize operator in order to track when the request has been completed (reached 100), and when we hit the completion criteria we will decrement the request counter with the decrementCounter function.
In order to emit result e.g. to notify someone else in the app for the state of the pending requests we can subscribe to the requestsInProgress Subject.
You can test it out either here or in this stackBlitz
let {
interval,
Subject,
BehaviorSubject
} = rxjs
let {
mergeMap,
map,
takeWhile,
finalize,
first,
distinctUntilChanged
} = rxjs.operators
// Imagine next lines as a service
// Subject responsible for managing strems
let requestMerger = new Subject();
// Subject responsible for tracking streams in progress
let requestsInProgress = new BehaviorSubject(0);
function incrementCounter() {
requestsInProgress.pipe(first()).subscribe(x => {
requestsInProgress.next(x + 1);
});
}
function decrementCounter() {
requestsInProgress.pipe(first()).subscribe(x => {
requestsInProgress.next(x - 1);
});
}
// Adds request to the request being processed
function addRequest(req) {
// The take while is used to complete the request when we have `value === 100` , if you are dealing with http-request `takeWhile` might be redudant, because http request complete by themseves (e.g. the finalize method of the stream will be called even without the `takeWhile` which will decrement the requestInProgress counter)
requestMerger.next(req.pipe(takeWhile(x => x < 100)));
}
// By subscribing to this stream you can determine if all request are processed or if there are any still pending
requestsInProgress
.pipe(
map(x => (x === 0 ? "Loaded" : "Loading")),
distinctUntilChanged()
)
.subscribe(x => {
console.log(x);
document.getElementById("loadingState").innerHTML = x;
});
// This Subject is taking care to store or request that are in progress
requestMerger
.pipe(
mergeMap(x => {
// when new request is added (recieved from the requestMerger Subject) increment the requrest being processed counter
incrementCounter();
return x.pipe(
finalize(() => {
// when new request has been completed decrement the requrest being processed counter
decrementCounter();
})
);
})
)
.subscribe(x => {
console.log(x);
});
// End of fictional service
// Button that adds request to be processed
document.getElementById("add-stream").addEventListener("click", () => {
addRequest(interval(1000).pipe(map(x => x * 25)));
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.6.6/rxjs.umd.min.js"></script>
<div style="display:flex">
<button id="add-stream">Add stream</button>
<h5>Loading State: <span id="loadingState">false</span> </h5>
</div>

Your problem is that each time your call your function, you are creating a new observable. Your life would be much easier if all calls of your function pushed all upload jobs through the same stream.
You can achieve this using a Subject.
I would suggest you push single "Upload Jobs" though a simple subject and design an observable that emits the state of all upload jobs whenever anything changes: A simple class that offers a createJob() method to submit jobs, and a jobs$ observable to reference the state:
class UploadService {
private jobs = new Subject<UploadJob>();
public jobs$ = this.jobs.pipe(
mergeMap(job => this.processJob(job)),
scan((collection, job) => collection.set(job.id, job), new Map<string, UploadJob>()),
map(jobsMap => Array.from(jobsMap.values()))
);
constructor() {
this.jobs$.subscribe();
}
public createJob(id: string) {
this.jobs.next({ id, progress: 0 });
}
private processJob(job: UploadJob) {
// do work and return observable that
// emits updated status of UploadJob
}
}
Let's break it down:
jobs is a simple subject, that we can push "jobs" through
createJob simply calls jobs.next() to push the new job through the stream
jobs$ is where all the magic happens. It receives each UploadJob and uses:
mergeMap to execute whatever function actually does the work (I called it processJob() for this example) and emits its values into the stream
scan is used to accumulate these UploadJob emissions into a Map (for ease of inserting or updating)
map is used to convert the map into an array (Map<string, UploadJob> => UploadJob[])
this.jobs$.subscribe() is called in the constructor of the class so that jobs will be processed
Now, we can easily derive your isUploading and cumulativeProgress from this jobs$ observable like so:
public isUploading$ = this.jobs$.pipe(
map(jobs => jobs.some(j => j.progress !== 100)),
distinctUntilChanged()
);
public progress$ = this.jobs$.pipe(
map(jobs => {
const current = jobs.reduce((sum, j) => sum + j.progress, 0) / 100;
const total = jobs.length ?? current;
return current / total;
})
);
Here's a working StackBlitz demo.

ReactiveX filtering on observable multiple times and merging

I have a problem creating the following observable.
I want it to receive a predefined array of values
And I want to filter by different things, and be able to work with these as individual observables.
And then when it comes time to merge these filtered observables, I want to preserve the order from the original one
//Not sure the share is necessary, just thought it would tie it all together
const input$ = Observable.from([0,1,0,1]).share();
const ones$ = input$.filter(n => n == 1);
const zeroes$ = input$.filter(n => n == 0);
const zeroesChanged$ = zeroes$.mapTo(2);
const onesChanged$ = ones$.mapTo(3);
const allValues$ = Observable.merge(onesChanged$,zeroesChanged$);
allValues$.subscribe(n => console.log(n));
//Outputs 3,3,2,2
//Expected output 3,2,3,2
EDIT: I am sorry I was not specific enough in my question.
I am using a library called cycleJS, which separates sideeffects into drivers.
So what I am doing in my cycle is this
export function socketCycle({ SOCKETIO }) {
const serverConnect$ = SOCKETIO.get('connect').map(serverDidConnect);
const serverDisconnect$ = SOCKETIO.get('disconnect').map(serverDidDisconnect);
const serverFailedToConnect$ = SOCKETIO.get('connect_failed').map(serverFailedToConnect);
return { ACTION: Observable.merge(serverConnect$, serverDisconnect$, serverFailedToConnect$) };
}
Now my problem arose when I wanted to write a test for it. I tried with the following which worked in the wrong matter(using jest)
const inputConnect$ = Observable.from(['connect', 'disconnect', 'connect', 'disconnect']).share();
const expectedOutput$ = Observable.from([
serverDidConnect(),
serverDidDisconnect(),
serverDidConnect(),
serverDidDisconnect(),
]);
const socketIOMock = {
get: (evt) => {
if (evt === 'connect') {
return inputConnect$.filter(s => s === 'connect');
} else if (evt === 'disconnect') {
return inputConnect$.filter(s => s === 'disconnect');
}
return Observable.empty();
},
};
const { ACTION } = socketCycle({ SOCKETIO: socketIOMock });
Observable.zip(ACTION, expectedOutput$).subscribe(
([output, expectedOutput]) => { expect(output).toEqual(expectedOutput); },
(error) => { expect(true).toBe(false) },
() => { done(); },
);
Maybe there is another way I can go about testing it?

When stream is partitioned, the timing guarantees between elements in different daughter streams is actually destroyed. In particular, even if connect events always come before disconnect events at the event source, the events of the connect Observable won't always come before their corresponding events items in the disconnect Observable. At normal timescales, this race condition probably quite rare but dangerous nonetheless, and this test shows the worst case.
The good news is that your function as shown is just a mapper, between events and results from handlers. If you can continue this model generally over event types, then you can even encode the mapping in a plain data structure, which benefits expressiveness:
const event_handlers = new Map({
'connect': serverDidConnect,
'disconnect': serverDidDisconnect,
'connect_failed': serverFailedToConnect
});
const ACTION = input$.map(event_handlers.get.bind(event_handlers));
Caveat: if you were reducing over the daughter streams (or otherwise considering previous values, like with debounceTime), the refactor is not so straightforward, and would also depend on a new definition of "preserve order". Much of the time, it would still be feasible to reproduce with reduce + a more complicated accumulator.

Below code might be able to give you the desire result, but it's no need to use rxjs to operate array IMHO
Rx.Observable.combineLatest(
Rx.Observable.from([0,0,0]),
Rx.Observable.from([1,1,1])
).flatMap(value=>Rx.Observable.from(value))
.subscribe(console.log)

RXJS Observable stretch

I have a Rx.Observable.webSocket Subject. My server endpoint can not handle messages receiving the same time (<25ms). Now I need a way to stretch the next() calls of my websocket subject.
I have created another Subject requestSubject and subscribe to this.
Then calling next of the websocket inside the subscription.
requestSubject.delay(1000).subscribe((request) => {
console.log(`SENDING: ${JSON.stringify(request)}`);
socketServer.next(JSON.stringify(request));
});
Using delay shifts each next call the same delay time, then all next calls emit the same time later ... thats not what I want.
I tried delay, throttle, debounce but it does not fit.
The following should illustrate my problem
Stream 1 | ---1-------2-3-4-5---------6----
after some operation ...
Stream 2 | ---1-------2----3----4----5----6-

Had to tinker a bit, its not as easy as it looks:
//example source stream
const source = Rx.Observable.from([100,500,1500,1501,1502,1503])
.mergeMap(i => Rx.Observable.of(i).delay(i))
.share();
stretchEmissions(source, 1000)
.subscribe(val => console.log(val));
function stretchEmissions(source, spacingDelayMs) {
return source
.timestamp()
.scan((acc, curr) => {
// calculate delay needed to offset next emission
let delay = 0;
if (acc !== null) {
const timeDelta = curr.timestamp - acc.timestamp;
delay = timeDelta > spacingDelayMs ? 0 : (spacingDelayMs - timeDelta);
}
return {
timestamp: curr.timestamp,
delay: delay,
value: curr.value
};
}, null)
.mergeMap(i => Rx.Observable.of(i.value).delay(i.delay), undefined, 1);
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.4.2/Rx.js"></script>
Basically we need to calculate the needed delay between emissions so we can space them. We do this using timestamp() of original emissions and the mergeMap overload with a concurrency of 1 to only subscribe to the next delayed value when the previous is emitted. This is a pure Rx solution without further side effects.

Here are two solutions using a custom stream and using only rxjs-operators - since it looks quite complicated I would not advice you to use this solution, but to use a custom stream (see 1st example below):
Custom stream (MUCH easier to read and maintain, probably with better performance as well):
const click$ = Rx.Observable
.fromEvent(document.getElementById("btn"), "click")
.map((click, i) => i);
const spreadDelay = 1000;
let prevEmitTime = 0;
click$
.concatMap(i => { // in this case you could also use "flatMap" or "mergeMap" instead of "concatMap"
const now = Date.now();
if (now - prevEmitTime > spreadDelay) {
prevEmitTime = now;
return Rx.Observable.of(i); // emit immediately
} else {
prevEmitTime += spreadDelay;
return Rx.Observable.of(i).delay(prevEmitTime - now); // emit somewhere in the future
}
})
.subscribe((request) => {
console.log(`SENDING: ${request}`);
});
<script src="https://unpkg.com/rxjs/bundles/Rx.min.js"></script>
<button id="btn">Click me!</button>
Using only RxJS Operators (contains issues, probably shouldn't use):
const click$ = Rx.Observable
.fromEvent(document.getElementById("btn"), "click")
.map((click, i) => i);
click$
// window will create a new substream whenever no click happened for 1001ms (with the spread out
.window(click$
.concatMap(i => Rx.Observable.of(i).delay(1000))
.debounceTime(1001)
)
.mergeMap(win$ => Rx.Observable.merge(
win$.take(1).merge(), // emitting the "first" click immediately
win$.skip(1)
.merge()
.concatMap(i => Rx.Observable.of(i).delay(1000)) // each emission after the "first" one will be spread out to 1 seconds
))
.subscribe((request) => {
console.log(`SENDING: ${request}`);
});
<script src="https://unpkg.com/rxjs/bundles/Rx.min.js"></script>
<button id="btn">Click me!</button>

Mark van Straten's solution didn't work completely accurately for me. I found a much more simple and accurate solution based from here.
const source = from([100,500,1500,1501,1502,1503]).pipe(
mergeMap(i => of(i).pipe(delay(i)))
);
const delayMs = 500;
const stretchedSource = source.pipe(
concatMap(e => concat(of(e), EMPTY.pipe(delay(delayMs))))
);

How to process RxJS stream n items at a time and once an item is done, autofill back to n again?

I have a stream of events and I would like to call a function that returns a promise for each of those events, the problem is that this function is very expensive, so I would like to process at most n events at a time.
This pebble diagram is probably wrong but this is what I would like:
---x--x--xxxxxxx-------------x-------------> //Events
---p--p--pppp------p-p-p-----p-------------> //In Progress
-------d--d--------d-d-dd------dddd--------> //Promise Done
---1--21-2-34-----------3----4-3210-------- //QUEUE SIZE CHANGES
This is the code that I have so far:
var n = 4;
var inProgressCount = 0;
var events$ = Rx.Observable.fromEvent(produceEvent, 'click')
.map((ev) => new Date().getTime());
var inProgress$ = events$.controlled();
var done$ = inProgress$
.tap(() => inProgressCount++)
.flatMap((timestamp) => Rx.Observable.fromPromise(expensiveComputation(getRandomInt(1, 5) * 1000, timestamp)));
done$.subscribeOnNext((timestamp) => {
inProgressCount--;
inProgress$.request(Math.max(1, n - inProgressCount));
});
inProgress$.request(n);
There are two issues with this code:
It's using the inProgressCount var which is updated with side
effect functions.
The done$ subscription is only called once when I request more than 1 item from the controlled stream. This is making the inProgressCount var to update incorrectly, this eventually limits the queue to one at a time.
You can see it working in here:
http://jsbin.com/wivehonifi/1/edit?js,console,output
Questions:
Is there a better approach?
How can I get rid of the inProgressCount variable?
Why is the done$ subscription only getting called once when requesting multiple items?
Update:
Answer to question #3: switchMap is the same as flatMapLatest, so that's why I was only getting the last one. Updated the code to flatMap instead of switchMap.

You actually don't need to use backpressure at all. There is an operator called flatMapWithMaxConcurrent that does this for you. It is essentially an alias for calling .map().merge(concurrency) and it only allows a maximum number of streams to be in flight at a time.
I updated your jsbin here: http://jsbin.com/weheyuceke/1/edit?js,output
But I annotated the important bit below:
const concurrency = 4;
var done$ = events$
//Only allows a maximum number of items to be subscribed to at a time
.flatMapWithMaxConcurrent(concurrency,
({timestamp}) =>
//This overload of `fromPromise` defers the execution of the lambda
//until subscription
Rx.Observable.fromPromise(() => {
//Notify the ui that this task is in progress
updatePanelAppend(inProgress, timestamp);
removeFromPanel(pending, timestamp);
//return the task
return expensiveComputation(getRandomInt(1, 5) * 1000, timestamp)
}));