Develop Reference

Struggling to get my promises to work in an array loop

I have been struggling with this issue for a week and have researched myself close to death. I am a total newbie. I have managed to grasp the crux of promises, but I am failing to see how to include this in a loop.
I have an app that is looking through an array. There is some validation of the array against a mongoose database (which is taking time to run). I am trying to push items into a new array based on some of this validation. I know the validation is working because of the console log in the loop. However my final array is not waiting for the loop to finish. Which means I need to put the loop into a promise, or well I think, but the issue is that I don't know how to resolve it. Current output is a blank array instead of the validated array. Here is my code:
//dummy data of an array - this is originally extracted from a mongoose DB and works (it's my first promise).
const appArray = ["5f8f25d554f1e43f3089ea5d",
"5f8f25e854f1e43f3089ea5e",
"5f8f25f454f1e43f3089ea5f",
"5f8f314ab92c7f406f28b83a",
"5f8fe50a9d44694cad91a01b",
"5f92e8a75d848870e015dff3",
"5f92e8b35d848870e015dff4",
"5f92e8cb5d848870e015dff5",
"5f8fe51d9d44694cad91a01c"];
//the second promise takes the array above and validates it against another collection on mongoose
function myPromise2 (response){
return new Promise((resolve, reject) => {
let appoints = [];
response.forEach(e => {
//loop through each item of the array and look against Appointment collection
Appointment.findById(e, function(err, foundApp){
//some validation supposed to happen here and then pushed into a new array
appoints.push(foundApp);
console.log(appoints);
})
})
//once completed supposed to resolve and return
resolve(appoints);
})
};
myPromise2(appArray).then((response) => {console.log(response)});

Here is an example which should work. Add a promise for each element to the array and then resolve the outer function if all promises have resolved.
// dummy data of an array - this is originally extracted from a mongoose DB and works (it's my first promise).
const appArray = ["5f8f25d554f1e43f3089ea5d",
"5f8f25e854f1e43f3089ea5e",
"5f8f25f454f1e43f3089ea5f",
"5f8f314ab92c7f406f28b83a",
"5f8fe50a9d44694cad91a01b",
"5f92e8a75d848870e015dff3",
"5f92e8b35d848870e015dff4",
"5f92e8cb5d848870e015dff5",
"5f8fe51d9d44694cad91a01c"];
// the second promise takes the array above and validates it against another collection on mongoose
function myPromise2 (response) {
let appoints = [];
response.forEach(e => {
appoints.push(new Promise((resolve) => {
//loop through each item of the array and look against Appointment collection
Appointment.findById(e, function(err, foundApp) {
//some validation supposed to happen here and then pushed into a new array
resolve(foundApp);
})
}))
})
return Promise.all(appoints)
};
myPromise2(appArray).then((response) => {console.log(response)});

Points to address:
Use the promise that mongoose provides through the .exec() method. This way you don't need new Promise
Collect these individual promises in an array (use .map instead of .forEach), and pass this array to Promise.all
If you do this, the code for myPromise2 reduces to the following:
function myPromise2 (response){
return Promise.all(response.map(e => Appointment.findById(e).exec()));
}

here is my suggestion:
const appArray = [
"5f8f25d554f1e43f3089ea5d",
"5f8f25e854f1e43f3089ea5e",
"5f8f25f454f1e43f3089ea5f",
"5f8f314ab92c7f406f28b83a",
"5f8fe50a9d44694cad91a01b",
"5f92e8a75d848870e015dff3",
"5f92e8b35d848870e015dff4",
"5f92e8cb5d848870e015dff5",
"5f8fe51d9d44694cad91a01c"
];
function myPromise2 (response){
return Promise.all(response.map(id) => {
return Appointment.findById(id).exec();
})
};
myPromise2(appArray)
.then(console.log) // response array
.catch(// handle errors)
// you can also async/await the calling part
you can also use one of:
Promise.allSettled
Prmoise.any (es2021)
Promise.race
it's just depends on how you would like to handle the responses/failures.

A good alternative to consider maybe Async/Await and have a look Async_await. This will hopefully answer all your issues
It's probably a good idea to look into how the JS event loop system works guide here,

How can I make an appendable observable queue?

I have an array where I need to make single requests using very single data set of this array. The problem that I found difficult to solve was to schedule the calls. Means, only when a request finishes, the next request starts. I was looking for an RxJs queue, but I couldn't simply find a solution.
Example:
function makeRequest(body): Observable<any> {
return someAsyncRequest(body);
}
array.forEach((entry) => {
makeRequest(entry);
});
// This is just an example how the setup is. This code does not work.
// What I need is a queue like feature in RxJs to append requests and wait before the previous one is finished.

You have quite few options, but I suppose concat or forkJoin fits you best. concat calls second API only after previous completes, while forkJoin will do the same, but only if none of them errors. If any of them errors, it will not return anything.
Example with concat:
concat(...array.map(entry => makeRequest(entry)).subscribe()
p.s. import concat as static operator:
import { concat } from 'rxjs'

if you want to have things go out one by one, but still receive all the results at once, i recommend concat -> reduce. looks like this:
concat(...array.map(entry => makeRequest(entry))).pipe(
reduce((completed, curResponse) => completed.concat([curResponse]), [])
).subscribe(allResponses => console.log(allResponses))
this structure achieves the single emission that forkjoin will give you but will do the requests one by one and then gather them once all complete. if you want the results one by one as they complete though, then just concat gets the job done as shown by others

ForkJoin can meet your requirment
‘forkJoin’ waits for each HTTP request to complete and group’s all the observables returned by each HTTP call into a single observable array and finally return that observable array.
public requestDataFromMultipleSources(): Observable<any[]> {
let response1 = this.http.get(requestUrl1);
let response2 = this.http.get(requestUrl2);
let response3 = this.http.get(requestUrl3);
return forkJoin([response1, response2, response3]);
}
subscribe to single observable array and save the responses separately.
this.dataService.requestDataFromMultipleSources().subscribe(responseList => {
this.responseData1 = responseList[0];
this.responseData2 = responseList[1];
this.responseData3 = responseList[2];
});

MDN Promise.all see this you can do it even without observable
Promise.all(array.map(item => makeRequest(item))).then(values =>{
// your logic with recieved data
})

Difference in performance (map(), forEach(), for())

Recently I had some issues using built-in map() function on array in JavaScript, which I managed to solve by using standard for loop. I've look through posts on StackOverflow to find out where is the difference between those two methods (and forEach() which I tested just for sake of it). I understand that map() is creating a new array by executing the function provided, whereas forEach() (and I believe for as well) are not creating a new array.
Could anyone please explain where is the difference in how the functions are executed?
Tested scenario: Frontend in ReactJS receives data from the backend over http request. The received data contains array of objects, where one property is image in base64 string. I defined the function which converted each image to image object (using Jimp), inverted the color, change to base64 and saved back in the array as JSX object to be displayed on the page. Three version of code looked as followed:
FOR LOOP:
console.log("before for");
for(n of result) {
console.log("inside for ", counter);
await this.processImage(n, counter++);
}
console.log("end for");
this.setState(() => {
console.log("Rendered");
return {
rows: result
}
})
FOREACH():
console.log("before foreach");
result.forEach(async (n) => {
console.log("inside foreach ", counter);
await this.processImage(n, counter++);
counter++;
})
console.log("end foreach");
this.setState(() => {
console.log("Rendered");
return {
rows: result
}
})
MAP():
console.log("before map");
result.map(async (n) => {
console.log("inside map ", counter);
await this.processImage(n, counter++);
counter++;
})
console.log("end map");
this.setState(() => {
console.log("Rendered");
return {
rows: result
}
})
I included the console.logs in the code above to show how I was testing the execution of the code. In each scenario what I got was (before, inside x3, end, rendered) in the same order. Unfortunately, map() and forEach() didn't perform the whole image processing and what I could see on the page instead of an image was super-long string. The for loop didn't fail a single time.
I understand in this situation I probably don't need to use map() as I don't have to create a new array. I would still like to know why the result was not always guaranteed, so I can avoid issues in the future.
I want to properly understand how those functions work, so I can use them correctly in the future. The documentation I read on is not very clear to me, I hope you guys can help!
Thanks

By using an async function inside .map and .forEach you fire and forget the asynchronous action, which means that you won't know when it finished. The async function does however return a Promise, and if you use .map you could collect them in an array, call Promise.all on that and await that:
await Promise.all(result.map(async (n) => {
console.log("inside map ", counter);
await this.processImage(n, counter++);
counter++; // why increase counter twice?
}));
// all processings are guaranteed to be done
This will execute all the processing in parallel, which is probably (way) faster than processing sequentially which you'd do with the for loop. Using a .forEach you aren't able to wait for all results to arrive, and therefore you probably don't want to use it in most cases.
If you arent doing asynchronous things, for and .forEach would behave nearly equal, except for arrays with empty slots (.forEach skips empty slots):
for(const el of Array(3))
console.log(el); // logs undefined three times
Array(3).forEach(console.log) // silence
.map behaves like .forEach just that it builds up an array with the returned values, just as you said.

should set state in callback after sorting function?

in my componentDidMount method I partition the data into two separate lists and then set state of the two lists so my UI can render them. I have two questions:
is the componentDidMount() method the appropriate place to partition a list received from an API call?
is it possible for the code to attempt to set the state before lodash has completed the partitioning (lets say its a really long list)? if so, I tried a .then() callback but I am getting an error saying .then() is not a function.
componentDidMount() {
const { data } = this.props;
let currentTime = moment();
let sortedPosts = _.partition(this.state.listViewData, function(o) {
return (o.isActive || (new Date(o.scheduledActiveEnd)) < currentTime);
}).then(() => {
this.setState({
futureListViewData: sortedPosts[0],
currentListViewData: sortedPosts[1]
})
})

is the componentDidMount() method the appropriate place to partition a list received from an API call?
Yes, componentDidMount is a good spot for putting asynchronous code, such as fetching data and then parsing the response.
is it possible for the code to attempt to set the state before lodash has completed the partitioning (lets say its a really long list)? if so, I tried a .then() callback but I am getting an error saying .then() is not a function.
No, _.partition is synchronous. It returns an array of arrays, not a promise. If you call _.partition on a large array, the thread will block until its done partioning the array.

How to create Observable from function?

I want to call a function (synchronously) and then use its return value as an initial emission (subsequently chaining some other operators on the resulting observable).
I want to invoke this function during subscription, so I can't just use Observable.of(() => getSomeValue()). I've seen bindCallback (previously fromCallback) but I don't think it can be used for this task (correct me if I'm wrong). I've seen start static operator in v4 docs but apparently it is not implemented in v5 (and no indication that its on the way). RxJava also has fromCallable operator that does exactly that afaik.
Only way I could think of is like this:
Observable.create((observer: Observer<void>) => {
let val = getSomeValue();
observer.next(val);
observer.complete();
})
which I think does just that. But this just seems so complicated for a simple thing that should probably have been like Observable.fromFunction(() => getSomeValue()) And what if I want to run it asynchronously, like start operator does? How can I do this in the current version of RxJS?

I tend to avoid any explicit use of Observable.create where ever possible, because generally it is a source of bugs to have to manage not just your event emission but also your teardown logic.
You can use Observable.defer instead. It accepts a function that returns an Observable or an Observable-like thing (read: Promise, Array, Iterators). So if you have a function that returns an async thing it is as easy as:
Observable.defer(() => doSomethingAsync());
If you want this to work with a synchronous result then do:
Observable.defer(() => Observable.of(doSomethingSync()));
Note: That like create this will rerun the function on each subscription. This is different then say the result of Observable.bindCallback which stores the function call result without re-executing the function. So if you need that sort of behavior you will need to use the appropriate multicasting operator.

An implementation of a fromFunction$ that I used in my project:
function fromFunction$<T>(factory: () => T): Observable<T> {
return Observable.create((observer: Subscriber<T>) => {
try {
observer.next(factory());
observer.complete();
} catch (error) {
observer.error(error);
}
});
}
Used like:
fromFunction$(() => 0).subscribe((value) => console.log(`Value is '${value}'`), null, () => console.log('Completed'));
fromFunction$(() => [1, 2, 3]).subscribe((value) => console.log(`Value is '${value}'`), null, () => console.log('Completed'));
fromFunction$(() => { throw 'Something' }).subscribe(null, (error) => console.error(`Error: ${error}`));
Gives:
Value is '0'
Completed
Value is '1,2,3'
Completed
Error: Something
Until such implementation exists.

Actually I think the best option is using Observable.create because it's the most universal solution for both synchronous and asynchronous initial values.
If you're sure you'll use a synchronous function you can use startWith() operator (this makes sence only if return value from getSomeValue() should be the same for all Observers).
Using Observable.bindCallback as a source Observable is of course doable however I personally recommend to avoid it because it makes your code very hard to understand and it's usually not necessary because you can use just Observable.create.