I have nested forEach loops out of which one of them contains another await call. I want my function doSomething to be executed only after the nested loop is done. As of now doSomething() gets executed first.
entity.forEach(function (dataItem) {
userParams.forEach(async function (userParamItem) {
const Marks = await _fetchMark(req, dataItem.MKTUNITID);
//do some manipulation to Marks
}
}
});
});
doSomething();
Whenever I have to use awaits inside a forEach (or any iterator loop; for, while, array-methods), I like to create a little array where I store the promises that are made.
I'll type out an example. I have a list of 10 urls I want to run a fetch towards, and store the results before running doSomething(). My runFetch()-function is asynchronous, and thus by default (unless I await it) returns a promise. After iterating, I will await ALL the promises at the same time, so they can run in parallell, but still wait for the slowest one.
const urls = [...] // my urls
const promises = []
for (const url of urls) {
promises.push( runFetch(url) )
}
const results = await Promise.all(promises)
//now that all my fetches are done, I can continue.
doSomething()
forEach can be bit tricky in such situations, you can simply use for loops instead as
they will not jump to next item until all promises are not resolved
for(i =0; i < entity.length; i++)
{
let dataItem = entity[i]
for(j =0; j < userParams.length; j++)
{
let userParamItem = userParams[i]
const Marks = await _fetchMark(req, dataItem.MKTUNITID);
//do some manipulation to Marks
}
}
Javascript does this because forEach is not promise-aware. It cannot support async and await so you cannot use await in forEach. Instead, use a for...of loop:
for(const dataItem of entity) {
for(const userParamItem of userParams) {
const Marks = await _fetchMark(req, dataItem.MKTUNITID);
//do some manipulation to Marks
}
}
});
doSomething();
This way the nested loops will execute first and doSomething() last.
Here is a nice post if you wanna learn more:
https://zellwk.com/blog/async-await-in-loops/
Related
I have some code that functions like so. It will iterate over an array with forEach, perform a heavy task that freezes the UI with the current iteration, push the returned value from that heavy task to an array, and then return the array being pushed to after the forEach has concluded. This is how it would work:
// generate a dummy array
var array = Array.from(Array(1000000).keys()):
function doSomething() {
// the array to push to
var pushArray = [];
array.forEach((item) => {
// perform a heavy task...
const output = doHeavyThing(item);
// ...and push the heavy task result to an array
pushArray.push(output);
// (this DOM call will not work, because doHeavyThing blocks the UI)
document.getElementById('updateDiv').innerHTML = item;
})
// return the array that was pushed to
return pushArray;
}
const result = doSomething();
return result;
Is there any way I can temporarily pause the execution of the forEach loop, unfreezing the UI, to allow for a DOM update? This would only require a couple of milliseconds.
I cannot use web workers in this situation, because the data being returned is so large that any calls to postMessage would crash the browser on mobile.
I also haven't been able to get setTimeout to work, because it is asynchronous - meaning I cannot return anything. I also cannot reliably use callbacks.
Does anyone have an idea on how to solve this problem? Clarifying questions are always welcome.
My doHeavyThing function is a function from the seek-bzip package:
bzip.decodeBlock(compressedDataBlock, 32);
I believe something like this would work to ensure the event loop can tick in between calls to doHeavyThing.
// create a promise that will resolve on the next tick of the event loop
function sleep() {
return new Promise(r => setTimeout(r));
}
// standin for doHeavyThing. This is enough to induce
// noticable delay on my machine, tweak the # of iterations
// if it's too fast to notice on yours
function doHeavyThing() {
for (let i = 0; i < 100; i++) { console.log(i); }
}
async function handleHeavyLifting() {
const array = Array.from(Array(1000).keys());
const result = [];
for (const item of array) {
document.getElementById('updateDiv').innerHTML = item;
result.push(doHeavyThing(item));
// let DOM updates propagate, other JS callbacks run, etc
await sleep();
}
return result;
}
handleHeavyLifting();
<div id="updateDiv">
None
</div>
I am having hard time in understanding how for...of differ from .forEach() in case of handling Promises.
Using this snippet:
const allSettled = async arr => {
const data = []
for (const promise of arr) {
try {
const result = await promise;
data.push(result);
} catch (e) {
data.push(e)
}
}
return data
}
I go over each promise in array, wait for it to settle and push the result to data. It executes sequentially.
If I have this snippet:
const badAllSettled = arr => {
const data = []
arr.forEach(async promise => {
try {
const result = await promise;
data.push(result);
} catch (e) {
data.push(e)
}
})
return data
}
I get empty array (because forEach does not wait for Promise to settle).
AFAIK for...of works with iterables, hence it might suspend and wait await to return. But how this flow works step-by-step I don't understand.
Thanks!
.forEach() is implemented like this:
Array.prototype.forEach = function forEach(fn) {
for (let i = 0; i < this.length; i++)
fn(this[i], i, this);
}
As you can see, when .forEach() is called, it just calls the callback synchronously several times. However, if the callback is an async function, it doesn't get awaited.
By contrast, when an await keyword is found inside an async function, all the code execution withing that function is paused until the promise settles, which means that any control flow (including for ... of loops) is also paused.
The following code behaves like .forEach() because it calls the async function which awaits the promise without awaiting the promise the function returns:
const allSettled = async arr => {
const data = []
for (const promise of arr) {
(async () => {
try {
const result = await promise;
data.push(result);
} catch (e) {
data.push(e)
}
)();
}
return data
}
for...of doesn't wait. The await keyword does the waiting. You have one of those here: const result = await promise;
The internals of forEach don't await the promise returned by the function you pass to forEach(here). (The use of await inside the function that generates that promise is irrelevant).
for (const item of array) {
// do something
}
does "something" for each element of the array. If you await something inside, it will await it until proceeding to the rest of the code.
array.forEach(callback)
synchronously calls the callback for each element of the array. If your callback happens to be asynchronous, it will call it and then immediately call the next callback. .forEach doesn't wait for each callback to resolve to something before calling the next callback.
Note that if you want your promises to run in parallel, perhaps for perf gains, you might wanna use Promise.all
// runs every fetch sequentially
// if each of them takes 1s, this will take 3s to complete
const results = []
for (const item of [1,2,3]) {
results.push(await fetchSomething(item))
}
// runs every fetch in parrallel
// if each of them takes 1s, this will take 1s to complete
const promises = [1,2,3].map(item => fetchSomething(item))
const results = await Promise.all(promises)
I'm trying to resolve this problem, I have a function that calls two other function that will push elements inside array. My problem is that in this way the first function function1 when it has finished calculating, it returns its value and the code continues. As soon as the second function finishes, it returns its value and the rest of the code is executed again.
What I would like to do is that the first function function1(..) is called as soon as it ends the second function function2(..) is called and when both arrays are full the code goes on. How can I do?
Thanks so much
async initialFunction(){
( async () => {
await function1(arrayPopulated1);
await function2(arrayPopulated2);
}) ();
// Code that use: newArray1 and newArray2 to write in the DB
}
async function1(array1){
for(let i = 0; i < array1.length; i++){
//operation there
// value = somedata
newArray1.push(value)
}
}
async function1(array2){
for(let i = 0; i < array2.length; i++){
//operation there
// value = somedata
newArray2.push(value)
}
}
EDIT:
The problem is that after function1 finishes its value returns to InitialFunction continue the code of this function and write in the db. It then returns the value of function2 to InitialFunction continues the code of this function and writes again to the DB.
I would like function1 to finish its for by writing to the newArray1 array, function2 to write the values to the newArray2 array and when both have been populated then InitialFunction writes to the db.
If you want both promises to complete before continuing, you can wrap both of the promises in a Promise.all.
Promise.all([promise1, promise2, promise3]).then((values) => {
console.log(values);
});
Checkout MDN link
You don't have to wrap the promises again in another async function
async initialFunction(){
const newArray1 = [];
const newArray2 = [];
await function1(arrayPopulated1);
await function2(arrayPopulated2);
// Save to db
}
async function1(array1){
for(let i = 0; i < array1.length; i++){
newArray1.push(value)
}
}
async function1(array2){
for(let i = 0; i < array2.length; i++){
newArray2.push(value)
}
}
I am thinking about a scenario of building up a promise queue:
//Let's assume that promises is an array of promises
var promiseQueue = [];
for (var promise of promises) {
if (promiseQueue.length) promiseQueue[promiseQueue.length - 1].then(promise);
promiseQueue.push(promise);
}
I am thinking about implementing a function called resolver:
function *resolve() {
var promise;
while (promise = yield) Promise.resolve(promise);
}
and then iterating it:
var promiseGenerator = resolve();
The problem is the for..of here which would be responsible for the actual iteration:
for (var r of promiseGenerator) {
}
At the code above the generator will be successfully iterated, but unfortunately I am not aware of a way to successfully pass a parameter to this generator at the iteration of for..of.
I would like to clarify that I do not need an alternative, I am perfectly aware that we can do something like this:
for (var p in promiseQueue) promiseGenerator.next(promiseQueue[p]);
I am specifically interested to know whether I can pass parameters to the generator when I execute a for..of cycle.
EDIT
The problem raised by amn is that in the example he/she was focusing on would always get undefined. That's true if we pass undefined to next(), but not true if we pass something else. The problem I was raising is that a for..of loop does not allow us to pass anything to yield, which is this specific question is all about, the example is a mere illustration of the problem, showing that the promises we would create will never be created in a for..of loop. However, there is life for Iterable objects outside the realm of for..of loops and we can pass defined values into the yield. An example with the criticized code chunk can look like:
function *resolve() {
var promise;
while (promise = yield) Promise.resolve(promise);
}
var responses = [];
var f = resolve();
var temp;
for (var i = 10; !(temp = f.next(i)).done; i--) responses.push(temp);
As we can see above, the yield above cannot be assumed ab ovo to be undefined. And of course we can pass some custom thenables, like
Promise.resolve({
then: function(onFulfill, onReject) { onFulfill('fulfilled!'); }
});
or even promises which were not resolved yet. The point of the example was to show that we cannot pass values to the yield using the for..of loop, which is quite a feature gap in my opinion.
No, it is not possible to pass arguments to next.
function* generateItems() { /* ... */ }
for (var item of generateItems()) {
console.log(item);
}
is mostly short for
function* generateItems() { /* ... */ }
var iterator = generateItems()[Symbol.iterator]();
do {
const result = iterator.next();
if (result.done) break;
const item = result.value;
console.log(item);
} while (true);
barring a few missing try/catch wrappers. You can see in the spec here that it calls .next with no arguments:
Let nextResult be ? Call(iteratorRecord.[[NextMethod]], iteratorRecord.[[Iterator]], « »).
e.g.
iterator.next.apply(iterator, []);
calling next() with an empty array of arguments.
Why does this snippet
const firstArray = ['toto', 'toto'];
const secondArray = ['titi', 'titi'];
firstArray.forEach(async (toto, i) =>
{
await secondArray.forEach(async titi =>
{
// async code
console.log(titi, i);
});
// async code
console.log(toto, i);
});
produce the following output:
Removing the await keyword produces the expected output
My guess is it resides in the await keyword's behaviour, as, without which, the produced output is the expected output.
EDIT: this is a purely trivial question. I want to understand why using await before a forEach provides this behaviour. There is no 'concrete code' behind this.
EDIT2: edited the snippet as the comments and answer reflected misunderstanding of my question
forEach works synchronously meaning it doesn't do anything with the return of each iteration (in this case, the promise) and it ignores it.
If you use instead:
for(let item of arrayItems)
or a normal for loop you should see the expected result working asynchronously.
I guess you want to await all promises before continuing. If should be the case, a combination of Array.prototype.reduce(...), Array.prototype.map(...) and Promise.all(...) will better suit:
const allPromises = firstArray.reduce((accumulatedPromises, toto) => {
const secondArrayPromises = secondArray.map((titi) => {
return // some promise;
});
const firstArrayPromise = // some promise;
return accumulatedPromises.concat(secondArrayPromises, firstArrayPromise);
}, []);
const finalPromise = Promise.all(allPromises);
Then, either await finalPromise if you're within an async function, or use .then(...).
Your outer forEach loop spits out two functions (note the async type you have; async functions automatically return a Promise) directly on the stack that then runs in sequence before finally outputting from the firstArray. So after both the functions are executed your output for the outer loop is printed. This is how the forEach loop works with async functions.
To use async/await with loops, you would want to have:
async function boot() {
for(let item of list) {
await getData(); // Here getData returns a promise
}
}
boot();
Another way could be to use bluebird for Promise:
let exec = [];
await Promise.map(list => getData());
// Here Promise.map iterates over list
// and returns a list of promises which are
// then resolved using await
The key to understanding is that await is asynchronous. So everything after will be executed no earlier than the current execution thread returns. Thus, the part
// async code
console.log(toto);
is put to a queue rather than being executed immediately.
Async/await essentially wraps your code like this:
const firstArray = ['toto', 'toto'];
const secondArray = ['titi', 'titi'];
firstArray.forEach((toto,i) => new Promise((resolve)=>
{
resolve((new Promise((resolve)=>resolve(secondArray.forEach(titi => new Promise((resolve)=>
{
resolve(console.log(titi, i));
}))))).then(()=>console.log(toto,i))
);}));
Now consider this code as well (setTimeout is pretty much a promise that is resolved after a given amount of time):
const firstArray = ['toto', 'toto'];
const secondArray = ['titi', 'titi'];
firstArray.forEach((toto, i) =>
{
secondArray.forEach(titi =>
{
console.log(titi, i);
});
setTimeout(()=>console.log(toto, i));
});