I have two functions, one function, asyncTest_returnPromise simply returns a promise, and another function, asyncTest, calls that promise 3 times by calling Promise.all(promises) on an array of promises. Within each promise in the array, I send one value ("Foo") to the console before the asyncTest_returnPromise function is called, and send another value ("Bar") to the console after it is called.
Given my understanding of the Promise.all function, I would have expected each promise within the array to be processed only after the subsequent promise had been fulfilled, meaning that as both console.log statements are within each promise in the array, the result would have been:
Foo
Bar
Foo
Bar
Foo
Bar
Instead, however, the output seems to be:
Foo
Foo
Foo
Bar
Bar
Bar
As all instances of "Foo" have been sent to the console before even the first "Bar", it seems to me these promises must be being processed concurrently.
Here are the two functions:
function asyncTest_returnPromise() {
return new Promise((resolve, reject) => {
resolve()
})
}
function asyncTest() {
var promises = []
for (i = 0; i < 3; i++) {
promises.push(new Promise((resolve, reject) => {
console.log("Foo")
asyncTest_returnPromise().then(_ => {
console.log("Bar")
resolve();
})
}))
}
Promise.all(promises)
}
asyncTest();
So I would like to know, have I misunderstood the purpose of Promise.all or how it works? Has some other part of my code caused all "foo" outputs before all the "bar" outputs? Is there something I need to do to have the entirety of each promise complete before moving on to the next one?
They are running asynchronously.
Each time around the loop you:
Log Foo
Create a promise (which resolves immediately)
Use then to queue up a function to run after the promise resolves
Then the loop goes around for the next iteration.
Afterwards, when the asyncTest function is finished , the event loop is free to look at the queued up functions.
Since all the promises are resolved, they each run (and log Bar).
If you want to wait for a promise before going around the loop again, then you should look at the await keyword which would let asyncTest pause (and go to sleep) until a promise resolved.
It's just how Event Loop works. Let me explain how it's processed in simple words:
First, the loop starts and runs 3 times. Each time it creates a new microtask and puts it in the microtask queue for later handling. The order in which those microtasks have been created is kept.
Now those microtasks are being handled one at a time, synchronously. Each of those microtasks (which are run synchronously as the Loop runs) executes console.log('Foo'); and then creates another microtask (another Promise) which are queued by the .then() call. So this additional microtask is put into the queue (behind those three first microtasks). It happens for every microtask created by the loop (so it's 3 times). This is where you have Foo printed 3 times.
Now that all our microtasks (3) have been processed, the Event Loop keeps going and takes the next microtasks for processing. This time it's a microtask that runs console.log('Bar');. It happens also three times. Here's where you get Bar printed 3 times.
Of course, there's more that happens in the Event Loop but this is essentially what happens with your Promises here.
Related
I can’t figure out how the code below works, my example of work contradicts the execution of the event loop.
async function main() {
for (let i = 0; i < 10; i++) {
await new Promise(res => setTimeout(res, 1000));
console.log("time + i");
}
}
main();
My example of how this code works:
function main added to Call stack.
call the loop for.
call function Promise which return setTimeout.
setTimeout added to Macrotask.
function main add to Microtask (await resolve Promise).
Some remarks about the steps you listed:
call function Promise which return setTimeout.
Promise is called as a function, but more specifically as a constructor
It doesn't return setTimeout, but ... a promise.
setTimeout added to Macrotask.
setTimeout is not added to a queue. setTimeout is executed, with res and a delay as argument. It returns immediately. The host has registered the callback and timeout using non-JavaScript technology. There is nothing about this in the queues yet.
function main add to Microtask
More precisely, the current execution context of main is saved. Once the awaited promise has resolved, only then will a job be put in the microtask queue with the purpose to resume the suspended execution of main.
Detailed steps
Here are the steps 3, 4 and 5 in more detail:
The Promise constructor is called, which immediately calls the callback given as argument.
setTimeout is called, which registers its callback argument res for later execution.
The setTimeout call immediately returns. Its return value is a unique timer ID, but that value is ignored by the Promise constructor
The Promise constructor returns a promise that is in a pending state.
The await saves the current execution context of main, and makes the main function return with a pending promise.
This is the end of this synchronous execution cycle, and JavaScript now monitors the job queues.
In a following step we have this:
The setTimeout implementation (non-JavaScript) sees that the 1000 delay has passed and places a job in a (macrotask) job queue.
The JavaScript engine gets the job from that queue, and executes it. In this case it means it executes res (starting from an empty call stack).
The call of res will fulfill the promise that was created with new Promise. This creates a job in a (microtaks) job queue to restore the execution context of main.
res returns and the callstack is empty again.
And then, as part of the same "task":
JavaScript reads the microtask queue and executes that job: the main execution context is restored (put on the callstack), with variable i equal to 0 and execution continuing with console.log
The loop makes its second iteration, and the operand of await is evaluated again, calling again the Promise constructor (see the steps at the top). The difference here is that when await makes main return (again) it is returning from a "resume" call that was made from a job, not from JavaScript code. This will be the case for all next executions of await.
And so it continues for all iterations of the (asynchronous) loop. When the loop has completed:
The resumed main function ends with an implicit return undefined. This resolves the promise that main had returned when it had executed await for the first time -- when it returned to the top-level script where main(); had been called.
As there is no code that is awaiting that promise to be resolved, nothing more happens.
This question already has answers here:
Correct way to write a non-blocking function in Node.js
(2 answers)
Closed 8 months ago.
function main() {
console.log("S-1");
setTimeout(() => {
console.log("setTimeout");
}, 0);
new Promise((resolve, reject) => {
for (let i = 0; i < 10000000000; i++) {}
resolve("Promise");
}).then((res) => console.log(res));
console.log("S-2");
}
// Invoke function
main();
When I run the main() function, the output I get is:
'S-1'
'S-2'
'Promise'
'setTimeout'
After 'S-1' is logged, there is a big time gap, and then after few seconds, the rest gets logged, at once.
Shouldn't the time gap occur after logging 'S-2'?
The function inside the Promise constructor runs synchronously, on the same (and only) main thread as the rest of the script. Constructing a Promise doesn't result in a separate thread being created.
If you do some heavy processing immediately inside a Promise constructor, that heavy processing will have to complete before control flow is yielded back to the outside of the Promise constructor. In this case, it means that the many iterations of the loop must finish before:
the constructor finishes completely
The constructor resolves to a Promise
That Promise gets a .then handler attached to it
Finally, the console.log("S-2"); line runs
To get the output you're desiring or expecting without moving the logs around, you'd have to offload the expensive code to a different environment, such as to a worker or a server (using a network request).
As #CertainPerformance had said, Javascript can only handle one call-stack. It needs to finish this thread once and for all. For promises and timeouts, it uses the job and task queues. The event loop prioritizes the job queue and then the task queue.
Lets try to understand what you wrote in terms of call-stack and queues.
function main() {
// 1. execute straight forward
console.log("S-1");
// 2. sets / schedules a timer, the anonymous function is then stored in the callback
setTimeout(() => {
console.log("setTimeout");
}, 0);
// 3. create a new Promise, with the constructor acting as executor
// executes immediately, as its designed to set the state of the Promise for the then() to react to.
new Promise((resolve, reject) => {
for (let i = 0; i < 10000000000; i++) {}
// 3.5 the resolve signals that the execution is finished
// either returns the promise or passes through the then()
resolve("Promise");
// 4. after the promise execution has been resolved,
// then() creates another anonymous function, and stores in the callback.
// at this point there are 2 entries in the callback
}).then((res) => console.log(res));
// 5. once the above Promise executor has been executed, execute this
console.log("S-2");
// By now, since both the timer and promise execution has been done,
// the timeout is put into the Task Queue and the Promise then() is put into the Job Queue.
}
// 0. puts this into the call stack and execute
main();
// 6. The event loop deques all of the call stack from the job queue and puts it into the main call stack to execute.
// hence, the then() is executed first
// 7. The event loop then deques all of the call stack from the task queue and puts it into the main call stack to execute.
// hence, the timeout function is executed next.
If you want to make it a callback.
function main() {
console.log("S-1");
setTimeout(() => {
console.log("setTimeout");
}, 0);
new Promise((resolve, reject) => {
setTimeout(() => {
resolve("promise");
}, 1000);
}).then((res) => console.log(res));
console.log("S-2");
}
// Invoke function
main();
Reference: https://medium.com/#idineshgarg/let-us-consider-an-example-a58bb1c11f55
I have the following:
for (let job of jobs) {
resets.push(
new Promise((resolve, reject) => {
let oldRef = job.ref
this._sequenceService.attachRef(job).then(() => {
this._dbService.saveDoc('job', job).then(jobRes => {
console.log('[%s / %s]: %s', oldRef, jobRes['jobs'][0].ref, this._moment.unix(job.created).format('DD/MM/YYYY HH:mm'))
resolve()
}).catch(error => {
reject(error)
})
}).catch(error => {
reject(error)
})
})
)
}
return Promise.all(resets).then(() => {
console.log('Done')
})
this._sequenceService.attachRef has a console.log() call.
When this runs, I am seeing all the console logs from the this._sequenceService.attachRef() call and then I see all the logs in the saveDoc.then() call. I was expecting to see them alternate. I understand that according to this article, promises don't resolve in order but I wouldn't expect my promise to resolve until I call resolve() so would still expect alternating logs, even if not in order.
Why is this not the case?
Your code can be written a lot cleaner by avoiding the promise anti-pattern of wrapping a promise in a new manually created promise. Instead, you just push the outer promise into your array and chain the inner promises to the outer one by returning them from inside the .then() handler. It can all be done as simply as this:
for (let job of jobs) {
let oldRef = job.ref;
resets.push(this._sequenceService.attachRef(job).then(() => {
// chain this promise to parent promise by returning it
// inside the .then() handler
return this._dbService.saveDoc('job', job).then(jobRes => {
console.log('[%s / %s]: %s', oldRef, jobRes['jobs'][0].ref, this._moment.unix(job.created).format('DD/MM/YYYY HH:mm'));
});
}));
}
return Promise.all(resets).then(() => {
console.log('Done')
}).catch(err => {
console.log(err);
});
Rejections will automatically propagate upwards so you don't need any .catch() handlers inside the loop.
As for sequencing, here's what happens:
The for loop is synchronous. It just immediately runs to completion.
All the calls to .attachRef() are asynchronous. That means that calling them just initiates the operation and then they return and the rest of your code continues running. This is also called non-blocking.
All .then() handlers are asynchronous. The earliest they can run is on the next tick.
As such this explains why the first thing that happens is all calls to .attachRef() execute since that's what the loop does. It immediately just calls all the .attachRef() methods. Since they just start their operation and then immediately return, the for loop finishes it's work pretty quickly of launching all the .attachRef() operations.
Then, as each .attachRef() finishes, it will trigger the corresponding .saveDoc() to be called.
The relative timing between when .saveDoc() calls finish is just a race depending upon when they got started (how long their .attachRef() that came before them took) and how long their own .saveDoc() call took to execute. This relative timing of all these is likely not entirely predictable, particularly if there's a multi-threaded database behind the scenes that can process multiple requests at the same time.
The fact that the relative timing is not predictable should not be a surprise. You're running multiple two-stage async operations purposely in parallel which means you don't care what order they run or complete in. They are all racing each other. If they all took exactly the same time to execute with no variation, then they'd probably finish in the same order they were started, but any slight variation in execution time could certainly change the completion order. If the underlying DB gets has lock contention between all the different requests in flight at the same time, that can drastically change the timing too.
So, this code is designed to do things in parallel and notify you when they are all done. Somewhat by definition, that means that you aren't concerned about controlling the precise order things run or complete in, only when they are all done.
Clarifying my comments above:
In this case the promises are at four levels.
Level 0: Promise created with Promise.all
Level 1: Promise created with new Promise for each Job
Level 2: Promise as generated by this._sequenceService.attachRef(job)
Level 3: Promise as generated by this._dbService.saveDoc('job', job)
Let's say we have two jobs J1 and J2
One possible order of execution:
L0 invoked
J1-L1 invoked
J2-L1 invoked
J1-L2 invoked
J2-L2 invoked
J1-L2 resolves, log seen at L2 for J1
J1-L3 invoked
J2-L2 resolves, log seen at L2 for J2
J2-L3 invoked
J1-L3 resolves, log seen at L3 for J1
J1-L1 resolves
J2-L3 resolves, log seen at L3 for J2
J2-L1 resolves
L0 resolves, 'Done' is logged
Which is probably why you see all L2 logs, then all L3 logs and then finally the Promise.all log
When I resolve the promise, the function should continue being executed:
function test(){
return new Promise(resolve=>{
resolve(5)
setTimeout(()=>{console.log(7)}, 2000}
})
}
And then I invoke the function like this
test().then(console.log);
So it prints 5, and then after 2 seconds, it prints 7, as I expected.
But if I work with sync thing after resolve() it works differently:
function test(){
return new Promise(resolve=>{
resolve(5)
for(let i = 0; i < 100; i++)
console.log(i)
})
}
Now it prints the numbers from 0 to 100, and then it prints 5. But I expected it to return 5 through resolve() and only then print all the rest stuff.
The code gives the expected outcome. When you resolve immediately with 5, the function isn't finished executing yet, so it runs the rest of the code. Once done, the call stack is empty and the asynchronous then gets its chance to run. The difference between the two programs is that the second one logs immediately because it does not have to wait for the function to finish to execute. It's synchronous.
I think it has to do with javascript's event loop. With the link you can watch amazing video about event loop in JS.
I'm not sure, but it seems like what's inside resolve() is called asynchronously, but for loop is sync, that's why what's inside resolve is put into a queue and for loop is executed immediately. After "normal" execution process finished, resolve does it's job, returning 5.
In the example with setTimeout you've put the code into queue, but for later(2000ms) execution. Since 5 doesn't do anything, it's returned right away(asynchronously though) and after 2 seconds your code is run. Wath the video.
I figured it out.
Nodejs runs code from left to right.
First it gives me a function with resolve reject as arguments, then it executes this function. It resolves the value. Then it encounters a synchronous for loop, therefore it has to execute it synced. Then it returns an already resolved promise that I apply method then on. then() executes the callback when the promise is resolved. It is already resolved. And it prints my resolved value
If you use callbacks it will be different:
function test(callback){
callback(7)
for(let i = 0; i<100; i++){
console.log(i)
}
}
it will first, print(7) and only then all the rest numbers
But it works only if you know that the callback will be invoked once. Because otherwise, it will break the logic.
So, a few things I wanted to make sure we're on the same page. Your test function's setTimeout is asynchronous. Yes, it's inside a promise, but you've sent it to the event loop in an uncontrolled manner.
setTimeout(()=>{console.log(7)}, 2000} //not controlled in your first function
Whatever you place in resolve is the only thing that gets controlled by the promise. The way you're using the promise is not how they are supposed to be used. You might have to go back to basics and read up on it some more if this confuses you.
edit: I seemed to have no articulated myself properly, and thus have garnered quite a bit of salt. Better explanation:
You should treat a promise how you would a return statement. A promise function should not perform any other output operation except a single resolve statement. This can then be synchronously controlled in subsequent operations in another function. Key word is another.
function test(){
return new Promise(resolve=>{
resolve(5)
})
}
function loopTest() {
test().then( (num) => {
console.log(num)
for (let i = 0; i < 10; i++) {
console.log(i)
}
})
} // what you were trying to actually accomplish
loopTest()
I am learning Promise, in order to understand it I read a bit about Event loop of JavaScript. This article briefly introduced the working of event loop such as call stack, event table and message queue.
But I don't know how the call stack deal with the line containing 'return', and what happens thereafter.
Below is an example that I wrote to hopefully understand how Promise works based on event loop. Also see http://jsbin.com/puqogulani/edit?js,console if you want to give it a go.
var p1 = new Promise(
function(resolve, reject){
resolve(0);
});
p1.then(function(val){
console.log(val);
p1.then(function(){
console.log("1.1.1");
p1.then(function(){
console.log("1.1.2");
p1.then(function(){
console.log("1.1.3");
});
});
});
p1.then(function(){
console.log("1.2");
})
return 30;
//return new Promise(function(resolve, reject){
// resolve(30);
//});
})
.then(function(val){
console.log(val/2);
});
p1.then(function(){
console.log("2.1");
});
console.log("Start");
As can be seen, there are two "return", using each of them will give a different output order. Specifically, when using return 30;, 1.1.2, 1.1.3 are after 15, but when using return new Promise(...), 1.1.2, 1.1.3 are before 15. So what exactly happened when the code reached two different 'return'?
The difference is described in http://promisesaplus.com/ under the promise resolution procedure.
For the first return value:
2.3.3.4 If then is not a function, fulfill promise with x.
For the second:
2.3.2 If x is a promise, adopt its state [3.4]:
2.3.2.2 If/when x is fulfilled, fulfill promise with the same value.
We can see this implemented q.js. This is one possible implementation but seems to explain the delay:
function coerce(promise) {
var deferred = defer();
Q.nextTick(function () {
try {
promise.then(deferred.resolve, deferred.reject, deferred.notify);
} catch (exception) {
deferred.reject(exception);
}
});
return deferred.promise;
}
When returning a promise from the then function, we have two separate promise objects: the one returned from the function passed to then, and the one returned from then. These need to be connected together so that resolving the first, resolves the second. This is done with promise.then(deferred.resolve, ...)
The first delay comes from Q.nextTick. This executes the function on the next iteration of the event loop. There's some discussion in the commit comments on why it's needed.
Calling promise.then adds a further delay of one iteration of the event loop. As required in the spec:
2.2.4 onFulfilled or onRejected must not be called until the execution context stack contains only platform code. [3.1].
The execution would go something like:
p1.then with function containing 1.1.1 is called
function containing 1.1.1 is queued
p1.then with function containing 1.2 is called
function containing 1.2 is queued
Promise resolving to 30 is returned
Q.nextTick function is queued
----------
1.1.1 is printed
p1.then with function containing 1.1.2 is called
function containing 1.1.2 is queued
1.2 is printed
Q.nextTick function is executed
promise.then(deferred.resolve, ...) is queued
----------
1.1.2 is printed
p1.then with function containing 1.1.3 is called
function containing 1.1.3 is queued
promise.then(deferred.resolve, ...) is executed
function containing val/2 is queued
----------
1.1.3 is printed
val/2 is printed