function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
let p = sleep(50);
p.then(() => console.log('a')).then(() => console.log('c'));
p.then(() => console.log('b')).then(() => console.log('d'));
Is this guaranteed to print "a, b, c, d" in that order?
As far as I can tell, "a" has to fire before "c" and "b" has to fire before "d", but beyond that, can the JS interpreter decide to execute the remainder in a different order?
The way that things are queued using setTimeout is exactly that - a queue. If two callbacks are queued with the same 'delay', the callback that was queued first, will fire first.
Edit: I failed to understand the OP's intention initially.
'Branching' promises is what is actually occurring here. Meaning - the 'then-able' being referenced in the first set of then-ables (for a & b) will fire the two provided callbacks at 'the same time' because they both reference the same promise - however - the tricky bit is that they execute in the order that they were queued using the .then(...) of the resolving promise object.
Then the following/subsequent callbacks are queued in their respective orders (c & d).
To answer the question directly: No. The nature of the async actions in a then-able could be anything. However, the functions provided in the OP's then-ables are essentially synchronous, resulting in the intuitive - but entirely misleading - logging order.
As far as I can tell, "a" has to fire before "c" and "b" has to fire before "d"
Yes, that much for certan.
beyond that, can the JS interpreter decide to execute the remainder in a different order?
Depends on whom you ask, but no there are more guarantees that make the output more predictable:
the Promise/A+ specification also demands that "all respective callbacks must execute in the order of their originating calls to then." So in your example, that means "a" has to fire before "b", because the callback was chained first to p.
the ECMAScript specification defines a job queue for promise callbacks, nailing down the order (imo unnecessarily). It conforms to Promises/A+ in that the "a" and "b" callbacks are queued in the order they were set up when the promise fulfills. It also means that after "a" returns and fulfills the promise, it will schedule "c", and after "b" returns and fulfills the promise, it will schedule "d", so their order is determined as well (for synchronous callbacks).
In general, don't rely on any scheduling algorithm, if you require a certain order then make it explicit.
Related
Within my Node.JS application I have written a function (findByReference) that goes to a database and asynchronously yields a fetched database row. I have written this function using Promises. Additionally, I have written an implementation of the Maybe monad and want my findByReference function to yield an instance of Maybe.
My code looks like the below:
findByReference(r)
.then(raw => raw ? Just(raw) : Nothing())
.then(row => {
(row instanceof Maybe) === true;
});
Without going into what Just and Nothing mean, the implication of this (because of how I've written Maybe) is that the row variable in the above code has a function on it called "then". To cut a long story short, it appears that Javascript is getting confused and is for some reason automatically calling MY "then" and instead of passing the Maybe is actually passing to the callback whatever MY "then" returns as the value of row. This is obviously leading to all manner of weird behaviour. If I simply remove the "then" function from my object then it all works as expected.
I am aware that if a Promise.then returns another Promise, then execution will pause until that promise is resolved. I have been unable to find any official documentation to back this up, but is it the case that this decision is simply based on the existence of a "then" function (the closest I have found is this https://developers.google.com/web/fundamentals/primers/promises which refers to the return value as "something Promise-like"). If this is the case, it would be my understanding that "then" as a function name is basically a reserved word in Javascript? I have seen other implementations of Maybe (such as this one https://www.npmjs.com/package/data.maybe) that use the word "chain" for a similar thing - I wondered if this is why?
Can anyone shed any light on if my deduction here is correct and if so is there any workaround I can use other than renaming my function?
FYI the only other SO question I've found that touches this problem is this one - Resolve promise with an object with a "then" function - but since that is angular-specific I don't believe this is a duplication.
Thanks in advance!
...the row variable in the above code has a function on it called "then". To cut a long story short, it appears that Javascript is getting confused and is for some reason automatically calling MY "then"...
It's not confused. :-) This is the definition of how promises work. JavaScript's promises work according to the Promises/A+ specification, which uses this terminology:
1.1 “promise” is an object or function with a then method whose behavior conforms to this specification.
1.2 “thenable” is an object or function that defines a then method.
If you have an object passing through a promise chain that's a thenable but not a promise, it's incompatible with promises.
So yes, in a sense, the then property of objects passing through promise chains is "reserved" by the Promises/A+ spec. You'll need to wrap your raw value in an object that doesn't have a then (and then unwrap it later). Or if you can, rename then in your design to remove the conflict.
Is increment an atomic operation in JavaScript? If one thread is accessing
++i; and at the same time another one starts to access the operation will there be any problems?
In JavaScript, a function always runs to completion. That means if a function is running, then it will run completely; only after that, the other function will be called. So, there is no chance of interleaving between statements (but in case of Java it is different).
If you are confused with asynchronous execution, then always remember async means later not parallel. So, coming to your problem, the answer is, No you will not face any problem, it will be a total atomic operation.
Javascript is single threaded, So you need to worry about deallocks or dirty read problems.
Why doesn't JavaScript support multithreading?
If one thread is accessing ++i; and at the same time another one starts to access the operation will there be any problems?
That won't happen with a simple variable like i, because JavaScript is defined such that there can only be one active thread (the agent's executing thread) in a realm at any given time. ("realm" - roughly speaking, a JavaScript global environment and the things within it, such as your variables.) So the issue simply doesn't arise with normal variables or object properties. Your function can't be interrupted during synchronous operation; JavaScript defines "run to completion" semantics: whenever a "job" (like triggering an event handler) is picked up from the job queue and executed, it runs to completion before any other job can be executed. (For an async function, the logic can only be suspended at await, return, or throw, not in the middle of a synchronous compound arithmetic operation. It can be in the middle of a compound arithmetic operation involving await. More about this below.)
The only place you'd have to worry about this is if you're using shared memory, where the actual memory is shared between realms and thus could indeed be accessed by multiple threads at the same time. But if you were doing that, you'd be dealing with a SharedArrayBuffer or a typed array using a SharedArrayBuffer, not a simple variable or property. But yes, if dealing with shared memory, you're exposed to all the "glorious" fun of CPU operation reordering, stale caches, and so on. That's part of why we have the Atomics object, including Atomics.add, which atomically adds a value to an element in a typed array using shared memory. (But beware naïve usage! After all, another thread could overwrite the value just after your add finishes, before you read it...) Atomics provides the bare building blocks necessary to ensure safe access to shared memory. (More about this in Chapter 16 of my book JavaScript: The New Toys: "Shared Memory and Atomics".)
Note: This all applies to standard JavaScript engines that comply with the specification, such as those found in web browsers and Node.js. Non-standard JavaScript environments, such as the the scripting support for JavaScript built into the Java Virtual Machine, may (of course) define alternative non-standard semantics.
Re async functions: There is no multithreading involved in async functions. But the fact the logic of the function is suspended at an await can cause some surprising behavior. That said, the place it can occur is clearly flagged with the await.
I wouldn't worry about the details below unless you have to. But for those who do...
Consider:
let a = 1;
async function one() {
return 1;
}
async function example() {
console.log(`Adding 1 to a`);
a += await one();
}
console.log(`Start, a = ${a}`);
Promise.all([
example(),
example(),
example(),
])
.then(() => {
console.log(`All done, a = ${a}`);
});
(Technically, we could just use a += await 1;, because await will wrap its operand in an implied Promise.resolve(x), but I thought it would be clearer to show an actual promise.)
That outputs:
Start, a = 1
Adding 1 to a
Adding 1 to a
Adding 1 to a
All done, a = 2
But wait, we added 1 to a three times, it should have ended up being 4, not 2?!?!
The key is in the await in this statement:
a += await one();
That's processed like this:
Get the current value of a and set it aside; call it atemp.
Call one and get its promise.
Wait for the promise to settle and get the fulfillment value; let's call that value addend.
Evaluate atemp + addend.
Write the result to a.
Or in code:
/* 1 */ const atemp = a;
/* 2 */ const promise = one();
/* 3 */ const addend = await promise; // Logic is suspended here and resumed later
/* 4 */ const result = atemp + addend;
/* 5 */ a = result;
(You can find this detail in EvaluateStringOrNumericBinaryExpression in the spec.)
Where we used example, we called it three times without waiting for its promise to settle, so Step 1 was run three times, setting aside the value of a (1) three times. Then later, those saved values are used and a's value is overwritten.
Again, there was no multithreading involved (and run-to-completion is fully intact), it's just that when an async function's logic reaches an await, return (explicit or implicit), or throw, the function exits at that point and returns a promise. If that was because of an await, the function's logic will continue when the promise the function awaited settles, and will (in the normal course of things) eventually settle the promise the async fucntion returned. But each of those things will happen to completion, and on a single active thread.
Javascript does not support multithreading. It may have web workers, but your question would not apply to this case, as workers do not share variables.
Yes there will be a problem. Even if Javascript is single threaded, i++ is (read + modify + write) 3 steps operation, So when anyone is taking the i variable from asyncfunction, the other one can set the modified i to the variable area. In order to solve this issue, you can use atomic variables instead of regular variable. When one process take the number with read another process can change the number with write during the first one modifying. No matter this is single thread or multithread, all the things happen the same. All the things I said here are retated to asyncfunction on Node.js.
In this answer, a promise chain is built recursively.
Simplified slightly, we have :
function foo() {
function doo() {
// always return a promise
if (/* more to do */) {
return doSomethingAsync().then(doo);
} else {
return Promise.resolve();
}
}
return doo(); // returns a promise
}
Presumably this would give rise to a call stack and a promise chain - ie "deep" and "wide".
I would anticipate a memory spike larger than either performing a recursion or building a promise chain alone.
Is this so?
Has anyone considered the memory issues of building a chain in this way?
Would memory consumption differ between promise libs?
a call stack and a promise chain - ie "deep" and "wide".
Actually, no. There is no promise chain here as we know it from doSomeThingAsynchronous.then(doSomethingAsynchronous).then(doSomethingAsynchronous).… (which is what Promise.each or Promise.reduce might do to sequentially execute handlers if it was written this way).
What we are facing here is a resolve chain1 - what happens in the end, when the base case of the recursion is met, is something like Promise.resolve(Promise.resolve(Promise.resolve(…))). This is only "deep", not "wide", if you want to call it that.
I would anticipate a memory spike larger than either performing a recursion or building a promise chain alone.
Not a spike actually. You'd slowly, over time, build a bulk of promises that are resolved with the innermost one, all representing the same result. When, at the end of your task, the condition is fulfilled and the innermost promise resolved with an actual value, all of these promises should be resolved with the same value. That would end up with O(n) cost for walking up the resolve chain (if implemented naively, this might even be done recursively and cause a stack overflow). After that, all the promises except for the outermost can become garbage collected.
In contrast, a promise chain that is built by something like
[…].reduce(function(prev, val) {
// successive execution of fn for all vals in array
return prev.then(() => fn(val));
}, Promise.resolve())
would show a spike, allocating n promise objects at the same time, and then slowly resolve them one by one, garbage-collecting the previous ones until only the settled end promise is alive.
memory
^ resolve promise "then" (tail)
| chain chain recursion
| /| |\
| / | | \
| / | | \
| ___/ |___ ___| \___ ___________
|
+----------------------------------------------> time
Is this so?
Not necessarily. As said above, all the promises in that bulk are in the end resolved with the same value2, so all we would need is to store the outermost and the innermost promise at one time. All intermediate promises may become garbage-collected as soon as possible, and we want to run this recursion in constant space and time.
In fact, this recursive construct is totally necessary for asynchronous loops with a dynamic condition (no fixed number of steps), you cannot really avoid it. In Haskell, where this is used all the time for the IO monad, an optimisation for it is implemented just because of this case. It is very similar to tail call recursion, which is routinely eliminated by compilers.
Has anyone considered the memory issues of building a chain in this way?
Yes. This was discussed at promises/aplus for example, though with no outcome yet.
Many promise libraries do support iteration helpers to avoid the spike of promise then chains, like Bluebird's each and map methods.
My own promise library3,4 does feature resolve chains without introducing memory or runtime overhead. When one promise adopts another (even if still pending), they become indistinguishable, and intermediate promises are no longer referenced anywhere.
Would memory consumption differ between promise libs?
Yes. While this case can be optimised, it seldom is. Specifically, the ES6 spec does require Promises to inspect the value at every resolve call, so collapsing the chain is not possible. The promises in the chain might even be resolved with different values (by constructing an example object that abuses getters, not in real life). The issue was raised on esdiscuss but remains unresolved.
So if you use a leaking implementation, but need asynchronous recursion, then you better switch back to callbacks and use the deferred antipattern to propagate the innermost promise result to a single result promise.
[1]: no official terminology
[2]: well, they are resolved with each other. But we want to resolve them with the same value, we expect that
[3]: undocumented playground, passes aplus. Read the code at your own peril: https://github.com/bergus/F-Promise
[4]: also implemented for Creed in this pull request
Disclaimer: premature optimization is bad, the real way to find out about performance differences is to benchmark your code, and you shouldn't worry about this (I've only had to once and I've used promises for at least 100 projects).
Is this so?
Yes, the promises would have to "remember" what they're following, if you do this for 10000 promises you'd have a 10000 long promise chain, if you don't then you won't (for example, with recursion) - this is true for any queueing flow control.
If you have to keep track of 10000 extra things (the operations) then you need to keep memory for it and that takes time, if that number is a million it might not be viable. This varies among libraries.
Has anyone considered the memory issues of building a chain in this way?
Of course, this is a big issue, and a use case for using something like Promise.each in libraries like bluebird over thenable chaining.
I've personally had in my code to avoid this style for a quick app that traverses all the files in a VM once - but in the vast majority of cases it's a non issue.
Would memory consumption differ between promise libs?
Yes, greatly. For example bluebird 3.0 will not allocate an extra queue if it detects a promise operation is already asynchronous (for example if it starts with a Promise.delay) and will just execute things synchronously (because the async guarantees are already preserved).
This means that what I claimed in my answer to the first question isn't always true (but is true in the regular use case) :) Native promises will never be able to do this unless internal support is provided.
Then again - it's no surprise since promise libraries differ by orders of magnitude from one another.
I just came out a hack that may help solving the problem: don't do recursion in the last then, rather, do it in the last catch, since catch is out of the resolve chain. Using your example, it would be like this:
function foo() {
function doo() {
// always return a promise
if (/* more to do */) {
return doSomethingAsync().then(function(){
throw "next";
}).catch(function(err) {
if (err == "next") doo();
})
} else {
return Promise.resolve();
}
}
return doo(); // returns a promise
}
To complement the awesome existing answers I'd like to illustrate the expression, which is the result of such an asynchronous recursion. For the sake of simplicity I use a simple function that computes the power of a given base and exponent. The recursive and base case are equivalent to those of the OP's example:
const powerp = (base, exp) => exp === 0
? Promise.resolve(1)
: new Promise(res => setTimeout(res, 0, exp)).then(
exp => power(base, exp - 1).then(x => x * base)
);
powerp(2, 8); // Promise {...[[PromiseValue]]: 256}
With the help of some substitution steps the recursive portion can be replaced. Please note that this expression can be evaluated in your browser:
// apply powerp with 2 and 8 and substitute the recursive case:
8 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 8)).then(
res => 7 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 7)).then(
res => 6 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 6)).then(
res => 5 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 5)).then(
res => 4 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 4)).then(
res => 3 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 3)).then(
res => 2 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 2)).then(
res => 1 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 1)).then(
res => Promise.resolve(1)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2); // Promise {...[[PromiseValue]]: 256}
Interpretation:
With new Promise(res => setTimeout(res, 0, 8)) the executor is invoked immediately and executes a non-bllocking computation (mimicked with setTimeout). Then an unsettled Promise is returned. This is equivalent with doSomethingAsync() of the OP's example.
A resolve callback is associated with this Promise via .then(.... Note: The body of this callback was substituted with the body of powerp.
Point 2) is repeated and a nested then handler structure is build up until the base case of the recursion is reached. The base case returns a Promise resolved with 1.
The nested then handler structure is "unwound" by calling the associated callback correspondingly.
Why is the generated structure nested and not chained? Because the recursive case within the then handlers prevents them from returning a value until the base case is reached.
How can this work without a stack? The associated callbacks form a "chain", which bridges the successive microtasks of the main event loop.
This promise pattern will generate a recursive chain. So, each resolve() will create a new stack frame (with its own data), utilizing some memory. This means that large number of chained functions using this promise pattern can produce stack overflow errors.
To illustrate this, I'll use a tiny promise library called Sequence, which I've written. It relies on recursion to achieve sequential execution for chained functions:
var funcA = function() {
setTimeout(function() {console.log("funcA")}, 2000);
};
var funcB = function() {
setTimeout(function() {console.log("funcB")}, 1000);
};
sequence().chain(funcA).chain(funcB).execute();
Sequence works great for small/medium sized chains, in the range of 0-500 functions. However, at about 600 chains Sequence starts degradating and generating often stack overflow errors.
The bottom line is: currently, recursion-based promise libraries are more suitable for smaller/medium sized function chains, while reduce-based promise implementations are ok for all cases, including larger chains.
This of course doesn't mean that recursion-based promises are bad. We just need to use them with their limitations in mind. Also, it's rare that you'll need to chain that many calls (>=500) via promises. I typically find myself using them for async configurations which utilize heavily ajax. But even if the most complex cases I haven't seen a situation with more than 15 chains.
On a side note...
These statistics were retrieved from tests performed with another of my libraries - provisnr - which captures the achieved number of function calls within a given interval of time.
This question already has answers here:
How do I access previous promise results in a .then() chain?
(17 answers)
Closed 8 years ago.
What is the correct pattern, when coding with promises, to access data coming from long before in a chain of promises?
For example:
do_A.then(do_B).then(do_C).then(do_D).then(do_E_WithTheDataComingFrom_A_And_C_OnlyWhen_D_IsSuccesfullyCompleted)
My current solution: passing along a single JSON structure through the chain, and let each step populate it.
Any opinion about that?
I don't think there's one "correct" pattern for this. Your solution sounds neat, however, it's a bit tightly coupled. It may work great for your situation, but I see a few problems with it as a general pattern:
Participating steps need to agree on the structure of the collector object.
Every step needs to participate in at least the forwarding of the object, which can get tedious if the chain is long and the need for previous data occurs sporadically. This is also inflexible to insertion of steps not written by you (chaining doesn't always happen linearly).
Said differently: Unless do_A|B|C|D and do_E are under your control, you'll need to wrap them with boilerplate to store off your collector object in a closure and convert to and from the functions' natural inputs and results, since the functions wont be "in on" your pattern.
On the other hand, if the functions are in on it, then the data-dependencies between the steps have effectively become hidden inside the functions. This may look clean, but it could become a maintenance problem. For example: if this is a team project, then someone might think they can re-order your steps, absent any no clue in the call-pattern what inputs do_E requires.
I would suggest a more straightforward approach using closures:
var a, c;
do_A()
.then(function(result) { a = result; return do_B(); })
.then(do_C)
.then(function(result) { c = result; return do_D(); })
.then(function() {
return do_E_WithTheDataComingFrom_A_And_C_OnlyWhen_D_Succeeds(a, c);
})
.catch(failed);
There's no collector object to define; do_A|B|C|D and do_E can be generic functions without knowledge of any pattern; there's no boilerplate unless returned data is relied on (do_B and do_D); and the data-dependencies (a and c) are explicit.
Yes, this is the correct way to chain state with actions.
Chaining .then statements is very common and is usually our building block when piping things around. It's at the very core of promises.
What you're doing is both correct and idiomatic.
For the curious spirit let's show this.
In order to verify this - we can check the promises specification.
We want to verify that:
It chains
In the case of a rejection, it doesn't call the handler in the chained then
It rejects the next promise returned from then with the same reason
It executes in sequence passing return value.
Let's verify these in order, using the specification - in particular .then:
1. It chains
7.1 then must return a promise [3.3].
Great, let's verify that it also chains on fullfillment
If either onFulfilled or onRejected returns a value x, run the Promise Resolution Procedure >[[Resolve]](promise2, x).
Great, so we know that when our promise resolves or rejects then our then handler is called with the appropriate parameter. So .then(do_A).then(do_B) will always work assuming do_A resolves.
2. In the case of a rejection, it doesn't call the handler in the chained then
7.iv. If onRejected is not a function and promise1 is rejected, promise2 must be rejected with the same reason.
Great, so it rejects and calls onRejected if it's there, if it doesn't it chains.
3. It rejects the next promise returned from then with the same reason
We just covered that in 2.
4. It executes in sequence passing return value.
That is again
If either onFulfilled or onRejected returns a value x, run the Promise Resolution Procedure [[Resolve]](promise2, x).
So, if you set onFulfilled it'll run the resolution process. The resolution process itself dictates:
The promise resolution procedure is an abstract operation taking as input a promise and a value, which we denote as [[Resolve]](promise, x). If x is a thenable, it attempts to make promise adopt the state of x, under the assumption that x behaves at least somewhat like a promise. Otherwise, it fulfills promise with the value x.
If/when resolvePromise is called with a value y, run [[Resolve]](promise, y).
Where y is the return value of x.
Great! so it works.
I have a simple program in node.js, such as:
// CODE1
step1();
step2();
var o = step3();
step4(o);
step5();
step6();
this program is meant to be run in a stand-alone script (not in a web browser),
and it is a sequential program where order of execution is important (eg, step6 needs to be executed after step5).
the problem is that step3() is an async function, and the value 'o' is actually passed to a given callback,
so I would need to modify the program as follows:
// CODE2
step1();
step2();
step3( function (o) {
step4(o);
step5();
step6();
})
it could make sense to call step4 from the callback function, because it depends on the 'o' value computed by step3.
But step5 and step6 functions do not depend on 'o', and I have to include them in that callback function only to preserve the order of execution: step3, then step4, then step5 then step6.
this sounds really bad to me.
this is a sequential program, and so I would like to convert step3 to a sync function.
how to do this?
I am looking for something like this (eg using Q):
// CODE3
step1();
step2();
var deferred = Q.defer();
step3(deferred.resolve);
deferred.blockUntilFulfilled() // this function does not exist; this is what i am looking for
var o = deferred.inspect().value
step4(o);
step5();
step6();
How to do this?
ps: there are advantages and disadvantages of using sync or async, and this is a very interesting discussion. however, it is not the purpose of this question. in this question, i am asking how can i block/wait until a promise (or equivalent) gets fulfilled. Please, please, please, do not start a discussion on whether sync/blocking is good or not.
It's impossible to turn an async operation into a sync operation in vanilla JavaScript. There are things like node-fibers (a C++ add-on) that will allow this, or various compile-to-JS languages that will make the async operations look sync (essentially by rewriting your first code block to the second), but it is not possible to block until an async operation completes.
One way to see this is to note that the JavaScript event loop is always "run to completion," meaning that if you have a series of statements, they will always be guaranteed to execute one after the other, with nothing in between. Thus there is no way for an outside piece of information to come in and tell the code to stop blocking. Say you tried to make it work like so:
stepA();
stepB();
while (global.operationIsStillGoing) {
// do nothing
}
stepC();
This will never work, because due to the run-to-completion nature of JavaScript, it is not possible for anything to update global.operationIsStillGoing during the while loop, since that series of statements has not yet run to completion.
Of course, if someone writes a C++ addon that modifies the language, they can get around this. But that's not really JavaScript any more, at least in the commonly understood sense of ECMAScript + the event loop architecture.