If I have the following :
var deferred = Q.defer();
deferred.resolve();
var a = deferred.promise.then(function() {
console.log(1);
});
console.log(2);
...why do I see 2, then 1 in the console?
I understand this output is correct according to the Promises spec, which says to call the function on the next tick (e.g. setTimeout()), even if it is already resolved, but I don't understand why.
I would like to have code that calls then on a series of promises synchronously, assuming that all the promises have been resolved.
My real use case is that I am trying to use Angular's implementation, $q, and I want all of the then callbacks to execute in the same $digest cycle, so that I don't get unnecessary subsequent $digest cycles.
Answer is consistency.
In real code, you don't have promises that are always immediately resolved when created, they would be pointless. So you have promises that sometimes may be immediately resolved.
In that case, you don't want to have a different flow. You want always the same, predictable flow. So you want the next function to be always called on next tick.
Don't use a promise when you don't need one.
It's a design mistake based on a set of opinions and assumptions. It has become latched because it was rushed out without full technical verification in a design by committee process that also had the back pressure of a lot of vendors implementing their own already with the same mistake making it hard to backtrack.
Once a standard goes out for JS to the web it can be revoked, even if it's broken with the idea being that webpages shouldn't break. If someone wrote a page today, then died, it should be possible to still view it in your browser in five years. It would be very problematic if when browsing the web you kept bumping into pages that didn't work with your browser.
In very simple use cases it doesn't do much harm and removes confusion about whether something is maybe async.
For increasingly non-trivial use cases it causes increasingly more harm and adds confusion. Initially it appears to make it easier to reason about your code but it sacrifices the less trivial usage for the least trivial usage.
It's overall much easier to reason about your code if it doesn't run things in the next tick that don't need to be async at all. This is a case of choosing to damage the language at level two to cater to level one users, at the expense of level two users and above, rather than working to help ascend level one users to level two users. It's a condescending or pessimistic design decision.
There's an intermediate solution where it's possible to run each task as if it runs after the current code runs to completion but things are scheduled in the correct order. This hasn't been implemented and it's also debatable as compromises don't always produce the best solutions. This compromise produces performance issues for direct return callstacks.
The way promises work means that the forward callstack is depth first and run to completion (isolated) but the return callstack is breadth first and run in pieces interleaved with other return callstacks. These are two radically different concept and behaviours. Traditionally with callbacks both run in the same way.
It also means you can't naively replace callbacks with promises or anything based on promises. Callbacks give you more options which promises take away. If you replace callbacks with promises without taking this difference into account you can create code that has stability issues and potential security issues as well since this behaviour of unordering events can cause the current code flow to unexpectedly jump track.
You can't rely on order which means there are cases with promises where if you ask for lots of things when you get them back you have to double check they're the thing you asked for where as you would not need to do this with callbacks. You may also need to buffer and reorder events which you would not need to do with callbacks. It can make benchmarks unreliable as well if you're not careful to isolate the two things being run.
This can also create serious performance bottlenecks that you can't always easily prevent. If you use promises a hundred returned results returned at once from a single event to an iterator, each taking one second per return call and their promise resolve depth is two, they'll all be broken into half with the first halves all run then the second halves. That means it will take 50.5 seconds before anything can finish where as with callbacks after 50 seconds half of them would already be finished. If the result of the task is passed along to another external service for processing then it leaves that service standing idle for 50 seconds when it could have been processing your results. This makes the promises terrible for when you want both low latency and high throughput under services that take load demonstrating the weakness of the design.
Not being able to naively replace callbacks with promises is one of the most devastating consequences of this design mistake which is also carried over to async/await. If you want to convert a callback library you can't simple change the syntax, you must carefully scrutinise the semantics at every point.
There is no plan to fix this. You can create your own promises and generators can be used to provide the same kind of syntax as async/await but with the same predictable and high performance behaviour of callbacks. You may however have problems playing nice with other libraries that still rely on native promises.
Related
This is a question about performance more than anything else.
Node exposes three different types of methods to accomplish various filesystem tasks:
Promises API (async)
Callback API (async)
Synchronous API (sync)
I've read more articles and stackoverflow answers than I can count, all of which claiming to never need the sync methods.
I recently wrote a script which required a couple directories to be made if they didn't already exist. During this, I noticed that if I used the async/await methods (primarily fs.promises.mkdir and fs.promises.access), the event loop would simply continue to the next async bit of code, regardless of the fact that the next bits require those directories. This is expected behavior, after all, it's async.
I understand this could be solved with a nice little callback hell sesh, but that isn't the question, whereas the idea that the promises api can be used over all other methods is.
The question then becomes:
Is it ever better to use Node's filesystem sync methods over the same async methods?
Is it ever truly required in situations like this to block the process?
Or said differently:
Is it possible to completely avoid sync methods and ONLY use the promises api (NOT promises + callbacks)?
It seems like using the sync methods (given my situation above, where the directories are required to be there before any other call is made) can be EXTREMELY useful to write readable, clear code, even though it may negatively impact performance.
With that being said, there's an overwhelming level of information to say that the sync api is completely useless and never required.
Again, this purely caters to the promises api. Yes, callbacks and promises are both async, but the difference between the job and message queues makes the both api's completely different in this context.
PS: For additonal context on examples, I've provided a code sample so you don't have to imagine my example ;)
Thanks! :)
// Checks if dir exists, if not, creates it. (not the actual code, just an example)
// Sync version
if (!fs.existsSync(dirPath)) {
fs.mkdirSync(dirPath);
}
// Async version
try {
await fs.promises.access(dirPath);
} catch {
await fs.promises.mkdir(dirPath);
}
It depends on the situation. The main benefit of the sync methods is that they allow for easier consumption of their results, and the main disadvantage is that they prevent all other code from executing while working.
If you find yourself in a situation where other code not being able to respond to events is not an issue, you might consider it to be reasonable to use the sync methods - if the code in question has no chance of or reason for running in parallel with anything else.
For example, you would definitely not want to use the sync methods inside, say, a server handling a request.
If your code requires reading some configuration files (or creating some folders) when the script first runs, and there aren't enough of them such that parallelism would be a benefit, you can consider using the sync methods.
That said, even if your current implementation doesn't require parallelism, something to keep in mind is that, if the situation changes and you find that you do actually need to allow for parallel processing, you won't have to make any changes to your existing code if you had started out by using the promise-based methods in the first place - and if you understand the language, using the Promises properly should be pretty easy, so if there's a chance of that, you might consider using the Promises anyway.
In Java we have "threads", in CPython we have threads (non-concurrent) and "processes".
In JS, when I kick off an async function or method, how do I officially refer to these "strands of executing code"?
I have heard that each such code block executes from start to finish*, meaning that there is never any concurrent** processing in JS. I'm not quite sure whether this is the same situation as with CPython threads. Personally I hesitate to use "thread" for what we have in JS as these "strands" are so different from Java concurrent threads.
* Just to clarify in light of Stephen Cleary's helpful response: I mean "each such synchronous code block". Obviously if an await is encountered control is released ...
** And obviously never any "true parallel" processing. I'm following the widely accepted distinction between "concurrent" (only one thread at any one time, but one "strand of execution" may give way to another) and "parallel" (multiple processes, implementing true parallel processing, often using multiple CPUs or cores or processes). My understanding is that these "strands" in JS are not even concurrent: once one AJAX method or Promise or async method/function starts executing nothing can happen until it's finished (or an await happens)...
In JS, when I kick off an async function or method, how do I officially refer to these "strands of executing code"?
For lack of a better term, I've referred to these as "asynchronous operations".
I have heard that each such code block executes from start to finish
This was true... until await. Now there's a couple of ways to think about it, both of which are correct:
Code blocks execute exclusively unless there's an await.
await splits its function into multiple code blocks that do execute exclusively (each await is a "split point").
meaning that there is never any concurrent processing in JS.
I'd disagree with this statement. JavaScript forces asynchrony (before async/await, it used callbacks or promises), so it does have asynchronous concurrency, though not parallel concurrency.
A good mental model is that JavaScript is inherently single-threaded, and that one thread has a queue of work to do. await does not block that thread; it returns, and when its thenable/Promise completes, it schedules the remainder of that method (the "continuation") to the queue.
So you can have multiple asynchronous methods executing concurrently. This is exactly how Node.js handles multiple simultaneous requests.
My understanding is that these "strands" in JS are not even concurrent: once one AJAX method or Promise or async method/function starts executing nothing can happen until it's finished...
No, this is incorrect. await will return control to the main thread, freeing it up to do other work.
the way things operate in JS means you "never have to worry about concurrency issues, etc."
Well... yes and no. Since JavaScript is inherently single-threaded, there's never any contention over data shared between multiple threads (obviously). I'd say that's what the original writer was thinking about. So I'd say easily north of 90% of thread-synchronization problems do just go away.
However, as you noted, you still need to be careful modifying shared state from multiple asynchronous operations. If they're running concurrently, then they can complete in any order, and your logic has to properly handle that.
Ideally, the best solution is to move to a more functional mindset - that is, get rid of the shared state as much as possible. Asynchronous operations should return their results instead of updating shared state. Then these concurrent asynchronous operations can be composed into a higher-level asynchronous operation using async, await, and Promise.all. If possible, this more functional approach of returning-instead-of-state and function composition will make the code easier to deal with.
But there's still some situations where this isn't easily achievable. It's possible to develop asynchronous equivalents of classical synchronous coordination primitives. I worked up a proof-of-concept AsyncLock, but can't seem to find the code anywhere. Well, it's possible to do this, anyway.
To date 24 people, no more no fewer, have chosen to look at this.
FWIW, and should anyone eccentric take an interest in this question one day, I propose "hogging strands" or just "strands" ... they hog the browser's JS engine, it would appear, and just don't let go, unless they encounter an await.
I've been developing this app where 3 "strands" operate more or less simultaneously, all involving AJAX calls and a database query... but in fact it is highly preferable if the 2nd of the 3 executes and returns before the 3rd.
But because the 3rd query is quite a bit simpler for MySQL than the 2nd the former tends to return from its AJAX "excursion" quite a bit earlier if everything is allowed to operate with unconstrained asynchronicity. This then causes a big delay before the 2nd strand is allowed to do its stuff. To force the code to perform and complete the 2nd "strand" therefore requires quite a bit of coercion, using async and await.
I read one comment on JS asynchronicity somewhere where someone opined that the way things operate in JS means you "never have to worry about concurrency issues, etc.". I don't agree: if you don't in fact know when awaits will unblock this does necessarily mean that concurrency is involved. Perhaps "hogging strands" are easier to deal with than true "concurrency", let alone true "parallelism", however... ?
Say I have a sample code running in NodeJS
function (){
///OPERATION 1
///OPERATION 2
}
Considering none of the operations require any sort of time out, by default would javascript run both at the same time or finish operation 1 then operation 2?
No two lines of JavaScript ever run simultaneously within the same process. Ever.
See concurrency vs parallelism.
Asynchronous code in Node.js - assuming there are no worker processes involved - is always running with concurrency and never parallelism. Concurrency is easier to program and helps us build complex machinery quickly. To "scale up" you may need to bring in worker processes to do work in parallel.
In your example, if both operations are synchronous, they will run in the order they are written (1, 2). In fact, if operation 1 is synchronous it will always run first no matter what. If both are asynchronous, then how you experience it depends on how long they each take to complete! If operation 1 is asynchronous but operation 2 is synchronous, then they will seem to run in reverse order (2, 1). This has to do with the way functions get scheduled on the event loop, so understanding that will help this all make sense.
Take a breath. Time for a deep dive.
To be clear, in reality, lines of code always get executed in order. We don't have GOTO and the JavaScript engine does not mysteriously jump around to different places. The key thing to understand is that when someone says a function is asynchronous it is really also partly synchronous. Something is happening synchronously. Otherwise it would be an empty function. Instead, it just means that only a tiny bit of work is done synchronously, usually that simply consists of scheduling work for later, and then the rest of it happens later.
So above when I said that if they are both asynchronous then "it depends", it's merely the completion or result of that function (which you experience via a callback or Promise) whose order is undefined in relation to the completion of other asynchronous functions. This isn't the case for fully synchronous functions simply because the world stops for synchronous functions.
If you have two functions both trying to retrieve the same data from two different sources, one from your hard disk and one from the internet, which will finish first? If they are both asynchronous, then it's a trick question. Probably the hard disk is faster, but don't bet your life on it. Still, one of them technically gets kicked off first, synchronously.
This paradigm of scheduling things for later and not waiting for the result before continuing (non-blocking) is one of the ways Node.js manages to have such great performance even without worker processes / parallelism. For I/O in particular, such as reading a file from the disk, there are "quiet periods" of inactivity where there is nothing for the process to do. In this case, waiting before continuing is a huge waste of time. It is more appropriate to use that opportunity to interleave other functions in the meantime. This is what asynchronous functions do. They delay work to allow us to interleave other work. Usually they achieve this via process.nextTick() or setImmediate().
All that said, nothing in life is free. Delaying work has a cost and if you misuse timers you will slow down your program. The goal is to make everything that has unavoidable delays (like I/O) asynchronous and almost nothing else. However asynchronous behavior "pollutes the stack". Anything that uses an asynchronous function becomes asynchronous in nature. You could return a value synchronously and pretend like it's not the case (make it invisible to the outside world), but that is usually a bad idea because then you cannot propagate errors or the result at all.
If you are still confused about how to look at a program and figure out when everything runs, have a look at async and await. It is a wonderful way to write asynchronous code that looks like more traditional synchronous code. And it is arriving in Node 7.
Pretty much usual scenario. I want to have some decoupled piece of code, that is triggering event when something is ready. This will happen only once for the whole application run.
On other side, there is another piece of code, where I want something else to happen when two or more events are triggered. I mean like all of them, like dependencies.
Alright, more async things together ... definitely promises right ?
Then I started thinking. Is it really wise to use pub/sub for one time events ? Wouldn't be better to just make accessible promise, that resolves once that event is about to be triggered ? However that would mean I need to kinda interconnect that decoupled code. One thing is having shared EventEmitter, but being dependent on some code to actually create promise ... that sounds rather bad.
So I am thinking about some kind of mix. Having module, where other modules can ask for "event" by it's name and obtaining prepared Promise object. Other module should then trigger that event and effectively fulfilling/rejecting that event this way.
var promisedLand = require('./promisedLand');
promisedLand.waitFor('event'); // returns promise
promisedLand.resolve('event', value);
promisedLand.reject('event', error);
What do you think about this solution ? Any chance there is some solution like this already available ?
Good question. Let me start by one thing: Promises are not event emitters.
Let me reiterate that, since it's a misconception that comes a lot. Promises are not event emitters. With progression, they can be hacked into a form of crippled event emitter but at the end of the day. Promises are not event emitters.
Promises
What are they? Promises are a "box" over a value which you can open at some point using the .then method and then put the result in another box. Nothing more, nothing less.
Like you said, promises are one time. If your event is a one time event - then promises are definitely ok. Essentially, your event is an eventuality and promises model it better most of the time.
Promises as emitters
The problem with using promises as event emitters is composition, progression events in promises simply did not compose very well. Promises chain and compose and events don't. This is why the Q library is dumping progression in favor of estimation in v2. This is why progression was never included in ECMAScript 6.
Event emitters are a perfectly fine abstraction on their own, use event emitters when they're the right tool to model your relation (pub-sub), use promises when they're the right tool to model your relation, use streams when they're the right tool to model your relation. Just don't use one tool for everything as this (from experience) will bring you nothing but a lot of pain.
What I described in my question is really cool though, what about that?
What you're looking for? Oh, that exist. It's quite wonderful actually though it also comes with its own sets of problems.
What you're looking for is called FRP - functional reactive programming. There are a lot of libraries that do that with the best one (in my opinion) being BaconJS.
FRP has the notion of observables you were talking about. Here is an example of a counter from the BaconJS website:
var up = $('#up').asEventStream('click');
var down = $('#down').asEventStream('click');
var counter =
// map up to 1, down to -1
up.map(1).merge(down.map(-1))
// accumulate sum
.scan(0, function(x,y) { return x + y });
// assign observable value to jQuery property text
counter.assign($('#counter'), 'text');
Very similar to promises in chaining, but does not represent a direct continuation but a continuous streaming and a sink.
FRP is a very common and developed paradigm in functional languages like Haskell, and is very applicable in JavaScript. It's not very common yet and it has its own shortcomings, but it definitely thinks like what you had in mind.
So, short recap:
Promises are not event emitters. Promises are awesome and majestic and solve a lot of interesting concurrency problems - but they are not a silver bullet for all your decoupled flow control.
FRP is this cool thing you came up with but didn't formulate yet. It has a notion of observables and events exactly as you described in your question.
Also, you can pat yourself on the back for thinking about a paradigm without knowing it on your own. You deserve it, honest.
Alright, I have made my own solution similar to the one presented in the question.
Welcome to the Promised Land
Aloha. This is my first question here after eons of lurking. I apologize for the verbose but direct form of this question -- as well as the limitations on acceptable answers -- but I am here to contribute. I have a problem that other questions (for example How to avoid long nesting of asynchronous functions in Node.js or Chained callback readability in Javascript?) do not answer to my satisfaction.
I want to perform several Javascript callbacks so that I have a sequence of four actions occurring one after another. Here is the source:
var self = this;
fade(div, fadeIn, -20, function() {
load(unload, dataOut, -10, function() {
load(self, dataIn, 10, function() {
fade(div, fadeOut, 20),0);
unload = self;}
);}
);}
);
The load() and fade() functions are very similar except for internal semantic details, and they look something very similar to this:
function load(menuCategory, data, step, callback) {
menuCategory.parentNode.style.width = data+"px";
data += step;
if(dataIn <= data && data <= dataOut) {
setTimeout(function() { load(menuCategory,data,step,callback) }, 15);
} else if(callback && (typeof(callback) == 'function')) {
callback();}
}
I have several questions associated with this mess, but first let me give you the answers I am NOT looking for. Answers using jQuery are not acceptable. Answers using other timing or "synchronicity" frameworks or modules are not acceptable. Answers questioning the reason for blocking execution are not acceptable. Answers using languages other than Javascript are unacceptable. Non-abstract answers are acceptable, but the final solution will need to be as abstract as possible, meaning the answer must be able to act on a variety of similar but slightly different actions.
Here is my primary question:
Is there a function/object I could use to apply these actions sequentially, to cut down on the callback hell going on here? In other words, what would an object look like if the object/function were able to synchronously execute each action as it was individually iterated through it (i.e. passed to it)?
Secondary question:
Is anyone aware of how much this callback business looks like goto() in other languages? Meaning, this nesting of callbacks feels very awkward and inelegant to me. Is there a Javascript language construct either being developed or already developed that will reduce this crazy business? Callback nesting is some of the hardest logic flow to follow. It just feels awkward.
My goal here is a tight, elegant, attractive, homegrown Javascript object/function -- that I can call in a for() loop or equivalent -- to apply these operations on each action in sequence. If you've read this far I want to thank you for your time and consideration. :)
using setTimeout(func(),0) in order to execute synchronously
No. Either use func() for a synchronous execution, or setTimeout(func, 0) to queue the execution (which makes it asynchronous).
Is setTimeout(func[…],0) the "most elegant" way to put this action onto the execution queue for rendered webpage objects? In other words, is there a "more elegant" way to block execution (i.e. do these things sequentially)?
Yes, this is the standard method. However, this does not block execution, and has nothing to do with sequencing. If the execution is synchronous, just execute it, if not you will have to deal with the callbacks and queuing a new task does not help.
Is anyone aware of how much this callback business looks like goto() in other languages? Meaning, this nesting of callbacks feels very awkward and inelegant to me. Is there a Javascript language construct either being developed or already developed that will reduce this crazy business?
No, I'm not aware of other languages, but afaik goto is a synchronous, control-flow-structuring statement as does not deal with asynchronous actions.
But no, there are no JavaScript language constructs ("syntactic sugar") that help us around the continuation-passing style with its callback hell.
Is there a function/object I could use to apply these actions sequentially, to cut down on the callback hell going on here? In other words, what would an object look like if the object/function were able to synchronously execute each action as it was individually iterated through it (i.e. passed to it)?
My goal here is a tight, elegant, attractive, homegrown Javascript object/function -- that I can call in a for() loop or equivalent -- to apply these operations on each action in sequence.
Again you want "sequencially" instead of "synchronous" :-) Yes, there are [handcoded/homegrown] solutions which make dealing with callbacks easier. As those callback organisation libraries you mentioned do not satisfy you, I'd recommend to look into the very elegant and attractive Promise Api and its Deferred objects (see futures and promises for the concept).
I'm unsure why you're using setTimeout at all here. Its not necessary to queue callbacks. The following structure will work, assuming that fade() and load() handle callback execution correctly.
fade(div, fadeIn, -20, function() {
load(unload, dataOut, -10, function() {
load(self, dataIn, 10, function() {
fade(div, fadeOut, 20),0);
});
)};
)};
Another topic you should look into is Promises, also called deferreds and futures. Its essentially a coding pattern that specifically handles nested callbacks. At root, you create a Promise object that gets returned immediately, then when your method is done executing it fires a method on the promise that can then execute your next function. Its a different way of writing javascript, but easy to understand and very useful.
As it turns out, the practice that fits as the "best" answer to my question is events. In particular the practice I'm referring to is best described by EventEmitter on Node.js, or creating custom events and event listeners on client browsers. While the Promise API and its deference model is intriguing, it uses extensions to the language instead of core Javascript, which is what I was looking for here in particular (and thus why none of the previous answers -- despite their clarity -- support exactly what I was looking for).
This link (http://wekeroad.com/2012/04/05/cleaning-up-deep-callback-nesting-with-nodes-eventemitter) provides a very good example of what I was looking for. As well, I should mention the book "Async Javascript" by Trevor Burnham (http://www.amazon.com/Async-JavaScript-Trevor-Burnham/dp/1475247362), and "Secrets of the JavaScript Ninja" by John Resig and Bear Bibeault (http://www.amazon.com/Secrets-JavaScript-Ninja-John-Resig/dp/193398869X). These books provide extremely clear working examples of custom events and event handlers in client browsers that provide an elegant way around this "callback hell" I was referring to.