Although I'm not having a practical problem, I'm in need of some explanation about what makes something asynchronous in standard Javascript in NodeJs. Take for example this piece of code:
function pause(ms) {
var dt = new Date()
while((new Date()) - dt <= ms) {
}
console.log("Third")
}
console.log("First")
pause(3000)
console.log("second")
The output is
First
Third
second
Since NodeJs is asynchronous I would have expected the outcome to be "First Second Third". The following code however is asynchronous:
console.log("First")
setTimeOut(function(){
console.log("Third")
}, 3000)
console.log("Second")
The output is:
First
Second
Third
The problem is I do not fully understand why. Is it because while loops are blocking, even though I've put it inside a function? If so, what else is blocking and what isn't? Has it something to do with the event loop? Are only I/O operations asynchronous? Is there an overview of what is and isn't or is there a general principle?
The JS/Node.js language by default is synchronous (or blocking if you prefer) - as are most programming languages - in that lines of code are executed one after another, which is why the output of the first example is "First", "Third", "Second".
console.log("First"); // Outputs "First"
pause(3000); // Outputs "Third"
console.log("second"); // Outputs "Second"
That is pause() is just a function that is called and executed synchronously; there is nothing special about it.
In the second example the languages event-driven model has been used via setTimeout(), which accepts a callback function as a parameter that is called when the timer expires. Which means the sequence of outputs is different.
So to answer your question: standard JavaScript code is synchronous, but many of the standard library functions are non-blocking. As indeed your own can be if written accordingly.
Which standard library functions? Well I/O and timers generally are non-blocking, but it really is a case of reading the documentation.
Related
Suppose we have these two async functions:
function myfu(f) {
setTimeout(function(){ f(true) }, 200)
}
function myfu2(x, f) {
setTimeout(function(){ f(500 + x) }, 200)
}
I was thinking about this syntax:
if (myfu.()) {
var d = myfu2.(55)
console.log(d) // outputs 555
}
which is amazingly simpler to type and read that this:
myfu(function (DOTCALL1) {
if (DOTCALL1) {
myfu2(55, function (DOTCALL2) {
d = DOTCALL2
console.log(d) // outputs 555
})
}
})
The conversion from .( syntax could be easily implemented by means of search/replace/regex.
I noticed that most of the time the result of my asynchronous functions is not used, and after it was called, the upper functions also exits immediately.
Although sometimes the upper function can do something after it called the first async(callback), usually it does nothing. Sometimes the asynchronous function could return something useful, but most of the time it just returns undefined and real stuff is returned with callback(result).
So I thought that maybe I could modify the code some how to make it easier to type and read. Which is illustrated by the example above.
My question is sipmle: is there already similar solution? I do not want to reinvent the wheel (if it is already invented). Or, if there is no such project, why not, why this approach, which seems so easy to implement and supposed to simplify development significantly is not used?
Please do not respond with the suggestion to "take a look at async.js". I am not asking about asynchronous programming patterns, asynchronous programming in general, not about arrow function notation =>, not about coffeScript. I am asking about a code conversion on the text level from proposed original syntax form to well known async ladder, aka "callback hell" notation.
What you are describing is very similar to streamline.js: https://github.com/Sage/streamlinejs.
I wrote it but I got the idea from narrative.js: See http://www.neilmix.com/narrativejs/doc/
So your intuition is right. This can be done. It is nevertheless a little more complicated than a few search/replace/regex operations, at least if you want to go beyond the obvious and support async calls in all JavaScript constructs. For example, try to rewrite:
while (asyncF1.() && asyncF2.()) asyncF3.();
I described the streamline transformation algorithm in a blog post: https://bjouhier.wordpress.com/2011/05/24/yield-resume-vs-asynchronous-callbacks/
I have this site bookmarked since last year where the author blogs about EcmaScript6 which allegedly allows you to obtain something similar to .NET's async / await pattern (so allowing code to invoke asynchronous methods like a "normal" one). Here is the link
ES6 introduces two small extensions to the language syntax:
function*: the functions that you declare with a little twinkling star are generator functions. They execute in an unusual way and return generators.
yield: this keyword lets you transfer control from a generator to the function that controls it.
And, even though these two language constructs were not orginally designed to have the async/await semantics found in other languages, it is possible to give them these semantics:
The * in function* is your async keyword.
yield is your await keyword.
Knowing this, you can write asynchronous code as if JavaScript had async/await keywords.
I never had the chance to try it out though, so YMMV. It sounds useful.
This question already has answers here:
Are all javascript callbacks asynchronous? If not, how do I know which are?
(5 answers)
Closed 8 years ago.
I am learning node.js. By now, I have understood what callback means but I am not sure whether a callback is going to be executed synchronously(the caller not proceeding ahead until the callback returns) or asynchronously(the caller making a callback and proceeding ahead with its code).
You cannot really tell by looking at the function invocation. Some callbacks are asynchronous, others are not. You will need to check the docs, it will should be stated there.
Often you can tell them apart by the function's signature. If a callback is expected to be called only once with the result of a computation, and neither the callback nor the function don't return anything, the callback is usually invoked asynchronously (setTimeout, readFile etc). If the function returns the result immediately, the is typically callback invoked multiple times synchronously (Array::sort, Array::map) and yields valuable values. Of course, the exception proves the rule, and sometimes you cannot tell easily, e.g. setInterval vs Array::forEach (both return nothing, and invoke the callback multiple times).
I'll try to rephrase your question, because there is a lot confusion in "Async function call" and "Async IO" terminology.
Let's write it that way: Given code below, and without looking at "foo" source code, can we run some test to say if output is going to be "Line 1\nLine2" or "Line 2\nLine1"?
var t1 = asyncTestCapture();
foo(function() {
console.log("Line 1");
});
console.log("Line 2");
var out12vsOut21 = asyncTestVerify(t1);
Answer number 1: don't try to rely on this order with exception for a well known standard function with callback parameters (Array's forEach, map, etc, String replacer, JSON reviver)
Answer number 2: (note that I tested the code very briefly and it's node.js specific, but I believe it can answer "out12vsOut21" question. Also note that I'm using undocumented node functions)
function asyncTestCapture() {
return {
activeHandles: process._getActiveHandles(),
activeRequests: process._getActiveRequests()
};
}
function asyncTestVerify(handles1) {
var handles2 = asyncTestCapture();
return handles2.activeHandles === handles1.activeHandles && handles2.activeRequests === handles1.activeRequests
}
I'll repeat again: if you need the code above there is something wrong in your design. You don't need to know "out12vsOut21" order, build your code the way it does not depend on "async or not foo is"
You can use a flag for that:
var async = false;
someFunction(function () {
console.log(async ? 'async' : 'sync');
});
async = true;
I have a simple program in node.js, such as:
// CODE1
step1();
step2();
var o = step3();
step4(o);
step5();
step6();
this program is meant to be run in a stand-alone script (not in a web browser),
and it is a sequential program where order of execution is important (eg, step6 needs to be executed after step5).
the problem is that step3() is an async function, and the value 'o' is actually passed to a given callback,
so I would need to modify the program as follows:
// CODE2
step1();
step2();
step3( function (o) {
step4(o);
step5();
step6();
})
it could make sense to call step4 from the callback function, because it depends on the 'o' value computed by step3.
But step5 and step6 functions do not depend on 'o', and I have to include them in that callback function only to preserve the order of execution: step3, then step4, then step5 then step6.
this sounds really bad to me.
this is a sequential program, and so I would like to convert step3 to a sync function.
how to do this?
I am looking for something like this (eg using Q):
// CODE3
step1();
step2();
var deferred = Q.defer();
step3(deferred.resolve);
deferred.blockUntilFulfilled() // this function does not exist; this is what i am looking for
var o = deferred.inspect().value
step4(o);
step5();
step6();
How to do this?
ps: there are advantages and disadvantages of using sync or async, and this is a very interesting discussion. however, it is not the purpose of this question. in this question, i am asking how can i block/wait until a promise (or equivalent) gets fulfilled. Please, please, please, do not start a discussion on whether sync/blocking is good or not.
It's impossible to turn an async operation into a sync operation in vanilla JavaScript. There are things like node-fibers (a C++ add-on) that will allow this, or various compile-to-JS languages that will make the async operations look sync (essentially by rewriting your first code block to the second), but it is not possible to block until an async operation completes.
One way to see this is to note that the JavaScript event loop is always "run to completion," meaning that if you have a series of statements, they will always be guaranteed to execute one after the other, with nothing in between. Thus there is no way for an outside piece of information to come in and tell the code to stop blocking. Say you tried to make it work like so:
stepA();
stepB();
while (global.operationIsStillGoing) {
// do nothing
}
stepC();
This will never work, because due to the run-to-completion nature of JavaScript, it is not possible for anything to update global.operationIsStillGoing during the while loop, since that series of statements has not yet run to completion.
Of course, if someone writes a C++ addon that modifies the language, they can get around this. But that's not really JavaScript any more, at least in the commonly understood sense of ECMAScript + the event loop architecture.
At a job interview, I was perplexed to hear "javascript can evaluate statements out of order." To what degree is this true? I can imagine a hundred ways to explicitly evaluate statements out of order -- such as in a time-sharing operating system. But he seemed to say that if I evaluate
console.log('a')
console.log('b')
that the Javascript spec somehow doesn't require that the output would be a first then b. I can imagine that the evaluator might try to evaluate the second statement if the IO of the first is blocking if the statements are functionally pure, i.e. no side effects, but side effects must always occur in sequence, correct? And of course IO is one big side effect.
To what extent can spec-compliant Javascript evaluate out of order? Or was this a case of miscommunication?
JavaScript is single threaded (web workers aside). Period. ECMA-262 Language Specification Edition 5.1 says nothing about out-of-order execution. In your simple example these two statements are guaranteed to be executed in the same order.
Moreover, single block of JavaScript code will never be interrupted by any other block of code, e.g. event handler. That's why long running blocks of code cause UI to freeze:
for(var i = 0; i < 1000000000; ++i) {
console.log(i);
}
It's guaranteed that the block of code above will never be interrupted or reordered. As long as the loop is running, all event handlers and timeouts wait for single thread. And of course, numbers will appear in correct order.
What might be executed out-of-order is an asynchronous timeout:
setTimeout(function() {
console.log('a');
}, 1);
setTimeout(function() {
console.log('b');
}, 1);
Here you might expect a to be printed first, but it's possible that JS engine will reorder these events. After all you schedule these calls to execute at almost the same point in time.
Clearly miscommunication.
The dude was maybe referring to JavaScript hoisting. You can read more about it here : http://www.adequatelygood.com/2010/2/JavaScript-Scoping-and-Hoisting
Furthermore, you can learn more particularities of the language here: http://bonsaiden.github.com/JavaScript-Garden/
For the most part, yes. With two major exceptions (leaving aside the obvious "define a function, call the function" which effectively "goes back" to the function body):
1: Hoisting. var statements are hoisted, so if you write
alert(a);
var a = 123;
You get undefined, and not an error message. This is because it is hoisted to
var a;
alert(a);
a = 123;
Similarly, function definitions are hoisted too. If you write:
foo(123);
function foo(num) {alert(num);}
It will work, because the function is hoisted. However, this does NOT work if you wrote
foo(123);
foo = function(num) {alert(num);}
Because that's an assignment of an anonymous function, not a function definition.
2: Asynchronous functions.
A common mistake among beginners is to write this:
var a = new XMLHttpRequest();
a.open("GET","sompage.php",true);
a.onreadystatechange = function() {
if( a.readyState == 4 && a.status == 200) {
myvar = "Done!";
}
};
a.send();
alert(myvar);
They expect the alert to say Done!, but instead they get an inexplicable error about it not being defined. This is because the myvar = "Done!" hasn't been run yet, despite appearing earlier in the script.
See also this anecdote from Computer Stupidities:
An introductory programming student once asked me to look at his program and figure out why it was always churning out zeroes as the result of a simple computation. I looked at the program, and it was pretty obvious:
begin
readln("Number of Apples", apples);
readln("Number of Carrots", carrots);
readln("Price for 1 Apple", a_price);
readln("Price for 1 Carrot", c_price);
writeln("Total for Apples", a_total);
writeln("Total for Carrots", c_total);
writeln("Total", total);
total := a_total + c_total;
a_total := apples * a_price;
c_total := carrots + c_price;
end;
Me: "Well, your program can't print correct results before they're computed."
Him: "Huh? It's logical what the right solution is, and the computer should reorder the instructions the right way."
I think they tried to put you on your wrong foot. Javascript is sequensial. Otherwise functions where you calculate values won't work out. What can be true is that console.log activates an Async task which means it can be executed in a different order. Like an Ajax call, a Webworker, a timeout or an interval.
If you do the following it will result B than A, this is not a sequensial code because in code. A comes for B but B is executed first.
setTimeout(function(){
console.log("A")
}, 5);
console.log("B");
We are working with node, mainly for an internal project and to understand the best way to use the technology.
Not coming from a specific asynchronous background the learning curve can be a challenge but we are getting used to the framework and learning the process.
One thing that has polarised us is when the best time to use synchronous code vs asynchronous code is. We are currently using the rule that if anything interacts with IO then it has to be asynchronous via call backs or the event emitter (thats a given), but other items which are not in any way using IO can be constructed as synchronous functions (this will depends as well on the heaviness of the function itself and how blocking it actually is) but is this the best approach to take when working with Node.js?
For instance, we are creating a Hal+JSON builder, which currently exists within our code base. It is synchronous simply because all it is doing is creating some rather smallish object literals and nothing more, there are no external dependencies and certainly no IO interactions.
Is our approach a good one to take or not?
Let's say you have two functions, foo and bar, which are executing synchronously:
function foo() {
var returnValue = bar();
console.log(returnValue);
}
function bar() {
return "bar";
}
In order to make the API "asynchronous" is to change it to use callbacks:
function foo() {
bar(function(returnValue) {
console.log(returnValue);
});
}
function bar(callback) {
callback("bar");
}
But the fact of the matter is, this code is still entirely synchronous. The callback is being executed on the same call stack, and no threading optimizations are being made, no scalability benefits are to be had.
It then becomes a question of code readablity and coding style. I personally find the typical var val = func(); type code more readable and readily understandable. The only drawback is, that if you one day would need to change the functionality of bar so, that it would need to perform some I/O activity or call some other function which is asynchronous, you need to change the API of baras well.
My personal preference: use traditional, synchnous patterns when applicable. Always use asynchronous style when I/O is involved or when in doubt.
Turning a synchronous function into an asynchronous one using process.nextTick() is an option, but one you should only use if your software is blocking for too long. Every time your synchronous function is running node can't do anything else as it's single threaded. This means that if your application is a server, it becomes unresponsive.
Before you split up any synchronous function, I would advise to first do some benchmarking and profiling so you can make an informed decision. Otherwise you run the risk of dong a premature optimisation
See here for a good discussion https://softwareengineering.stackexchange.com/questions/80084/is-premature-optimization-really-the-root-of-all-evil
Here's how you can make your function asyncronous, in the sense that you will allow node to do other things http://howtonode.org/understanding-process-next-tick
I don't what's "Hal+Json builder", but here's what I think about NodeJS. You should write your code as asynchronous as possible, pushing it to it's limits. The simple reason is that asynchronous code trades performance for responsiveness, which is in most cases more important.
Of course if certain operations are really quick, then there is no need for asynchronous code. For example consider this code:
var arr = []; // assume array is nonempty
for (var i = 0, l = arr.length; i < l; i++) {
arr[ i ].doStuff( );
}
If arr is small and doStuff is a quick operation, then you should not bother in writing this code in asynchronous way. However if it takes some time, then you should consider writing it like that:
var arr = []; // assume array is nonempty
var process_array_element = function( ) {
if (arr.length) {
arr.pop().doStuff( );
process.nextTick( process_array_element );
}
};
process_array_element( );
Now this code is truely asynchronous. It will take even more time for it to complete the job, but in the meantime it won't block entire server. We've traded performance for responsiveness.
Conclusion: It depends on your situation! :)