i have a node.js server that i want to be able to handle exceptions without crashing, and i've got code kinda like the below. What i'm wanting to know, with all the event-driven awesomeness and callbacks and lambdas and all that, will my exceptions still be caught by my main entry point?
try {
http.get(..., function(results) {
// Might get an exception here
results.on('data', function () {
// Might also get an exception here
});
results.on('end', function () {
// Might also get an exception here
});
});
} catch(e) {
// Will the exceptions from the lambdas be caught here?
console.log('Nicely caught error: (' + e.name + '): ' + e.message);
}
Thanks
It depends on the flow of control. Node.js puts emphasis on being asynchronous and one of the main drawbacks of asynchronicity is that the code doesn't flow in a way that you might be used to with a synchronized language.
In a synchronous language the caller is blocked while a function is waiting for some data. This makes the programmers job fairly simple because they can be guaranteed that when the function that's waiting for data returns, there will be data for the caller to consume.
It's the exact opposite in an asynchronous language, or with non-blocking I/O. In this case, the caller is blocked for the duration of the function call, however functions don't have to wait for data or I/O to complete before returning. This makes things slightly harder on the programmer because when an function call returns there are no guarantees about whether there will be data available. Hence, non-blocking I/O typically implies callback functions that get called when data is available to act on.
try/catch blocks work with the call stack. That is, when an exception is thrown the runtime will unwind the call stack until it finds a catch block that surrounds the call that threw the exception. But, since http.get is a non-blocking call it exits immediately after registering some callbacks and processing continues. The callbacks are called in a separate "thread" and therefore the calls aren't nested within the original try/catch block.
A diagram would really help explain things here but unfortunately I don't have one available to me.
The error handling style for the node.js standard library is to call the same callback, but pass a non-null first argument representing the error. If you have exceptional conditions in your asynchronous code, keep it in that format.
A throw would climb up the caller chain, which normally isn't aware of what callbacks are doing (for example, the tcp layer doesn't care that its data is parsed as http). Throwable exceptions are a bad fit for asynchronous programming.
In your example code, none of the potential exceptions thrown from within the http.get callback will land in your catch block. The stack of the callback is built from node's event loop when data is available to read.
There is a way to catch uncaught exceptions in node:
process.on("uncaughtException", function (err) {
console.log("uncaught exception: " + err);
});
This will sort of work for your example, depending on where the exception is.
The trouble is, uncaught exceptions can unravel node's inner workings in surprising ways, so you really don't want to depend on this. The only reliable way to catch all possible exceptions in a way that you can deal with them is to put a try/catch around every entry point from the event loop.
This sounds pretty tedious, but it usually isn't that bad. In your example program you are using node's API for HTTP requests, which is still a very low-level interface. For most things, you'd want to wrap up this exception catching functionality once and use it as a library.
Related
I am aiming to understand what exactly is it about certain JavaScript functions that make them asynchronous. You could have a function like this:
function someRandomFunction () {
for (let i of everything) {
iterator++
something = true
hello = that + whatever
}
}
Nothing about this function is asynchronous. It does a lot of stuff but it does it very quickly.
But then take a Node.js function like this:
fs.readFile('path/to/file', (err, data) => {
// do something
})
This function is declared to be asynchronous. But why is it? What is the reason behind it?
Is it because it takes a certain amount of time for reading a file to complete, therefore it's asynchronous? Why is that asynchronous when looping through some variables and doing some calculations is not asynchronous?
The 'idea' of something being asynchronous means "we've relinquished control to some other operational code, and we have no idea when that other code will allow us to operate again".
So in your bottom sample, you give control over to node's filesystem operations to read a file. We don't know how large that file is, so there is no way to anticipate how much time it will take to complete. So JavaScript allows us to provide a "callback" that get's fired when the async operation is complete. When the callback is executed, control is returned to our code.
The top example is synchronous because your code maintains control over the operation and Javascript executes code synchronously by nature.
A function is either explicitly synchronous or asynchronous; there's no "certain amount of time" that automatically makes it asynchronous.
The important thing to remember with Node is that it's "async I/O", which means nothing is ever async without some input/output to/from the process. You normally see this with file reading/writing, database calls, and anything that requires a network call.
This is a basic but fundamental question. Passing a function to another function doesn't make the code asynchronous automatically. It's all about attemting to work with another piece of code, mostly of unknown origins, probably but not necessarily related with operating system's IO mechanism. In another words you ask something to happen out of the realm of the currently exectuing JS context by handing a task to known or unknown resources which are possibly running on a separate thread.
As i said this can be an IO operation or a database access of which you have no idea on
What code is running
At which thread it is running
How long it will take
Whether it will succeed or not
Behind the curtains these alien codes finally trigger the event queue to invoke the callback function through a driver. In some pure functional languages like Haskell such activities are strictly kept away from the context and must be handled by monadic type functions. In JS this is basically done by asynchronous workflow by callbacks, promises or async-awiat mechanism.
Once you hand out a task to the outer world then there shall be no way for it to interfere with your synchronous code.
In terms of JavaScript, the difference between synchronous and async functions is where and when the code is executed. Synchronous functions are executed immediately one after the other in the current call stack. Async functions are passed off to the event loop, and values returned once the current call stack has completed execution.
Caveat being, nothing in JavaScript is truly asynchronous due to the fact that JS only executes on a single process thread
Could someone please give me some clarity if Node's Crypto module is blocking or not? From what I thought I understood, some core modules can yield to a separate thread in c++.
I'm mainly interested in:
crypto.createHmac
crypto.createSign
crypto.createVerify
If it does process on a separate thread, then fine. If not, then I might have to think about using the cluster module. I'm attempting to avoid blocking the main event loop.
Here is a trivial example of how I would implement crypto.createHmac.
function createHmac(algo, secret, data, callback) {
var cryptoStream = crypto.createHmac(algo, secret);
// Emitters
cryptoStream
.on('error', function (err) {
return callback(err);
});
// Write the data
cryptoStream.write(data, 'utf8', function(){
cryptoStream.end();
return callback(null, cryptoHmacStream.read());
});
}
The "crypto.create*" functions you list are synchronous and blocking, however, they are quite trivial and just set up some basic data structures. They do not perform any heavy crypto computation, which happens later in asynchronous non-blocking methods. Thus, these are fine to call in the main event loop. They are no different than doing basic string manipulation, regex matching, etc in terms of overhead and do not require special treatment. You do not need the cluster module. Your snippet is already asynchronous and non-blocking because the cryptoStream is asynchronous by way of the event emitter pattern. The write method is asynchronous, which is the important part.
To contradict the previous answer, as far as I understand, the crypto.createHmac method does all computation on the main thread in C++, but this is still the same thread as the event loop.
Your snippet is not actually asynchronous and not actually non-blocking. The interface only looks asynchronous. It will still use up time on your event loop.
For truly asynchronous crypto methods and better throughput, use crypto-async.
Recently, I've been read the TJ's blog article: "Farewell Node.js".
I'm not quite understand about the Node fails part. Here it is:
Error-handling in Go is superior in my opinion. Node is great in the sense that you have to think about every error, and decide what to do. Node fails however because:
you may get duplicate callbacks
you may not get a callback at all (lost in limbo)
you may get out-of-band errors
emitters may get multiple “error” events
missing “error” events sends everything to hell
often unsure what requires “error” handlers
“error” handlers are very verbose
callbacks suck
What specific problem is being referred to when the author writes "you may not get a callback at all (lost in limbo)"?
It means the error is lost in limbo since the operating function did not "get a callback", viz., the error is "swallowed", since there is no callback to handle it.
var foo = function(onSuccess, onFailure) {
// ...
// uh-oh, I failed
if(onFailure) {
onFailure(err);
}
else {
// well, that probably wasn't too important anyway...
}
}
foo(function() { console.log("success!"); } /* no second argument... */);
Note in synchronous coding (say, most Java) it's much harder for this to happen. Catch blocks are much better enforced and if an exception escapes anyway, it goes to the uncaught exception handler which by default crashes the system. It's like this in node too, except in the above paradigm where an exception isn't thrown it's likely swallowed.
Strong community convention could solve it in my trivial example above, but convention can not completely solve this in general. See e.g. the Q promise library which supports a done method.
Q.fcall(promisedStep1)
.then(promisedStep2)
.then(promisedStep3)
.then(promisedStep4)
.then(function (value4) {
// Do something with value4
})
.catch(function (error) {
// Handle any error from all above steps
})
.done();
The done call there instructs the promise chain to throw any unhandled exceptions (if the catch block were missing, or the catch block itself throws). But it is fully the responsibility of the programmer to call done, as it must be, since only the programmer knows when the chain is complete. If a programmer forgets to call done the error will sit dangling in the promise chain. I have had real production bugs caused by this; I agree, it's a serious problem.
I'll be honest that a lot of that block in the post doesn't make much sense to me. But I'm an experienced Node.js programmer and this is the only thing I can think that could mean.
Is node's javascript environment single threaded, or does everything happen at the same time? Or (more likely) do neither of those statements explain what's going on with node.
I'm new to node and trying to understand how it processes callbacks. My googling on the subject hasn't proven fruitful, and it seems like there's multiple audiences using terms like "threads, locking, and single threaded" with difference contexts for each audience, and I don't have enough experience with node to correctly parse what I'm reading.
From what I've read, node's javascript execution environment is, like a browser's, single threaded. That is, despite everything being designed around asynchronous callbacks, everything happens in a deterministic order, and there's never two threads modifying the same variable or running statements at the same time. I've also read this means a node programmer-user doesn't have to worry about locking semantics.
If I'm in browser-land and use one of the popular javascript libraries to setup a non-dom-event-handler callback, something like
console.log("Here");
$.each([1,2,3],function(){
console.log("-- inside the callback --");
});
console.log("There");
My output is consistently
Here
-- inside the callback --
-- inside the callback --
-- inside the callback --
There
However, if I do something like this with a callback in node js (running it as a shell script from the command line)
var function example()
{
var fs = require('fs');
console.log("Here");
fs.readdir('/path/to/folder', function(err_read, files){
console.log('-- inside the callback --');
});
console.log("There");
for(var i=0;i<10000;i++)
{
console.log('.');
}
}
example();
console.log("Reached Top");
I consistently (seemingly — see "not much experience" above) get results like this
Here
There
.
. (repeat 10,000 times)
Reached Top
-- inside the callback --
That is, node finishes executing the function example before calling the callback.
Is this deterministic behavior in node? Or are there times where the callback will be called before the example function finishes? Or will it depend on the implementation in the library using the callback?
I understand the idea behind node is to write event based code — but I'm trying to understand what node is really doing, and what behavior can be relied on and what can't.
Is node's javascript environment single threaded, or does everything happen at the same time?
Node has single threaded program execution model, meaning that only one instruction will be executed at any time within one node process. The execution will continue until the program yields the control. This can happen at the end of the program code, or when callback reaches it's end.
In the first case:
console.log("Here");
$.each([1,2,3],function(){
console.log("-- inside the callback --");
});
console.log("There");
they key is the fact that $.each uses callbacks synchronously, so effectively it calls it's callbacks in deterministic order.
Now in the second case, fs.readdir uses callbacks asynchronously - it puts the callback waiting for the event to be triggered (that is, when reading the directory finishes). That can happen at any time. The calling function, however, doesn't yield the control so example will always finish before any of the callbacks is called.
In one sentence:
All the code of the function/global scope is executed before any callbacks defined in that function/global scope.
In this case, you are calling a function that is doing IO and is asynchronous. That is why you are seeing output the way you are.
Because the rest of your code is executing inline - on the single execution thread, it is completed before the IO events are given a chance to make their way onto the event loop.
I would say expect this behavior, but don't depend on it with absolute certainty as it does depend on the implementation of the library using the callback. An (bad, evil) implementor could be making an synchronous request to see if there is any work to be done and, if not, callback immediately. (Hopefully you'll never see this, but…).
I would like to add a bit to #dc5's answer. On it's website they describe node as
Node.js uses an event-driven, non-blocking I/O model
The event driven part is very significant. It usually clears my doubts when I am having trouble understanding the execution model of the node programs.
So taking your example what is happening is :
Here is printed to the console.
An async call is made to the files.
Node attaches a listener to the async call.
Continues down the execution line and There is printed.
Now it could be that the async call is completed before the current piece of code that is being executed but node js completes the task at hand and then returns to service the async call.
So instead of waiting for anything it keeps on executing in line. Which is why node js execution loop is never idle.
I am trying to grasp on Javascript Asynchronous functions and callbacks.
I got stuck on the concept of callback functions, where I am reading on some places: they are use to have sequential execution of code (mostly in context of jquery e.g animate)and some places specially in the context of Nodejs; they are use to have a parallel execution Asynchronous and avoid blocking of code.
So can some expert in this topic please shed light on this and clear this fuzz in my mind (examples??).
so I could make my mind for the usage of callback function
or that is solely depends on the place of where you are calling/placing a callback function in your code? .
Thanks,
P.S: I am scared that this question would be close as subjective but still I could expect concrete answer for this (perhaps some examples)
Edit: actually this is the example from internet which makes me ambigous:
function do_a(){
// simulate a time consuming function
setTimeout( function(){
console.log( '`do_a`: this takes longer than `do_b`' );
}, 1000 );
}
function do_b(){
console.log( '`do_b`: this is supposed to come out after `do_a` but it comes out before `do_a`' );
}
do_a();
do_b();
Result
`do_b`: this is supposed to come out after `do_a` but it comes out before `do_a`
`do_a`: this takes longer than `do_b`
when JS is sequential then do_b should always come after do_a according to my understanding.
The core of JavaScript is largely synchronous, in that functions complete their task fully, before completing. Prior to the advent of AJAX, it was really only setTimeout and setInterval that provided asynchronous behavior.
However, it's easy to forget that event handlers are, effectively async code. Attaching a handler does not invoke the handler code and that code isn't executed until some unknowable time in the future.
Then came AJAX, with its calls to the server. These calls could be configured to be synchronous, but developers generally preferred async calls and used callback methods to implement them.
Then, we saw the proliferation of JS libraries and toolkits. These strove to homogenize different browsers' implementations of things and built on the callback approach to async code. You also started to see a lot more synchronous callbacks for things like array iteration or CSS query result handling.
Now, we are seeing Deferreds and Promises in the mix. These are objects that represent the value of a long running operation and provide an API for handling that value when it arrives.
NodeJS leans towards an async approach to many things; that much is true. However this is more a design decision on their part, rather than any inherent async nature of JS.
Javascript is always a synchronous(blocking) single thread language but we can make Javascript act Asynchronous through programming.
Synchronous code:
console.log('a');
console.log('b');
Asynchronous code:
console.log('a');
setTimeout(function() {
console.log('b');
}, 1000);
setTimeout(function() {
console.log('c');
}, 1000);
setTimeout(function() {
console.log('d');
}, 1000);
console.log('e');
This outputs: a e b c d
In node long running processes use process.nextTick() to queue up the functions/callbacks. This is usually done in the API of node and unless your programming(outside the api) with something that is blocking or code that is long running then it doesn't really effect you much. The link below should explain it better then I can.
howtonode process.nextTick()
jQuery AJAX also takes callbacks and such as it its coded not to wait for server responses before moving on to the next block of code. It just rememebers the function to run when the server responds. This is based on XMLHTTPRequest object that the browsers expose. The XHR object will remember the function to call back when the response returns.
setTimeout(fn, 0) of javascript will run a function once the call stack is empty (next available free tick) which can be used to create async like features.setTimeout(fn, 0) question on stackoverflow
To summerise the async abilities of javascript is as much to do with the environments they are programmed in as javascript itself. You do not gain any magic by just using lots of function calls and callbacks unless your using some API/script.
Jquery Deferred Object Is another good link for async capabilities of jQuery. Googling might find you information on how jQuery Deferred works also for more insight.
In JavaScript the term "asynchronous" typically refers to code that gets executed when the call stack is empty and the engine picks a job from one of its job queues for execution.
Once code is being executed, it represents a synchronous sequence of execution, which continues until the call stack is empty again. This sequence of execution will not be interrupted by events in order to execute some other JavaScript code (when we discard Web Workers). In other words, a single JavaScript environment has no preemptive concurrency.
While synchronous execution is ongoing, events might be registered as jobs in some job queues, but the engine will not process those before first properly execution what is on the call stack. Only when the call stack is empty will the engine take notice of the job queues, pick one according to priority, and execute it (and that is called asynchronous).
Callbacks
Callbacks can be synchronous or asynchronous -- this really depends on how they are called.
For instance, here the callback is executed synchronously:
new Promise(function (resolve) { /* .... */ });
And here the callback is executed asynchronously:
setTimeout(function () { /* ... */ });
It really depends on the function that takes the callback as argument; how it deals with eventually calling that callback.
Ways to get code to execute asynchronously
The core ECMAScript language does not offer a lot of ways to do this. The well-known ones are offered via other APIs, such as the Web API, which are not part of the core language (setTimeout, setInterval, requestAnimationFrame, fetch, queueMicrotask, addEventListener, ...).
Core ECMAScript offers Promise.prototype.then and (depending on that) await. The callback passed to then is guaranteed to execute asynchronously. await will ensure that the next statements in the same function will be executed asynchronously: await makes the current function return, and this function's execution context will be restored and resumed by a job.
It also offers listening to when the garbage collector is about to garbage collect an object, with FinalizationRegistry.
Web Workers
Web Workers will execute in a separate execution environment, with their own call stack. Here preemptive concurrency is possible. When the term asynchronous is used in the JavaScript world, it typically does not refer to this kind of parallelism, although the communication with a Web Worker happens via asynchronous callback functions.
Javascript by default is "synchronous", it's the web APIs that handle "asynchronous" behaviour.
As for the setTimeout example,
console.log(...), in the global scope works straight away, while those inside functions wrapped inside setTimeout, wait inside the callback queue only to be pushed back on the call stack once ready. Thus they take time. Also, the time specified is not exact but the minimum time after which that piece of code can run anytime.
Thanks !