How and when is the single threaded asynchronous processing model of nodejs a better approach than the multithreaded approach of the known server Gurus like PHP, Java and C#?. Can someone please explain to me simply and clearly?
My question is how technically is the single threaded asynchronous processing model a better approach ?
Grasping the Node JS alternative to multithreading
Node.js was created explicitly as an experiment in async processing. The theory was that doing async processing on a single thread could provide more performance and scalability under typical web loads than the typical thread-based implementation.
The single threaded, async nature does make things complicated. But do you honestly think it's more complicated than threading? One race condition can ruin your entire month! Or empty out your thread pool due to some setting somewhere and watch your response time slow to a crawl! Not to mention deadlocks, priority inversions, and all the other gyrations that go with multithreading.
But is it really single threaded. Read this article https://softwareengineeringdaily.com/2015/08/02/how-does-node-js-work-asynchronously-without-multithreading/
Node.js is built on top of Google's V8 engine, which in turns compiles JavaScript. As many of you already know, JavaScript is asynchronous in nature. Asynchronous is a programming pattern which provides the feature of non-blocking code i.e do not stop or do not depend on another function / process to execute a particular line of code.Asynchronous is great in terms of performance, resource utilization and system throughput. But there are some drawbacks:
Very difficult for a legacy programmer to proceed with Async.
Handling control flow is really painful.
Callbacks are dirty.
NodeJS is single threaded and it is not a deterrent or a performance block really. The single threaded event loop is super efficient and is much less complicated than deploying effective multithreading. Multi-threading does not always mean better performance.
Having said that, if you do need to handle heavy concurrency, then you can employ the services of the cluster module which splits multiple NodeJS processes across available CPU cores, all the while maintaining a link with a master process which can be used to control/offload processing tasks.
Node was built from the ground up with asynchronicity in mind, leveraging the event loop of JavaScript. It can handle a lot of requests quickly by not waiting around for the request when there are certain kinds of work being done for the request, such as database requests.
Imagine you have a database operation that takes 10 seconds to complete, represented by a setTimeout
router.route('/api/delayed')
.get(function(req, res) {
setTimeout(function() {
res.end('foo');
}, 10000);
});
router.route('/api/immediate')
.get(function(req, res) {
res.end('bar');
});
or a back end framework that does not support asynchronous execution, this situation is an anti-pattern: the server will hang as it waits for the database operation to complete and then fulfill the request. In Node, it fires off the operation and then returns to be ready to field the next incoming request. Once the operation finishes, it will be handled in an upcoming cycle of the event loop and the request gets fulfilled.
As long as we only write non-blocking code, our Node server will perform better than other backend languages
After reading about it in the book: Web Development with MongoDB and Node.js 2nd Edition by Maithun Satheesh, Jason Krol and Bruno Joseph D'mello, I finally came across a clear advantage
To understand this, we should understand the problem that Node.js
tries to resolve. It tries to do asynchronous processing on a single
thread to provide more performance and scalability for applications
that are supposed to handle too much web traffic. Imagine web
applications that handle millions of concurrent requests; if the
server makes a new thread for handling each request that comes in, it
will consume a lot of resources and we would end up trying to add
more and more servers to increase the scalability of the application.
The single threaded asynchronous processing model has its advantage
in the previous context, and you can process much more concurrent
requests with less number of server-side resources.
And I notice that one can process much more concurrent requests with less serverside resources
my 2 pence worth.... i am not sure sure "if the the single-threaded approach of nodejs is better" : simply put, nodejs does not support multi-threading. that can translate loosely to " everything runs in a single thread". Now, I am not quite sure how it can "compare" to a multi-threaded system , as a "multi" threaded system can support both as a single thread (like nodejs ) and multiple threads . Its all in your application design , and the platform capabilities that are available to you.
What is more important, in my opinion, is the ability to support multi-tasking, in an asynchronous way .Nodejs, does provide support for multi- tasking, in a simplified and easy-to use package. It does have limitations on due to the lack of native support for multi-threading. to take advantage of the multi-tasking ( and not worry.. much ) about multi-threading, think along the lines of designing your serverside application as performing little chunks of work over a long period of tie , and each chunk of work is invoked, and consuming events generated from the clientside . Think an event-driven design/architecture ( simple switch/case for loops, callbacks, and data checkpointing to files or database, can do the trick). And I will bet my tiny dollar , that if you get your application to work in this fashion, sans multi-threading, it will be a much better design , more robust, and if you migrate it ( and adapt for multi-threading ) it run like on an SpaceX buster!
While multi-threading is aways a plus for serverside implementation, it is also a powerful beast that requires a lot of experience and respect to tame and harness ( something that nodejs shields/protects you from)
Another way to look at is is this : Multi-tasking is a perspective ( of running several tasks) at the application level , which multi-threading is a perspective, at a lower level : Multi-tasking can be mapping on to different implementations, with multi-threading being one of them .
Multi-threading capability
Truth : Node.js ( currently ) does not provide native support for multi-threading in the sense of low level execution/processing threads. Java, and its implementations /frameworks, provides native support for multi-threading, and extensively too ( pre-emption, multi-tenancy, synchronous multi-threading, multi-tasking, thread pools etc )
Pants on Fire(ish) : lack of multi-threading in Nodejs is a show stopper. Nodejs is built around an event driven architecture , where events are produced and consumed as quickly as possible. There is native support for functional call backs. Depending on the application design, this highlevel functionality can support what could otherwise be done by thread. s
For serverside applications, at an application level , what is important is the ability to perform multiple tasks, concurrently :ie multi-tasking. There are several ways to implement multi-tasking . Multi-threading being one of them, and is a natural fit for the task. That said, the concept of “multi -threading “ is is a low level platform aspect. For instance multi-threaded platform such as java , hosted/running on a single core process server ( server with 1 CPU processor core) still supports multi-multi at the application level, mapped to multi-threading at the low level , but in reality , only a single thread can execute at any ontime. On a multi-core machine with sa y 4 cores , the same multi-tasking at application level is supported , and with up to 4 threads can executing simultaneously at any given time. The point is, in most cases, what really matters is the support for mult-tasking, which is not always synonymous with multi-threading.
Back to node.js , the real discussion should be on application design and architecture , and more specifically, support for MULTI-TASKING. In general, there is a whole paradigm shift between serverside applications and clientside or standalone applications, more so in terms of design, and process flow. Among other things, server side applications need to run along side other applications( onthe server ), need to be resilient and self contained (not affect the rest o f the server when the application fails or crashes ) , perform robust exception handling ( ie recover from errors, even critical ones ) and need to perform multiple tasks .
Just the ability to support multi-tasking is a critical capability for any server side technology . And node.js has this capability, and presented in a very easy to use packaging . This all means that design for sever side applications need to focus more on multi-tasking, and less on just multi-threading. Yes granted, working on a server-side platform that supports multi-threading has its obvious benefits ( enhanced functionality , performance) but that alone does not resolve the need to support multi-tasking at the application level . Any solid application design for server side applications ,AND node.js must be based on multi-tasking through event generation and consumption ( event processing). In node.js , the use of function callbacks, and small event processors (as functions ), with data checkpointing ( saving processing data , in files or data bases) across event processing instances is key.
What else for Node.js vs Java
a whole lot more! Think scalability , code management , feature integration , backward, forward compatibility , return on investment , agility, productivity …
,… to cut on “verbosity” of this article , pun intended :), we will leave it at this for now :)
whether you agree or not, please shoot the messenger ( Quora) and not the opinions!
Related
So, maybe this question is too noob and novice to be asked but I still have no clue why LIBUV got a place in Node JS Architecture? So here is my understanding of NodeJs architecture.
Node Js is built over V8
V8 is capable of running code written with EcmaScript standards.
V8 is written in C++.
So if you want to give any new functionality we can embed V8 in our C++ project and attach new code with new Embedded V8 in C++.
Now here is the doubt,
Since V8 supports EcmaScript Javascript that means it has the capability to run callbacks written with the standards of EcmaScript.
So we can add code for File System Access, HTTP server & DB access in C++ since there are libraries (header files) that gives that functionality since Java is written in C++ (correct me if I am wrong) and Java has the capability to do the same.
Now if we can add this functionality in C++ where does the place for Libuv come into the picture of NodeJs architecture.
Thanks in advance and
Happy Coding :)
Check the docs below -
https://nodejs.org/en/docs/meta/topics/dependencies/#libuv
Another important dependency is libuv, a C library that is used to
abstract non-blocking I/O operations to a consistent interface across
all supported platforms. It provides mechanisms to handle file system,
DNS, network, child processes, pipes, signal handling, polling and
streaming. It also includes a thread pool for offloading work for some
things that can't be done asynchronously at the operating system
level.
So to sum it up, V8 provides the functionalities related to running JS files, but to use system resources like Network, Files, etc., libuv is used. Also it provides a threading model for accessing the resources mentioned.
The libuv module has a responsibility that is relevant for some particular functions in the standard library. for SOME standard library function calls, the node C++ side and libuv decide to do expensive calculations outside of the event loop entirely.They make something called a thread pool that thread pool is a series of four threads that can be used for running computationally intensive tasks such as hashing functions.
By default libuv creates four threads in this thread pool. So that means that in addition to that thread used for the event loop there are four other threads that can be used to offload expensive calculations that need to occur inside of our application. Many of the functions include in the node standard library will automatically make use of this thread pool.
Now the presence of this thread pool is very significant. Well clearly Node.js is not truly single threaded
Libuv also gives node access to the operating system’s underlying file system such as networking. So just as the node standard library has some functions that make use of libuv thread pool it also has some functions that make use of code that is built into the underlying operating system through libuv.
Simple Http request
const https=require(“https”)
const start=Date.now()
https.request(“https://www.google.com”,res=>{
res.on(“data”,()=>{} )
res.on(“end”,()=>{console.log(Date.now()-start) }) }).end()
So in this case libuv sees that we are attempting to make an HTTP request. Neither libuv nor node has any code to handle all of this low level operations that are involved with a network request. Instead libuv delegates the request making to the underlying operating system. So it's actually our operating system that does the real HTTP request Libuv is used to issue the request and then it just waits on the operating system to emit a signal that some response has come back to the request. So because Libuv is delegating the work done to the operating system the operating system itself decides whether to make a new threat or not. Or just generally how to handle the entire process of making the request.
If anyone stumbles upon this and since it lacks a good answer to the OP's question, I will try to take on this.
TLDR;
Javascript language is not asynchronous
Javascript language is not multi-threaded
Callbacks themselves are not asynchronous, they are just mean to piggyback your code to an asynchronous operation.
Let's go over your doubts one by one.
1. Since V8 supports EcmaScript Javascript that means it has the capability to run callbacks written with the standards of EcmaScript.
Callbacks don't mean that the operation is asynchronous. A callback has got nothing to do with asynchronous execution. Callback is just a way to piggyback your function so that it executes after 'something asynchronous'.
// example of synchronous callback
function main(cb) {
console.log('main code of the function');
cb(); // callback invocation here
}
main(function () {
console.log('in callback');
});
Now an example of asynchronous callback
function getDataFromNetwork(url, cb) {
ajaxCall(url).then(cb);
}
getDataFromNetwork('http://some-endpoint', function (data) {
console.log(data);
});
This is an asynchronous call with a callback. Here getDataFromNetwork function is asynchronous not the callback. The point is that callbacks are just a mechanism of running a code after something. In an asynchronous operation, this becomes a necessity. How else we are going to do that? right?
No!
Nowadays we have async-await where you can run a code after the asynchronous function completes without using callbacks.
So you get that? Callbacks are not asynchronous. And that's not the point of having libuv.
2. So we can add code for File System Access, Http server & DB access in C++ since there are libraries (header files) that gives that functionality since Java is written in C++ (correct me if I am wrong) and Java has the capability to do the same.
Yes we can add lots of code for File System Access, Http server. But Why? We do already have a lot of libraries to do that. And yes its already written in C thats how NodeJS executes them.
Java already has that?
Right, but thats also a part of JVM rather than the core Java language, just like libuv is part of NodeJS runtime rather than the core Javascript language. In this regard both Java and NodeJS are similar. Its just that Java has its own C++ layer and NodeJS borrows libuv for that. BTW libuv was primarily built for NodeJS
3. Now if we can add these functionality in C++ where does the place for Libuv come in to the picture of NodeJs architecture.
I answered how these functionalities are already in C++, now lets see where libuv fits in this picture of the whole architecture.
Lets take an ajax/network call for example. Who do you think executes this?
NodeJS? No, It just gives instruction to its C++ API (Node API).
then is it Node API? No, It just gives instruction to the libuv
then is it libuv? Yes, it is
Same goes for timers, file access, child processes etc.
Also think when a lot of network calls, file access are fired within a NodeJS program, on what process it runs? who schedules them? who notify about the results and failure.
This is a lot to do. Java has its own thread pool to do that. Java has its own schedular to schedule the threads. and since Java provides threads to end user(programmers) as well. It makes sense to implement all that stuff using Java threads.
But NodeJS is single-threaded. Why it should have threads to execute I/O operations when it can borrow it from another library without making them a part of Javascript? After all, we aren't going to provide threads to the programmer so why bother?
Also Historically, Javascript was only meant to run in browsers. The only asynchronous operation browsers had access to were network requests, no file access, no DB. So we did not have a lot of bedrock already to build upon.
Since node runs a single threaded model with event looping I wonder how node prevents the entire application to fail if you write a code like:
while(true){ doSomething()}
where doSomething is a synchronous function (a blocking piece of code)
Note that it doesn't make any sense to write a function like doSomething but nothing prevents you to make a mistake
The problem here is that, since it's single threaded, it won't allow any other parts of the application to run (for instance, a web server would stop accepting new connections) because this function would never end. In a Multi threaded environment you would loose this thread alone.
Is there anything that node can do for you to prevent these kind of problems?
I wonder how node prevents the entire application to fail if you write an infinite loop
nodejs does not prevent such an infinite loop. It will just run that loop forever or until some resource is exhausted (if the loop is consuming some resource like memory).
If node can't prevent this kind of situations, is this a design fault or there's no way to prevent these kind of problems?
I don't think most people consider it a design fault - though that's purely an opinion and different people may have a different opinion. It is a consequence of the way nodejs was designed which has many other benefits.
The only way to prevent such problems is to not write faulty code that does this. Honestly, it's not too hard to avoid writing this type of code once you're aware that it's an issue to avoid.
The problem here is that, since it's single threaded, it won't allow any other parts of the application to run (for instance, a web server would stop accepting new connections) because this function would never end. In a Multi threaded environment you would loose this thread alone
Correct. This is something you learn when coding in nodejs. I've never found it a hard thing to avoid. nodejs is an single-threaded event driven system, not a multi-threaded system. As such, you program with events, not long running loops that poll or check conditions. It is a rather straightforward concept to learn and use once you understand this is how nodejs works. It is different than some other environments. But, how to use asynchronous operations in nodejs is just something you have to learn to program in that environment. It's not avoidable and is just part of the character of nodejs. There is no way that nodejs could have the type of architecture it has without having to learn this to program in it. If you want a different architecture (for whatever personal reason), then pick a different environment, not nodejs.
The single-threadedness massively simplifies many other things (far, far fewer opportunities for race conditions) and improves scalability in some circumstances (with asynchronous I/O) vs. threaded environments. For situations where you want multiple CPUs to be applied to your problem, it is generally straightforward in node.js to either use the built-in clustering module or to fire up worker processes and feed them work. Data is often shared among multiple processes via some sort of database (either file-based or RAM-based) that handles much of the multi-process synchronization for you.
It doesn't. This seems like less of a question and more an open statement. Node will loop infinitely and all your parallel code will stop running.
it's not possible to find such issue in the node.js program itself. however a node.js script with an infinite loop will use lead to 100% cpu . so this can be monitored and you can use tools to restart the program. I don't recommand to do this, you should fix your infinite loop first, but it s sometimes hard to find the issue with large codebase. last time it happened to be I used a remote debugger to find the infinite loop.
I have read about Web Workers on http://www.whatwg.org/specs/web-apps/current-work/multipage/workers.html and I think I understand their purpose, but I am wondering if one of the main purposes of web workers, namely "allows long tasks to be executed without yielding to keep the page responsive." could be already achieved without web workers?
Like Registering Callbaks also allow long tasks to be executed, and only interrupt when they are ready, wtihout blocking, isn't that the same?
Callbacks allow you to manage concurrency. That is handling tasks. Not always in an easy way.
Not only do webworkers allow you to do concurrency in an easier way, they also let you have parallelism, that is tasks really running in parallel : they don't necessarily block each other and they don't block the UI.
In order to have a long javascript based running task in your browser before web worker, you had to micro-manage it to cut it in small parts in order to allow the UI to keep responsive. And of course having more than one long running task was more complex.
We know web browsers increased a lot over the past few years and it is primarily because of lot of work done on its engines, ex- V8 (Google), Chakra (Microsoft). The JavaScript so far runs in a single thread. The problem with single-threaded architecture is that it blocks the code and UI becomes unresponsive in case of running the complex script. There are various ways to solve this problem:
Offload work to the server, but to make apps faster fat client is preferred
Use asynchronous calls, but many complex ecosystem of async calls & promises could
lead into callback hell
Leverage multi-threading. Interesting!
Web Workers solve this issue by providing the capability of multi-threading in JavaScript.
Let's say I'm building a 3-tier web site, with Mongo DB on the back end and some really lightweight javascript in the browser (let's say just validation on forms, maybe a couple of fancy controls which fire off some AJAX requests).
I need to choose a technology for the 'middle' tier (we could segment this into sub-tiers but that detail isn't the focus here, just the overall technology choice), where I want to crunch some raw data coming out of the DB, and render this into some HTML which I push to the browser. A fairly typical thin-client web architecture.
My safe choice would be to just implement this middle tier in Java, using some libraries like Jongo to talk to the Mongo DB and maybe Jackson to marshal/unmarshal JSON to talk to my fancy controls when they make AJAX requests. And some Java templating framework for rendering my HTML on the server.
However, I'm really intrigued by the idea of throwing all this out the window and using Node.js for this middle tier, for the following reasons:
I like javascript (the good parts), and let's say for this application's business logic it would be more expressive than Java.
It's javascript everywhere. No need to switch between languages, and indeed the OO and functional paradigms, when working anywhere on the stack. There's no translation plumbing between the tiers, JSON is supported natively everywhere.
I can reuse validation logic on the client and server.
If in the future I decide to do the HTML rendering client-side in the browser, I can reuse the existing templates with something like Backbone with a pretty minimal refactoring / retesting effort.
If you're at this point and like Node, all the above will seem obvious. So I should choose Node right?
BUT... this is where it falls down for me: as we all know Node is based around a single-threaded async I/O web server model. This is great for my scalability and performance in terms of servicing requests for data, but what about my business logic? What about my template rendering? Won't this stuff cause a huge bottleneck for all requests on the single thread?
Two obvious solutions come to mind, but neither of them sits right:
Keep the 'blocking' business logic in there and just use a cluster of Node instances and a load balancer, to service requests in true parallel. Ok great, so why isn't Node just multi-threaded in the first place? Or was this always the idea, to Keep It Simple Stupid and avoid the possibility of multi-threaded complexity in the base case, making the programmer do the extra setup work on top of this if multi-core processing power is desired?
Keep a single node instance, and keep it non-blocking by just calling out to some java implementation of my business logic running on some other, muti-threaded, app server. Ok, this option completely nullifies every pro I listed of using Node (in fact it adds complexity over just using Java), other than the possible gains in performance and scalability for CRUD requests to the DB.
Which leads me finally to the point of my question - am I missing some huge important piece of the Node puzzle, have I just got my facts completely wrong, or is Node just unsuitable for crunching business logic on the server? Put another way, is Node just useful for sitting over a database and servicing many CRUD requests in a more performant and scalable way than some other implementation which blocks on I/O? And you have to do all your business logic in some tier below, or even client-side, to maintain any reasonable levels of performance and scalability?
Considering all the buzz over Node, I'd rather hoped it brought more to the table than this. I'd love to be convinced otherwise!
On any given system you have N cpus available (1-64, or whatever N happens to be). In any CPU-intensive application, you're going to be stuck with a throughput of N cpus. There's no magical way to fix that by adding more than N threads/processes/whatever. Either your code has to be more efficient, or you need more CPUs. More threads won't help.
One of the little-appreciated facts about multiple-CPU performance is that if you need to run N+1 CPU-intensive operations at the same time, your throughput per CPU goes down quite a bit. A CPU-intensive process tends to hang on to that CPU for a very long time before giving it up, starving the other tasks pretty badly. In the majority of cases, it is blocking I/O and the concomitant task-switching that makes modern OS multitasking work as well as it does. If more of our every-day common tasks were CPU-bound, we would discover we needed a lot more CPUs in our machines than we do currently.
The nice thing that Node.js brings to the server party efficiency-wise is a thorough use of each thread. Ideally, you end up with less task switching. This isn't a huge win, but having N threads handling N*C connections asynchronously is going to have a performance advantage over N*C blocking threads running on the same number of CPUs. But the bottom line on CPUs remains the same: if you have more than N worth of actual CPU work to be done, you're going to feel some pain.
The last time I looked at the Node.js API there was a way to launch a server with one listener plus one worker thread per CPU. If you can do that, I would be inclined to go with Node.js provided a few caveats are met:
The Javascript-everywhere approach buys you some simplicity. For something complicated, I would be concerned about the asynchronous programming style making things harder rather than easier.
The template-processing and other CPU-intensive tasks aren't appreciably slower in Node.js than your other language/platform choices.
The database drivers are reliable.
There is one downside that I can see:
If a thread crashes, you lose all of the connections being serviced by that thread.
Finally, try to remember that programmer time is generally more expensive than servers or bandwidth.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How to decide when to use Node.js?
Sorry if I'm a bit ambiguous, but I'm trying to understand the real advantages of using Node.js instead of other server-side language.
I'm a JavaScript enthusiast, so I'm probably going to play with Node.js, but I want to know if I should use it in my projects.
It's evented asynchronous non-blocking I/O build ontop of V8.
So we have all the performance gain of V8 which is the Google JavaScript interpreter. Since the JavaScript performance race hasn't ended yet, you can expect Google to constantly update performance on V8 (for free).
We have non-blocking I/O which is simply the correct way to do I/O. This is based on an event loop and using asynchronous callbacks for your I/O.
It gives you useful tools like creating a HTTP server, creating a TCP server, handling file I/O.
It's a low level highly performant platform for doing any kind of I/O without having to write the entire thing in C from scratch. And it scales very well due to the non-blocking I/O.
So you want to use Node.js if you want to write highly scaling and efficient applications using non-blocking I/O whilst still having a high level scripting language available. If needed, you can hand optimise parts of your code by writing extensions in C.
There are plenty of OS libraries for Node.js that will give you abstractions, like Express.js and now.
You don't want to use Node.js if you want (slow) high level abstractions to do everything for you. You don't want to use Node.js if you want RAD. You don't want to use Node.js if you can't afford to trust a young platform, either due to having to write large pieces of code yourself to do things that are build into other frameworks or because you can't use Node.js, because the API isn't stable yet or it's a sub 1.0 release.
The two most oft-quoted advantages are:
JavaScript is both server-side and client-side. There are fewer things to learn, less context switching, and the ability to reuse code across the two sides.
Uses non-blocking I/O, and Chrome's V8 engine, to provide fast, highly scalable servers.
For me though, the most interesting part is the amount of activity happening in this area. There are a lot of very interesting ideas under development for node - be sure to check out the list of Node.js modules.
When you're (or even if you are not) a JavaScript enthusiast you can/should use Node.js for a number of reasons:
It's a low-level, lightweight and standalone framework which brings power of JavaScript to the server-side environment.
If you like more higher level abstraction then there is a large number of modules and the npm package manager where you can find wide range of ready-to-use applications.
Fast/unencumbered development process - for example, you don't need tons of additional tools in order to start writing serious stuff.
Big open source based community full of enthusiasts and very talented people.
Made for creating real-time web oriented applications - that's where the (near) future is.
Personally, I'd most likely use Node.js when:
I want to write a server that doesn't use the HTTP protocol.
I'm prototyping a server implementation.
I'm writing a server that isn't expecting a ton of traffic (although I've never profiled a Node.js implementation next to, say, a matching C++ implementation).
I want to get active in the community (which is apparently growing quite rapidly).