We have a Requirement to send Mass notifications from Server to multiple clients (around 500 Clients) at once. We have implemented Node.js but we are not sure what is the performance bottlenecks as Socket connection from all the Clients to the server is open all the times. We are using this for Java Web application
Please let us know if anybody has any exposure on this
Thanks sai
This question is impossible to answer in any detail as you have provided no details about your code, not to even mention any code example.
But from my experience when people have problems with concurrency using Node and they are below 10,000 concurrent connections which you should handle easily (see this answer) then the usual suspect is that they are using blocking function calls.
Are you using any blocking code in your app? If so then here's your problem. If you have any blocking code in your Node app (like calling functions with "Sync" in their name) then you will have serious problems with concurrency. You should never use blocking functions anywhere else than on the first tick of the event loop (and if you don't know what it means then you should never use them at all).
Also, since you write that you're using Node for Java Web application it can be more complicated than that. If your Node app is connecting to Java app that spawns a new thread for every connection then the bottleneck may be in your Java app and not your Node app. As you provide no details about our architecture, it's impossible to say. But one this is for sure: if you want to handle massive concurrency then you cannot use threads. You need to use event loops. There's a reason why nginx or Redis are single-threaded. Threads don't scale, especially for I/O-bound use cases. Profile your code and see if it's your Java or Node code that needs fixing.
Related
I have a node.js app running in Azure as a webApp. On startup it connects to an external service using a websocket subscription. Specifically I'm using the reconnecting-websockets NPM package to wrap it to handle disconnects.
The problem I am having is that because there are 2 instances of the app running on Azure (horizontal scaling for failover) I end up with two subscriptions at any one time.
Is there an obvious way to solve this problem?
For extra context, this is a problem for 2 reasons:
I pay for each message received and am over quota
When messages are received I process then and do database updates, these are also being duplicated.
You basically want to have an AppService with potentially multiple instances, but you don't want your application to run in parallel. At least you don't want two have two subscriptions. Ideally you don't want to touch your application code.
An easy way to implement this would be to wrap your application into a continuous WebJob, and set its scale property to singleton.
Here is one tutorial on how to set up a nodejs webjob: https://morshemesh.medium.com/continuous-deployment-of-web-jobs-with-node-js-2308f95e63b1
You can then use a settings.job file to control that your webjob only runs on a single instance at any one time. Or you can use the Azure Portal to set the value when you manually deploy the Webjob.
{
"is_singleton": true
}
https://github.com/projectkudu/kudu/wiki/WebJobs-API#set-a-continuous-job-as-singleton
PS: Don't forget to enable Always On. It is also mentioned in the docs. But you probably already need that for your current deployment.
If you don't want your subscription to be duplicated then it stands to reason that you only want one process subscribing to the external websocket connection.
Since you mentioned that messages received will be updated in the db, then it makes sense that this would be an isolated backend process since you made it clear that you have multiple instances running for the frontend server (and whether or not a separate backend).
Of course if you want more redundancy, you could use a load balancer with simple distribution of messages to any number of instances behind. Perhaps some persistent queueing system if you feel that it's needed.
If you want these messages to be propagated to the client (not clear from the question), this will be a bit more annoying. If it's a one-way simple channel, then you could consider using SSE which is a rather simple protocol. If it's bilateral then I would myself probably consider running a STOMP server with intermediary broker (like RabbitMq) and connect directly from the client (i.e. the browser, not the server generating the frontend) to the service.
Not sure if you're well versed with Java, but I made some app that you could use for reference in case interested when we had to prepare some internal demos: https://github.com/kimgysen/zwoop-backend/tree/develop/stomp-api/src/main/java/be/zwoop
For all intents and purposes, I'm not sure if all this is worth the hustle for you, it sounds like you're on a tight budget and that you're looking for simple solutions without too much complexity. Have you considered giving up on load balancing the website (is the load really that high?), I don't have enough background knowledge on your project to judge, I believe. But proper caching optimization and initially scaling vertically may be sufficient at the start (?).
Personally I would start simple and gradually increase complexity when needed.
I'm just throwing ideas at you, hopefully it is helpful in any way to have a few considerations.
Btw, I don't understand why other answers on this question were all deleted (?).
Since node runs a single threaded model with event looping I wonder how node prevents the entire application to fail if you write a code like:
while(true){ doSomething()}
where doSomething is a synchronous function (a blocking piece of code)
Note that it doesn't make any sense to write a function like doSomething but nothing prevents you to make a mistake
The problem here is that, since it's single threaded, it won't allow any other parts of the application to run (for instance, a web server would stop accepting new connections) because this function would never end. In a Multi threaded environment you would loose this thread alone.
Is there anything that node can do for you to prevent these kind of problems?
I wonder how node prevents the entire application to fail if you write an infinite loop
nodejs does not prevent such an infinite loop. It will just run that loop forever or until some resource is exhausted (if the loop is consuming some resource like memory).
If node can't prevent this kind of situations, is this a design fault or there's no way to prevent these kind of problems?
I don't think most people consider it a design fault - though that's purely an opinion and different people may have a different opinion. It is a consequence of the way nodejs was designed which has many other benefits.
The only way to prevent such problems is to not write faulty code that does this. Honestly, it's not too hard to avoid writing this type of code once you're aware that it's an issue to avoid.
The problem here is that, since it's single threaded, it won't allow any other parts of the application to run (for instance, a web server would stop accepting new connections) because this function would never end. In a Multi threaded environment you would loose this thread alone
Correct. This is something you learn when coding in nodejs. I've never found it a hard thing to avoid. nodejs is an single-threaded event driven system, not a multi-threaded system. As such, you program with events, not long running loops that poll or check conditions. It is a rather straightforward concept to learn and use once you understand this is how nodejs works. It is different than some other environments. But, how to use asynchronous operations in nodejs is just something you have to learn to program in that environment. It's not avoidable and is just part of the character of nodejs. There is no way that nodejs could have the type of architecture it has without having to learn this to program in it. If you want a different architecture (for whatever personal reason), then pick a different environment, not nodejs.
The single-threadedness massively simplifies many other things (far, far fewer opportunities for race conditions) and improves scalability in some circumstances (with asynchronous I/O) vs. threaded environments. For situations where you want multiple CPUs to be applied to your problem, it is generally straightforward in node.js to either use the built-in clustering module or to fire up worker processes and feed them work. Data is often shared among multiple processes via some sort of database (either file-based or RAM-based) that handles much of the multi-process synchronization for you.
It doesn't. This seems like less of a question and more an open statement. Node will loop infinitely and all your parallel code will stop running.
it's not possible to find such issue in the node.js program itself. however a node.js script with an infinite loop will use lead to 100% cpu . so this can be monitored and you can use tools to restart the program. I don't recommand to do this, you should fix your infinite loop first, but it s sometimes hard to find the issue with large codebase. last time it happened to be I used a remote debugger to find the infinite loop.
JavaScript is strong with me. .NET, notsomuch...
I'm trying to figure out edgeJS on NodeJS and while I've got some sense of how to patch together communication between the two I can't seem to figure out how to maintain a listener in NODE for my Edge functions to send events to. I can only seem to get my functions to fire and hand off control to Edge while the function is running, but I would like Edge to tell NODE to do something basic for starters like console.log in NODE instead of Console.Write in .NET.
Is there an example out there that can help me? The Edge documentation seems to be above my head in this regard.
Thanks!
Disclaimer - I am new to Edge JS.
But Edge Js is for in process communication not cross process. When you say you would like to call console.log in node js - what I thought you want is call an independent nodejs application from C# ? I don't think that is possible using Edge js but you can always make http call and host a web service and that is what is shown in one of the performance graph that the in process and out of process calls difference.
Quoting from InfoQ article. Which shows we are talking about in process communication where the code of ne is nested in another and not separate.
There are two main benefits of using Edge.js to run Node.js and .NET in one process instead of splitting the application into two processes: better performance, and reduced complexity. Performance measurements of one scenario show that in-process Edge.js call from Node.js to C# is 32 times faster than the same call made using HTTP between two local processes. Dealing with a single process instead of two processes with a communication channel between them reduces the deployment and maintenance complexity you need to handle.
Based on my understanding, only I/O in NodeJS is non-blocking. If we do, for example, lots of heavy math operations, other users cannot access to the server until it's done.
I was wondering if there is a non-blocking way to do heavy computation in NodeJS? Just curious.
If you have long-running calculations that you want to do with Node, then you will need to start up a separate process to handle those calculations. Generally this would be done by creating some number of separate worker processes and passing the calculations off to them. By doing this, you keep the main Node event loop unblocked.
On the implementation side of things, you have two main options.
The first being that you manually spin off child processes using Node's child-process API functions. This one is nice because your calculations wouldn't even have to be javascript. The child process running the calculations could even be C or something.
Alternatively, the Web Worker specification, has several implementations available through NPM if you search for 'worker'. I can't speak to how well these work since I haven't used any, but they are probably the way to go.
Update
I'd go with option one now. The current child process APIs support sending messages and objects between processes easily in a worker-like way, so there is little reason to use a separate worker module.
You can use Hook.io to run a separate node process for your heavy computation and communicate between the two. Hook.io is particularly useful because it has auto-healing meshes meaning that if one of your hooks (processes) crashes it can be restarted automatically.
Use multiple NodeJS instances and communicate over sockets.
Use multiple node instances and communicate over node-zeromq, HTTP, plain TCP sockets, IPC (e.g. unix domain sockets), JSON-RPC or other means. Or use the web workers API as suggested above.
The multiple instances approach has its advantages and disadvantages. Disadvantages are that there is a burden of starting those instances and implementing own exchange protocols. Advantages are that scaling to many computers (as opposed to many cores/processors within a single computer) is possible.
I think this is way to late, but this is an awesome feature of nodejs you've to know about.
The only way abot multithreading is by spawning new processes, right so far.
But nodejs has an implemented message feature between spawned node-forks.
http://nodejs.org/docs/latest/api/child_processes.html#child_process.fork
Great work from the devs, you can pass objects etc. as message to your childs and backwards
You can use node cluster module.
https://nodejs.org/api/cluster.html
I would use JXCore - it's a polished nodejs based engine which runs your code but has several options including the multi threading you are searching for. Running this in production is a charm!
Project's source: https://github.com/jxcore/jxcore
Features include:
Support for core Node.JS features
Embeddable Interface
Publish to Mobile Platforms (Android, iOS ..)
Supports Multiple JavaScript Engines
Multi-threading Capabilities
Process Configuration & Monitor
In-memory File System
Application Packaging
Support for the latest JavaScript features (ES6, ASM.JS ...)
Support for Universal Windows Platform (uwp) api
I am a web developer writing a lot of code in jquery.
I am trying to learn YUI3 and I notice the 'node' module there. I also heard about something called Node.js. Are the YUI3 node module and Node.js related ?
The official Node.js website does not seems to have much info.
What exactly is Node.js?
It is server-side does that mean we can used Node.js instead say PHP ?
What kind of applications are being developed using Node.js?
Is it worth for a web developer to invest time learning this ?
Node.js is actually a JavaScript framework for asynchronous servers. It runs server-side, rather than client-side like the YUI3 widgets library, and, chances are, if you don't need to roll your own high-performance evented sockets/asynchronous HTTP/etc. server and are just looking to write run-of-the-mill websites, then there is no need to learn Node.js.
(that said, Node.js is still pretty cool.)
Node.js is an evented I/O JavaScript server platform. It makes it relatively easy to create things like sockets and quickly handle many concurrent connections.
Node is similar in design to and influenced by systems like Ruby's Event Machine or Python's Twisted. Node takes the event model a bit further—it presents the event loop as a language construct instead of as a library. In other systems there is always a blocking call to start the event-loop. Typically one defines behavior through callbacks at the beginning of a script and at the end starts a server through a blocking call like EventMachine::run(). In Node there is no such start-the-event-loop call. Node simply enters the event loop after executing the input script. Node exits the event loop when there are no more callbacks to perform. This behavior is like browser javascript—the event loop is hidden from the user.
About Node.js
the biggest connection between YUI3 and node.js is yeti
a command-line tool for launching JavaScript unit tests in a browser and reporting the results without leaving your terminal.