JavaScript is strong with me. .NET, notsomuch...
I'm trying to figure out edgeJS on NodeJS and while I've got some sense of how to patch together communication between the two I can't seem to figure out how to maintain a listener in NODE for my Edge functions to send events to. I can only seem to get my functions to fire and hand off control to Edge while the function is running, but I would like Edge to tell NODE to do something basic for starters like console.log in NODE instead of Console.Write in .NET.
Is there an example out there that can help me? The Edge documentation seems to be above my head in this regard.
Thanks!
Disclaimer - I am new to Edge JS.
But Edge Js is for in process communication not cross process. When you say you would like to call console.log in node js - what I thought you want is call an independent nodejs application from C# ? I don't think that is possible using Edge js but you can always make http call and host a web service and that is what is shown in one of the performance graph that the in process and out of process calls difference.
Quoting from InfoQ article. Which shows we are talking about in process communication where the code of ne is nested in another and not separate.
There are two main benefits of using Edge.js to run Node.js and .NET in one process instead of splitting the application into two processes: better performance, and reduced complexity. Performance measurements of one scenario show that in-process Edge.js call from Node.js to C# is 32 times faster than the same call made using HTTP between two local processes. Dealing with a single process instead of two processes with a communication channel between them reduces the deployment and maintenance complexity you need to handle.
Related
I have a node.js app running in Azure as a webApp. On startup it connects to an external service using a websocket subscription. Specifically I'm using the reconnecting-websockets NPM package to wrap it to handle disconnects.
The problem I am having is that because there are 2 instances of the app running on Azure (horizontal scaling for failover) I end up with two subscriptions at any one time.
Is there an obvious way to solve this problem?
For extra context, this is a problem for 2 reasons:
I pay for each message received and am over quota
When messages are received I process then and do database updates, these are also being duplicated.
You basically want to have an AppService with potentially multiple instances, but you don't want your application to run in parallel. At least you don't want two have two subscriptions. Ideally you don't want to touch your application code.
An easy way to implement this would be to wrap your application into a continuous WebJob, and set its scale property to singleton.
Here is one tutorial on how to set up a nodejs webjob: https://morshemesh.medium.com/continuous-deployment-of-web-jobs-with-node-js-2308f95e63b1
You can then use a settings.job file to control that your webjob only runs on a single instance at any one time. Or you can use the Azure Portal to set the value when you manually deploy the Webjob.
{
"is_singleton": true
}
https://github.com/projectkudu/kudu/wiki/WebJobs-API#set-a-continuous-job-as-singleton
PS: Don't forget to enable Always On. It is also mentioned in the docs. But you probably already need that for your current deployment.
If you don't want your subscription to be duplicated then it stands to reason that you only want one process subscribing to the external websocket connection.
Since you mentioned that messages received will be updated in the db, then it makes sense that this would be an isolated backend process since you made it clear that you have multiple instances running for the frontend server (and whether or not a separate backend).
Of course if you want more redundancy, you could use a load balancer with simple distribution of messages to any number of instances behind. Perhaps some persistent queueing system if you feel that it's needed.
If you want these messages to be propagated to the client (not clear from the question), this will be a bit more annoying. If it's a one-way simple channel, then you could consider using SSE which is a rather simple protocol. If it's bilateral then I would myself probably consider running a STOMP server with intermediary broker (like RabbitMq) and connect directly from the client (i.e. the browser, not the server generating the frontend) to the service.
Not sure if you're well versed with Java, but I made some app that you could use for reference in case interested when we had to prepare some internal demos: https://github.com/kimgysen/zwoop-backend/tree/develop/stomp-api/src/main/java/be/zwoop
For all intents and purposes, I'm not sure if all this is worth the hustle for you, it sounds like you're on a tight budget and that you're looking for simple solutions without too much complexity. Have you considered giving up on load balancing the website (is the load really that high?), I don't have enough background knowledge on your project to judge, I believe. But proper caching optimization and initially scaling vertically may be sufficient at the start (?).
Personally I would start simple and gradually increase complexity when needed.
I'm just throwing ideas at you, hopefully it is helpful in any way to have a few considerations.
Btw, I don't understand why other answers on this question were all deleted (?).
Scenario: I'm new to JavaScript (have experience C/C++/Java), and working on a React UI with NodeJS server wherein the server will get updates on the displayed data at specific intervals (say 5 secs). The server is supposed to push the data to React UI to update the displayed data quickly.
Contenders: I came across 2 JS libraries Websockets (is lib a right term for websocket?) and Socket.io to achieve this. To me they both appear same.
Did I Google?: Yes, but with my lack of knowledge of JS, I couldn't understand the difference.
Problem: What is the difference between these 2 libs (libs?) and how can I decide which one is more suitable here?
Scenario: I'm new to JavaScript (have experience C/C++/Java), and working on a React UI with NodeJS server wherein the server will get updates on the displayed data at specific intervals (say 5 secs). The server is supposed to push the data to React UI to update the displayed data asap.
This is how I would do it.
Contenders: I came across 2 JS libraries Websockets (is lib a right term for websocket?) and Socket.io to achieve this. To me they both appear same.
In short, I am aware of two different ways of communicating between the server and client: WEBSOCKET and SERVER-SENT EVENTS (SSE). They are slightly different so it would be highly advisable to Google both options to find out which would suit you better.
Socket.io is essentially a library to assist with the creation/management of websockets. Imagine it to be your .NET Framework when you could have written a system yourself in C instead.
Did I Google?: Yes, but with my lack of knowledge of JS, I couldn't understand the difference.
Here are some links that can help you:
SSE:
https://www.w3schools.com/html/html5_serversentevents.asp
https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
WEBSOCKET
https://www.html5rocks.com/en/tutorials/websockets/basics/
https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_client_applications
COMPARISON
https://www.google.com.au/search?safe=off&ei=MmNLWuGkLsHQ0AS87J_4Bw&q=websocket+vs+sse&oq=websocket+vs+sse
Problem: So can you please help me understand the difference between
these 2 libs (libs?) and help me decide which one is more suitable
here?
As I have mentioned before, you have technically only mentioned one package library (socket.io) which manages the websockets via their API. I would assume the other method would be to do it manually using the native JavaScript method. It is up to you which one you choose and you'll surely find that there are advantages and disadvantages with both.
We have a Requirement to send Mass notifications from Server to multiple clients (around 500 Clients) at once. We have implemented Node.js but we are not sure what is the performance bottlenecks as Socket connection from all the Clients to the server is open all the times. We are using this for Java Web application
Please let us know if anybody has any exposure on this
Thanks sai
This question is impossible to answer in any detail as you have provided no details about your code, not to even mention any code example.
But from my experience when people have problems with concurrency using Node and they are below 10,000 concurrent connections which you should handle easily (see this answer) then the usual suspect is that they are using blocking function calls.
Are you using any blocking code in your app? If so then here's your problem. If you have any blocking code in your Node app (like calling functions with "Sync" in their name) then you will have serious problems with concurrency. You should never use blocking functions anywhere else than on the first tick of the event loop (and if you don't know what it means then you should never use them at all).
Also, since you write that you're using Node for Java Web application it can be more complicated than that. If your Node app is connecting to Java app that spawns a new thread for every connection then the bottleneck may be in your Java app and not your Node app. As you provide no details about our architecture, it's impossible to say. But one this is for sure: if you want to handle massive concurrency then you cannot use threads. You need to use event loops. There's a reason why nginx or Redis are single-threaded. Threads don't scale, especially for I/O-bound use cases. Profile your code and see if it's your Java or Node code that needs fixing.
I have been using JS for simple front-end scripting for a while now, but am absolutely new to Node.js. After some surfing, I found out certain stuff about Node.js that it is fast, event-driven,uses modules, can be used both on server and client side, can be run from command line, etc.
As a project, the following task has been given to me:
"To develop a graph algorithm (such as minimum spanning tree) in javascript using node.js. Use the larger of the following graphs as inputs: http://snap.stanford.edu/data/ " [the link contains data from various network sites organised as nodes and edges and stored in .txt files]
Now I know how to implement a graph algorithm in a language (such as C), can even do it in JS using arrays. But I need some help regarding the "using node.js" part of the problem. What is its purpose in the problem ? Which of its features should I look up ?
Typically JS was made to run inside a browser.
Node.js is actually a javascript runtime invokable. You can invoke it from commandline. This means you can execute files of code from commandline like many other languages which you might be already familiar with. Beyond, that there is nothing much from your context.
But, yes it is fast, event-based, async and like server-scripting languages has server-handling capabilities inbuilt. That said it can be used for non-server contexts as well. Like computation in your case.
Node JS helps you to run backend logic, which is written on Javascript Language.
For example, in PHP, when you write backend code, you need some kind of application which will get all clients requests and run specific code to handle it. In PHP it will be done via Apache Server. In Java it will be done via Glassfish/JBoss/Tomcat.
Node JS is something like them, but for Javascript code.
Based on my understanding, only I/O in NodeJS is non-blocking. If we do, for example, lots of heavy math operations, other users cannot access to the server until it's done.
I was wondering if there is a non-blocking way to do heavy computation in NodeJS? Just curious.
If you have long-running calculations that you want to do with Node, then you will need to start up a separate process to handle those calculations. Generally this would be done by creating some number of separate worker processes and passing the calculations off to them. By doing this, you keep the main Node event loop unblocked.
On the implementation side of things, you have two main options.
The first being that you manually spin off child processes using Node's child-process API functions. This one is nice because your calculations wouldn't even have to be javascript. The child process running the calculations could even be C or something.
Alternatively, the Web Worker specification, has several implementations available through NPM if you search for 'worker'. I can't speak to how well these work since I haven't used any, but they are probably the way to go.
Update
I'd go with option one now. The current child process APIs support sending messages and objects between processes easily in a worker-like way, so there is little reason to use a separate worker module.
You can use Hook.io to run a separate node process for your heavy computation and communicate between the two. Hook.io is particularly useful because it has auto-healing meshes meaning that if one of your hooks (processes) crashes it can be restarted automatically.
Use multiple NodeJS instances and communicate over sockets.
Use multiple node instances and communicate over node-zeromq, HTTP, plain TCP sockets, IPC (e.g. unix domain sockets), JSON-RPC or other means. Or use the web workers API as suggested above.
The multiple instances approach has its advantages and disadvantages. Disadvantages are that there is a burden of starting those instances and implementing own exchange protocols. Advantages are that scaling to many computers (as opposed to many cores/processors within a single computer) is possible.
I think this is way to late, but this is an awesome feature of nodejs you've to know about.
The only way abot multithreading is by spawning new processes, right so far.
But nodejs has an implemented message feature between spawned node-forks.
http://nodejs.org/docs/latest/api/child_processes.html#child_process.fork
Great work from the devs, you can pass objects etc. as message to your childs and backwards
You can use node cluster module.
https://nodejs.org/api/cluster.html
I would use JXCore - it's a polished nodejs based engine which runs your code but has several options including the multi threading you are searching for. Running this in production is a charm!
Project's source: https://github.com/jxcore/jxcore
Features include:
Support for core Node.JS features
Embeddable Interface
Publish to Mobile Platforms (Android, iOS ..)
Supports Multiple JavaScript Engines
Multi-threading Capabilities
Process Configuration & Monitor
In-memory File System
Application Packaging
Support for the latest JavaScript features (ES6, ASM.JS ...)
Support for Universal Windows Platform (uwp) api