On Node.js api there lots of ifs and one can easily send request with some undefined var and crash the whole server until it re-starts again - something that could take up to 20 seconds.
I know that it should be checked if a variable is defined before working with it. But its very easy to forget something and keep working with an undefined var.
Is there a global definition to the server to avoid such of a drop down?
The easiest solution I could think of is implementing a cluster, in which only one process will go down, not the whole server. You could also make the process to go up again automatically. See more here
const cluster = require('cluster');
const http = require('http');
const numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
console.log(`Master ${process.pid} is running`);
// Fork workers.
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on('exit', (worker, code, signal) => {
console.log(`worker ${worker.process.pid} died`);
});
} else {
// Workers can share any TCP connection
// In this case it is an HTTP server
http.createServer((req, res) => {
res.writeHead(200);
res.end('hello world\n');
}).listen(8000);
console.log(`Worker ${process.pid} started`);
}
In any application there are a lot of "ifs" and assumptions. With JavaScript, being weakly typed and dynamic, you can really shoot yourself in the foot.
But the same rules apply here as any other language. Practice defensive programming. That is, cover all the bases in each function and statement block.
You can also try out programming Nodejs with Typescript. It ads some static type checking and other nice features that help you miss your foot when you shoot. You can also use (I think) Flow to statically type check things. But these won't make you a better programmer.
One other suggestion is to design your system as a SOA. So that one portion going down doesn't necessarily affect others. "Microservices" being a subset of that.
First, defensive programming and extensive testing are your friends. Obviously, preventing an issue before it happens is much better than trying to react to it after it happens.
Second, there is no foolproof mechanism for catching all exceptions at some high level and then putting your server back into a known, safe state. You just can't really do that in any complex server because you don't know what you were in the middle of when the exception happened. People will often try to do this, but it's like proceeding with a wounded server that may have some messed up internals. It's not safe or advisable. If a problem was not intercepted (e.g. exception caught or error detected) at the level where it occurred by code that knows how to properly handle that situation, then the only completely safe path forward is to restart your server.
So, if after implementing as much defensive programming as you possibly can and testing the heck out of it, you still want to prevent end-user downtime from a server crash/restart, then the best way to do that is to assume that a given server process will occasionally need to be restarted and plan for that.
The simplest way to prevent end-user downtime when a server process restarts is to use clustering and thus have multiple server processes with some sort of load balancer that both monitors server processes and routes incoming connections among the healthy server processes. When one server process is down, it is temporarily taken out of the rotation and other server processes can handle new, incoming connections. When the failed server process is successfully restarted, it can be added back to the rotation and be used again for new requests. Clustering in this way can be done within a single server (multiple processes on the same server) or across servers (multiple servers, each with server processes on them).
In some cases, this same process can even be used to roll out a new version of server code without any system downtime (doing this requires additional planning).
Related
Let's say I have a parent node process that forks a child process. I want to create a request-response mechanism between them, so that the child can send a message (JSON) to the parent and wait for a response and the other way as well.
I don't want to use HTTP because I want the mechanism to be as tight and low-latency as possible, and HTTP usually has some overhead. Also, HTTP is a client-server protocol and I want something bi-directional. (I'd be happy to know if the overhead is negliglbe or as bad as using pipes, TCP sockets directly, or WebSockets in this situation; it'd make my job easier).
The end-result I want is something like this. Note that this code is in TypeScript for clarity, not because the answer has to be in TypeScript too.
class Communicator {
// Callback for when a message is received by the Communicator
// Resolve returns a response to the sender, reject returns an error
onMessage : (data : any) => Promise<any>;
// Sends an object to the other process, and returns a promise
// that resolves with the response or rejects with the error reason
send(data : any) : Promise<any>;
}
In these case, I'm going to assume we use the native process.send mechanism, but if another system like WebSockets are better, it'd be a bit different:
// PARENT
let cp = require("child_process").fork("./child.js");
let comm = new Communicator(cp);
await comm.send({property : 1});
.
// CHILD
let comm = new Communicator(process);
await comm.send({property : 1});
The request/response system should support both Success-type and Error-type responses, as well as Error non-responses that indicate the underlying protocol failed.
How should I implement this in Node? There are lots of issues I can think of if I tried to do this naively, since multiple requests and responses can be running in parallel. Is there a library that already does this? Should I just use HTTP?
One possible solution to the problem appears to be to use WAMP over WebSockets:
WebSocket request-response subprotocol
However, since in this case we're talking about IPC and processes running inside a single machine, we're not really constrained with regard to the transfer protocol (we could even use raw sockets if we really wanted). There is also a built-in messaging system between parent and child processes.
So maybe it's possible to use WAMP over something else, or come up with another solution.
With regard to WAMP, one package we can use is autobahn.
I've a client to server Websocket connection which should be there for 40 seconds or so. Ideally it should be forever open.
The client continually sends data to server and vice-versa.
Right now I'm using this sequence:
var socket;
function senddata(data)
{
if (!socket)
{
socket = new WebSocket(url);
socket.onopen = function (evt) {
socket.send(data);
socket.onmessage = function (evt) {
var obj = JSON.parse(evt.data);
port.postMessage(obj);
}
socket.oneerror = function (evt) {
socket.close();
socket = null;
}
socket.onclose = function(evt){
socket = null;
}
}
}
else
{
socket.send(data);
}
}
Clearly as per current logic, in case of error, the current request data may not be sent at all.
To be frank it sometimes gives error that websocket is still in connecting state. This connection breaks often due to networking issues. In short it does not work perfectly well.
I've read a better design : How to wait for a WebSocket's readyState to change but does not cover all cases I need to handle.
Also I've Googled about this but could not get the correct procedure for this.
So what is the right way to send regular data through Websockets which handles well these issues like connection break etc?
An event you don't seem to cover is onclose. Which should work really well, since it's called whenever the connection terminates. This is more reliable than onerror, because not all connection disruptions result in an error.
I personally use Socket.IO, it enables real-time bidirectional event-based communication between client and server.
It is event driven. Events such as
on connection :: socket.on('conection',callback);
and
on disconnect :: socket.on('disconnect',callback);
are built in with socket.io so it can help you with your connection concerns. Pretty much very easy to use, check out their site if you are interested.
I use two-layer scheme on client: abstract-wrapper + websocket-client:
The responsibilities of the websocket-client are interacting with a server, recovering the connection and providing interfaces (event-emitter and some methods) to abstract-wrapper.
The abstract-wrapper is a high-level layer, which interacts with websocket-client, subscribes to its events and aggregating data, when the connection is temporary failed. The abstract-wrapper can provide to application layer any interface such as Promise, EventEmitter and so on.
On application layer, I just work with abstract-wrapper and don't worry about connection or data losing. Undoubtedly, it's a good idea to have here information about the status of connection and data sending confirmation, because it's useful.
If it is necessary, I can provide some code for example
This apparently is a server issue not a problem in the client.
I don't know how the server looks like here. But this was a huge problem for me in the past when I was working on a websocket based project. The connection would continuously break.
So I created a websocket server in java, and that resolved my problem.
websockets depend on lots of settings, like if you're using servlets then servlet container's settings matter, if you're using some php etc, apache and php settings matter, for example if you create a websocket server in php and php has default time-out of 30 seconds, it will break after 30 seconds. If keep-alive is not set, the connection wont stay alive etc.
What you can do as quick solution is
keep sending pings to a server after a certain amount of time (like 2 or 3 seconds, so that if a websocket is disconnected it is known to the client so it could invoke onclose or ondisconnect, I hope you know that there is no way to find if a connection is broken other than failing to send something.
check server's keep-alive header
If you have access to server, then it's timeouts etc.
I think that would help
Okay, I've spent over a week trying to figure this out to no avail, so if anyone has a clue, you are a hero. This isn't going to be an easy question to answer, unless I am being a dunce.
I am using node-http-proxy to proxy sticky sessions to 16 node.js workers running on different ports.
I use Socket.IO's Web Sockets to handle a bunch of different types of requests, and use traditional requests as well.
When I switched my server over to proxying via node-http-proxy, a new problem crept up in that sometimes, my Socket.IO session cannot establish a connection.
I literally can't stably reproduce it for the life of me, with the only way to turn it on being to throw a lot of traffic from multiple clients to the server.
If I reload the user's browser, it can then sometimes re-connect, and sometimes not.
Sticky Sessions
I have to proxy sticky sessions as my app authenticates on a per-worker basis, and so it routes a request based on its Connect.SID cookie (I am using connect/express).
Okay, some code
This is my proxy.js file that runs in node and routes to each of the workers:
var http = require('http');
var httpProxy = require('http-proxy');
// What ports the proxy is routing to.
var data = {
proxyPort: 8888,
currentPort: 8850,
portStart: 8850,
portEnd: 8865,
};
// Just gives the next port number.
nextPort = function() {
var next = data.currentPort++;
next = (next > data.portEnd) ? data.portStart : next;
data.currentPort = next;
return data.currentPort;
};
// A hash of Connect.SIDs for sticky sessions.
data.routes = {}
var svr = httpProxy.createServer(function (req, res, proxy) {
var port = false;
// parseCookies is just a little function
// that... parses cookies.
var cookies = parseCookies(req);
// If there is an SID passed from the browser.
if (cookies['connect.sid'] !== undefined) {
var ip = req.connection.remoteAddress;
if (data.routes[cookies['connect.sid']] !== undefined) {
// If there is already a route assigned to this SID,
// make that route's port the assigned port.
port = data.routes[cookies['connect.sid']].port;
} else {
// If there isn't a route for this SID,
// create the route object and log its
// assigned port.
port = data.currentPort;
data.routes[cookies['connect.sid']] = {
port: port,
}
nextPort();
}
} else {
// Otherwise assign a random port, it will/
// pick up a connect SID on the next go.
// This doesn't really happen.
port = nextPort();
}
// Now that we have the chosen port,
// proxy the request.
proxy.proxyRequest(req, res, {
host: '127.0.0.1',
port: port
});
}).listen(data.proxyPort);
// Now we handle WebSocket requests.
// Basically, I feed off of the above route
// logic and try to route my WebSocket to the
// same server regular requests are going to.
svr.on('upgrade', function (req, socket, head) {
var cookies = parseCookies(req);
var port = false;
// Make sure there is a Connect.SID,
if (cookies['connect.sid'] != undefined) {
// Make sure there is a route...
if (data.routes[cookies['connect.sid']] !== undefined) {
// Assign the appropriate port.
port = data.routes[cookies['connect.sid']].port;
} else {
// this has never, ever happened, i've been logging it.
}
} else {
// this has never, ever happened, i've been logging it.
};
if (port === false) {
// this has never happened...
};
// So now route the WebSocket to the same port
// as the regular requests are getting.
svr.proxy.proxyWebSocketRequest(req, socket, head, {
host: 'localhost',
port: port
});
});
Client Side / The Phenomena
Socket connects like so:
var socket = io.connect('http://whatever:8888');
After about 10 seconds on logging on, I get this error back on this listener, which doesn't help much.
socket.on('error', function (data) {
// this is what gets triggered. ->
// Firefox can't establish a connection to the server at ws://whatever:8888/socket.io/1/websocket/Nnx08nYaZkLY2N479KX0.
});
The Socket.IO GET request that the browser sends never comes back - it just hangs in pending, even after the error comes back, so it looks like a timeout error. The server never responds.
Server Side - A Worker
This is how a worker receives a socket request. Pretty simple. All workers have the same code, so you think one of them would get the request and acknowledge it...
app.sio.socketio.sockets.on('connection', function (socket) {
// works... some of the time! all of my workers run this
// exact same process.
});
Summary
That's a lot of data, and I doubt anyone is willing to confront it, but i'm totally stumped, don't know where to check next, log next, whatever, to solve it. I've tried everything I know to see what the problem is, to no avail.
UPDATE
Okay, I am fairly certain that the problem is in this statement on the node-http-proxy github homepage:
node-http-proxy is <= 0.8.x compatible, if you're looking for a >=
0.10 compatible version please check caronte
I am running Node.js v0.10.13, and the phenomena is exactly as some have commented in github issues on this subject: it just drops websocket connections randomly.
I've tried to implement caronte, the 'newer' fork, but it is not at all documented and I have tried my hardest to piece together their docs in a workable solution, but I can't get it forwarding websockets, my Socket.IO downgrades to polling.
Are there any other ideas on how to get this implemented and working? node-http-proxy has 8200 downloads yesterday! Sure someone is using a Node build from this year and proxying websockets....
What I am look for exactly
I want to accomplish a proxy server (preferrably Node) that proxies to multiple node.js workers, and which routes the requests via sticky sessions based on a browser cookie. This proxy would need to stably support traditional requests as well as web sockets.
Or...
I don't mind accomplishing the above via clustered node workers, if that works. My only real requirement is maintaining sticky sessions based on a cookie in the request header.
If there is a better way to accomplish the above than what I am trying, I am all for it.
In general I don't think node is not the most used option as a proxy server, I, for one use nginx as a frontend server for node and it's a really great combination. Here are some instructions to install and use the nginx sticky sessions module.
It's a lightweight frontend server with json like configuration, solid and very well tested.
nginx is also a lot faster if you want to serve static pages, css. It's ideal to configure your caching headers, redirect traffic to multiple servers depending on domain, sticky sessions, compress css and javascript, etc.
You could also consider a pure load balancing open source solution like HAProxy. In any case I don't believe node is the best tool for this, it's better to use it to implement your backend only and put something like nginx in front of it to handle the usual frontend server tasks.
I agree with hexacyanide. To me it would make the most sense to queue workers through a service like redis or some kind of Message Query system. Workers would be queued through Redis Pub/Sub functionality by web nodes(which are proxied). Workers would callback upon error, finish, or stream data in realtime with a 'data' event. Maybe check out the library kue. You could also roll your own similar library. RabbitMQ is another system for similar purpose.
I get using socket.io if you're already using that technology, but you need to use tools for their intended purpose. Redis or a MQ system would make the most sense, and pair great with websockets(socket.io) to create realtime, insightful applications.
Session Affinity(sticky sessions) is supported through Elastic LoadBalancer for aws, this supports webSockets. A PaaS provider(Modulus) does this exactly. Theres also satalite which provides sticky sessions for node-http-proxy, however I have no idea if it supports webSockets.
I've been looking into something very similar to this myself, with the intent of generating (and destroying) Node.js cluster nodes on the fly.
Disclaimer: I'd still not recommend doing this with Node; nginx is more stable for the sort of design architecture that you're looking for, or even more so, HAProxy (very mature, and easily supports sticky-session proxying). As #tsturzl indicates, there is satellite, but given the low volume of downloads, I'd tread carefully (at least in a production environment).
That said, since you appear to have everything already set up with Node, rebuilding and re-architecting may be more work than it's worth. Therefore, to install the caronte branch with NPM:
Remove your previous http-node-proxy Master installation with npm uninstall node-proxy and/or sudo npm -d uninstall node-proxy
Download the caronte branch .zip and extract it.
Run npm -g install /path/to/node-http-proxy-caronte
In my case, the install linkage was broken, so I had to run sudo npm link http-proxy
I've got it up and running using their basic proxy example -- whether or not this resolves your dropped sessions issue or not, only you will know.
I'm quite new to Node.js and I have a request for an application that would receive a payload of UDP packets and process it.
I'm talking about more than 400 messages per second, which would reach something like 200.000 messages/minute.
I have written a code to setup a UDP server (grabbed from docs here http://nodejs.org/api/all.html#all_udp_datagram_sockets actually) but it's losing something around 5% of the packets.
What I really need to develop is a server which would get the packet and send it to another worker do the job with the message. But looks like threading in node.js is a nightmare.
This is my core as is:
var dgram = require("dgram");
var fs = require("fs");
var stream = fs.createWriteStream("received.json",{ flags: 'w',
encoding: "utf8",
mode: 0666 });
var server = dgram.createSocket("udp4");
server.on("message", function (msg, rinfo) {
console.log("server got: " + msg + " from " +
rinfo.address + ":" + rinfo.port);
stream.write(msg);
});
server.on("listening", function () {
var address = server.address();
console.log("server listening " +
address.address + ":" + address.port);
});
server.bind(41234);
// server listening 0.0.0.0:41234
You are missing concepts, NodeJS is not meant to be multi-thread in terms of you mean, requests should be handled in a cycle. No other thread exists so no context-switches happens. In a multi-core environment, you can create a cluster via node's cluster module, I have a blog-post about this here.
You set the parent proceses to fork child processes, and ONLY child processes should bind to a port. Your parent proceses will handle the load balancing between children.
Note: In my blog post, I made i < os.cpus().length / 2; but it should be i < os.cpus().length;
I wrote a soap/xml forwarding service with a similar structure, and found that the info would come in 2 packets. I needed to update my code to detect 2 halves of the message and put them back together. This payload size thing may be more of an HTTP issue than a udp issue, but my suggestion is that you add logging to write out everything you are receiving and then go over it with a fine tooth comb. It looks like you would be logging what you are getting now, but you may have to dig into the 5% that you are losing.
How do you know its 5%?
if you send that traffic again, will it always be 5%?
are the same messages always lost.
I built a UDP server for voip/sip call data using ruby and Event Machine, and so far things have been working well. (I'm curious about your test approach though, I was doing everything over netcat, or a small ruby client, I never did 10k messages)
A subtle tip here. Why did you use UDP for a stream? You need to use TCP for streams. The UDP protocol sends datagrams, discrete messages. It will not break them apart on you behind the scenes. What you send is what you receive with UDP. (IP fragmentation is a different issue I'm not talking about that here). You don't have to be concerned with re-assembling a stream on the other side. That's one of the main advantages of using UDP instead of TCP. Also, if you are doing localhost to localhost you don't have to worry about losing packets due to network hiccups. You could lose packets if you overflow the network stack buffers though so give yourself big ones if you are doing high speed data transfer. So, forget about the stream, just use UDP send:
var udp_server = dgram.createSocket({
type: 'udp4',
reuseAddr: true,
recvBufferSize: 20000000 // <<== mighty big buffers
});
udp_server.send("Badabing, badaboom", remote_port, remote_address);
Go was developed by Google to deal with the proliferation of languages that occurs in modern tech shops. (it's crazy, I agree). I cannot use it because its culture and design prohibit using Exceptions which are the most important feature modern languages have for removing the huge amount of clutter added by old fashion error handling. Other than that, it's fine but that's a show stopper for me.
I'm making a multiplayer (2 player) browser game in JavaScript. Every move a player makes will be sent to a server and validated before being transmitted to the opponent. Since WebSockets isn't ready for prime time yet, I'm looking at long polling as a method of transmitting the data and node.js looks quite interesting! I've gone through some example code (chat examples, standard long polling examples and suchlike) but all the examples I've seen seem to broadcast everything to every client, something I'm hoping to avoid. For general server messages this is fine but I want two players to be able to square off in a lobby or so and go into "private messaging" mode.
So I'm wondering if there's a way to implement private messaging between two clients using nodejs as a validating bridge? Something like this:
ClientA->nodejs: REQUEST
nodejs: VALIDATE REQUEST
nodejs->ClientA: VALID
nodejs->ClientB: VALID REQUEST FROM ClientA
You need some way to keep track of which clients are in a lobby together. You can do this with a simple global array like so process.lobby[1] = Array(ClientASocket, ClientBSocket) or something similar (possibly with some additional data, like nicknames and such), where the ClientXSocket is the socket object of each client that connects.
Now you can hook the lobby id (1 in this case) onto each client's socket object. A sort of session variable (without the hassle of session ids) if you will.
// i just made a hashtable to put all the data in,
// so that we don't clutter up the socket object too much.
socket.sessionData['lobby'] = 1;
What this allows you to do also, is add an event hook in the socket object, so that when the client disconnects, the socket can remove itself from the lobby array immediately, and message the remaining clients that this client has disconnected.
// see link in paragraph above for removeByValue
socket.on('close', function(err) {
process.lobby[socket.sessionData['lobby']].removeByValue(socket);
// then notify lobby that this client has disconnected.
});
I've used socket in place of the net.Stream or request.connection or whatever the thing is.
Remember in HTTP if you don't have keep-alive connections, this will make the TCP connection close, and so of course make the client unable to remain within a lobby. If you're using a plain TCP connection without HTTP on top (say within a Flash application or WebSockets), then you should be able to keep it open without having to worry about keep-alive. There are other ways to solve this problem than what I've shown here, but I hope I got you started at least. The key is keeping a persistent object for each client.
Disclaimer: I'm not a Node.js expert (I haven't even gotten around to installing it yet) but I have been reading up on it and I'm very familiar with browser js, so I'm hoping this is helpful somehow.