Develop Reference

Closing single node server connection closes them all

I barely ask any questions on Stack Overflow, but this one is beyond me. I guess I'm missing something basic as I'm pretty new to Node server.
Our application is pretty basic. The server is supposed to receive a handful of text lines (data), merge and parse them, and once the connection is closed (data sending is over) it sends the data to the api.
var net = require('net');
var fs = require('fs');
const axios = require('axios')
const server = new net.Server();
server.listen(PORT, IP);
server.on("connection", client => {
client.write("Hello\n");
console.log('connected');
let received = "";
client.on("data", data => {
received += data
console.log("Partial data is: " + data);
});
client.on("close", () => {
received = received.toString('utf8');
fs.appendFile('log.txt', received, function (err) {});
received = received.replace(/(?:\r\n|\r|\n)/g, "||");
axios.post(APIADDRESS, {data: received});
console.log('Full data is: '+ {data: received});
});
});
To send the data I'm simply running a netcat or nc using the netcat ipaddress port, that's not a problem. It's connecting fine, status message is received.
The thing is - once I open two or more connections from two DIFFERENT SSh servers something weird happens. I can send the line after line just fine. The server reports back "partial data" debug without problem, for both of them.
However, once I close one of the connections (ctrl+c) they BOTH close.
In the end, only the data from the manually closed connection is received. The other one, from a separate nc on a separate ssh server never reaches the client.on("close") part, it seems. It's just terminated for no reason.
Any ideas? I don't even know where to start.
//EDIT
Just tested it from my pc and some ssh mobile app using separated SSH servers. As soon as ctrl+c is sent at any device it closes the connection for all clients.
//Forgot to mention I'm running pm2 to keep the server up. Once I turned on the script by hand, ignoring pm2 - it works fine. Weird. It is happening because of PM2.

I would guess that you have Putty configured to ‘Share SSH connections if possible’. Per some doc, when doing so:
When this mode is in use, the first PuTTY that connected to a given server becomes the ‘upstream’, which means that it is the one managing the real SSH connection. All subsequent PuTTYs which reuse the connection are referred to as ‘downstreams’: they do not connect to the real server at all, but instead connect to the upstream PuTTY via local inter-process communication methods.
So, if you Ctrl+C the PuTTY session that is managing the actual shared connection, they both lose their connection.
You could presumably disable this shared connection feature at either the client or server end of things since both must be enabled for sharing to occur.

To anyone coming here in the future.
If you are using pm2 with --watch enabled and the text log file is in the same folder as your main server script... That's the reason why it drops the connection after a single client disconnects. It just detects that the log has changed.
I'm not facepalming, that's not even funny.

How synchronise socketIO connection ID's on client and server?

I have a javascript GameClient that uses SocketIO to send messages to a nodeJs server. Multiple users can open the GameClient separately and send messages to the server.
GameClient
GameClient ---> NodeJS Server
GameClient
The server can send messages to specific clients using io.to(socketid).emit(). The code looks something like this:
CLIENT
this.socket = io({ timeout: 60000 })
this.socket.on('connect', () => Settings.getInstance().socketid = this.socket.id)
this.socket.on('reconnect', (attemptNumber:number) => console.log("reconnecting..."))
const json = JSON.Stringify({socketid:this.socket.id, name:"Old Billy Bob"})
this.socket.emit('user created', json)
SERVER (simplified for clarity, just keeping track of one user here)
user = {}
io.on('connection', (socket) => {
console.log('new connection')
socket.on('disconnect', () => {
console.log('user disconnected')
});
socket.on('user created', (json) => {
user = JSON.parse(json)
});
});
// demo code, send a message to our user
io.to(user.socketid).emit("message to one user")
PROBLEM
When the client browser tab becomes inactive for any reason at all, the client disconnects and reconnects and gets a new socket connection ID. This actually happens a lot in Chrome and Safari.
The server only knows the old connection id, so now it can't send direct messages any more. How do I keep the socket connection id synchronised on the client and server?
Since the server also gets a reconnected event, how does it know which user reconnected?

The answer to your question is quite simple: you need a way to identify who is who. And that is not socket.id because this only identifies sockets, not users, as you've already noticed.
So you need some authentication mechanism. Once a user authenticates you can reuse his true id (whether it is simply a name or an integer in a database is irrelevant). And then on the server side you keep a collection of pairs (true_id, socket_id). And whenever a message comes to that user, you broadcast it to all matched socket.io objects.
Edit: So here's the flow:
Client authenticates with the server, the server sends him his own true_id, which the client stores somewhere. The client may also store some session_id or maybe some other mechanism that will allow him fast reauthentication in case of disconnection (note: do not store credentials, its a security issue).
The server keeps track of (true_id, socket_id) pairs in the form of a double way, mutlivalue map (it's an implementation detail what kind of data structure should be used here, maybe two {} objects is enough). If a connection dies then (true_id, socket_id) entry is removed. Note that for a given true_id there still may be some other socket_id alive. And so it doesn't mean that the user disconnected. It only means that this particular channel is dead.
Users don't care about socket_id, they only care about true_id. What you emit is {target_id: true_id, ...} instead of {target_id: socket_id, ...} on the client side, when you want to send a direct message.
When the server receives such message with true_id inside, it retrieves all (true_id, socket_id) pairs and passes the message to all of these sockets (note: maybe you don't even need socket_id, you can simply store socket objects here). Although this is a business logic: do you allow multiple connections per user? I would. There are many edge cases here (like for example a client thinks that he disconnected, but the server thinks he is still connected, etc) and making this 100% correct is unfortunately impossible (due to the nature of networking). But with a bit of effort it is possible to make it work 99% of the time.
If a connection dies then it is your client's responsibility to automatically reconnect and reauthenticate. New socket_id for old true_id is generated on the server side.
Let me emphasize this again: clients don't care about socket_id at all. Because that doesn't identify them. This only identifies a channel. And only the server cares about this information.

Change will on a mqtt connection

We are using the mqtt.js (https://www.npmjs.com/package/mqtt) client to connect to AWS IoT service.
We can connect no problem and pass in last will with the following code
var clientOptions = {
will: {
topic: "logout",
payload: JSON.stringify({ _id: User.me._id, viewing: User.me.viewing })
}
};
client = mqtt.connect(signedUrl, clientOptions);
Now I want to update the will portion of the options--change the payload to have a new viewing property.
Is there a way to update the will without disconnecting and triggering the old will?

There is no way to change the Last Will and Testament, it can only be set in the connect packet.
But the Last Will and Testament should only fire if the client times out, not on a clean disconnect. This means you should be able to tell the client to disconnect and reconnect with a new LWT without triggering the old one to be published.
It you use clean session false and subscribe at QOS1 or better then you should not miss any messages when you reconnect as the broker should queue and deliver them on reconnect.

Correct way to handle Websocket

I've a client to server Websocket connection which should be there for 40 seconds or so. Ideally it should be forever open.
The client continually sends data to server and vice-versa.
Right now I'm using this sequence:
var socket;
function senddata(data)
{
if (!socket)
{
socket = new WebSocket(url);
socket.onopen = function (evt) {
socket.send(data);
socket.onmessage = function (evt) {
var obj = JSON.parse(evt.data);
port.postMessage(obj);
}
socket.oneerror = function (evt) {
socket.close();
socket = null;
}
socket.onclose = function(evt){
socket = null;
}
}
}
else
{
socket.send(data);
}
}
Clearly as per current logic, in case of error, the current request data may not be sent at all.
To be frank it sometimes gives error that websocket is still in connecting state. This connection breaks often due to networking issues. In short it does not work perfectly well.
I've read a better design : How to wait for a WebSocket's readyState to change but does not cover all cases I need to handle.
Also I've Googled about this but could not get the correct procedure for this.
So what is the right way to send regular data through Websockets which handles well these issues like connection break etc?

An event you don't seem to cover is onclose. Which should work really well, since it's called whenever the connection terminates. This is more reliable than onerror, because not all connection disruptions result in an error.

I personally use Socket.IO, it enables real-time bidirectional event-based communication between client and server.
It is event driven. Events such as
on connection :: socket.on('conection',callback);
and
on disconnect :: socket.on('disconnect',callback);
are built in with socket.io so it can help you with your connection concerns. Pretty much very easy to use, check out their site if you are interested.

I use two-layer scheme on client: abstract-wrapper + websocket-client:
The responsibilities of the websocket-client are interacting with a server, recovering the connection and providing interfaces (event-emitter and some methods) to abstract-wrapper.
The abstract-wrapper is a high-level layer, which interacts with websocket-client, subscribes to its events and aggregating data, when the connection is temporary failed. The abstract-wrapper can provide to application layer any interface such as Promise, EventEmitter and so on.
On application layer, I just work with abstract-wrapper and don't worry about connection or data losing. Undoubtedly, it's a good idea to have here information about the status of connection and data sending confirmation, because it's useful.
If it is necessary, I can provide some code for example

This apparently is a server issue not a problem in the client.
I don't know how the server looks like here. But this was a huge problem for me in the past when I was working on a websocket based project. The connection would continuously break.
So I created a websocket server in java, and that resolved my problem.
websockets depend on lots of settings, like if you're using servlets then servlet container's settings matter, if you're using some php etc, apache and php settings matter, for example if you create a websocket server in php and php has default time-out of 30 seconds, it will break after 30 seconds. If keep-alive is not set, the connection wont stay alive etc.
What you can do as quick solution is
keep sending pings to a server after a certain amount of time (like 2 or 3 seconds, so that if a websocket is disconnected it is known to the client so it could invoke onclose or ondisconnect, I hope you know that there is no way to find if a connection is broken other than failing to send something.
check server's keep-alive header
If you have access to server, then it's timeouts etc.
I think that would help

Error while getting messges from sockets in javascript

Hi i am having trouble with creating a socket communication from java script code.
I am always getting error while sending a message or closing the socket from server.
My Socket server code.
// Start listening for connections.
while (true)
{
Console.WriteLine("Waiting for a connection...");
// Program is suspended while waiting for an incoming connection.
Socket handler = listener.Accept();
data = null;
// An incoming connection needs to be processed.
while (true)
{
int bytesRec = handler.Receive(bytes);
data += Encoding.ASCII.GetString(bytes, 0, bytesRec);
break;
}
// Show the data on the console.
Console.WriteLine("Text received : {0}", data);
// Echo the data back to the client.
byte[] msg = Encoding.ASCII.GetBytes(data);
handler.Send(msg);
handler.Shutdown(SocketShutdown.Both);
handler.Close();
}
JavaScript code:
var connection = new WebSocket('ws://Myip:11000', ['soap', 'xmpp']);
// When the connection is open, send some data to the server
connection.onopen = function () {
connection.send('Ping'); // Send the message 'Ping' to the server
connection.send('your message');
};
// Log errors
connection.onerror = function (error) {
console.log('WebSocket Error ' + error);
};
connection.onclose = function (msg) {
console.log('WebSocket Error ' + msg);
};
It gets connected to server socket, but always gets error while closing or sending a message from server.

If this is really your actual code:
handler.Send(msg);
handler.Shutdown(SocketShutdown.Both);
handler.Close();
…then it's pretty broken. First, you can't assume that Socket.Send() actually sends all the bytes you asked it to. You have to check the return value, and keep sending until you've actually sent all the data.
Second, the initiation of a graceful closure should use SocketShutdown.Send, not SocketShutdown.Both. Specifying "Both" means (among other things) that you're not going to wait for the other end to negotiate the graceful closure. That you're just done and won't even receive any more data, in addition to being done sending.
And of course, the code is calling Close() before the other end has in fact acknowledged the graceful closure (by itself sending any remaining data it wanted to send and then shutting down with "Both").
Is all this the reason for your problem? I can't say for sure, since I have no way to test your actual code. But it's certainly a reasonable guess. If you tear down the connection without waiting after you try to send something, there's not any guarantee that the data will ever leave your machine, and in any case the other end could easily see the connection reset before it gets a chance to process any data that was sent to it.
There aren't a huge number of rules when it comes to socket programming, but what rules exist are there for a reason and are generally really important to follow. You should make sure your code is following all the rules.
(The rest of the code is also different from what I'd consider the right way to do things, but the problems aren't entirely fatal, the way that the over-eager connection destruction is).

I am afraid WebSocket does not work that way.
When the Javascript code connects to the server, it will send a HTTP request as ASCII text. That request will include a HTTP header Sec-WebSocket-Protocol: soap, xmpp, as you are requiring those protocols in your WebSocket creation.
Since your server code does not reply with an appropiate HTTP response accepting the websocket connection, the connection will fail. When you try to send data back, the client will not recognize it as a HTTP response and a error will be thrown.
A websocket is not a regular socket connection, it won't work that way. It requires HTTP negotiation and there is a schema for data framing. I recommend you to go through this article that explains very well how it works: http://chimera.labs.oreilly.com/books/1230000000545/ch17.html
If you are interested in learning how to develop a server, take a look to this tutorial in MDN: https://developer.mozilla.org/en-US/docs/WebSockets/Writing_WebSocket_server I also have an open source WebSocket server in C# you can take a look if you like.