I am struggling with my node.js hobby project due to a "write after end" Error. I have a created a node.js webserver that amongst other things, sends commands received from a HTML page onwards to another process using the following code:
var netSocket = require('net').Socket();
netSocket.connect(9090);
netSocket.write(messages);
netSocket.end();
This works until the traffic starts to increase (i.e. the amount of messages being sent and or the size of the messages). At this point I get the following error:
Error: write after end
at writeAfterEnd (_stream_writable.js:132:12)
at Socket.Writable.write (_stream_writable.js:180:5)
at Socket.write (net.js:615:40)
at Socket.<anonymous> (/Users/mark/Documents/GitHub Repos/voice_controlled_zumo/speech_module/web_server_HTTPS.js:66:15)
at Socket.emit (events.js:95:17)
at Socket.onevent (/Users/mark/Documents/GitHub Repos/voice_controlled_zumo/node_modules/socket.io/lib/socket.js:327:8)
at Socket.onpacket (/Users/mark/Documents/GitHub Repos/voice_controlled_zumo/node_modules/socket.io/lib/socket.js:287:12)
at Client.ondecoded (/Users/mark/Documents/GitHub Repos/voice_controlled_zumo/node_modules/socket.io/lib/client.js:193:14)
at Decoder.Emitter.emit (/Users/mark/Documents/GitHub Repos/voice_controlled_zumo/node_modules/socket.io/node_modules/socket.io-parser/node_modules/component-emitter/index.js:134:20)
My guess is that the server at 9090 is being overwhelmed by the amount of traffic, giving rise to the error. As a complete newbie in the node.js world I'd really appreciate any hints for how I could resolve this issue.
Note also that the webserver is serving pages over SSL (in case that makes any difference).
Thanks for taking the time to read this!
Mark
NodeJS is a non-blocking async platform.
In your case,
netSocket.write(messages);
is an async method; therefore, netSocket.end() is called before write is complete.
The correct use would be:
netSocket.write(messages, function(err) { netSocket.end(); });
The second argument here is a callback function that will be called once the 'write' method finishes its job.
I would recommend you read/watch more about NodeJS, async styles and callbacks.
Here is a great place to start: https://www.youtube.com/watch?v=GJmFG4ffJZU
And of course, the NodeJS API docs regarding net sockets.
First, I think there is misinformation in another answer about socket.write() and socket.end(). It is perfectly normal and OK to do them back to back in the same tick:
socket.write(everythingIPlanToSend);
socket.end();
You do not need to provide a callback to write. The callback will tell you when the data has been fully flushed out the connection, but that is an optional notification that typical programs don't need to concern themselves with.
However, looking at your stack trace, I think your event handling control flow is broken in this manner. You have a socket.io client connected, and that emits events which you listen for. When those events fire you send them onward to your upstream server. Then you end the upstream socket connection. At that point, you must unbind (removeListener) your socket.io connection listener so that as more events arrive you don't attempt to send them up the connection you have already closed.
Another way to say it is whenever you call .end() on your upstream socket, you have to make sure that future incoming events from the browser do not make use of that same socket. OR you have to change your code to use the same upstream socket for all events from the corresponding browser socket (which is probably more efficient/correct), BUT in that case don't call .end() on it until the browser socket actually disconnects.
I had similar issue with node module compression, after i update it to the latest version 1.6, the issue was resolved
npm install compression#1.6.0
Related
I have a working WebRTC JavaScript application. Here is the problem: if during a web call there is a bad network connection, the call is stopped without WebRTC attempting to reconnect.
I would like to improve the code of my application by adding an automatic reconnection attempt system, however, in order to do so I need to understand some theory about WebRTC (and I think this can be very useful for many other developers).
Here are my questions:
Does WebRTC have a native functionality to attempt reconnection if the network is bad or should I listen for some "disconnection tigger" and call "a function" to start a new connection from zero?
If things cannot be done magically, what is/are the right "disconnection tigger/s" and "the function" from which the reconnection attempt process should restart? Is there something that can (or should) be taken from the previous connection?
I have read about an {iceRestart: true} parameter. Should this be used for the first call (then WebRTC will magically handle disconnection by attempting to reconnect) or should I use it ONLY when my code attempts to reconnect (on the 2nd, 3rd times...)?
What is the difference between connectionState "disconnected", "failed" and "closed" and does it have something to do with attempting to reconnect with bad network?
What is the best way to avoid attempting to reconnect in an infinite loop if there is no hope to reconnect (i.e: internet completely down)?
What is the difference between oniceconnectionstatechange and onconnectionstatechange? which is relevant in my case?
Thanks!
Luca
I was able to find the (JavaScript) solution through experimenting...
1) Does WebRTC have a native functionality to attempt reconnection if the network is bad or should I listen for some "disconnection tigger" and call "a function" to start a new connection from zero?
Yes, it does it by default in JavaScript, unless your code handles disconnection by proactively terminating the call through additional lines of instructions.
2) If things cannot be done magically, what is/are the right "disconnection tigger/s" and "the function" from which the reconnection attempt process should restart? Is there something that can (or should) be taken from the previous connection?
Things already happen under the hood (by magic). If the code terminates the call, it is probably because the disconnection trigger (ICE connection state = disconnected OR connection state = disconnected) triggers some additional code from the app you copy/pasted from somewhere.
3) I have read about an {iceRestart: true} parameter. Should this be used for the first call (then WebRTC will magically handle disconnection by attempting to reconnect) or should I use it ONLY when my code attempts to reconnect (on the 2nd, 3rd times...)?
Not useful in this scenario.
4) What is the difference between connectionState "disconnected", "failed" and "closed" and does it have something to do with attempting to reconnect with bad network?
You have to listen for connectionState = disconnected, the other ones don't matter for this purpose.
5) What is the best way to avoid attempting to reconnect in an infinite loop if there is no hope to reconnect (i.e: internet completely down)?
No problem, the reconnection WebRTC handles automatically will not cost anything in terms of signaling, therefore, you can try to reconnect as many times as you want, the user will eventually exit the call on his own if things are too slow.
6) What is the difference between oniceconnectionstatechange and onconnectionstatechange? which is relevant in my case?
No difference in this case, the only difference is that the ice state change is triggered right before the connection state change.
--
I hope this will be helpful to somebody!
I saw on the node.js documentation that "Listener functions must only perform synchronous operations".
But, when i tried to use fs.writeFileSync, it doesn't works :
net.createServer(function (socket)
{
[...]
socket.on('error', (err) => {
**fs.writeFileSync('C:\temp\test.txt', err.message);**
console.log('Connection error:', err.message);
});
}).listen(port);
Maybe I didn't understand something ?
The console.log just allows my script to not crash when the socket is abruptly closed, for example, due to an RST sent by a Load Balancer.
In production, I need to trace in a file when i get an error from the listening port (using TCP). But if I add the fs.writeFileSync line, my script keeps crashing and I don't get any log in my file. Is there any way to do this?
This quote from the documentation only applies to the event that it describes - process.on('exit'):
Listener functions must only perform synchronous operations. The Node.js process will exit immediately after calling the 'exit' event listeners causing any additional work still queued in the event loop to be abandoned.
This is not a general statement - you can trigger asynchronous operations in event listeners just fine (but note that the EventEmitter will not wait for your async work to finish - so be wary of concurrency issues). For logging, consider opening a file just once and appending to avoid additional overhead.
Usually, a logging library can take care of this for you. For example, winston can write directly to files. A popular alternative approach is to just log everything to stdout using a more light-weight library such as pino, and run Node.js under a process supervisor such as PM2 or docker that already captures the standard output and writes it to a file automatically.
First of all, I am well aware that Node.js is non-blocking before anything else, but in this very specific case, it must be blocking and waiting here.
I have an authentication process that works that way (using APIs, I didn't design this so I cannot modify the way the auth works):
I send a PUT request to a REST API, I get a HTTPResponse code that determines if the API understood the request.
The server I just requested through its API sends the full response (including error codes, etc) through an XMPP protocol.
This means, when I send the request, I cannot know what happened next, and must wait for the XMPP event to trigger (basically, an on("message", callback) event).
I'd like to know how to work with this with Node.js.
Two more things to know:
1) I'm working on a client/server architecture, and my Node.js server is doing this authentication process, and sending a response through the websocket to the client and waiting for a socket answer is out of the question (not my call, but my boss wants this process to be done in one pass).
2) It must not be done with the client socket and must go through the full Node.js process for various reasons.
Thanks a lot for your help people! \o/
Sorry for not answering previously, we had some severe hardware failure at work.
Anyway, I'm not answering one of your comments directly because I found a solution I prefer, even if I thank you for your help. I've decided to use a promise and to wait for its answer to be sure to get a proper response.
Here is the code:
var answer = await new Promise((accept, reject) => {
// If there are no stanza in 30 seconds, then the process failed or the stanza got missed.
setTimeout(() => {
reject("timed out");
}, (30 * 1000));
// Waiting for the xmpp event to trigger.
xmpp.on("stanza", function(stanza) {
// Processing of the received stanza goes here.
});
});
#gkatzioura solution was interesting, but this looked a little bit heavy on bandwidth and we are working on a large scale applications, or maybe I didn't fully understand it, then it is my mistake.
#pspi solution was also interesting but this would be a problem considering the XMPP event listener is inside the request, and the PUT request needs to send a body on its end() event and here it wouldn't really work for what I want to do. I think that's because the original post I made was somewhat unclear.
Thanks a lot for your help guys! :]
I don't know enough XMPP, but would this just be case of "putting dependent logic inside callback".
request.put(..., function () {
// dependent xmpp logic inside request callback
xmpp.on(..., function () {
// xmpp and put response received, now talk back to socket client
socket.send(...);
});
});
In your case I would proceed with the event emitter (or anything in a publish subscribe fashion).
Fire your http call and inside the handler add an emitter listener with a check if the events is for the corresponding authentication.
Meanwhile your xmpp connection once it receives the authorization it shall emit a message.
The listener will receive the message successfully and will use the callback of the http call.
I'm trying to find the definition of connect_timeout, when is it fired, what's the use for it?
Reading here : http://socket.io/docs/client-api/#manager(url:string,-opts:object)
Right now I have an app I tried to run without turning on the server, and it tries to connect and the event "Reconnecting" is fired for 4 attempts, one every 2 seconds. Then it says "Failed Reconnecting" when it hits the 4 attempt mark and fires the event "reconnect_failed".
I haven't been able to hit the connect_timeout event. How do I do that? When does it happen?
I was hoping Socket.IO had some sort of function of "CONNECTING" and then if it failed it would continue attempting "CONNECTING" and if that failed it would say "CONNECTION FAILED" and if it connected successfully at some point, it would then call "RECONNECTING" instead and if that failed after a certain amount of attempts it would say "RECONNECTING FAILED". Is that something that has to be programmed by me? I haven't seen it built in.
Connection timeout is when a client is connected to the server and it takes too long for a response to be received, causing the client to disconnect because it has stopped receiving anything from the server. This can be caused by faulty internet or if the client loses connection to the server (i.e. the clients internet becomes disconnected). This is true for most server based communications and is likely the same for socket.io.
I'm trying to use node.js as a server with socket.io (over a https connection server object, client connecting with option secure: true) with a JavaScript front end.
On the server if a login attempt fails I am calling:
socket.emit('unauthorized');
socket.disconnect();
On the client side I have:
this.Socket = io.connect(server, { secure: true });
this.Socket.on('disconnect', this.Socket_Disconnect);
this.Socket.on('unauthorized', this.Socket_Unauthorized);
I am able to see the disconnect event firing (with arguments set as ['booted'] regardless of whether i pass any arguments to socket.disconnect(), but the unauthorized event never makes it. Am I attempting to pass this message the wrong way?
The rationale I have here is that I want to be able to send different types of disconnect events like:
logout - clients should stop trying to connect until a user logs in
server reboot - clients should reattempt connection until they get through
I need to be able to tell on the client side why a client was disconnected by the server, is there at least some way to make sure a message has sent before calling socket.disconnect()?
Edit: this appears specific to the node.js cluster code - when the application is run on a single thread it works as expected.
Solved this by setting up a Redis server and configuring socket.io to use it within node.js if anyone finds this looking for a solution to the issue.