Develop Reference

Adding information about the client on the initial Websocket handshake

I'm currently creating a distributed chat application. Everything works fine, meaning it's possible to send messages between the clients and the server and have it broadcasted appropiately.
However, at the moment only the actual message is sent to the server. I would like to add information about the user sending the message aswell.
I could add this information whenever I send a new message, but I would prefer if I could add this information during the initial handshake and then save this information on the backend.
I've thought about sending some information in the URL, but as I only instantiate the websocket once, this does not seem like the way to go. Similarly, I thought about adding the information as the body of the request, but I read that having a body on a GET request is usually not recommended.
So my question is, am I trying to do something that I should not be going for? Should I just send information about the client on each new message that is sent to the server?
Currently, my client looks like this:
const socket = new WebSocket("ws://localhost:8080/ws");
const connect = (cb) => {
console.log("Attempting Connection...")
socket.onopen = () => {
console.log("Successfully Connected");
}
socket.onmessage = (msg) => {
console.log(msg)
cb(msg);
}
socket.onclose = (event) => {
console.log("Socket Closed Connection: ", event);
}
socket.onerror = (error) => {
console.log("Socket Error: ", error);
}
};
const sendMsg = (msg) => {
console.log("Sending msg: ", msg);
socket.send(msg);
}
And the initial connection on the backend is handled by the following:
func serveWs(pool *websocket.Pool, w http.ResponseWriter, r *http.Request) {
fmt.Println("WebSocket Endpoint Hit")
conn, err := websocket.Upgrade(w, r)
if err != nil {
fmt.Fprintf(w, "%+V\n", err)
}
client := &websocket.Client{
Name: "?????", // Obviously, I would like the actual name to be here.
Conn: conn,
Pool: pool,
}
pool.Register <- client
client.Read()
}

The normal thing to do here as selbie pointed out is to expect from the client a special first message through WebSocket. If that message is not recived or does not meet the requirements the server ends the WebSocket conn.
Using socket.onopen is very common for this task.
More on why you cant put headers:
HTTP headers in Websockets client API

How can I work around the promises that serialport nodejs package works with

I've started a project that requires communication between an arduino and a local nodejs server (unrelated the data will be sent via an http request or a socket to the actual remote server later on). I'm using the node package serialport. At the beginning of the serial communication, the server needs to "find" the arduino. I've decided on the following negotiation codex:
1) the server sends a "c" character (as in connect) which the arduino is listening for
2) the arduino replies to all "c"s with another "c" which the server will be listening for
in other words when both sides receive a "c" that means the serial connection works
However, due to serialport using promises I can't go through all available ports and check if there's an arduino (which replies with "c") there.
Here's what I've come up with so far:
var SerialPort = require('serialport');
var Readline = require('#serialport/parser-readline');
async function tryPort(path) {
var port = new SerialPort(path, {
baudRate: 9600
});
port.on('error', function (err) {
console.log(err);
});
port.pipe(new Readline({ delimiter: '\n' })).on('data', (data)=>{
console.log(port);
console.log(data);
if (data == 'c') {
return port;
}
port.close();
});
port.write("c", function (err) {
if (err) console.log(err);
});
}
async function connect() {
var connection, ports = await SerialPort.list();
for(i=0;i<ports.length;i++){
connection = await tryPort(ports[i].path);
}
setTimeout(() => {
if (!connection) {
console.log("no port/response found");
}else{
console.log(connection);
}
}, 3000);
}
connect();
I went with the assumption the variable 'connection' will be assigned the value of the port that responded correctly last because that port will take the longest to finish. Unfortunately, it seems this won't work with promises... So I'm wondering if there's any other way to accomplish it?

WebSocket stops working in Vue/Node application

I have a Node/Vue application. I am consuming a WebSocket from Binance, a crypto exchange. I can see the quotes on the server console as I log them, I can send them to the browser for a short period of time before the client stops logging them.
Browser just using WebSocket API
Node using ws library
Node code, this I am running as it's own service as its just this.
'use strict';
const WebSocket = require('ws');
const binanceWS = new WebSocket('wss://stream.binance.com:9443/ws/btcusdt#trade')
const server = new WebSocket.Server({ port: 5002 });
//websocket connection event will return a socket you can later use
binanceWS.on("open", function() {
console.log("connected to Binance");
});
binanceWS.on('message', function(data){
console.log(data);
server.on('connection', function connection(ws){
console.log("Connected a new client");
ws.send(data);
});
server.on('closed', function (id){
console.log("connection closed");
console.log(id);
});
server.on('error', function (err){
console.log(err)
})
})
On the Client side I am using Vue and in the app.js file I have this on the created hook.
let socket = new WebSocket("ws://localhost:5002")
socket.addEventListener('message', function(event){
let quotes = JSON.parse(event.data);
console.log(quotes.p)
});
socket.addEventListener('error', function(event){
console.log("closing because " + event);
})
Right now I am only listening to the consoles in the above app.vue file.
What I see in the browser console is a lot of quotes, then they stop after a second or 2. There can be over a thousand quotes in some times. Then on occasion I see a console.log('created') that I have in a child component of app.vue. In many cases this is the last thing in the console after hundreds of quotes.
In the console.log for the server I see a lot of sessions being created with one page refresh. So much that it fills my console.
So I'm not sure I am creating the connections correcly, I am not sure if Vue is somehow stopping the console.log's?
I don't see any errors anywhere and the entire time in my server console the Binance API continues streaming.

you have to write server event listener outside binance on message handler;
then you can pass messages from binance to the server by emitting new event to the server
on receiving message from binance you can send data to all connection on the server
Or Try this code I think it will work :
'use strict';
const WebSocket = require('ws');
const binanceWS = new WebSocket('wss://stream.binance.com:9443/ws/btcusdt#trade')
const server = new WebSocket.Server({ port: 5002 });
server.on('connection', function connection(ws){
console.log("Connected a new client");
});
server.on('closed', function (id){
console.log("connection closed");
console.log(id);
});
server.on('error', function (err){
console.log(err)
})
//websocket connection event will return a socket you can later use
binanceWS.on("open", function() {
console.log("connected to Binance");
});
binanceWS.on('message', function(data){
console.log(data);
server.clients.forEach(function each(client) {
if (client.readyState === WebSocket.OPEN) {
client.send(data);
}
});
})

How to use server-sent-events in express.js

I setup my REST server with express.js. Now I want to add sse to this server. After I implemented this sse package, I get an error. I know that I get this error, when would try to use res.send twice, but I am not.
ERROR: Error: Can't set headers after they are sent.
at ServerResponse.OutgoingMessage.setHeader (http.js:690:11)
at ServerResponse.header (/home/root/node_modules/express/lib/response.js:718:10)
at ServerResponse.send (/home/root/node_modules/express/lib/response.js:163:12)
at app.get.str (/home/root/.node_app_slot/main.js:1330:25)
at Layer.handle [as handle_request] (/home/root/node_modules/express/lib/router/layer.js:95:5)
at next (/home/root/node_modules/express/lib/router/route.js:131:13)
at sse (/home/root/node_modules/server-sent-events/index.js:35:2)
at Layer.handle [as handle_request] (/home/root/node_modules/express/lib/router/layer.js:95:5)
at next (/home/root/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (/home/root/node_modules/express/lib/router/route.js:112:3)
Is it possible that I can't use the express methods anymore within the sse function? For example:
app.get('/events', sse, function(req, res) {
res.send('...');
});
Furthermore, I found this solution and this. Is it possible to make sse with the res.write function or in another way without using another package?

I disagree with using Socket.IO to implement basic Server-Sent Events. The browser API is dead simple and the implementation in Express requires only a couple of changes from a normal response route:
app.get('/streaming', (req, res) => {
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('Connection', 'keep-alive');
res.flushHeaders(); // flush the headers to establish SSE with client
let counter = 0;
let interValID = setInterval(() => {
counter++;
if (counter >= 10) {
clearInterval(interValID);
res.end(); // terminates SSE session
return;
}
res.write(`data: ${JSON.stringify({num: counter})}\n\n`); // res.write() instead of res.send()
}, 1000);
// If client closes connection, stop sending events
res.on('close', () => {
console.log('client dropped me');
clearInterval(interValID);
res.end();
});
});
Set the appropriate headers as per the spec
Use res.flushHeaders() to establish SSE connection
Use res.write() instead of res.send() to send data
To end stream from the server, use res.end()
The snippet above uses setInterval() to simulate sending data to the client for 10 seconds, then it ends the connection. The client will receive an error for the lost connection and automatically try to re-establish the connection. To avoid this, you can close the client on error, or have the browser send a specific event message that the client understands means to close gracefully. If the client closes the connection, we can catch the 'close' event to gracefully end the connection on the server and stop sending events.
express: 4.17.1
node: 10.16.3

You can definitely achieve this without other packages.
I wrote a blog post about this, part 1 sets out the basics.
You mustn't close the SSE as that breaks the functionality. The whole point is that it is an open HTTP connection. This allows for new events to be pushed to the client at any point.

This adds a complete, runnable example (with client to read the stream) to John's excellent answer and makes a tweak, adding the Connection: keep-alive header.
server.js:
const express = require("express");
const fs = require("fs").promises;
const path = require("path");
const app = express();
app
.set("port", process.env.PORT || 5000)
.get("/", (req, res) => {
fs.readFile(path.join(__dirname, "client.html"))
.then(file => res.send(file.toString()))
.catch(err => res.status(404).send(err.message))
;
})
.get("/stream", (req, res) => {
res.set({
"Access-Control-Allow-Origin": "*",
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"Content-Type": "text/event-stream",
});
res.flushHeaders();
let counter = 0;
const interval = setInterval(() => {
res.write("" + counter++);
}, 1000);
res.on("close", () => {
clearInterval(interval);
res.end();
});
})
.listen(app.get("port"), () =>
console.log(`server listening on port ${app.get("port")}`)
)
;
client.html:
<!DOCTYPE html>
<html lang="en">
<head></head>
<body>
<script>
(async () => {
const response = await fetch("/stream");
if (!response.ok) {
throw Error(response.status);
}
for (const reader = response.body.getReader();;) {
const {value, done} = await reader.read();
if (done) {
break;
}
document.body.innerText = new TextDecoder().decode(value);
}
})();
</script>
</body>
</html>
After node server.js, navigate your browser to localhost:5000. You can also test the stream directly with curl localhost:5000/stream.
I won't repeat the notes from John's answer, but, in short we set the necessary headers and flush them to begin the connection, then use res.write to send a chunk of data. Call res.end() to terminate the connection on the server or listen for res.on("close", ...) for the client closing the connection.
The client uses fetch and response.body.getReader() which can be read with const {value, done} = await reader.read() and decoded with TextDecoder().decode(value).
See also https://masteringjs.io/tutorials/express/server-sent-events
Express 4.17.1, Node 15.2.0, Chrome 89.0.4389.128 (Official Build) (64-bit)

It appears from the documentation on the library you're using that you should use a res.sse when using that as middleware on a function. See:
https://www.npmjs.com/package/server-sent-events
But, all this is actually doing from their code is wrapping res.write as you mentioned. See:
https://github.com/zacbarton/node-server-sent-events/blob/master/index.js#L11

Self-promotion: I wrote the ExpreSSE package that provides middlewares for working with SSE in express, you can find it on npm: #toverux/expresse.
A simple example:
router.get('/events', sse(/* options */), (req, res) => {
let messageId = parseInt(req.header('Last-Event-ID'), 10) || 0;
someModule.on('someEvent', (event) => {
//=> Data messages (no event name, but defaults to 'message' in the browser).
res.sse.data(event);
//=> Named event + data (data is mandatory)
res.sse.event('someEvent', event);
//=> Comment, not interpreted by EventSource on the browser - useful for debugging/self-documenting purposes.
res.sse.comment('debug: someModule emitted someEvent!');
//=> In data() and event() you can also pass an ID - useful for replay with Last-Event-ID header.
res.sse.data(event, (messageId++).toString());
});
});
There is also another middleware to push the same events to multiple clients.

New Answer:
Just use socket.io, it's so much easier and better!
https://www.npmjs.com/package/socket.io#in-conjunction-with-express
basic setup:
const express = require('express');
const PORT = process.env.PORT || 5000;
const app = express();
const server = require('http').createServer(app);
const io = require('socket.io')(server);
// listen to socket connections
io.on('connection', function(socket){
// get that socket and listen to events
socket.on('chat message', function(msg){
// emit data from the server
io.emit('chat message', msg);
});
});
// Tip: add the `io` reference to the request object through a middleware like so:
app.use(function(request, response, next){
request.io = io;
next();
});
server.listen(PORT);
console.log(`Listening on port ${PORT}...`);
and in any route handler, you can use socket.io:
app.post('/post/:post_id/like/:user_id', function likePost(request, response) {
//...
request.io.emit('action', 'user liked your post');
})
client side:
<script src="/socket.io/socket.io.js"></script>
<script src="https://code.jquery.com/jquery-1.11.1.js"></script>
<script>
$(function () {
var socket = io();
$('form').submit(function(e){
e.preventDefault(); // prevents page reloading
socket.emit('chat message', $('#m').val());
$('#m').val('');
return false;
});
socket.on('chat message', function(msg){
$('#messages').append($('<li>').text(msg));
});
});
</script>
full example: https://socket.io/get-started/chat/
Original Answer:
Someone (user: https://stackoverflow.com/users/451634/benny-neugebauer | from this article: addEventListener on custom object) literally gave me a hint on how to implement this without any other package except express! I have it working!
First, import Node's EventEmitter:
const EventEmitter = require('events');
Then create an instance:
const Stream = new EventEmitter();
Then create a GET route for event streaming:
app.get('/stream', function(request, response){
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive'
});
Stream.on("push", function(event, data) {
response.write("event: " + String(event) + "\n" + "data: " + JSON.stringify(data) + "\n\n");
});
});
In this GET route, you are writing back that the request is 200 OK, content-type is text/event-stream, no cache, and to keep-alive.
You are also going to call the .on method of your EventEmitter instance, which takes 2 parameters: a string of the event to listen for and a function to handle that event(that function can take as much params as it is given)
Now.... all you have to do to send a server event is to call the .emit method of your EventEmitter instance:
Stream.emit("push", "test", { msg: "admit one" });
The first parameter is a string of the event you want to trigger (make sure that it is the same as the one in the GET route). Every subsequent parameter to the .emit method will be passed to the listener's callback!
That is it!
Since your instance was defined in a scope above your route definitions, you can call the .emit method from any other route:
app.get('/', function(request, response){
Stream.emit("push", "test", { msg: "admit one" });
response.render("welcome.html", {});
});
Thanks to how JavaScript scoping works, you can even pass that EventEmitter instance around to other function, even from other modules:
const someModule = require('./someModule');
app.get('/', function(request, response){
someModule.someMethod(request, Stream)
.then(obj => { return response.json({}) });
});
In someModule:
function someMethod(request, Stream) {
return new Promise((resolve, reject) => {
Stream.emit("push", "test", { data: 'some data' });
return resolve();
})
}
That easy! No other package needed!
Here is a link to Node's EventEmitter Class: https://nodejs.org/api/events.html#events_class_eventemitter
My example:
const EventEmitter = require('events');
const express = require('express');
const app = express();
const Stream = new EventEmitter(); // my event emitter instance
app.get('/stream', function(request, response){
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive'
});
Stream.on("push", function(event, data) {
response.write("event: " + String(event) + "\n" + "data: " + JSON.stringify(data) + "\n\n");
});
});
setInterval(function(){
Stream.emit("push", "test", { msg: "admit one" });
}, 10000)

How to properly close Node.js Express server?

I need to close server after getting callback from /auth/github/callback
url. With usual HTTP API closing
server is currently supporting with server.close([callback])
API function, but with node-express server i’m getting TypeError: Object function app(req, res){ app.handle(req, res); } has no method 'close'
error. And I don't know how to find information to solve this problem.
How should I close express server?
NodeJS configuration notes:
$ node --version
v0.8.17
$ npm --version
1.2.0
$ npm view express version
3.0.6
Actual application code:
var app = express();
// configure Express
app.configure(function() {
// … configuration
});
app.get(
'/auth/github/callback',
passport.authenticate('github', { failureRedirect: '/login' }),
function(req, res) {
res.redirect('/');
setTimeout(function () {
app.close();
// TypeError: Object function app(req, res){ app.handle(req, res); } has no method 'close'
}, 3000)
}
);
app.listen('http://localhost:5000/');
Also, I have found ‘nodejs express close…’ but I don't sure if I can use it with code I have: var app = express();.

app.listen() returns http.Server. You should invoke close() on that instance and not on app instance.
Ex.
app.get(
'/auth/github/callback',
passport.authenticate('github', { failureRedirect: '/login' }),
function(req, res) {
res.redirect('/');
setTimeout(function () {
server.close();
// ^^^^^^^^^^^
}, 3000)
}
);
var server = app.listen('http://localhost:5000/');
// ^^^^^^^^^^
You can inspect sources: /node_modules/express/lib/application.js

In express v3 they removed this function.
You can still achieve the same by assigning the result of app.listen() function and apply close on it:
var server = app.listen(3000);
server.close((err) => {
console.log('server closed')
process.exit(err ? 1 : 0)
})
https://github.com/visionmedia/express/issues/1366

If any error occurs in your express app then you must have to close the server and you can do that like below-
var app = express();
var server = app.listen(process.env.PORT || 5000)
If any error occurs then our application will get a signal named SIGTERM. You can read more SIGTERM here - https://www.gnu.org/software/libc/manual/html_node/Termination-Signals.html
process.on('SIGTERM', () => {
console.info('SIGTERM signal received.');
console.log('Closing http server.');
server.close((err) => {
console.log('Http server closed.');
process.exit(err ? 1 : 0);
});
});

I have answered a variation of "how to terminate a HTTP server" many times on different node.js support channels. Unfortunately, I couldn't recommend any of the existing libraries because they are lacking in one or another way. I have since put together a package that (I believe) is handling all the cases expected of graceful express.js HTTP(S) server termination.
https://github.com/gajus/http-terminator
The main benefit of http-terminator is that:
it does not monkey-patch Node.js API
it immediately destroys all sockets without an attached HTTP request
it allows graceful timeout to sockets with ongoing HTTP requests
it properly handles HTTPS connections
it informs connections using keep-alive that server is shutting down by setting a connection: close header
it does not terminate the Node.js process

calling server.close does the job
server.close((err) => {
console.log('server closed')
process.exit(err ? 1 : 0)
})
also it is good to listen for system(user) signals and shutdown gracefully on them too, for that you should listen on both SIGTERM and SIGINT
const port = process.env.PORT || 5000;
const server = app.listen(port);
console.log(`listening on port:${port}`);
for (let signal of ["SIGTERM", "SIGINT"])
process.on(signal, () => {
console.info(`${signal} signal received.`);
console.log("Closing http server.");
server.close((err) => {
console.log("Http server closed.");
process.exit(err ? 1 : 0);
});
});

Old question but now Node v18.2.0 introduced server.closeAllConnections(). It should be noted that server.close never runs its callback when the browser sends the request Connection: keep-alive, because server.close only stops the server from accepting new connections, it does not close old connections.
Before Node v18.2.0 I tackled this problem by waiting 5 seconds for the server to shutdown, after which it would force exit.
This code encompasses both situations
process.on('SIGINT', gracefulShutdown)
process.on('SIGTERM', gracefulShutdown)
function gracefulShutdown (signal) {
if (signal) console.log(`\nReceived signal ${signal}`)
console.log('Gracefully closing http server')
// closeAllConnections() is only available from Node v18.02
if (server.closeAllConnections) server.closeAllConnections()
else setTimeout(() => process.exit(0), 5000)
try {
server.close(function (err) {
if (err) {
console.error('There was an error', err)
process.exit(1)
} else {
console.log('http server closed successfully. Exiting!')
process.exit(0)
}
})
} catch (err) {
console.error('There was an error', err)
setTimeout(() => process.exit(1), 500)
}
}

Most answers call process.exit(), I don't think this is a good idea. You probably need to perform some teardown, also it's simply not needed.
const server = app.listen(port);
server.on('close', () => {
// Perform some teardown, for example with Knex.js: knex.destroy()
});
// FYI Docker "stop" sends SIGTERM
// If SIGTERM is not catched, Docker is forced to kill the container after ~10s
// and this provokes an exit code 137 instead of 0 (success)
process.on('SIGTERM', () => server.close());
Check Express.js 4.x documentation: https://expressjs.com/en/advanced/healthcheck-graceful-shutdown.html#graceful-shutdown