Inspired by How to share sessions with Socket.IO 1.x and Express 4.x? i implemented socket authentication in some "clean" way where is no need to use cookie-parser and to read cookies from headers, but few items remain unclear to me. Example use last stable socket.io version 1.3.6.
var express = require('express'),
session = require('express-session'),
RedisStore = require('connect-redis')(session),
sessionStore = new RedisStore(),
io = require('socket.io').listen(server);
var sessionMiddleware = session({
store : sessionStore,
secret : "blabla",
cookie : { ... }
});
function socketAuthentication(socket, next) {
var sessionID = socket.request.sessionID;
sessionStore.get(sessionID, function(err, session) {
if(err) { return next(err); }
if(typeof session === "undefined") {
return next( new Error('Session cannot be found') );
}
console.log('Socket authenticated successfully');
next();
});
}
io.of('/abc').use(socketAuthentication).on('connection', function(socket) {
// setup events and stuff
});
io.use(function(socket, next) {
sessionMiddleware(socket.request, socket.request.res, next);
});
app.use(sessionMiddleware);
app.get('/', function(req, res) { res.render('index'); });
server.listen(8080);
index.html
<body>
...
<script src="socket.io/socket.io.js"></script>
<script>
var socket = io('http://localhost:8080/abc');
</script>
</body>
So io('http://localhost:8080/abc'); from client-side will send initial HTTP handshake request to server, from where server can gather cookies and many others request informations. So server has access to that initial request via socket.request.
My first question is why handshake request is not in scope of express-session middleware?(More generally in scope of app.use middlewares?) In some way i expected this app.use(sessionMiddleware); to fire before that initial request, and then to access easily to socket.request.session
Second, what are the scenarios in which middlewares defined with io.use() will fire? Only for initial HTTP handshake request? It seems like io.use() is used for socket related stuff(question is: what stuff), while app.use for standard requests.
I'm not quite sure why in the above example io.use() is fired before io.of('/abc').use(). Intentionally i wrote that order putting io.of('/abc').use() first to see will it work and it work.
Should have been written conversely.
Lastly, socket.request.res like pointed also from some people in linked question, sometimes is undefined causing app to broke, problem can be solved by providing empty object instead of socket.request.res, like: sessionMiddleware(socket.request, {}, next); which seems to me like a dirty hack. For what reasons socket.request.res yield to undefined?
Despite #oLeduc is kind of correct, there are a few more things to explain..
Why the handshake's request is not in scope of express-session middleware?
The biggest reason here is that the middleware in express is designed to handle request specific tasks. Not all, but most of the handlers use the standard req, res, next syntax. And sockets are "request-less" if I can say. The fact that you have socket.request is due to the way the handshake is made, and that it is using HTTP for that. So the guys at socket.io hacked that first request into your socket class so that you can use it. It was not designed by the express team to ever work with sockets and TCP.
What are the scenarios in which middlewares defined with io.use() will fire?
io.use is a close representation of the express use middleware way. In express, the middleware is executed on each request, right? But sockets do not have requests and it will be awkward to use middleware on each socket emit, so they've made it to be executed on each connection. But as well as the express middleware is stacked and used before the actual request is handled (and responded), Socket.IO uses the middleware on connection and even before the actual handshake! You can intercept the handshake if you want to, using that kind of middleware, which is very handy (in order to protect your server from spamming). More on this can be found in the code of passport-socketio
Why io.use() fires before io.of('/abc').use()?
The real explanation on this can be found here, which is this code:
Server.prototype.of = function(name, fn){
if (String(name)[0] !== '/') name = '/' + name;
if (!this.nsps[name]) {
debug('initializing namespace %s', name);
var nsp = new Namespace(this, name);
this.nsps[name] = nsp;
}
if (fn) this.nsps[name].on('connect', fn);
return this.nsps[name];
};
And in the beginning of the code, there is this line of code:
this.sockets = this.of('/');
So, there is a default namespace created at the very beginning. And right there, you can see that it has immediately a connect listener attached to it. Later on, each namespace gets the very same connect listener, but because Namespace is EventEmitter, the listeners are added one after another, so they fire one after another. In other words, the default namespace has it's listener at first place, so it fires first.
I don't think this is designed on purpose, but it just happened to be this way :)
Why is socket.request.res undefined?
To be honest, I'm not pretty sure about that. It's because of how engine.io is implemented - you can read a bit more here. It attaches to the regular server, and sends requests in order to make a handshake. I can only imagine that sometimes on errors the headers are separated from the response and that's why you won't get any. Anyways, still just guessing.
Hope information helps.
Why the handshake's request is not in scope of express-session middleware?
Because socket.io will attach to a http.Server which is the layer under express. It is mentioned in a comment in the source of socket.io.
The reason for this is because the first request is a regular http request used to upgrade the reqular stateless http connection into a state-full websocket connection. So it wouldn't make much sense for it to have to go through all the logic that applies to regular http requests.
What are the scenarios in which middlewares defined with io.use() will fire?
Whenever a new socket connection is created.
So every time a client connects it will call the middlewares registed using io.use(). Once the client is connected however, it is not called when a packet is received from the client. It doesn't matter if the connection is initiated on a custom namespace or on the main namespace, it will always be called.
Why io.use() fires before io.of('/abc').use()?
Namespaces are a detail of socket.io's implementation, in reality, websockets will always hit the main namespace first.
To illustrate the situation, look at this snippet and the output it produces:
var customeNamespace = io.of('/abc');
customeNamespace.use(function(socket, next){
console.log('Use -> /abc');
return next();
});
io.of('/abc').on('connection', function (socket) {
console.log('Connected to namespace!')
});
io.use(function(socket, next){
console.log('Use -> /');
return next();
});
io.on('connection', function (socket) {
console.log('Connected to namespace!')
});
Output:
Use -> /
Main namespace
Use -> /abc
Connected to namespace!
See the warning that the socket.io team added to their documentation:
Important note: The namespace is an implementation detail of the Socket.IO protocol, and is not related to the actual URL of the underlying transport, which defaults to /socket.io/….
Why is socket.request.res undefined?
As far as I know it should never be undefined. It might be related to your specific implementation.
Related
I am writing my first very simple express server for data a collection purpose. This seems like a beginner question but I failed to find an answer so far. The data is very small (less than 500 integers) and will never grow, but it should be able to be changed through POST requests.
I essentially (slightly simplified) want to:
Have the data in a .json file that is loaded when the server starts.
On a POST request, modify the data and update the .json file.
On a GET request, simply send the .json containing the data.
I don't want to use a database for this as the data is just a single small array that will never grow in size. My unclarities are mainly how to handle modifying the global data and file reading / writing safely, i.e. concurrency and how exactly does Node run the code.
I have the following
const express = require('express');
const fs = require('fs');
let data = JSON.parse(fs.readFileSync('./data.json'));
const app = express();
app.listen(3000);
app.use(express.json());
app.get("/", (req, res) => {
res.sendFile('./data.json', { root: __dirname });
});
app.post("/", (req, res) => {
const client_data = req.body;
// modify global data
fs.writeFileSync("./data.json", JSON.stringify(data), "utf8");
});
Now I have no idea if or why this is safe to do. For example, modifying the global data variable and writing to file. I first assumed that requests cannot run concurrently without explicitly using async functions, but that seems to not be the case: I inserted this:
const t = new Date(new Date().getTime() + 5000);
while(t > new Date()){}
into the app.post(.. call to try and understand how this works. I then made simultaneous POST requests and they finished at the same time, which I did not expect.
Clearly, the callback I pass to app.post(.. is not executed all at once before other POST requests are handled. But then I have a callback running concurrently for all POST requests, and modifying the global data and writing to file is unsafe / a race condition. Yet all code I could find online did it in this manner.
Am I correct here? If so, how do I safely modify the data and write it to file? If not, I don't understand how this code is safe at all?
Code like that actually opens up your system to race conditions. Node actually runs that code in a single-threaded kind of way, but when you start opening files and all that stuff, it gets processed by multiple threads (opening files are not Node processes, they are delegated to the OS).
If you really, really want to use files as your global data, then I guess you can use an operating system concept called Mutual Exclusions. Basically, its a 'lock' used to prevent race conditions by forcing processes to wait while something is currently accessing the shared resource (or if the shared resource is busy). In Node, this can be implemented in many ways, but one recommendation is to use async-mutex library to handle concurrent connections and concurrent data modifications. You can do something like:
const express = require('express');
const fs = require('fs');
const Mutex = require('async-mutex').Mutex;
// Initializes shared mutual exclusion instance.
const mutex = new Mutex()
let data = JSON.parse(fs.readFileSync('./data.json'));
const app = express();
app.listen(3000);
app.use(express.json());
app.get("/", (req, res) => {
res.sendFile('./data.json', { root: __dirname });
});
// Turn this into asynchronous function.
app.post("/", async (req, res) => {
const client_data = req.body;
const release = await mutex.acquire();
try {
fs.writeFileSync('./data.json', JSON.stringify(data), 'utf8');
res.status(200).json({ status: 'success' });
} catch (err) {
res.status(500).json({ err });
finally {
release();
}
});
You can also use Promise.resolve() in order to achieve similar results with the async-mutex library.
Note that I recommend you to use a database instead, as it is much better and abstracts a lot of things for you.
References:
Node.js Race Conditions
I'm must say I'm very new to back end development,
I'm currently working on an exercise project of making a fake money poker website. I use Node.js socket.io/express-session/passport
At first, I mainly used express with a HTTP server listening on one port. Averagely Like this:
const express = require("express")
const app = express()
app.get('/home',connectEnsureLogin.ensureLoggedIn("/loginPage"),function(req, res) {
//console.log(req.user.username+": sessionId: "+req.sessionID);
return res.sendFile( __dirname+"/website/index.html");
}
);
const PORT = process.env.PORT || 5000;
app.listen(PORT, () => console.log("Poker site Server started on ${PORT})")
The website wasn't working very fast. When a client joined a poker table they needed to ask the server every second for new updates on the state of the game so that was a lot of HTTP requests coming into my server. So I decided without much theoretical certitude that it seemed like a good idea: To have the server use socket.io sockets to hand info for clients that are in poker tables, but when they are not in poker tables and are just browsing the site I use a HTTP server to handle their request. Code wise I feel I haven't really managed to do this correctly. My code with Express, express-session, and passport combined makes sure only to hand information to users authenticated. But since The socket.io servers seem totally separate from all the express code, they don't share the same authentication functionality as the express code. So I need to somehow link my express and socket.io code so I can check if a client is authenticated before handing him any info via sockets. here is the system I'm currently using I didn't put all my code but I tried to summarize the essential parts:
const express = require('express');
const app = express();
//i creat the http server that is somehow linked with my express app when this server is listening
//it will call express handling methods.
const http = require('http').Server(app);
const io = require('socket.io')(http);
const path = require("path");
const passport = require("passport");
const connectEnsureLogin = require('connect-ensure-login');
const AccountInfo = require("./AccountInfo").AcccountInfo;
const expressSession = require('express-session')({
secret: process.env.SESSION_SECRET,
resave: false,
saveUninitialized: false
});
//passport setup
passport.use(AccountInfo.createStrategy());
passport.serializeUser(AccountInfo.serializeUser());
passport.deserializeUser(AccountInfo.deserializeUser());
//body parser
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
//Sessions
app.use(expressSession);
//!!!!here is where I connect socket.io with the sessions i found this in another forum.
// thanks to this code I can access the session that a client is using when their socket connects.
io.use(function(socket, next) {
expressSession(socket.request, socket.request.res, next);
});
//so when a clients socket connects i save his socket.id to his session.
io.on('connection',function(socket) {
console.log(`socket.io connected: ${socket.id}`);
// save socket.io socket in the session
socket.request.session.socketio = socket.id;
socket.request.session.save();
});
//once the clients socket is connected directly after the clients sends a HTTP "PUT" request
//and this code answers it.
app.post('/Table/ConnectSocketToTable',Utilities.ensureLoggedIn(),function(req, res)
{
//I retrieve the socket using the socket.id I had saved in the session.
let socket = io.sockets.sockets.get(req.session.socketio);
let player = GetPlayerFromAnyTable(req.user.username);
if(player==null)//the player can't be in two tables at once
{
//since now we are in an express callback, express made sure that the client is indeed
//authenticated with the middle-ware: "Utilities.ensureLoggedIn()" also just before I made sure
//the client is not in another table. So we are good to go we can now link the socket to the table
//and have the client receive all the info about the state of his table
socket.join("table-"+req.session.passport.table);
req.user.socket = socket;
let table = GetTable(req.session.passport.table);
table.sitPlayer(req.user);
}
else
{
//the player is already connected so we just update his socket to a new one
player.requestUnseat=false;
player.account.socket =io.sockets.sockets.get(req.session.socketio);
}
socket.on('chatMessage', function(data,time) {
socket.to("table-"+req.session.passport.table).emit("chatMessage",req.user.username,data,time);
console.log(`send chat message : ${data}`);
});
socket.on('disconnect', function() {
GetTable(req.session.passport.table).requestUnsitUsername(req.user.username);
console.log(req.user.username +" was disconnected so now requesting unsit");
});
console.log("the socket of "+req.user.username+" has being connected to table-"+req.session.passport.table);
return res.sendStatus(200);
});
So for me, the way I'm doing this seems pretty bad since "app.post('/Table/ConnectSocketToTable'...)" and "io.on('connection',...)" are two different request listening functions I feel I should probably just do everything in one.
So should I do all the checks in the "io.on('connection',...)" function and somehow manage to make sure the client is authenticated within the callback of io.on('connection',callback) ?
or should I find a way to make the socket connection happen in the initial HTTP call the client uses to join a table, which is what I initially wanted?
But really I'm kinda lost because I'm telling myself maybe I don't even need Express anymore and I should just use socket.io for everything. I seem to clearly lack the general understanding that would allow me to know what approach I should be going for so any help is welcome. I started doing this self-made exercise to get into server-side development but also if there is any other recommended exercise to start up with back-end development I'm definitely interested in hearing about it.
From random testing I found out how to authenticate to my express session from the socket code you don't actually have to do it in the callback of io.on('connection',callback) you just need to add a few more middleware functions like this:
//connecting express sessions
io.use(function(socket, next) {
expressSession(socket.request, socket.request.res, next);
});
//connecting passport
io.use(function(socket, next) {
passport.initialize()(socket.request, socket.request.res, next);
});
//connecting passport sessions
io.use(function(socket, next) {
passport.session()(socket.request, socket.request.res, next);
});
//check if client is authenticated returns error if authentication failed
io.use((socket, next) => {
console.log("started socket Connection");
if(!socket.request.isAuthenticated&&socket.request.isAuthenticated())
{
socket.request.session.socketio = socket.id;
socket.request.session.save();
console.log("table "+socket.request.session.passport.table);
console.log("user.username "+socket.request.user.username);
console.log(`is authentificated`);
next();
}
else
{
console.log(`failed socket connection`);
next(new Error("unauthorized"));
}
});```
I am trying to make some global variables in io.use method of socket io server side library. the purpose is to have variables that are available all throughout the request lifecycle (whatever request means in terms of websockets). I am using express framework and socket.io library. The snippet is as follows :-
io.use(function(socket, next) {
_io = io;
_sid = socket.id;
_varName = socket.handshake.query.varName;
next();
});
What I am trying to have is something like res.locals for websockets i.e. global variables scoped to each individual websocket connection.
the problem that I am facing using above snippet is that these variables hold value for the last received request and is not reliable across multiple requests.
also, is io.use the best place to do jwt verification if I want to do jwt based auth on each incoming message?
Global variables got very limited usecases. This isnt one. SocketIO allows you to put all your handlers into the socket callback. Thats quite good as it allows you to closure the variables you want:
io.on("connection", function(socket, next) {
var sid = socket.id;
var varName = socket.handshake.query.varName;
socket.on("message", function(){
console.log(varName);
});
//...
});
If thats not an option then might invent your own locals object:
io.use(function(socket, next) {
socket.locals = {
sid: socket.id,
varName: socket.handshake.query.varName
};
next();
});
... However i dont see any sense in copying properties that are part of the socket anyway.
I've been looking over this basic example in order to set up a simple server in Node, however I am really struggling to understand where the 'request' and 'response' parameters are coming from. Where do they point to and how?
// Load the http module to create an http server.
var http = require('http');
function onRequest(request, response) {
response.writeHead(200, {"Content-Type": "text/plain"});
response.end("Hello World\n");
}
// Configure our HTTP server to respond with Hello World to all requests.
var server = http.createServer(onRequest);
// Listen on port 8000, IP defaults to 127.0.0.1
server.listen(8000);
Usually when passing paramaters into a function I pass something I can see, like a variable equal to 5, or 'Hello', but in this case I'm not...
Sorry if this is not explained very well!
When you call createServer, you are passing the function onRequest to it:
var server = http.createServer(onRequest);
This pattern is known as a callback: you pass a function to someone else, with the expectation that they will call your function if something interesting has happened.
In essence, you are saying to Node,
Hey, please create an HTTP server for me. Whenever you receive a request, call my function onRequest with the request and response objects passed as parameters.
Another way to do this is to listen to the request event, which takes the same parameters in its callback.
The parameters are documented as being http.IncomingMessage and http.ServerResponse. You can call them whatever you want, but request and response are the idiomatic parameter names. (Some people use req and res because they are shorter to type.)
Create server and send response :
1).Create server
var http = require('http');
var server = http.createServer ( function(request,response){
response.writeHead(200,{"Content-Type":"text\plain"});
response.end("Hello");
});
server.listen(8000);
console.log("Server running on port 8000");
2).Save above code and run in cmd .
3).Open browser and go to http://localhost:8000/
Now you see the "Hello"
I'm creating a simple testing platform for an app and have the following code setup as my server.js file in the root of my app:
var restify = require('restify'),
nstatic = require('node-static'),
fs = require('fs'),
data = __dirname + '/data.json',
server = restify.createServer();
// Serve static files
var file = new nstatic.Server('');
server.get(/^\/.*/, function(req, res, next) {
file.serve(req, res, next);
});
// Process GET
server.get('/api/:id', function(req, res) {
// NEVER FIRES
});
It serves static files perfectly, however, when I try to make a call to the /api it just hangs and times out. Imagine I'm missing something stupid here, any help would be greatly appreciated.
node-static is calling next with an error, which means it's never yielding to other handlers.
You can move your other handlers above node-static or ignore it's errors by intercepting it's callback.
I made a working version here: http://runnable.com/UWXHRONG7r1zAADe
You may make yourself sure the api get call is caught by moving the second get before the first. The reason is your api calls routes are already matched by the first pattern.