Develop Reference

detect a request using NodeJS without express or accessing the request object outside an express middleware [duplicate]

This question already has answers here:
Access current req object everywhere in Node.js Express
(2 answers)
Closed 8 months ago.
This post was edited and submitted for review 8 months ago and failed to reopen the post:
Original close reason(s) were not resolved
I have the following lines of code:
const express = require('express');
const app = express()
// ... defining the routes, app.get('/api/users', (req, res, next)=>{ }) ...etc
app.listen(3000, ()=> console.log('Listening on port 3000...'))
module.exports = app
I want to be able to read the request object outside an express middleware.
I have another file called mongoose_models.js, inside that file, I don't have the access to the express middleware arguments (req, res, next).
And the only option I have for reading the request body from that file is to import the app and somehow read the request Object.
NodeJs is event-driven, so there must be a way somehow to do so, for instance, inside the file mongoose_models.js I would have maybe something like this code:
// mongoose_models.js
// ... some code
const app = require('../app.js')
app.on('request', (req)=>{
// here I have the request
})
or maybe if express supports:
// mongoose_models.js
// ... some code
const { req } = require('express')
console.log(req.body) // ? maybe something like that ?
or maybe if express supports too:
// mongoose_models.js
// ... some code
const app = require('../app.js')
app.onRequest((req, res) => {
// here I have the access to the request object
})
Is there a way to reach the request object without having to be inside an express middleware in NodeJS?
edit:
Some of you asked me to provide the source code, unfortunately, I wanted to provide a stackblitz or code sandbox instance, but I didn't know how to set up the connections to the database.
Anyway, the following is the file structure of the sample project:
app.js file (full code):
const express = require('express')
const app = express()
const mongoose = require('mongoose')
const RoomModel = require('./mongoose_models')
app.use((req, res, next) => {
// this middleware is the "protect" middleware, it validates a JWT (JSON web token), decodes it, and then stores the user it finds to the req object:
// .... etc some code
// decode the JWT .. some code
// find the user in the DB const userDoc = await UserModel.findOne({ _id: decodedJWT.id )})
const userDoc = {
id: 'abc-123-edf-cds-123-321-qu5-eu4-dc9-182',
name: 'Murat',
// and some other fields ... etc
}
req.$loggedInUser = userDoc
})
app.get('/rooms', async(req, res, next) => {
const docs = await RoomModel.find({})
res.status(200).json({
message: 'here are all the rooms',
results: docs.length,
data: docs,
})
})
app.post('/rooms', async(req, res, next) => {
const doc = await RoomModel.create(req.body)
res.status(201).json({
message: 'the new room which got created:',
data: doc,
})
})
// connecting to the database:
mongoose.connect(
'mongodb+srv://USERNAME:PASSWORD#YOUR_CLUSTER.mongodb.net/?retryWrites=true&w=majority'
)
// starting the HTTP service:
app.listen(3000, () => console.log('app listening on port 3000...'))
mongoose_models.js file (full code):
const mongoose = require('mongoose')
const roomSchema = new mongoose.Schema({
name: String,
by: mongoose.Schema.ObjectId,
})
roomSchema.pre('save', function(next) {
// Here I want to make the by field be the req.$loggedInUser.id but I can't because I have no way to read the request object
const doc = this
// doc.by = req.$loggedInUser.id // < ----- 👈👈👈 HERE, I can't reach the req object
next()
})
const RoomModel = mongoose.model('Room', roomSchema, 'rooms')
module.exports = RoomModel

NodeJS is event driven, so there must be a way somehow to do so, for
instance, inside the file mongoose_models.js I would have maybe
something like this code:
// mongoose_models.js
// ... some code
const app = require('../app.js')
app.on('request', (req)=>{
// here I have the request
})
This approach is, essentially, middleware. So write is as middleware.
const myMiddleware = (req, res, next) => {
// here you have the request
next(); // go to next middleware
}
module.exports = myMiddleware
Attaching something to listen for requests is done with use (for non-method specific functions) and post, get, etc. There is no on method or onRequest method.
// mongoose_models.js
// ... some code
const { req } = require('express')
console.log(req.body) // ? maybe something like that ?
The request object doesn't exist until the client makes a request to the server.
You get a new request object each time a request is made.
The server might be handling multiple requests at the same time.
So no, you can't do anything like that.
Is there a way to reach the request object without having to be inside an express middleware in NodeJS?
No.

Express.js: How to refresh on DB change in backend?

I decided to make a janky chat site type thing to get me started working with requests and such.
My approach was to create an express.js server that takes in requests when the '/messageReciever' is posted to.
app.post("/messageReciever", (req, res) => {
logMessage(req.body.message);
});
The next step was to make a 'client' that could send information to this end point:
var XMLHttpRequest = require("XMLHttpRequest").XMLHttpRequest;
function makePostRequest(url, json)
{
let http = new XMLHttpRequest();
http.open("POST", url, true);
http.setRequestHeader("Content-Type", "application/json");
http.send(JSON.stringify(json));
}
function sendMessage(url, message)
{
makePostRequest(url, {message: message});
logMessage(message);
}
Both of these are fine. The issue I'm running into is, once I receive the post request I want to refresh the main page of my site (to show the messages)
app.get('/', (req, res) => {
res.render('index', data = retrieveMessages());
});
I've tried basically everything I've found online:
res.redirect('back');
res.redirect(req.get('referer'));
res.redirect(req.originalUrl)
I used res.redirect('back') previously in my code, and it works. The issue is that I'm trying to refresh someone's connection to a site based on someone else's connection; meaning I can't use the response information like I normally could.
I've tried looking for ways to refresh pages from outside functions but I can't find anything.
(I realize that there are easier ways to make a chat site that don't include weirdly sending data back and forth between two server's)

You can use a package called socket.io. Socket.io allows you to send requests to a client once the server has some data.
Example:
Server:
// Define express
const express = require('express');
const app = express();
// Create the server
const http = require('http');
const server = http.createServer(app);
// Define socket.io
const io = require('socket.io')(server);
// Define the port for the server to listen on
let port = 3000;
function logMessage(message, id) {
...
io.emit('message_sent_' + id, { message }); // Emit that a message was sent to the clients
}
function recieveMessages(id) {
// Get the messages somehow
}
app.post('/messageReciever', (req, res) => {
// req.body.message is your message and req.cookies.id is the clients random ID
logMessage(req.body.message, req.cookies.id);
});
app.get('/', (req, res) => {
res.cookie('id', 'some-generated-id'); // Set a cookie for the unique ID to fetch user messages
res.render('index', { data: retrieveMessages() });
});
// Get the server listening to incoming requests
server.listen(port, () => console.log('my app is online');
Client:
<!doctype html>
<html>
<body>
...
</body>
<script src="/socket.io/socket.io.js"></script>
<script>
const socket = io.connect();
socket.on('message_sent_' + 'some-id', function(data) {
// Do something with the data
});
</script>
</html>
References:
https://socket.io/docs/v4/
http://expressjs.com/
https://marques-robinson-project.medium.com/chat-app-with-socket-io-and-express-using-node-js-2293b87f47c3

How can I verify a HMAC SHA256 in JavaScript?

I'm writing a script in Node.js that integrates Todoist, Notion and Discord. I'm using Heroku for hosting because I want to use webhooks (from Todoist). The idea is that every time I add or complete a task, a notification is fired by Todoist to my app and after verifying that the request is legitimate, I update my Notion database and send messages through a discord bot.
The problem I encounter is when I have to check the HMAC SHA256. I'm still quite new to JavaScript and even newer to hashing and secret keys, so please bear with me.
The Todoist documentation says:
To verify each webhook request was indeed sent by Todoist, an X-Todoist-Hmac-SHA256 header is included; it is a SHA256 Hmac generated using your client_secret as the encryption key and the whole request payload as the message to be encrypted. The resulting Hmac would be encoded in a base64 string.
After a lot of research, I tried to use a function I found as the verified answer in another question, but this happens: throw new Error('Malformed UTF-8 data');
This is what I have so far:
const express = require('express');
const Discord = require('discord.js');
const client = new Discord.Client();
var CryptoJS = require('crypto-js');
require('dotenv').config();
const PORT = process.env.PORT || 3000;
app = express();
app.use(express.urlencoded({ extended: true }));
app.use(express.json());
process.on('unhandledRejection', error => {
// Will print "unhandledRejection err is not defined"
console.log('unhandledRejection', error.message);
});
function sign_string(message, key){
var secret_key = CryptoJS.enc.Base64.parse(key).toString(CryptoJS.enc.Utf8);
var hash = CryptoJS.HmacSHA256(message, secret_key);
return CryptoJS.enc.Base64.stringify(hash);
}
client.on('ready', () => {
client.users.fetch(process.env.MY_USER_ID).then(user => user.send('Hey, The bot is up!'));
});
app.get('', (req, res) => {
res.send('Hello World');
});
app.post('', (req, res) => {
if(req.get('User-Agent') === 'Todoist-Webhooks') {
var delivered_hmac = req.get('X-Todoist-Hmac-SHA256');
var computed_hmac = sign_string(JSON.stringify(req.body), process.env.TODOIST_CLIENT_SECRET);
if(delivered_hmac === computed_hmac) {
if(req.body.event_name === 'item:added' && req.body.event_data.description === '') {
// add task to notion
// idea: ask user for data
} else {
if(req.body.event_name === 'item:completed' && req.body.event_data.description !== '') {
// complete task on notion
}
}
client.users.fetch(process.env.MY_USER_ID).then(user => user.send('You can update your tasklist if you want'));
res.status(200).send('Event handled');
} else {
client.users.fetch(process.env.MY_USER_ID).then(user => user.send('A 403 (Unauthorized) status code has been sent to a request'));
res.status(403).send('Unauthorized');
console.log(`delivered_hmac: ${delivered_hmac}\ncomputed_hmac: ${computed_hmac}\n`)
console.log(req.body);
}
} else {
client.users.fetch(process.env.MY_USER_ID).then(user => user.send('A 400 (Bad request) status code has been sent to a request'));
res.status(400).send('Bad request');
}
//handle notion
})
client.login(process.env.BOT_TOKEN);
app.listen(PORT, () => {
console.log(`App up at port ${PORT}`);
});

How to make a post request to mongo db atlas using node.js and express.js

I am trying to make a simple post request to Mongo Db Atlas using Node.js, Express.js and Mongoose.js. I fill out the form and send the request but it just keeps loading, no erros, nothing. ANyone have any ideas?
//Require assets
const express = require('express');
const app = express();
const mongoose = require('mongoose');
let port = 3000;
const bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
const MongoClient = require('mongodb').MongoClient;
const uri = 'mongodb+srv://michael:<PASSWORD-GOES-HERE>#around-town-vsisv.mongodb.net/admin';
const client = new MongoClient(uri, { useNewUrlParser: true });
client.connect(err => {
const collection = client.db('around_town_db').collection('events');
// perform actions on the collection object
client.close();
});
mongoose.Promise = global.Promise;
var eventSchema = mongoose.Schema({
eventName: String,
})
var eventData = mongoose.model('Event', eventSchema);
//Load index page using endpoint
app.get('/', (req, res) => {
res.sendFile(__dirname + '/index.html');
});
//Post using endpoint
app.post('/addevent', (req, res) => {
var userData = {
eventName: req.body.eventName
}
new eventData(userData)
.save()
.then(res => {
res.send('item saved to database');
})
.catch(err => {
res.status(400).send('unable to save to database');
});
});
//Listen on port 3000
app.listen(port, () => {
console.log('Server listening on port ' + port);
});
Below is a simple form for submitting the post request. Any ideas on what I am missing?
<!DOCTYPE html>
<html>
<head>
<title>Add a local event</title>
</head>
<body>
<div class="app">
<h1>Add an event</h1>
<form method="post" action="/addevent">
<label>Enter event Name</label><br>
<input type="text" name="eventName" placeholder="Enter event name..." required>
<input type="submit" value="Add Event">
</form>
</div>
</body>
</html>

OK, so it looks like you have two problems. First, you're not letting Mongoose itself connect to your database, and second you're overwriting the response variable. If you note your .then() callback in the userData.save() promise chain, you're calling that result variable 'res' as well as the response variable 'res' from the route callback.
To fix the first problem, somewhere in your app startup, one time call (instead of your current MongoClient code)
mongoose.connect(uri)
You can also use mongoose.createConnection() if you want more fine-grained control over the connection, or need to make multiple connections to different databases.
Second problem, your post handler should look like this:
//Post using endpoint
app.post('/addevent', (req, res) => {
var userData = {
eventName: req.body.eventName
}
new eventData(userData)
.save()
.then(result => { // note the use of a different variable name
res.send(result); // also, you generally want to send *something* down that lets the user know what was saved. Maybe not the whole object, but this is illustrative and the client will at least need to know something (e.g. the id) to refer to the object by in the future.
})
.catch(err => {
res.status(400).send('unable to save to database');
});
});
Unrelated to your problem, but most folks write Express APIs in a RESTful manner. If you want to follow that convention, then your routes should look more like this:
GET /events // list the events
GET /events/:eventId // get a single event by ID
POST /events // add a new event
PUT /events/:eventId // update an existing event
DELETE /events/:eventId // remove an event

Try something like follow:
app.post('/addevent', (req, res) => {
var userData = new eventData({
eventName: req.body.eventName
})
userData.save()
.then(res => {
res.send('item saved to database');
})
.catch(err => {
res.status(400).send('unable to save to database');
});
});

Emiting websocket message from routes

I'm trying to setup my server with websockets so that when I update something via my routes I can also emit a websocket message when something on that route is updated.
The idea is to save something to my Mongo db when someone hits the route /add-team-member for example then emit a message to everyone who is connected via websocket and is a part of whatever websocket room that corresponds with that team.
I've followed the documentation for socket.io to setup my app in the following way:
App.js
// there's a lot of code in here which sets what to use on my app but here's the important lines
const app = express();
const routes = require('./routes/index');
const sessionObj = {
secret: process.env.SECRET,
key: process.env.KEY,
resave: false,
saveUninitialized: false,
store: new MongoStore({ mongooseConnection: mongoose.connection }),
secret : 'test',
cookie:{_expires : Number(process.env.COOKIETIME)}, // time im ms
}
app.use(session(sessionObj));
app.use(passport.initialize());
app.use(passport.session());
module.exports = {app,sessionObj};
start.js
const mongoose = require('mongoose');
const passportSocketIo = require("passport.socketio");
const cookieParser = require('cookie-parser');
// import environmental variables from our variables.env file
require('dotenv').config({ path: 'variables.env' });
// Connect to our Database and handle an bad connections
mongoose.connect(process.env.DATABASE);
// import mongo db models
require('./models/user');
require('./models/team');
// Start our app!
const app = require('./app');
app.app.set('port', process.env.PORT || 7777);
const server = app.app.listen(app.app.get('port'), () => {
console.log(`Express running → PORT ${server.address().port}`);
});
const io = require('socket.io')(server);
io.set('authorization', passportSocketIo.authorize({
cookieParser: cookieParser,
key: app.sessionObj.key, // the name of the cookie where express/connect stores its session_id
secret: app.sessionObj.secret, // the session_secret to parse the cookie
store: app.sessionObj.store, // we NEED to use a sessionstore. no memorystore please
success: onAuthorizeSuccess, // *optional* callback on success - read more below
fail: onAuthorizeFail, // *optional* callback on fail/error - read more below
}));
function onAuthorizeSuccess(data, accept){}
function onAuthorizeFail(data, message, error, accept){}
io.on('connection', function(client) {
client.on('join', function(data) {
client.emit('messages',"server socket response!!");
});
client.on('getmessage', function(data) {
client.emit('messages',data);
});
});
My problem is that I have a lot of mongo DB save actions that are going on in my ./routes/index file and I would like to be able to emit message from my routes rather than from the end of start.js where socket.io is connected.
Is there any way that I could emit a websocket message from my ./routes/index file even though IO is setup further down the line in start.js?
for example something like this:
router.get('/add-team-member', (req, res) => {
// some io.emit action here
});
Maybe I need to move where i'm initializing the socket.io stuff but haven't been able to find any documentation on this or perhaps I can access socket.io from routes already somehow?
Thanks and appreciate the help, let me know if anything is unclear!

As mentioned above, io is in your global scope. If you do
router.get('/add-team-member', (req, res) => {
io.sockets.emit('AddTeamMember');
});
Then every client connected, if listening to that event AddTeamMember, will run it's associated .on function on their respective clients. This is probably the easiest solution, and unless you're expecting a huge wave of users without any plans of load balancing, this should be suitable for the time being.
Another alternative you can go:
socket.io lib has a rooms functionality where you can join and emit using the io object itself https://socket.io/docs/rooms-and-namespaces/ if you have a knack for this, it'd look something like this:
io.sockets.in('yourroom').broadcast('AddTeamMember');
This would essentially do the same thing as the top, only instead of broadcasting to every client, it'd only broadcast to those that are exclusive to that room. You'd have to basically figure out a way to get that users socket into the room //before// they made the get request, or in other words, make them exclusive. That way you can reduce the amount of load your server has to push out whenever that route request is made.
Lastly, if neither of the above options work for you, and you just absolutely have to send to that singular client when they initiate it, then it's going to get messy, because you have to have some sort of id to that person, and since you have no reference, you'd have to store all your sockets upon connection, and then make a comparison. I do not fully recommend something like this, because well, I haven't ever tested it, and don't know what type of repercussions could happen, but here is a jist of an idea I had:
app.set('trust proxy', true)
var SOCKETS = []
io.on('connection', function(client) {
SOCKETS.push(client);
client.on('join', function(data) {
client.emit('messages',"server socket response!!");
});
client.on('getmessage', function(data) {
client.emit('messages',data);
});
});
router.get('/add-team-member', (req, res) => {
for (let i=0; i< SOCKETS.length; i++){
if(SOCKETS[i].request.connection.remoteAddress == req.ip)
SOCKETS[i].emit('AddTeamMember');
}
});
Keep in mind, if you do go down this route, you're gonna need to maintain that array when users disconnect, and if you're doing session management, that's gonna get hairy really really quick.
Good luck, let us know your results.

Yes, it is possible, you just have to attach the instance of socket.io as long as you get a request on your server.
Looking to your file start.js you just have to replace your functions as:
// Start our app!
const app = require('./app');
app.app.set('port', process.env.PORT || 7777);
const io = require('socket.io')(app.app);
const server = app.app.listen(app.app.get('port'), () => {
server.on('request', function(request, response){
request.io = io;
}
console.log(`Express running → PORT ${server.address().port}`);
});
now when you receive an event that you want to emit some message to the clients you can use your io instance from the request object.
router.get('/add-team-member', (req, res) => {
req.io.sockets.emit('addteammember', {member: 6});
//as you are doing a broadcast you just need broadcast msg
....
res.status(200)
res.end()
});
Doing that i also were able to integrate with test framework like mocha, and test the events emited too...
I did some integrations like that, and in my experience the last thing to do was emit the msg to instances in the socket.
As a good practice the very begining of middleware functions i had were doing data validation, data sanitization and cleaning data.
Here is my working example:
var app = require('../app');
var server = require('http').Server(app);
var io = require('socket.io')(server);
io.on('connection', function(client) {
client.emit('connected');
client.on('disconnect', function() {
console.log('disconnected', client.id);
});
});
server.on('request', function(request, response) {
request.io = io;
});
pg.initialize(app.config.DATABASEURL, function(err){
if(err){
throw err;
}
app.set('port', process.env.PORT || 3000);
var server1 = server.listen(app.get('port'), function(){
var host = 'localhost';
var port = server1.address().port;
console.log('Example app listening at http://%s:%s', host, port);
});
});

Your io is actually the socket object, you can emit events from this object to any specific user by -
io.to(userSocketId).emit('eventName', data);
Or you can broadcast by -
io.emit('eventName', data);
Just create require socket.io before using it :)

You can use emiter-adapter to emit data to client in other process/server. It use redis DB as backend for emitting messages.

I did something similar in the past, using namespaces.
Let's say your client connect to your server using "Frontend" as the namespace.
My solution was to create the instance of socket.io as a class in a separate file:
websockets/index.js
const socket = require('socket.io');
class websockets {
constructor(server) {
this.io = socket(server);
this.frontend = new Frontend(this.io);
this.io.use((socket, next) => {
// put here the logic to authorize your users..
// even better in a separate file :-)
next();
});
}
}
class Frontend {
constructor(io) {
this.nsp = io.of('/Frontend');
[ ... ]
}
}
module.exports = websockets;
Then in App.js
const app = require('express')();
const server = require('http').createServer(app);
const websockets = require('./websockets/index');
const WS = new websockets(server);
app.use('/', (req, res, next) => {
req.websocket = WS;
next();
}, require('./routes/index'));
[ ... ]
Finally, your routes can do:
routes/index.js
router.get('/add-team-member', (req, res) => {
req.websocket.frontend.nsp.emit('whatever', { ... });
[ ... ]
});