I am using Express.js as http server. Defined all my routes.
Most endpoints need to verify session before returning a response. E.g. below code serves users in the system and list of services respectively:
function getUsers(req, res, next) {
verifyUser(req, res, next, function () {
//serve users
});
}
function getServices(req, res, next) {
verifyUser(req, res, next, function () {
//serve services
});
}
You probably noticed there is a verifyUser function which validates the session. Which is as below.
function verifyUser(req, res, next, callback) {
var sessionKey = req.cookies.sessionKey;
var user = users.userBySession(sessionKey);
if (user) {
callback(req, res, next, user);
} else {
res.status(401).send({
message: 'Unauthorized'
});
}
}
As you can see I keep passing in req, res and next parameters along with a callback whenever I use this function.
I tried to use apply function to make it easier. Changed my getUsers function like this:
function getUsers(req, res, next) {
verifyUser
.apply(null, arguments, function () {
//serve users
});
}
The problem with this approach is callback is not passed into verifyUser function. And I don't really like passing null as scope with each call.
How can I achieve this by writing less and better code ? Any ideas?
You could use bind to create a 'partial function':
// create bound responseHelper object
var responseHelper = verifyUser.bind(null, req, res, next);
// usage
responseHelper(getUsersCallback); // same as verifyUser(req, res, next, getusersCallBack);
I think you're looking to turn verifyUser into a middleware function.
function verifyUser (req, res, next) {
var user = // yadda yadda session stuff
if (user) {
req.user = user; // [1] what you do to the req object here...
} else {
return res.status(401).send({ message: "No way Smokey Joe"});
/**
* alternatively, do something like
* var err = new Error("Not authorized");
* err.statusCode = 401;
* return next(err);
*
* this will kick off Express' error handling mechanism,
* which you should read about in the docs (see the link below)
*/
}
next();
// very important to call next after this verifyUser has done its job
// if you don't, the next middleware won't go off,
// and the request will just hang
}
function getUsers (req, res, next) {
// [2] will show up on the req object here, assuming you chain these
// two functions together as middleware
}
app.get("/users", verifyUser, getUsers);
app.get("/services", verifyUser, getServices);
// here's a route that needs no session auth, so no need to verifyUser
app.get("/latest-posts", getLatestPosts);
When you tell Express to use a function or attach a function to a route path via get('/my/route', hanlderFun) or some such, you've basically turned handlerFun into a middleware.
You can define however many middleware as handlers on a route as you like, and they'll all execute in turn as long as you keep calling next.
app.post("/checkout", verifyUser, tallyCart, checkInventory, doPayment, sendInvoice);
The job of next is to pass control from the current middelware to the next one. It's an object
You can do other stuff with next, too, which you should read up on in the docs.
http://expressjs.com/en/guide/writing-middleware.html
http://expressjs.com/en/guide/using-middleware.html
The docs on routing have good info on middleware as well:
http://expressjs.com/en/guide/routing.html
For extra credit, check out error handling middleware, too:
http://expressjs.com/en/guide/error-handling.html
Related
I'm struggling a lot trying to understand the logic behind the ExpressJS module, in particular i'm focusing on the implementation of the middlewares chain.
My objective here is to understand how is it possible to implement the logic of a server listening for requests, and once arrived, passes the request packet through an array of functions each of which:
should be able to modify the packet for the next middleware
should be able to send the response back but it should not prevent other middlewares to respond as well, so there should be only one response at the end of the chain (i guess)
should access some function to call in order to run the next middleware in chain, but without the need to pass it the request and response parameters (e.g. just calling "next( )" )
Note on point 1
I imagine that with languages like javascript is possible to pass the object around using concepts like closure, but i would like to understand it in a language-independent way and use javascript features only if it provides clever ways to handle it simply, otherwise i'll write the logic as i was using any other language.
So my big questions are:
How should the request handling be implemented?
How do i allow every middleware to modify the packet?
How do i pass the updated packet to the next middleware?
How do i implement the "next" function to pass to every middleware?
Thank you
Broadly, this is how a chain of middlewares with a next function can be called.
function myMiddleware(req, res, next) {
// do something with req or res
next();
}
const mws = [
myMiddleware,
anotherMiddleware
];
/**
* Calls a chain of middlewares.
*
* mws is an array of middlewares
*/
function callMwChain(req, res, mws) {
if (mws.length === 0) {
// We're done here
return;
}
/**
* Take the first middleware
*/
const firstMw = mws[0];
/**
* Call it, and give it a next function that continues the chain
*/
firstMw(req, res, () => {
callMwChain(req, res, mws.slice(1));
});
}
I've written this as a recursive function, but it can also be rewritten as stack. This is easier though.
Express middlewares have a lot of bonus magic, so their implementation is going to be more complex, but this is broadly how it works.
Note that there's no event-loop in sight.
Here's a little working server that supports middleware and GET routes. It uses a similar calling convention on the route handlers as Express (req, res, next), but has far, far, far fewer features than Express. But, hopefully you can see how the dispatch function cycles through the routes, advancing to the next route only if the previous handler calls next().
You can actually run this code and define routes and play with it (though it is definitely barebones basic):
const http = require('http');
function dispatch(req, res, array, index, cb) {
console.log(req.method, req.url, array, index);
if (array && array.length > index) {
let item = array[index];
if (!item.path || item.path.toLowerCase() === req.url.toLowerCase()) {
item.fn(req, res, function() {
// previous route called next(), so we advance to the next item in the array
++index;
dispatch(req, res, array, index, cb);
});
return;
}
}
cb();
}
const server = http.createServer((req, res) => {
// incoming request here, initiate calling the middleware
dispatch(req, res, server.middleware, 0, function() {
if (req.method === "GET") {
dispatch(req, res, server.gets, 0, function() {
// done with gets here, apparently didn't send a response
res.statusCode = 404;
res.end();
})
} else {
// ... fill in code for other methods here
res.statusCode = 404;
res.end();
}
});
});
// server route implementation
// save arrays for each type of route handler
// upon a request, run all the middleware requests, then advance
// to the routes for the right type of request and see if any of them
// match the URL. If so, run them.
server.middleware = [];
server.gets = [];
server.posts = [];
server.use = function(fn) {
server.middleware.push({path: "", fn: fn});
}
server.get = function(path, fn) {
server.gets.push({path, fn});
}
// route definitions
server.use((req, res, next) => {
console.log(req.url);
req.myCustom = "hi";
next();
});
server.use((req, res, next) => {
console.log(req.myCustom);
next();
});
server.get("/", (req, res, next) => {
res.write("Hello World");
res.end();
});
server.listen(80);
As stated, multiple callback functions can be provided and behave like middleware to handle a request. They can be in the form of a function, an array of functions, or combinations of both, as shown in the following examples.
For example:
app.get('/example/b', function (req, res, next) {
console.log('the response will be sent by the next function ...')
next()
}, function (req, res) {
res.send('Hello from B!')
})
What's the purpose of this? Can't we simply use:
app.get('/example/b', function (req, res) {
console.log('the response will be sent by the next function ...')
res.send('Hello from B!')
})
The multiple functions would more likely be used when you already have a previously defined function that you probably intend to use in multiple places. For example:
app.get("/somePath", checkAuth, function(req, res) {
// you know it's already authenticated here
});
app.get("/someOtherPath", checkAuth, function(req, res) {
// you know it's already authenticated here
});
function checkAuth(req, res, next) {
if (some logic here) {
// allow handler chain to continue
next();
} else {
// auth error
res.status(401).end();
}
}
Of course, you could also use middleware for checking authentication, but the above example allows you to target just a few specific routes with some middleware that you may use in multiple places.
As you have already observed, if you don't intend to use the function anywhere else, then you may as well just put the logic into your one handler.
Yes you can, the purpose is for example, to handle errors, the middleware sequence in express allows you to use this way. For example, see this way to set up the express config:
app.use(logger.connectLogger());
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({
extended: false
}));
app.use(routes);
app.use(errorConnect);
http.createServer(app).listen(config.port, function () {
logger.getLogger().info('My backend listening on port: ' + config.port);
});
My routes module have all the matching route -> callback:
// Methods exposed for backend API.
router.get('/status', ips.getStatus);
router.route('/ip')
.post(ips.keepIps)
.get(ips.getIps)
// NOT ALLOWED
.put(returnNotAllowed)
.delete(returnNotAllowed);
// Methods exposed and used by IP's frontend.
router.route('/front/ip')
.get(front.getIpsByFront)
.post(front.keepIpsByFront);
router.post('/login', login.login);
....
For example in one of those callbacks, I have the next way to manage an incoming request:
/**
* Login user from frontend.
*/
exports.login = function(req, res, next) {
var username = req.body.username + '#System',
password = req.body.password,
server = req.body.server,
auth = 'Basic ' + new Buffer(username + ':' + password).toString('base64');
loginApi.login(auth, server)
.then(function(result) {
res.statusCode = 200;
res.send(result);
})
.catch(function(error) {
next({
statusCode: 403,
message: 'Error in login'
});
});
};
When I catch an error, I call next with a custom error object, and after this, if you back and watch the config (first paragraph) you can see that I added to the express middleware an error manage with errorConnect. In my opinion this is a usefull way to understand what you are asking because if I understand well you had doubts with next()
exports.signin = function(req, res, next) {
passport.authenticate('local', function(err, user, info) {
if (err || !user) {
res.status(400).send(info);
} else {
// Remove sensitive data before user.password = undefined; user.salt = undefined;
req.login(user, function(err) {
if (err) {
res.status(400).send(err);
} else {
res.json(user);
}
});
}
})(req, res, next);
};
This a piece of code in the "MEAN Web Development" book by AmosQ.Haviv.Who could tell me what the method passport.authenticate()'tail:(req, res, next) means?Is that a Closure?
passport.authenticate() will probably take in the settings 'local' (a domain ? ) and the calllback to create a new function that will do the authentication.
This function will indeed create a closure over 'local' and the callback.
The new auth function will expect 3 parameters: the original req(uest), res(ponse) object and a next parameter and is immediately called using the ( req, res, next ) syntax.
So it's very likely that this specific function will do the login and then run the callback used to create the auth function, passing req and res back into the callback.
Look at it as a way to use the passport.authenticate() method to create different versions of logins you can use. One for 'local', one for 'otherDomain', etc.
I use node.js and express v4.12. I want to decorate all app.get calls by custom logic.
app.get(/*getPath*/, function (req, res, next) {
// regular logic
});
and my custom logic
customFunc() {
if (getPath === 'somePath' && req.headers.authorization === 'encoded user'){
//costum logic goes here
next();
} else {
res.sendStatus(403);
}
}
The idea is to execute custom logic before code that I already have but I need access to req, res and next objects inside my custom function. And another problem that I need to have app.get arguments to work with requested pattern inside custumFunc.
I tried to implement decorator pattern just like this:
var testfunc = function() {
console.log('decorated!');
};
var decorator = function(f, app_get) {
f();
return app_get.apply(this, arguments);
};
app.get = decorator(testfunc, app.get);
But javascript throws an error.
EDIT
In case app.use() I can only get req.path like /users/22 but when I use it like app.get('/users/:id', acl, cb) I can get req.route.path property and it equals '/users/:id'and this is what I need for my ACL decorator. But I don't want to call acl function for each endpoint and try to move it to app.use() but whith req.route.path property.
Example of implementing your middleware:
app.use(function(req, res, next) {
if (req.path==='somePath' && req.headers.authorization ==='encoded user'){
//costum logic goes here
next();
} else {
res.sendStatus(403);
}
});
If you want to pass middleware only in one route you can implement like this:
app.get(/*getPath*/, customFunc, function (req, res, next) {
// regular logic
});
You are trying to build a middleware. Simply add your decorator to the application via app.use.
I am currently working on a project to develop an API manager to control an existing API.
It contains a list of "before" and "after" middlewares, which are used to do things like security checking and logging. And a "service" middleware to do http request to the existing API. But the problem is that I want to make the order the middleware being executed to be dynamic, meaning that I could load some configuration file to change the order the middleaware get executed every time the request comes in.
here is my previous code:
'use strict';
// Loading the express library
var express = require('express');
var app = express();
var service = require('./routes/index');
// Testing configurable middleware
var confirguration = {
before1: {
priority: 100,
enable: true
},
before2: {
priority: 80,
enable: true
},
service: {
priority: 50,
enable: true
},
after1: {
priority: 30,
enable: true
},
after2: {
priority: 10,
enable: true
}
}
var before1 = require('./example_middleware/before1');
var before2 = require('./example_middleware/before2');
var after1 = require('./example_middleware/after1');
var after2 = require('./example_middleware/after2');
// Fake request to simulate the /service
var fakeRequest = require('./example_middleware/fake_request');
// Function to sort the order of the middleware to be executed
var sortConfig = function(confirguration){
var sortable = [];
for (var middleware in confirguration)
// To make middlewares configurable
if (confirguration[middleware]['enable'] == true){
sortable.push([middleware, confirguration[middleware]['priority']]);
}
sortable.sort(function(a, b) {return b[1] - a[1]});
return sortable;
}
// var sortedConfig = [];
var middlewareSet = new Array();
app.use('/test', function(request, response, next){
var middleware;
var sortedConfig = sortConfig(confirguration);
for (var i in sortedConfig){
switch(sortedConfig[i][0]){
case 'before1':
middleware = before1;
break;
case 'before2':
middleware = before2;
break;
case 'service':
middleware = fakeRequest;
break;
case 'after1':
middleware = after1;
break;
case 'after2':
middleware = after2;
break;
}
// console.log(sortedConfig[i][0]);
// Execute the middleware in expected order
middlewareSet.push(middleware);
}
// request.sortedConfig = sortedConfig;
console.log(middlewareSet);
console.log('middleware list sorted');
next();
});
app.use('/test', middlewareSet);
But I keep getting the same error message coming from the app.use() at the last line:
app.use() requires middleware functions
It works if I use:
app.use('/test', [before1, before2, fakeRequest, after1, after2]);
But it's not dynamic though, what did I misunderstand? There must be a way to do this in express.js.
Thanks in advance.
EDIT:
I modified my code according to Ryan's answer, here is the code:
var async = require('async');
app.use('/test', configurableMiddleWare);
function configurableMiddleWare(req, res, next) {
var operations = [];
var middleware;
var sortedConfig = sortConfig(confirguration);
// push each middleware you want to run
sortedConfig.forEach(function(fn) {
switch(fn[0]){
case 'before1':
middleware = before1;
break;
case 'before2':
middleware = before2;
break;
case 'service':
middleware = fakeRequest;
break;
case 'after1':
middleware = after1;
break;
case 'after2':
middleware = after2;
break;
}
operations.push(middleware); // could use fn.bind(null, req, res) to pass in vars
});
console.log('middleware list sorted');
// now actually invoke the middleware in series
async.series(operations, function(err) {
if(err) {
// one of the functions passed back an error so handle it here
return next(err);
}
// no errors so pass control back to express
next();
});
}
Just to make sure I haven't made any mistakes in my test middleware, here is an example of one of them:
'use strict';
var express = require('express');
var router = express.Router();
router.route('/')
.all(function(request, response, next){
console.log('This is middleware BEFORE1');
next();
});
module.exports = router;
Now, when I run my application, I got the following error from npm:
TypeError: Cannot call method 'indexOf' of undefined
at Function.proto.handle (/Users/jialunliu/Documents/SOA_project/FAT-LADY/node_modules/express/lib/router/index.js:130:28)
at router (/Users/jialunliu/Documents/SOA_project/FAT-LADY/node_modules/express/lib/router/index.js:35:12)
at /Users/jialunliu/Documents/SOA_project/FAT-LADY/node_modules/async/lib/async.js:610:21
at /Users/jialunliu/Documents/SOA_project/FAT-LADY/node_modules/async/lib/async.js:249:17
at iterate (/Users/jialunliu/Documents/SOA_project/FAT-LADY/node_modules/async/lib/async.js:149:13)
at async.eachSeries (/Users/jialunliu/Documents/SOA_project/FAT-LADY/node_modules/async/lib/async.js:165:9)
at _asyncMap (/Users/jialunliu/Documents/SOA_project/FAT-LADY/node_modules/async/lib/async.js:248:13)
at Object.mapSeries (/Users/jialunliu/Documents/SOA_project/FAT-LADY/node_modules/async/lib/async.js:231:23)
at Object.async.series (/Users/jialunliu/Documents/SOA_project/FAT-LADY/node_modules/async/lib/async.js:608:19)
at configurableMiddleWare (/Users/jialunliu/Documents/SOA_project/FAT-LADY/app.js:135:11)
Which is coming from the line
async.series(operations, function(err){})
I am keep getting this kind of error message, saying the function could not read from this array of functions "operations"....
I think you are on the right track, you will just need to tweak a few things. I would register one top level function with app.use() and then do all of your dynamic stuff within that function. Updating my answer to a working example. Be sure to install async first npm install --save async
// define all middleware functions
var middleware = {
mw1: function(req, res, next) {
console.log('mw 1');
next();
},
mw2: function(req, res, next) {
console.log('mw 2');
next();
},
mw3: function(req, res, next) {
console.log('mw 3');
next();
},
mw4: function(req, res, next) {
console.log('mw 4');
next();
}
};
// register our "top level function"
app.use(configurableMiddleware);
var requestCount = 1; // this is just for the working example
function configurableMiddleware(req, res, next) {
var isEvenRequest = requestCount++ % 2 === 0; // simple logic to alternate which "configurable" middleware to use
var operations; // in the real world you could build this array dynamically, for now we just hardcode two scenarios as an example
// Each request to http://localhost:3000 will alternate which middleware is used, so you will see a different log each time
if(isEvenRequest) {
console.log('Even request should log mw2 and mw4');
// .bind(null, req, res) makes sure that the middleware gets the request and response objects when they are invoked,
// as of this point they still haven't been invoked...
operations = [middleware.mw2.bind(null, req, res), middleware.mw4.bind(null, req, res)];
}
else {
console.log('Odd request should log mw1 and mw3');
operations = [middleware.mw1.bind(null, req, res), middleware.mw3.bind(null, req, res)];
}
// invoke each middleware in series - you could also do async.parallel if the order of middleware doesn't matter
// using the async module: https://github.com/caolan/async
async.series(operations, function(err) {
if(err) {
console.log('There was a problem running the middleware!');
return next(err);
}
// all middleware has been run
next();
});
}
For more info on .bind() see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/bind
connect-sequence: a dedicated node module for that specific purpose:
You can just use the module connect-sequence which is designed for that purpose:
npm install --save connect-sequence
see:
the npmjs page: https://www.npmjs.com/package/connect-sequence
or the github project: https://github.com/sirap-group/connect-sequence
and then, here an example of usage:
/**
* Product API
* #module
*/
var ConnectSequence = require('connect-sequence')
var productsController = require('./products.controller')
module.exports = productRouter
function productRouter (app) {
app.route('/api/products/:productId')
.get(function (req, res, next) {
// Create a ConnectSequence instance and setup it with the current `req`,
// `res` objects and the `next` callback
var seq = new ConnectSequence(req, res, next)
// build the desired middlewares sequence thanks to:
// - ConnectSequence#append(mid0, ..., mid1),
// - ConnectSequence#appendList([mid0, ..., mid1])
// - and ConnectSequence#appendIf(condition, mid)
if (req.query.filter) {
seq.append(productsController.filter)
}
if (req.query.format) {
seq.append(
productsController.validateFormat,
productsController.beforeFormat,
productsController.format,
productsController.afterFormat
)
}
// append the productsController.prepareResponse middleware to the sequence
// only if the condition `req.query.format && req.formatedProduct` is true
// at the moment where the middleware would be called.
// So the condition is tested after the previous middleware is called and thus
// if the previous modifies the `req` object, we can test it.
seq.appendIf(isProductFormatted, productsController.prepareResponse)
seq.append(productsController.sendResponse)
// run the sequence
seq.run()
})
app.param('productId', function (req, res, next, id) {
// ... yield the product by ID and bind it to the req object
})
function isProductFormatted (req) {
return Boolean(req.formatedProduct)
}
}
This is open source, PR are welcome!
If you like and use connect-sequence, but if you find bug or need some new features, feel free to post issues or submit pull requests!
Based on the idea behind #Ryan's code I came up with this function. It executes a list of middleware in order binding the variables as needed, allowing everything to be executed by just executeMiddlewareList([middleware1, middleware2...], req, res, next);. For each middlewarereq, res is passed and the callback from async.eachSeries. This means when next() is called inside the middleware, then next one will be handled from the list. If middleware throws an error with next(err), execution will stop and you can manually handle this.
function executeMiddlewareList (middlewareList, req, res, next) {
async.eachSeries(middlewareList, function(middleware,callback) {
middleware.bind(null,req,res,callback)()
}, function(err) {
if (err) return res.status(500).json({error: err});
next();
})
}
function testMid (number) {
return function (req, res, next) {
log.debug('req.test from', req.test, " to ", number);
req.test=number;
next();
}
}
router.get('/test', function(req, res, next) {
m.executeMiddlewareList([test(1), test(2)], req, res, next);
//Output: req.test from undefined to 1
// req.test from 1 to 2
}, function (req,res) {
//Do stuff after the executeMiddlewareList, req.test = 2
})
Finally, I find the answer according the Ryan's, the code would look like this:
function configurableMiddleWare(req, res, next) {
var operations = [];
var middleware;
var sortedConfig = sortConfig(confirguration);
// push each middleware you want to run
sortedConfig.forEach(function(fn) {
switch(fn[0]){
case 'before1':
middleware = before1;
break;
case 'before2':
middleware = before2;
break;
case 'service':
middleware = fakeRequest;
break;
case 'after1':
middleware = after1;
break;
case 'after2':
middleware = after2;
break;
}
console.log(fn[0]);
console.log(middleware);
operations.push(middleware.bind(null, req, res)); // could use fn.bind(null, req, res) to pass in vars
});
console.log('middleware list sorted');
// now actually invoke the middleware in series
async.series(operations, function(err) {
if(err) {
// one of the functions passed back an error so handle it here
return next(err);
}
console.log('middleware get executed');
// no errors so pass control back to express
next();
});
}
app.use('/test', configurableMiddleWare);
The key step is indeed the operations.push(middleware.bind(null, req, res));
Which to be honest, I don't understand what does it mean exactly. I know this is passing "req" and "res" variable into the middleware, but I don't get what the point of the "null" in the front. Much appreciated if someone could help me clarify this.