I just brought up this issue with Express and I am interested in what StackOverflow thinks about it:
https://github.com/strongloop/express/issues/2831
my question is why Express opts to disallow the developer from passing data directly between middleware functions and basically forces you to assign temporary data to the request object in what I have always considered to be a quite awkward assignment.
to be more specific:
passing data between middleware functions usually involves doing this
req.specialData = {}
next();
however, it would be perhaps much easier and more performant (!) if this were possible
next(null,data);
or
function mySpecialMiddleWare(req,res,next,data){}
//now, call the above function
mySpecialMiddleWare(req,res,next, {some:data});
the problem is that Express uses a, what I believe to be, stupid way of determining if the call is to a normal middleware function or the next error first middleware function, by checking to see if the function.length > 3 or function.length === 4...
is there any chance what I am saying makes any sense?
wouldn't it be better/easier/faster/stronger to allow direct passing of data via middleware functions instead of assigning data awkwardly to req ??
perhaps Express already has this capability and I am just misinformed?
my question is why Express opts to disallow the developer from passing data directly between middleware functions and basically forces you to assign temporary data to the request object in what I have always considered to be a quite awkward assignment.
So I think the API is the way it is to encourage most middleware to be modular, re-usable, and loosely coupled. That means the generally should do something without being concerned too much about what other middleware might run before them or after them. In order to achieve this and create an ecosystem of loosely-compatible middleware functions, express keeps the API fairly generic. There are pros and cons to this. But as a direct answer to your first question, I would say to keep the interface simple, consistent, and flexible as well as try to avoid strict ordering dependencies.
As in your case, there might be implicit dependencies between middlewares. Another common example would be typically your session middleware has an implicit dependency that your cookie middleware runs before it. Having these all be implicit can be argued to create bugs and unnecessary troubleshooting. On the other hand, it enables application developers to more easily mix and match middlewares that might otherwise have no awareness of one another.
In hindsight I think both I as well as some of the express maintainers would agree that using function arity (number of expected arguments) for semantic API reasons was an odd and poor choice on TJ's part. I think if the project were to be rewritten a more explicit API would be defined for error handling.
wouldn't it be better/easier/faster/stronger to allow direct passing of data via middleware functions instead of assigning data awkwardly to req ??
Better - this is highly arguable and opinion based. There is a lot to be said for the simplicity of it and the evidence is the huge ecosystem and usage. There are alternatives available such as hapi, restify, etc, though so you might consider them.
Easier - probably not. What's there is pretty easy.
Faster - probably not in any meaningful way. Not sure why you think your version would be faster, but best to bring metrics when you make such claims.
Stronger - If by "stronger" you mean more explicit that's probably true but there's a reason some people still prefer JavaScript even though TypeScript all the way up through Haskell exist and are definitely "stronger" in some sense of the word.
There might be advantages to what Express is doing, however, it is worth noting that Express allows passing data between middlewares using res.locals. With that said, your question inspired me to make this library. Expressjs-plus.
With it, you will be able to pass data between your middlewares easily without having to use the request nor the response object.
You can use regular javascript functions e.g. function(varInResLocals, varInReqParams, callback) instead of express middleware signature.
Here is a working example for you.
var express = require('express');
var ExpressPlus = require('expressjs-plus').ExpressPlus;
var app = express();
// simple handler example
var userHandler = function(param, paramsArray, req, res){
if(param !== 'user') return false;
paramsArray.push("USER WAS FOUND!");
return true;
};
// this handler allows you to pass res.locals properties between your middlewares seemingly,
// it the parameter was found in locals, it attaches it to paramsArray.
var resLocalsHandler = function(param, paramsArray, req, res){
if(param in res.locals){
paramsArray.push(res.locals[param]);
return true;
}else return false;
};
var appPlus = new ExpressPlus(app, [userHandler, resLocalsHandler], []);
var regularFunction = function(user, id, cb){
return cb(null, { response: {user: user, id: id}, status: 200, resLocalsVar: "passVar" });
};
// resLocalsVar was passed in a previous method
var regularFunction2 = function(resLocalsVar, user, id, cb){
// now you can have access to it
console.log(resLocalsVar);
return cb(null);
};
// the responder at the end will use res.locals.status and res.locals.response to issue an HTTP response
app.use(appPlus.GMV(regularFunction), appPlus.GMV(regularFunction2), appPlus.responder);
// adds error handlers, it will add a default error handler along with the list of error handlers passed
// in this case, no error handlers were passed
appPlus.setErrorHandlers();
app.listen(3001, function(){
console.log('Listening!');
});
Related
In NodeJS, using Javascript I have a "res" object that can be either an Express or Fastify response, is there any way to check which of the two is this response?
There are 3 ways to solve this problem: the bad way, the less-awful way, and the way that results in actual clean code.
1. The bad way: duck typing
The most intiutive way is using a technique called "duck typing" to tell what kind of object you're dealing with. Look at res, check what properties it has, and infer the type based on that. In particular, you'll be interested in the differences between the objects to be able to tell them apart, so search for special features that the other framework doesn't have.
Example checks:
function isExpress(res) {
return (typeof res === 'object' && typeof res.sendFile === 'function');
}
function isFastify(res) {
return (typeof res === 'object' && typeof res.hasHeader === 'function');
}
However, there's a problem: what if you get a res that is extended by a middleware for express.js or for fastify, which adds the property? Then, depending on the ordering of your if-elses, you may confuse one for the other.
2. The less-awful way: decorating the object yourself
It would be better if we could tell the objects apart regardless of what middlewares are applied - so that a feature being added doesn't break our type recognition logic.
Instead, we can purposefully extend the res object using a special, additional property, so that it's obvious from looking at that property whether we're dealing with express or fastify. To do this, we need to make sure that:
express will decorate res with a property that says "I'm express"
fastify will decorate res with a property that says "I'm fastify"
Quite literally:
const FRAMEWORK_TYPE = Symbol('type of framework that res comes from');
expressApp.use(function(req, res, next) {
res[FRAMEWORK_TYPE] = 'express';
next();
});
fastifyApp.decorateReply(FRAMEWORK_TYPE, 'fastify');
Then, you can refer to res[FRAMEWORK_TYPE] and check the hint that you've left for yourself directly.
3. The clean way: design patterns
The way you've formulated the original problem is: how to check what type of instance we're dealing with. However, the purpose of this check is most likely to be able to integrate different frameworks (express and fastify) with a single piece of code that does something else (a handler).
This problem is usually solved with the Adapter design pattern - have a layer that speaks a common API and hides the complexity of having different possible implementations. For instance, say your handler needs the request body and headers, and you need to be able to respond via JSON regardless of framework. Then, you could define the handler signature to be:
({ body, headers }) => responseObject
Next, call that handler from both frameworks. You'll notice that you need to refer to different parts of req and res in case of express and fastify in order to call the handler properly. Also, in case of fastify, you'll be able to just Promise.resolve(responseObject), but for express, you may need to do res.json(responseObject) instead.
This way, you decouple your business logic (handler) from the infrastructure layer (web server).
I'm fairly new to JS especially Node and Express. I am following some tutorials on how to build an API and at the same time learning about JS special features such as let/const/var, arrow functions etc.
In many tutorials I have seen things likes this :
somecode.then((result) => {someothercode})
With: "somecode" being for example a get request
Is "result" the name of the returned value or is it a convention that JS developper use?
By that I mean, does this for example work?
somecode.then((foo) => {someothercode})
Also for req, res variables what does this mean?
app.get("/users/:userId", [
usersController.getById
]);
Here is the getById function (using once again the "result"):
exports.getById = (req, res) => {
userModel.findById(req.params.userId).then((result) => {
res.status(200).send(result);
});
};
the getById method defined in the controller needs (req, res), does that mean, when i call it like the code above, the req and res arguments are implicitly used?
Also it needs a parameter :
req.params.userId
which is in the url of the route, how does it pass to another file?
I have a route.js file that uses a controller.js file that uses a model.js. How does the param go from route to controller?
And it won't work if I change the param name right? for example:
req.params.id
Sorry for long post, I'm trying to understand JS logic to get some good habits and write clean code.
Thanks!
Is "result" the name of the returned value or is it a convention that JS developper use? By that I mean, does this for example work?
From my experience, yes - result is often used. Often times you'll see thing like value, response, but ultimately it can be whatever you define. I would recommend sticking to convention, and also check out the MDN Promise tutorial if you are starting out with understanding NodeJS asynchronous operations.
Also for req, res variables what does this mean?
app.get("/users/:userId", [
usersController.getById
]);
That is a middleware chain. Check out the Express docs for more information.
the getById method defined in the controller needs (req, res), does that mean, when i call it like the code above, the req and res arguments are implicitly used? Also it needs a parameter :
req.params.userId
which is in the url It won't work if I change the param name right? for example:
req.params.id
Yes, that is using a named parameter. Without the full router code, it is hard to know how the getById method is linked to the defined route. The Express routing documentation will likely be a good start on that.
Is "result" the name of the returned value or is it a convention that JS developper use?
result is the name of a new variable you are creating to represent the value passed in from the Promise resolution. Yes, your foo example will work.
(req, res) => {} is the same (mostly) as a function that looks like this:
function getById(req, res) {...}
req, and res are just representational of the values that will be passed to this function. They could just as easily have been called (foo, bar).
It looks like you're struggling with understanding callback functions. Consider the following code then please crack open the source code for the packages that you are using. and it looks like you are using express.js
function something(callback) {
var x = 5;
var y = 'anything';
callback(x, y);
}
something(function(req, res) {
console.log(req);
console.log(res);
});
the something function is created and inside of that function scope, var x and y are created with any type. then when we invoke or use something function we are passing a function as a variable that gets passed in as variable callback then it can be used since it is a function so we call callback with x and y which can literally be any value and for effect, I am passing back a number and a string as req and res.
It's just a convention. Note that the code:
somecode.then((result) => {someothercode});
Is actually:
somecode.then(myFunction);
Since somecode is a Promise, your function may be called with zero or one argument. It is up to you to name this argument:
function myFunction (foo) {
// use foo here
}
somecode.then(myFunction);
Of course, unlike some other languages, javascript does not force you to name your function. You can just use a nameless (anonymous) function:
somecode.then(function(mango) { /* use mango here */ })
Arrow functions is a new syntax allowing you to write anonymous functions in a shorter style (it also behaves slightly differently with regards to scope and the value of this)
Express.js and http.Server
In node's http.Server library and Express.js framework, each server request will call a function you define and pass it two arguments: the request object and the response object. The variables req and res are just conventions people use when writing their own request handler functions. You can name them anything you like. For example you may prefer to use request and response instead or rx and tx:
app.get('/say/hello', (rx, tx) => tx.send('Hello'));
How many arguments do I write a callback function with??
The best way to know is to read the documentation of the module you are using. It is not the only way to know - you can of course read the source code instead. But it is often easier to read the documentation. Because of this, javascript modules tend to have really good documentation (otherwise they would be unusable and ignored by the community).
Express.js will actually pass three arguments to your callback (not two!!) - request, response and next where next is a function you can call if you want Express to continue processing instead of replying to the request. One interesting feature of javascript is that you are allowed to call functions with fewer or more arguments and it is not considered a syntax error:
function example (x) {}
example(); // not an error
example(1); // not an error
example(1,2,3,4); // also not an error
Express uses this feature by always calling your callback with three arguments while allowing you to declare said callback with only two arguments if you don't need the third, next argument.
My html page makes an AJAX request to Node to get process.env. I do not have passwords or keys there. But, likely will need it in the future. Is there a secure way I can do this without templating?
client-side
// source.html
$.get( "/env", function( data ) {console.log(data);});
Server side
// index.js
app.get('/source', function(req, res){
res.sendFile(__dirname + '/source.html');
});
app.get('/env', function(req, res) {
res.send(process.env);
});
I looked into EJS templating, but that seemed overkill for passing process.env as described in Variables between Node.js Server and Client.
Ideas:
App.get('/source', app.use(process.env.SOCKET_ADDR))? Am I overlooking an express route passing JS object option?
Not clean, but I could make routes for environment variables individually rather than calling the whole object. On the same thought, make a new env object that contains no sensitive variables.
Suck it up and template...
It sounds like you have two concerns: security (possibly not relevant now, but will be in the future) and presentation (to template or not to template?). It's still not clear to me what your presentation needs are, but I'll offer some suggestions about that. We'll start with security.
Given that you may, in the future, have environment variables that expose details that are sensitive, I recommend whitelisting the environment variables that you return. It will prevent (or at least reduce the likelihood) of sensitive information being exposed accidentally in the future. Whitelisting means you are explicit about what you let through (blacklisting means you are explicit about what you do not let through). Here's how I would implement a whitelist of environment variables to return (this just returns JSON; we'll talk about presentation later):
const envWhitelist = new Set([
"TERM", "SSH_TTY", "PATH", "SHELL",
"EDITOR", "LANG", "NODE_PATH",
]);
app.get('/env.json', function(req, res) {
res.send(Object.keys(process.env)
.filter(k => envWhitelist.has(k))
.reduce((a, k) => (a[k] = process.env[k], a), {})
);
});
NOTE: I'm using some ES6 features here. Particularly arrow notation functions and the Set object. This works in current versions of Node (v4.2.2 and up).
This allows you to specify what environment variables you consider "safe" to expose to the world. I would be very careful here, you might be surprised what environment variables could lead an experienced hacker to a successful exploit. For example, I wouldn't expose the PATH, or TERM, or frankly just about anything that I wasn't setting myself. So be careful here.
I think blacklisting in this instance would be very dangerous (if you add an environment variable that contains sensitive information, you will have to remember every time to add it to the blacklist), but for completeness, here it is:
const envBlacklist = new Set([
"SOME_SECRET_VAR", "DB_PASSWORD",
"WHATEVER_ELSE",
]);
app.get('/env.json', function(req, res) {
res.send(Object.keys(process.env)
.filter(k => !envBlacklist.has(k))
.reduce((a, k) => (a[k] = process.env[k], a), {})
);
});
Okay, now let's talk about presentation. So far, we've just been returning JSON (which is why I changed the endpoint to be GET /env.json), which lends itself to front-end presentation. I.e., you could use an AJAX call and then dynamically construct HTML with jQuery or a templating library. But if you wanted to get HTML from the server, it would be easy enough to do, even without a templating library. Here's a whitelisted example using an HTML <dl>:
const envWhitelist = new Set([
"TERM", "SSH_TTY", "PATH", "SHELL",
"EDITOR", "LANG", "NODE_PATH",
]);
app.get('/env', function(req, res) {
const keys = Object.keys(process.env)
.filter(k => envWhitelist.has(k));
res.send('<dl>' +
keys.map(k => '<dt>' + k + '</dt><dd>' + process.env[k] + '</dd>')
.join('') + '</dl>');
});
I'm not sure I think this example is better than using templates, but it is one way to avoid templating.
I am currently starting with node.js, so I am for the first time using Js beyond dom manipulation.
I came across a code piece like below. I cant understand it. What is happening? is it a key value object? Is an anonymous function being passed to 'new'?
module.exports = {
'new': function(req, res) {
res.view();
},
/**
* Overrides for the settings in `config/controllers.js`
* (specific to UserController)
*/
_config: {}
};
As others have said, this is ultimately just creating an object called module.exports then assigning two properties to it. One is another object called _config and the other is a function called new that expects two arguments.
That's the plain JavaScript explanation.
In node.js, you're also seeing a few conventions in play, which I'll describe below.
One convention is module.exports.
This is the object that will be made available when some other code loads this file using require(). It would work something like this:
var m = require('yourmodule.js');
m.new(req, res);
Another convention is the pair of arguments: req, res.
These are usually parameters that represent a request (like an http.IncomingMessage) and a response (like a http.ServerResponse).
Putting it all together, this module is probably defining a Controller that will receive http requests, and render them as responses. It currently does this for new, and there are probably routes configured elsewhere that call this method when a user requests something like 'http://server.come/user/new'.
Looks like basic JavaScript.
An object named module has a property named exports that is an object.
This object has a property named new whose value is an anonymous function.
In theory you could invoke the method like this:
module.exports.new(someRequest, someResponse);
I've just completed writing my first Node application. It's working well but as I read more from the community and absorb details I've come to the conclusion that my app is a bit different from the examples I'm reading.
So I dug deeper into finding solutions and reading a lot of howtonode.org. I stumbled on the concept of control-flow, promises and the Q-library. I'm starting to digest it all, I think I get it but I wanted to confirm my research and findings. Before I decide to restructure everything with Qlibrary I'm wondering if I've at least got my core app and methodology down pat, it will basically delegate how I move forward at this point.
app.js
// Require global libraries
var app = {
config : require('../../config.json'), // my custom config files
restify : require('restify'),
path : require('path'),
mongo : require('mongodb'),
model : require('./models.js'),
q : require('q'),
api : require('./api_util.js'),
underscore : require('underscore')
};
app.server.pre(function (request, response, next) {
request.app = app;
return next();
});
app.server = app.restify.createServer(app.config.api);
require('controller.js')(app);
app.server.listen(app.config.api.port);
controller.js
module.exports = function Controller(app, parcel, controller_name) {
app.server.get({
path : '/my/controller/',
version : '1.0.0',
name : 'My REST GET Controller'
}, function(request, response, next){
var model = new request.app.model();
// do some other stuff
));
}
I've taken the approach of encapsulating all my libraries and important components in the app variables and pass it into my controller so that I can call the references directly and only pass in one variable between each callback.
For every request after that I'm accessing the app variable from the request value that I attached to the pre in app.js (hence the request.app.model() function within the route callback function.
I'm wondering if I should be concerned about structuring my code this way. Its made it very easy to pass the reference back and forth without having to worry about global variables or any of that stuff. I read something slightly similar where someone hinted that this is a smart direction because it really helps with namespacing and getting around the whole global variable declaration issue (which is a big no-no).
If I was pushed in the right direction I think Q library and everything else will fall into place rather nicely.
Thanks!