I'm making a search-page on my server. When the endpoint is reached and the user waits for the search function to return the results and render the page Express falls through to the 404 handler instead, and I get the following error when I suppose the render function is called:
Error: Can't set headers after they are sent.
What am I doing wrong?
router.get("/", async (req, res) => {
try {
const queryString = req.query.q;
const user = helper.checkAndGetUser(req, res);
let s = String(queryString), searchedTags = [""];
if(s.indexOf(",") > -1){
searchedTags = s.replace(" ", "").split(",");
}
const options = {
"query": {tags: {$all: searchedTags}, _forSale: true}
};
const results = await Search.search(options).then(result => result).catch(err => {
throw err;
});
//This res.render -call is called after the 404 splat-route.
return res.render("partial/search.pug", {user: user, search: {
query: queryString,
results: results
}});
//If I'd use res.send for debugging, it is instead called before the splat-route, like the following:
return res.send(results);
} catch(err) {
next(err);
}
});
module.exports = router;
I register the router:
const search = require("./search.js");
app.use("/search", search);
Followed by the 404 splat-route:
app.get("*", async (req, res, next) => {
const user = helper.checkAndGetUser(req, res);
res.status(404);
res.render("partial/404.pug", {user: user});
});
To clarify:
My question is how can I make the res.render function get called just as the res.send function?
UPDATE [2017-10-05]:
I continued with another part of the site, a similar endpoint, and discovered that sending the result provided by the promise worked as expected if using res.send but not res.render. Using res.render the 404-handler kicked in again. Can this be a bug in Express?
This happens if you attempt to write to res after it is sent, so you must be calling additional code after res.render() or you already responded before calling that.
change it to return res.render(...) so it exits the functions, otherwise it will continue through the function and hit other res.render()s etc.
Something is up with that error handler also. I will update my post in a few mins with tips (on phone). It should probably have (req, res, next) and call return next(err) and pass it to your error handling middleware.
Here is the pattern I like to use in async/await Express:
// these routes occur in the order I show them
app.get('/route', async (req, res, next) => {
try {
const data = 'asdf'
const payload = await something(data)
.then((result) => createPayload(result))
// remember, if you throw anywhere in try block, it will send to catch block
// const something = willFail().catch((error) => {
// throw 'Custom error message:' + error.message
// })
// return from the route so nothing else is fired
return res.render('route', { payload })
} catch (e) {
// fire down to error middleware
return next(e)
}
})
// SPLAT
app.get('*', async (req, res, next) => {
// if no matching routes, return 404
return res.status(404).render('error/404')
})
// ERRORS
app.use(async (err, req, res, next) => {
// if err !== null, this middleware fires
// it has a 4th input param "err"
res.status(500).render('error/500')
// and do whatever else after...
throw err
})
Note: next() callback called without param is treated as no error, and proceeds to the next middleware. If anything is passed in, it will fire the error middleware with the param as the value of err in the error handling middleware. You can use this technique in routes and other middlewares, as long as the error middleware comes last. Mind your use of return with res.send/render() to prevent double setting headers.
NEW:
Something looks a little bit off with that .then() having a callback in it. I don't see logically where err would come from since the value of the resolved promise goes into the .then() function as result. At this point, it is suspect and should be removed or refactored if possible. This part here:
try {
let results = [];
await Search.search(options).then(result => {
results = result;
}, err => {
throw err;
});
console.log("res.render");
return res.render("partial/search.pug", {user: user, search: {
query: string,
results: results
}});
} catch(err) {
next(err);
}
First, here is about what I would expect to see with async/await syntax:
router.get("/", async (req, res, next) => {
try {
const queryString = req.query.q;
const user = helper.checkAndGetUser(req, res);
let s = String(queryString), searchedTags = [""];
if (s.indexOf(",") > -1) {
searchedTags = s.replace(" ", "").split(",");
}
const options = {
"query": { tags: { $all: searchedTags }, _forSale: true }
};
// If a promise is ever rejected inside a try block,
// it passes the error to the catch block.
// If you handle it properly there, you avoid unhandled promise rejections.
// Since, we have async in the route function, we can use await
// we assign the value of Search.search(options) to results.
// It will not proceed to the render statement
// until the entire promise chain is resolved.
// hence, then(data => { return data }) energizes `results`
const results = await Search.search(options)
.then(data => data)
// If any promise in this chain is rejected, this will fire
// and it will throw the error to the catch block
// and your catch block should pass it through to your
// error handling middleware
.catch(err => { throw 'Problem occurred in index route:' + err });
return res.render("partial/search.pug", {
user: user, search: {
query: string,
results: results
}
});
} catch (err) {
// look at the top how we added next as the 3rd, callback parameter
return next(err);
}
});
module.exports = router;
Error handler:
// notice how we add `err` as first parameter
app.use((err, req, res, next) => {
const user = helper.checkAndGetUser(req, res);
res.status(404);
res.render("partial/404.pug", {user: user});
});
From the Express docs:
Define error-handling middleware functions in the same way as other middleware functions, except error-handling functions have four arguments instead of three: (err, req, res, next). For example:
app.use(function (err, req, res, next) {
console.error(err.stack)
res.status(500).send('Something broke!')
})
http://expressjs.com/en/guide/error-handling.html
That might be your true issue because the error handler should only fire if next() is called with any input, but yours appears to be firing every time like a normal middleware, so I suspect it's because there is no err parameter on that middleware function, so it is treated as a normal one.
The Default Error Handler
Express comes with a built-in error handler, which takes care of any errors that might be encountered in the app. This default error-handling middleware function is added at the end of the middleware function stack.
If you pass an error to next() and you do not handle it in an error handler, it will be handled by the built-in error handler; the error will be written to the client with the stack trace. The stack trace is not included in the production environment.
If you call next() with an error after you have started writing the response (for example, if you encounter an error while streaming the response to the client) the Express default error handler closes the connection and fails the request.
So when you add a custom error handler, you will want to delegate to the default error handling mechanisms in Express, when the headers have already been sent to the client:
// code example in docs
Note that the default error handler can get triggered if you call next() with an error in your code more than once, even if custom error handling middleware is in place.
I also recommend using that splat route app.get('*', async (req, res, next) => {}) right above the error handler middlware (aka as the last loaded route in your list). This will catch all unmatched routes, such as /sih8df7h6so8d7f and forward the client to your 404. I think the error handler middlware is more suited for error 500 and clean formatted type errors because it gives you a function that can parse the value of next(err) anytime it is called from a route.
I usually do this for authentication failures with JSON web token (as the first line of code inside every auth required route):
if (!req.person) return res.status(403).render('error/403')
I realize some of this may fry your wig wholesale, so try all this stuff out and see each piece working before you determine if you would like to utilize it or not.
After a few days of going through the code again and again I stumbled upon a problem in the checkAndGetUser-function, that when run without the user being signed in, and since it's faster than the async call to the DB, triggered the splat endpoint and thus showed the 404 page.
I believe the reason for not triggering the splat endpoint when the res.render call was replaced by res.send was that the res.send function is that much faster than the render-call, since it doesn't have to parse any HTML.
Thank you #agm1984 for providing very useful information about the Express framework, if anyone else are having the same or a similar problem, make sure to read through his post.
Related
exports.checkTokenMW = async (req, res, next) => {
try{
const token = req.cookies["access"]
const authData = await this.verifyAccessToken(token);
console.log(authData, "checkingTokenmw")
res.json(authData)
}catch(err){
res.status(401);
}
next()
};
exports.verifyAccessToken = async (accessToken) => {
return new Promise((resolve, reject) => {
jwt.verify(accessToken, keys.accessTokenSecret, (err, authData) => {
if(err){
console.log(err)
return reject(createError.Unauthorized());
}
console.log(authData, "accesstoken")
return resolve(authData);
});
})
};
exports.verifyRefreshToken = async (refreshToken, res) => {
return new Promise((resolve, reject) =>{
//const refreshToken = req.body.refreshToken;
jwt.verify(refreshToken, keys.refreshTokenSecret, (err, authData) =>{
if (err) { reject(createError.Unauthorized()); return }
const userId = authData.userId
return resolve(userId)
})
})
};
exports.logOut = async (req, res) => {
try{
console.log("----------- logout")
const refreshToken = req.cookies["refresh"];
if(!refreshToken) {throw createError.BadRequest();}
const user = await this.verifyRefreshToken(refreshToken);
res.clearCookie("refresh")
res.clearCookie("access")
res.sendStatus(204);
res.redirect("/");
return res.send()
}catch(err){
console.log(err)
}
};
**The routes file has code something like this **
app.get('/api/logout', authService.checkTokenMW,authService.logOut)
I have been trying to tackle this error from a while not exactly sure what header is setting itself multiple times
**Here is the error **
**
> Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
> at ServerResponse.setHeader (_http_outgoing.js:561:11)
> at ServerResponse.header (E:\COURSE-WEBSITE\main-backend\wiz_backend\node_modules\express\lib\response.js:771:10)
> at ServerResponse.append (E:\COURSE-WEBSITE\main-backend\wiz_backend\node_modules\express\lib\response.js:732:15)
> at ServerResponse.res.cookie (E:\COURSE-WEBSITE\main-backend\wiz_backend\node_modules\express\lib\response.js:857:8)
> at ServerResponse.clearCookie (E:\COURSE-WEBSITE\main-backend\wiz_backend\node_modules\express\lib\response.js:804:15)
> at exports.logOut (E:\COURSE-WEBSITE\main-backend\wiz_backend\controllers\auth-controller.js:131:13)
> at processTicksAndRejections (internal/process/task_queues.js:95:5) {
> code: 'ERR_HTTP_HEADERS_SENT'
> }
**
The problem is inside the middleware function. Once you check for authData you don't need to send it back to response using res.json(authData). Because after that response will be sent and your next() function will be triggered anyways. Since next() will be called your route handler will try to also send another response which is the conflict you face. At the same time, in catch block, you need to have a return statement, so the function execution will be stopped, and it will not reach till next()
exports.checkTokenMW = async (req, res, next) => {
try{
const token = req.cookies["access"]
const authData = await this.verifyAccessToken(token);
console.log(authData, "checkingTokenmw")
// res.json(authData) // <- remove this line
}catch(err){
return res.status(401); // <- here
}
next()
};
The specific error you report is caused when your code tries to send more than one response to a given incoming request. You get to send one and only one response per request. So, all code paths, including error code paths have to be very careful to make sure that you are sending exactly one response.
Plus, if you have already sent a response, do not call next() because that will continue routing to other requests and eventually attempt to send some response (a 404 if no other handlers match).
With these in mind, you have several places your code needs fixing.
In your checkTokenWM:
exports.checkTokenMW = async (req, res, next) => {
try{
const token = req.cookies["access"]
const authData = await this.verifyAccessToken(token);
console.log(authData, "checkingTokenmw")
res.json(authData)
}catch(err){
res.status(401);
}
next()
};
You are calling res.json(authData) and then also calling next(). But, if the purpose of this is to be middleware that continues routing, then you should not be sending any response here if the token passes.
Then, in the catch block, you're setting a status, but not sending a response - that's not technically wrong, but probably not what you want. I'd suggest fixing both of those like this:
exports.checkTokenMW = async (req, res, next) => {
try{
const token = req.cookies["access"]
const authData = await this.verifyAccessToken(token);
console.log(authData, "checkingTokenmw");
// the access token verified so continue routing
next();
}catch(err){
// token did not verify, send error response
res.sendStatus(401);
}
};
In your logout:
exports.logOut = async (req, res) => {
try{
console.log("----------- logout")
const refreshToken = req.cookies["refresh"];
if(!refreshToken) {throw createError.BadRequest();}
const user = await this.verifyRefreshToken(refreshToken);
res.clearCookie("refresh")
res.clearCookie("access")
res.sendStatus(204);
res.redirect("/");
return res.send()
}catch(err){
console.log(err)
}
};
You are attempting to send three responses upon successful logout and no responses upon error.
First off, res.sendStatus(204) sets the status AND sends an empty response. res.status(204) would just set the status for a future call that actually sends the response if that's what you meant to do. But, with that status, you can't then do res.redirect().
It's not entirely clear what you're trying to do here. If you want to redirect, then that needs to be a 3xx status so you can't use 204. I'm going to assume you want to do a redirect.
I'd suggest fixing by changing to this:
exports.logOut = async (req, res) => {
try{
console.log("----------- logout")
const refreshToken = req.cookies["refresh"];
if(!refreshToken) {throw createError.BadRequest();}
const user = await this.verifyRefreshToken(refreshToken);
res.clearCookie("refresh");
res.clearCookie("access");
res.redirect("/");
}catch(err){
// can't call logout, if you weren't logged in
res.sendStatus(401);
}
};
FYI, most apps won't make it an error to call logout if you weren't logged in. They would just clear the cookies and redirect to home either way as it's really no issue whether they were previously logged in or not. The problem with doing it your way is if the cookies somehow get corrupted, then your code doesn't let the user attempt to clear things up by logging out and logging back in.
So, I'd probably just skip the token check entirely:
exports.logOut = async (req, res) => {
res.clearCookie("refresh");
res.clearCookie("access");
res.redirect("/");
};
And, also I'd change this:
app.get('/api/logout', authService.checkTokenMW,authService.logOut)
to this:
app.get('/api/logout', authService.logOut);
i'm working on node.js using Express to built a backend.
i'm intended to handle status 500 error that may happened.
router.put('/test', async (req, res) => {
try {
return res.send(await request.updateTest(req.body, 1))
} catch(err) {
console.log(err)
return res.status(500).send(err.stack)
}
})
this is my example of the code. it's do work perfectly. but when i'm try to make unknown error from the database query, i want to log the error and return status 500 as response with the error detail.
but i'll need to add try and catch every time i'm build a new controller/routes
is there anyway i could express them in form of middleware instead of write try and catch everytime?
this is an example of code i've try to make it as middleware but it's has no work and no effect when called.
error.js
module.exports = function (err, req, res, next) {
console.log(err)
res.status(500).send({
error: 'Internal Server Error',
message: err.stack
})
next(err)
}
main.js
const errorHandler = require('./error')
const { corsOption } = require('./cors')
const cors = require('cors')
const test = require('./test')
module.exports = function (app) {
app.use(cors(corsOption))
app.use(errorHandler)
app.use('/api/test', test)
}
is there anyway that i can do for this to work?
Your global error handler should be placed after all other middlewares/routes:
app.use(middleware)
// all other middlewares
app.use('/api/test', test)
// all other routes
// error handler
app.use(function (err, req, res, next) {
res.status(500).json({
error: err.message,
});
});
To avoid adding try/catch to everything, better to wrap your route handler to catch the errors (asyncWrapper):
app.use(middleware)
// all other middlewares
const asyncWrapper = (cb) => {
return (req, res, next) => cb(req, res, next).catch(next);
};
const test = async (req, res) => {
return res.send(await request.updateTest(req.body, 1))
}
// wrap your handler to catch errors (Async functions return a promise)
app.use('/api/test', asyncWrapper(test))
// all other routes
// error handler
app.use(function (err, req, res, next) {
res.status(500).json({
error: err.message,
});
});
There are two approaches to resolve unhandled exceptions in Node.js
Using try-catch blockwhich is already you are using
Using Process i.e use Process to handle exception. A process is a global object that provides information about the current Node.js process. The process is a listener function that is always listening to the events. The most effective and efficient approach is to use Process. If any uncaught or unhandled exception occurs in your code flow, that exception will be caught in code
process.on('uncaughtException', function(err) {
// Handle the error safely
console.log(err)
})
The above code will be able to handle any sort of unhandled exception which occurs in Node.js. see this Process Events
I have set up api end points like the following
// doesn't require to go through auth middleware
app.get('/api/list', asyncErrorHandler(async (req, res) => {
// stuffs here
if (somethingWrong)
throw new Error("Something is wrong for sure")
else
res.status(200).send("Only List for you okay?")
}, false))
// requires to go through auth middleware
app.get('/api/show', asyncErrorHandler(async (req, res) => {
// stuffs here
if (somethingWrong)
throw new Error("Something is wrong for sure")
else
res.status(200).send("Here you go my dude! wink")
}))
// auth function
const auth = async (req, res, next) => {
//stuffs here
if(notProperUser) throw new Error("Bruh! You not supposed to be here!")
else next()
}
How do I set up my asyncErrorHandler so that the following things occur in the block?
const asyncErrorHandler = (fn, authParam = true) => (req, res, next) => {
// if authPram = true
// promise resolve auth then = > fn = > if error is thrown,
// catch error and break promise chain
// log e and return res.status(e.status).send("error")
// (assume we have a custom error with status)
// else
// same as if block but skip auth and only execute fn callback function
// need to be executed
}
Cannot use the following
app.get('/api/show', auth, asyncErrorHandler(async (req, res) => {
// stuffs here
if (somethingWrong)
throw new Error("Something is wrong for sure")
else
res.status(200).send("Here you go my dude! wink")
}))
Because error thrown at auth layer won't be caught by asyncErrorHandler. Error at auth needs to get caught by asyncErrorHandler as there needs to be central error handling of all layers.
I want to know how we can accommodate the authParam flag and extra auth middleware with something like the following
const asyncErrorHandler = (fn, authParam = true) => (req, res, next) => {
// following code is incomplete - MISSING
// authParam check and based on that
// resolve auth and fn TWO promises sequentially
// if auth throws error fn should NOT execute
// and error at auth needs to be caught
// then sent to the client
Promise.resolve(fn(req, res, next)).catch((e) => {
res.status(e.status).send("Error")
})
}
How can I do something like above that will check authParam and based upon that resolve auth and fn accordingly? What changes would the above code require?
I know the difference between a callback and a middleware next() function.
If a write a custom remote-method in Loopback, I can easily send errors in callback e.g callback(error,null) but in remote hooks or observers whenever I send error in next() function e.g
var err = new Error('This is error');
next(err)
it always says that Internal server error but it does not tell me what error is.
In order to view error I have to login to server and view logs.
Please tell me how can I send error as a response in next() function so that the on frontend I know what error has occurred.
Maybe use a middleware to hook in:
app.use( function(err,req,res){
res.json(err);
});
(This needs to he the last middleware defined...)
basically you can define callbacks with err and result.
For example in loopback,
if I have a model call "Action" you can simply send err or result to front end using json.
app.get('/your/api/call', function (req, res, next) {
var getTeam = function (cb) {
app.models.Team.find({}, function (err, teams) {
if (err) {
cb(err);
} else {
cb(null, teams);
}
});
};
async.waterfall([
getTeam
], function (err, team, role) {
if (err){
res.send(err); //send error to front end
} else {
res.send(team); //send result to front end
}
});
});
This approach can use with "app.use" function in root level also.
I am using Express.js as http server. Defined all my routes.
Most endpoints need to verify session before returning a response. E.g. below code serves users in the system and list of services respectively:
function getUsers(req, res, next) {
verifyUser(req, res, next, function () {
//serve users
});
}
function getServices(req, res, next) {
verifyUser(req, res, next, function () {
//serve services
});
}
You probably noticed there is a verifyUser function which validates the session. Which is as below.
function verifyUser(req, res, next, callback) {
var sessionKey = req.cookies.sessionKey;
var user = users.userBySession(sessionKey);
if (user) {
callback(req, res, next, user);
} else {
res.status(401).send({
message: 'Unauthorized'
});
}
}
As you can see I keep passing in req, res and next parameters along with a callback whenever I use this function.
I tried to use apply function to make it easier. Changed my getUsers function like this:
function getUsers(req, res, next) {
verifyUser
.apply(null, arguments, function () {
//serve users
});
}
The problem with this approach is callback is not passed into verifyUser function. And I don't really like passing null as scope with each call.
How can I achieve this by writing less and better code ? Any ideas?
You could use bind to create a 'partial function':
// create bound responseHelper object
var responseHelper = verifyUser.bind(null, req, res, next);
// usage
responseHelper(getUsersCallback); // same as verifyUser(req, res, next, getusersCallBack);
I think you're looking to turn verifyUser into a middleware function.
function verifyUser (req, res, next) {
var user = // yadda yadda session stuff
if (user) {
req.user = user; // [1] what you do to the req object here...
} else {
return res.status(401).send({ message: "No way Smokey Joe"});
/**
* alternatively, do something like
* var err = new Error("Not authorized");
* err.statusCode = 401;
* return next(err);
*
* this will kick off Express' error handling mechanism,
* which you should read about in the docs (see the link below)
*/
}
next();
// very important to call next after this verifyUser has done its job
// if you don't, the next middleware won't go off,
// and the request will just hang
}
function getUsers (req, res, next) {
// [2] will show up on the req object here, assuming you chain these
// two functions together as middleware
}
app.get("/users", verifyUser, getUsers);
app.get("/services", verifyUser, getServices);
// here's a route that needs no session auth, so no need to verifyUser
app.get("/latest-posts", getLatestPosts);
When you tell Express to use a function or attach a function to a route path via get('/my/route', hanlderFun) or some such, you've basically turned handlerFun into a middleware.
You can define however many middleware as handlers on a route as you like, and they'll all execute in turn as long as you keep calling next.
app.post("/checkout", verifyUser, tallyCart, checkInventory, doPayment, sendInvoice);
The job of next is to pass control from the current middelware to the next one. It's an object
You can do other stuff with next, too, which you should read up on in the docs.
http://expressjs.com/en/guide/writing-middleware.html
http://expressjs.com/en/guide/using-middleware.html
The docs on routing have good info on middleware as well:
http://expressjs.com/en/guide/routing.html
For extra credit, check out error handling middleware, too:
http://expressjs.com/en/guide/error-handling.html