I have a node web server with several api endpoints. One of those endpoints needs to make a request to another server in order to assemble its own response. I've seen a few different answers with links to libraries that allow purely synchronous web requests, but they're all associated with warnings that say not to use them in production.
Here is an example of what my server request handler looks like:
app.get('/my_api_endpoint', function (req, res) {
// Need to get data from other webservice using data
// from this request.
data = request_to_other_web_service(req.params.some_value);
res.status(200);
res.send("This is the " + data);
res.end();
});
Is there a standard pattern for handling requests like this?
The reason why you are getting errors/warnings when trying to synchronously call an HTTP request is that from the get-go node.js was built with the core philosophy of callbacks & the event loop. If you are trying to write asynchronous code in a synchronous way... May i point you in the direction of async/await. You could you use npm install node-fetch
const fetch = require('node-fetch');
app.get('/my_api_endpoint', async function (req, res) {
// Need to get data from other webservice using data
// from this request.
data = await fetch("http://...");
res.status(200);
res.send("This is the " + data);
res.end();
});
In general all javascript libraries/frameworks are friendly to non-synchronous operations. Those that are not will be ignored by the community. Express already allow you to perform asynchronous operations.
Have you ever wondered why express requires you to "return" responses by calling methods on res instead of simply returning?:
// frameworks in some other languages:
#Path("/my_api_endpoint")
Response myEndpoint() {
data = request_to_other_web_service(some_value);
return data; // Javascript frameworks don't use returns!!
}
It is because all javascript web frameworks are async friendly. The reason you use a callback (a function that you call back at some later time) to return from a route is because you can call functions inside other functions. In Express you call res.send() after you've performed async operations. How you do it depends on the API you're using:
API that accepts a callback:
app.get('/my_api_endpoint', function (req, res, next) {
request(req.params.some_value, function (error, data) {
if (!error) {
res.status(200);
res.send("This is the " + data);
}
else {
next(error);
}
}
});
API that returns a promise:
app.get('/my_api_endpoint', function (req, res, next) {
request(req.params.some_value)
.then(function (data) {
res.status(200);
res.send("This is the " + data);
})
.catch(function (error) {
next(error);
});
});
Using async/await with API that returns a promise:
app.get('/my_api_endpoint', async function (req, res, next) {
try {
let data = await request(req.params.some_value)
res.status(200);
res.send("This is the " + data);
}
catch (error) {
next(error);
}
});
Stream based API:
app.get('/my_api_endpoint', async function (req, res, next) {
let dataStream = request(req.params.some_value);
let buffer = "";
dataStream.on('data', function (data) {
buffer += data.toString('utf8');
});
dataStream.on('end', function () {
res.status(200);
res.send("This is the " + buffer);
});
dataStream.on('error', function () {
next("Stream error");
});
});
Express can handle any asynchronous program flow.
Related
Where should i put async and await?
result is
work2
work1
app.post('/upload', (req, res) => {
const txtupload = multer().any()
let validate = true
txtupload(req,res, (err)=>{
console.log('work1')
validate = false
})
if(validate){
console.log('work2')
//code
}
});
The upload function from multer doesn't return a promise, so async/await isn't applicable. Instead, issue your response in the callback.
Your real question seems to be: How do I use multer to handle file uploads? For that we go to the multer documentation. Adapting that documentation to your code:
app.post('/upload', (req, res) => {
const txtupload = multer().any();
txtupload(req,res, (err) => {
if (err) {
// ...it failed, send a failure response via `res`...
} else {
// ...it worked, send a success response via `res`...
}
});
// Don't do anything here, the upload hasn't been processed yet
});
But refer to the examples in the documentation, there are other patterns for using multer.
Since it's an arrow function, you have to put it before the parameters:
app.post('/upload', async (req, res) => { ... }
I have a general question on how you handle services and routes in node.js. Would you handle the response directly in the service or would you leave that to the route? Here's what i mean in code
Like this
Route
router.get('/', (req, res, next) ==> {
someService.someMethod(req, res);
});
Service
const someMethod = (req, res) => {
try {
var something = await someOtherMethod(req.body.someParameter);
return res.status(200).send(something.data);
} catch (err) {
return res.status(500).send({msg: err.message});
}
}
Or this
Router
router.get('/', (req, res, next) ==> {
try {
var something = await someService.someMethod(req.body.someParameter);
res.status(200).send(something.data);
} catch (err) {
res.status(500).send({msg: err.message})
}
});
Service
const SomeMethod = (Input) => {
return someOtherMethod(Input);
}
The first way would make the routers much simpler and cleaner especially if the use the service in multiple routes, but on the downside I always need to supply the res and req and I will run into problems if I want to use the service internally. I'm tending to the second method.
How do you design your services?
I would go for router.get('/', RootController)
const RootController = (req, res) => {
// extract what you need from the request
const param = req.body.param;
// calculate what you need in a pure function `businessLogic`
const result = businessLogic(param);
// send the response
return res.send(result);
}
This way you get a separation of concerns - your root controller is responsible only for handling / requests - getting a response for a request. All "business logic" is done in a pure function (you can easily test it without any HTTP request contexts/mocks, it can be reused somewhere else, for example in different controller).
I use the following architecture:
1. Route
2. Controller
3. Services
Your route is the one validating the input, your controller is the one handling all the logics and calling the services and returning the final result to your route.
As stated, multiple callback functions can be provided and behave like middleware to handle a request. They can be in the form of a function, an array of functions, or combinations of both, as shown in the following examples.
For example:
app.get('/example/b', function (req, res, next) {
console.log('the response will be sent by the next function ...')
next()
}, function (req, res) {
res.send('Hello from B!')
})
What's the purpose of this? Can't we simply use:
app.get('/example/b', function (req, res) {
console.log('the response will be sent by the next function ...')
res.send('Hello from B!')
})
The multiple functions would more likely be used when you already have a previously defined function that you probably intend to use in multiple places. For example:
app.get("/somePath", checkAuth, function(req, res) {
// you know it's already authenticated here
});
app.get("/someOtherPath", checkAuth, function(req, res) {
// you know it's already authenticated here
});
function checkAuth(req, res, next) {
if (some logic here) {
// allow handler chain to continue
next();
} else {
// auth error
res.status(401).end();
}
}
Of course, you could also use middleware for checking authentication, but the above example allows you to target just a few specific routes with some middleware that you may use in multiple places.
As you have already observed, if you don't intend to use the function anywhere else, then you may as well just put the logic into your one handler.
Yes you can, the purpose is for example, to handle errors, the middleware sequence in express allows you to use this way. For example, see this way to set up the express config:
app.use(logger.connectLogger());
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({
extended: false
}));
app.use(routes);
app.use(errorConnect);
http.createServer(app).listen(config.port, function () {
logger.getLogger().info('My backend listening on port: ' + config.port);
});
My routes module have all the matching route -> callback:
// Methods exposed for backend API.
router.get('/status', ips.getStatus);
router.route('/ip')
.post(ips.keepIps)
.get(ips.getIps)
// NOT ALLOWED
.put(returnNotAllowed)
.delete(returnNotAllowed);
// Methods exposed and used by IP's frontend.
router.route('/front/ip')
.get(front.getIpsByFront)
.post(front.keepIpsByFront);
router.post('/login', login.login);
....
For example in one of those callbacks, I have the next way to manage an incoming request:
/**
* Login user from frontend.
*/
exports.login = function(req, res, next) {
var username = req.body.username + '#System',
password = req.body.password,
server = req.body.server,
auth = 'Basic ' + new Buffer(username + ':' + password).toString('base64');
loginApi.login(auth, server)
.then(function(result) {
res.statusCode = 200;
res.send(result);
})
.catch(function(error) {
next({
statusCode: 403,
message: 'Error in login'
});
});
};
When I catch an error, I call next with a custom error object, and after this, if you back and watch the config (first paragraph) you can see that I added to the express middleware an error manage with errorConnect. In my opinion this is a usefull way to understand what you are asking because if I understand well you had doubts with next()
What is the best practice to send two independed MongoDB results in Express application via HTTP Method?
Here is a short example which makes it clear:
//app.js
var express = require('express');
var app = express();
var testController = require('./controllers/test');
app.get('/test', testController.getCounts);
...
Following getCounts() function wouldn't work because I can't send the response twice.
///controllers/test
exports.getCounts = function(req,res) {
Object1.count({},function(err,count){
res.send({count:count});
});
Object2.count({},function(err,count){
res.send({count:count});
});
};
Anyway, I would like to have those two counts in one response object.
Should I call Object2.count in the callback of Object1 even if they are not dependent to each other?
Or should I re-design it somehow else?
Thank you!
You should use Promise to achieve this task :
function getCount(obj) {
return new Promise(function (resolve, reject) {
obj.count({}, function(err,count) {
if(err) reject();
else resolve(count);
});
});
}
With Promise.all you can trigger the two request and retrieve the results in order to add it to the response
exports.getCounts = function(req,res) {
Promise.all([getCount(Object1), getCount(Object2)])
.then(function success(result) {
res.send({'count1':result[0], 'count2':result[1]});
});
});
When you call res.send you will end the response for the request. You could instead use res.write, which will send a chunk to the client, and when done call res.end;
Example:
app.get('/endpoint', function(req, res) {
res.write('Hello');
res.write('World');
res.end();
});
However, it seems like you are trying to send json back to the client which raises and problem: writing to object separately will not be valid json.
Example:
app.get('/endpoint', function(req, res) {
res.write({foo:'bar'});
res.write({hello:'world'});
res.end();
});
The response body will now be: {foo:'bar'}{hello:'world'} which is not valid json.
There will also be a race condition between the two db queries, which means that you are not certain about the order of the data in the response.
Suggestion:
exports.getCounts = function(req,res) {
var output = {};
Object1.count({},function(err,count){
output.count1 = count;
Object2.count({},function(err,count){
output.count2 = count;
res.send(output);
});
});
};
//Response body
{
count1: [value],
count2: [value]
}
var database = require('database');
var express = require('express');
var app = express();
var cors = require('cors');
app.use(cors());
var bodyParser = require('body-parser');
var urlencodedParser = bodyParser.urlencoded({
extended: false
});
app.post('/dosomething', urlencodedParser, function(req, res) {
if (!req.body.a) {
res.status(500).send(JSON.stringify({
error: 'a not defined'
}));
return;
}
firstAsyncFunction(req.body.a, function(err, result) {
if (err) {
res.status(500).send('firstAsyncFunction was NOT a success!');
} else {
if (result.b) {
secondAsyncFunction(result.b, function(err, data) {
if (err) {
res.status(500).send('secondAsyncFunction was NOT a success!');
return;
}
res.send('EVERYTHING WAS A SUCCESS! ' + data);
});
}
else {
res.status(500).send('result.b is not defined');
}
}
});
});
function firstAsyncFunction(param, callback) {
//Some network call:
// Return either return (callback(null,'success')); or return (callback('error'));
var query = database.createQuery(someOptionsHere);
database.runDatabaseQuery(query, function(err, entities, info) {
if (err) {
return (callback('error'));
}
return (callback(null, 'success'));
});
};
function secondAsyncFunction(param, callback) {
//Some network call:
// Return either return (callback(null,'success')); or return (callback('error'));
var query = database.createQuery(someOptionsHere);
database.runDatabaseQuery(query, function(err, entities, info) {
if (err) {
return (callback('error'));
}
return (callback(null, 'success'));
});
};
var server = app.listen(process.env.PORT || 3000, function() {
var host = server.address().address;
var port = server.address().port;
console.log('App listening at http://%s:%s', host, port);
});
module.exports = app;
I have here a basic express http server. This server has one route, dosomething, which makes two network calls and tells the user if they were a success or not.
This is my entire webserver (this is a bare bones server of my actual server for example purposes). I am now concerned with this server crashing. Reading the docs for express I see there is a default error handler which will catch errors and prevent the server from crashing (http://expressjs.com/en/guide/error-handling.html). I have added the code:
function defaultErrorHandler(err, req, res, next) {
if (res.headersSent) {
return next(err);
}
res.status(500);
res.render('error', { error: err });
}
app.use(defaultErrorHandler);
This still crashes my server though. For example. I had a problem with my database returning an improper JSON response and inside of my firstAsyncFunction (not shown in the code) I tried to parse the JSON and it caused an error telling me it was improper JSON and the server crashed and was unable to take requests anymore until I restarted it. I would like to avoid this and have the default error handler send out a generic response back to the user when this occurs. I thought if I specified the defaultErrorHandler and put it inside of app.use that it would capture and handle all errors, but this does not seem to be the case? Inside of my async function for example you can see I am looking if an error was returned and if it was I send an error back to the user, but what if some other error occurs, how can I get express to capture and handle this error for me?
The defaultErrorHandler cannot handle exceptions that are thrown inside asynchronous tasks, such as callbacks.
If you define a route like:
app.get('/a', function(req, res) {
throw new Error('Test');
});
An error will be thrown, and in this case defaultErrorHandler will successfully catch it.
If the same exception occurs in an async manner, like so:
app.get('/a', function(req, res) {
setTimeout(function () {
throw new Error('Test');
}, 1000);
});
The server will crush, because the callback is actually in another context, and exceptions thrown by it will now be caught by the original catcher. This is a very difficult issue to deal with when it comes to callback.
There is more than one solution though. A possible solution will be to wrap every function that is prone to throw error with a try catch statement. This is a bit excessive though.
For example:
app.get('/a', function(req, res) {
setTimeout(function () {
try {
var x = JSON.parse('{');
}
catch (err) {
res.send(err.message);
}
}, 1000);
});
A nicer solution:
A nicer solution, would be to use promises instead, if it's possible, then for example you can declare a single errorHandler function like so:
function errorHandler(error, res) {
res.send(error.message);
}
Then, let's say you have to following function with fetches stuff from the database (I used setTimeout to simulate async behavior):
function getStuffFromDb() {
return new Promise(function (resolve, reject) {
setTimeout(function () {
resolve("{");
}, 100);
});
}
Notice that this function returns an invalid JSON string. Your route will look something like:
app.get('/a', function(req, res) {
getStuffFromDb()
.then(handleStuffFromDb)
.catch(function (error) { errorHandler(error, res) });
});
function handleStuffFromDb(str) {
return JSON.parse(str);
}
This is a very simplified example, but you can add a lot more functionality to it, and (at least theoretically) have a single catch statement which will prevent your server from crushing.