Im' trying to understand using promises with Google Cloud Functions a bit better. I just learned about the 'finally' method on promises, which is called after all promises in the chain are fully resolved or rejected. In a http function is it good practice to put response.send() inside of the finally method?
The below code uses request-promise-native for the http request. In the first .then() I call parseSchedule, which uses the cheerio web scraping api to loop through some data and on a website, and add it to the scheduledGames array (synchronously, I think).
I return from that and the then log that data to the console in writeDB, but one thing I noticed is that I see response.send() log 'execution finished' before I see the data from scheduleGames in the log. Is that correct?
Should I be using the 'finally' block like this?
Thanks,
const options = {
uri: 'https://www.cbssports.com/nba/schedule/' + urlDate,
Connection: 'keep-alive',
transform: function (body) {
return cheerio.load(body);
}
};
return request(options)
.then(parseSchedule)
.then(writeSchedule)
.catch((err) => console.log("there was an error: " + err))
.finally(res.send("execution finished"));
function parseSchedule($){
const scheduledGames = [];
$('tbody').children('tr').each((i, element) => {
const gameTime = $(element).children('td').eq(2).find('a').text()
const scheduledGame = { gameTime: gameTime};
scheduledGames.push(scheduledGame);
});
return scheduledGames;
}
function writeDB(scheduledGames){
console.log(scheduledGames);
}
}
It typically makes more sense to send a success response at the time in the promise chain when everything is successful, or send an error response in a catch handler. If you do these two things, it doesn't make sense to use finally at all, since success and error are the only two cases you really need to handle. Unless you have some special case, stick to just success and error.
I am having an issue where Intellij is warning me about 'throw' of exception caught locally. After doing some digging on why this is not ok it makes sense, errors should not be used for flow control. The problem I am facing though is in async I cannot reject the promise without throw something locally and I get the warning.
Some of my example code.
Top level takes in the request and awaits for the response from the controller:
Router.post("/", async (req, res) => {
try {
let itemController = new ItemController(req.body);
let response = await itemController.request();
res.send(response);
} catch (error) {
res.status(500).send({ error: error});
}
});
The controller takes in the request and awaits on other functions to get some data.
async request() {
try {
await isValidItem();
return await this.initialize();
} catch(error) {
throw error;
}
}
Then I have a function which gets manufacturer ID for this item and I run into my problem. If the SQL query doesn't fail and nothing is in the response I need to throw a local error so that the request can fail gracefully. and send a proper 500 error to the client.
async queryManufacturerID() {
try {
let result = await this.queryManufacturerID(this.itemID, this.brand);
if (result === false) {
throw new Error("this item has no manufacturer ID");
} else {
this.manufacturerID = result["manufacturerItemID"];
}
} catch (error) {
throw error;
}
}
My problem is I know I can adjust this so other functions that get a reply from this can know that this function failed without a error but that would have to happen in this entire chain to prevent throwing locally. Seems like a lot of bloat.
The only thing that makes this code bloaty and the IDE complain is not throwing errors, but this:
try {
//...
} catch (error) {
throw error;
}
It's a no op. You can safely remove it without changing the logic.
The only case where you should use catch is when you actually plan to handle the error and get the execution back on track.
After doing some digging on why this is not ok it makes sense, errors should not be used for flow control
I disagree. Errors are a part of proper flow control, they allow you to handle unplanned things. Throwing an error if something unexpected occured makes sense, also in this case.
i am trying to save something to a database using mongoose. The thing is i need to make sure the save was completed before i move on in the program and close the connection. Knowing that save is async in mongoose i tried using this code:
saveFunction = function(song){
song.save(function(err, userObj){
if(err){
console.log('Error' + err);
} else{
console.log('saved successfully:', userObj);
}
});
};
database.prototype.add= function(newSong){
mongoose.connect(url);
var song = new songModel({id : newSong.getId(),
title : newSong.getTitle(),
artist : newSong.getArtist,
genre : newSong.getGenre(),
rating : newSong.getRating(),
link : newSong.getLink()});
console.log("before async");
async.parallel([function (callback){
saveFunction(song);
callback();
}],function(){
mongoose.connection.close();
console.log('closed connection');
});
console.log("after async");
nextFreeId++;
};
^songModel is defined globally.
I tried lots of different methods and changed lots of thing but i always get and error of somekind. With this code i get a process.nexttick(function() throw err ) error. I just can't get it to work. Can someone tell me whats wrong or provide me with working code?
I think optimally the console should look like this:
before async
saved successfully
closed connection
after async
Thanks!
EDIT: Open to other alternatives to async aswell. I just want to get this code to work any way possible. I just need to save/find something/remove something and it needs to wait with the rest of the execution of the program until the save/find/removal is done. I'm getting pretty desperate, lost nearly a day on this problem alone on a tight shedule :(
You need to return a callback from your save function.
saveFunction = function(song,callback){
song.save(function(err, userObj){
if(err){
console.log('Error' + err);
return callback(true,err)
} else{
console.log('saved successfully:', userObj);
return callback(null);
}
});
};
Edit
From your comment, the behavior you are expecting will never occur. You are expecting
console.log("before async");
async.parallel -> do your bits
console.log('closed connection');
console.log("after async");
However this will never happen because async.parallel is an asynchronous call, which means that the execution does not wait for it to finish before moving onto the next command. The behavior you are seeing is
console.log("before async");
async.parallel -> starts
console.log("after async");
async.parallel -> console.log('closed connection');
Node is doing the first log, starting the async.parallel, then console.logging "after async". Then when async.parallel gets to its callback function, it prints "closed connection", so it appears after "after async" because it was executed after.
Any logic you want to perform that relies on the result of async.parallel must happen in the callback function. Furthermore, async.parallel is used when you want to run 2 or more functions asynchronously, then execute a callback once they are all finished. Your solution does not require async.parallel. You can replace that with:
saveFunction(song,function(err){
if(err){
//failed to save song
}
mongoose.connection.close(); //you do not need to do this anyway
console.log('closed connection');
nextFreeId++;
//anything else you need to do here
});
In a Parse custom webhook, which is of the form:
app.post('/receiveSMS', function(req, res) {
Where receiveSMS is hooked up to the Twilio api and this method is properly called (I have logs to prove it), but I'm trying to query on my tables within this method and it doesn't seem to be working.
Is this allowed, or is there anything special I need to do to make this work?
var contactObj = Parse.Object.extend("Contact");
var contactQuery = new Parse.Query(contactObj);
console.log(req.body.From);
contactQuery.each(function(contact) {
and the body of the each call never gets called.
Is this allowed, and if so, what am I doing wrong here?
Update -- The entirety of the webhook code block is:
app.post('/receiveSMS', function(req, res) {
console.log('receive SMS');
console.log(req.body.Body);
res.send('Success');
if(req.body.Body.toLowerCase() == "in" || req.body.Body.toLowerCase() == "out") {
twilio.sendSMS({
From: "(xxx) xxx-xxxx",
To: req.body.From,
Body: "It's been noted, and notifications have been sent. Check us out!"
}, {
success: function(httpResponse) {
console.log(httpResponse);
response.success("SMS Sent!");
},
error: function(httpResponse) {
console.error(httpResponse);
response.error("Uh OH, something went wrong");
}
});
if(req.body.Body.toLowerCase() == "in") {
console.log("in was received");
// eventQuery
var contactObj = Parse.Object.extend("Contact");
var contactQuery = new Parse.Query(contactObj);
console.log(req.body.From);
// contactQuery.equalTo("phone", req.body.From);
contactQuery.first({
success: function(contact) {
console.log("found contact");
console.log(contact);
}, error: function(error) {
alert("Error: " + error.code + " " + error.message);
}
});
}
}
});
This code is called and the logs "console.log('receive SMS')" and the like are all called, except for what is inside the query's first call.
Queries on tables is fine, but you can't use the each() function, as that is restricted to only work in background jobs.
You'll have to use find() or first() or get() depending on your needs.
UPDATE
OK, after seeing your full code I have some ideas as to why it isn't working. First off you're sending res.send("Success"); before you're finished, I'm not positive but I think this causes it to stop running the rest of your code (haven't checked, could be wrong).
Also you're doing multiple async operations without chaining them so the contactQuery.first() will run before the twilio.sendSMS() is finished.
Inside twilio.sendSMS() you're calling response.success() / response.error(). These are for cloud methods, not web hooks, so I expect these would be throwing errors server-side (check the logs on the Dashboard).
Inside contactQuery.first() you are using alert() which isn't supported in cloud code.
I'm not sure if those mistakes will be caught early and throw errors or if they'll raise run-time exceptions, but they should be fixed, your code re-deployed and try again. Then report any errors in the server logs.
Yes, it's allowed, I'm using the same web hooks.
My guess is that you probably have defined security restriction on your Contact class that prevent the query to fetch anything. What's the security setting on this class ?
You can either try to relax the constrains, or login as a dummy user, and execute the query (approach that I chose).
cheers
-A
I'm new to node.js although I'm pretty familiar with JavaScript in general. My question is regarding "best practices" on how to handle errors in node.js.
Normally when programming web servers, FastCGI servers or web pages in various languages I'm using Exceptions with blocking handlers in a multi-threading environment. When a request comes in I usually do something like this:
function handleRequest(request, response) {
try {
if (request.url=="whatever")
handleWhateverRequest(request, response);
else
throw new Error("404 not found");
} catch (e) {
response.writeHead(500, {'Content-Type': 'text/plain'});
response.end("Server error: "+e.message);
}
}
function handleWhateverRequest(request, response) {
if (something)
throw new Error("something bad happened");
Response.end("OK");
}
This way I can always handle internal errors and send a valid response to the user.
I understand that with node.js one is supposed to do non-blocking calls which obviously leads to various number of callbacks, like in this example:
var sys = require('sys'),
fs = require('fs');
require("http").createServer(handleRequest).listen(8124);
function handleRequest(request, response) {
fs.open("/proc/cpuinfo", "r",
function(error, fd) {
if (error)
throw new Error("fs.open error: "+error.message);
console.log("File open.");
var buffer = new require('buffer').Buffer(10);
fs.read(fd, buffer, 0, 10, null,
function(error, bytesRead, buffer) {
buffer.dontTryThisAtHome(); // causes exception
response.end(buffer);
}); //fs.read
}); //fs.open
}
This example will kill the server completely because exceptions aren't being catched.
My problem is here that I can't use a single try/catch anymore and thus can't generally catch any error that may be raised during the handling of the request.
Of course I could add a try/catch in each callback but I don't like that approach because then it's up to the programmer that he doesn't forget a try/catch. For a complex server with lots of different and complex handlers this isn't acceptable.
I could use a global exception handler (preventing the complete server crash) but then I can't send a response to the user since I don't know which request lead to the exception. This also means that the request remains unhandled/open and the browser is waiting forever for a response.
Does someone have a good, rock solid solution?
Node 0.8 introduces a new concept called "Domains". They are very roughly analogousness to AppDomains in .net and provide a way of encapsulating a group of IO operations. They basically allow you to wrap your request processing calls in a context specific group. If this group throws any uncaught exceptions then they can be handled and dealt with in a manner which gives you access to all the scope and context specific information you require in order to successfully recover from the error (if possible).
This feature is new and has only just been introduced, so use with caution, but from what I can tell it has been specifically introduced to deal with the problem which the OP is trying to tackle.
Documentation can be found at: http://nodejs.org/api/domain.html
Checkout the uncaughtException handler in node.js. It captures the thrown errors that bubble up to the event loop.
http://nodejs.org/docs/v0.4.7/api/process.html#event_uncaughtException_
But not throwing errors is always a better solution. You could just do a return res.end('Unabled to load file xxx');
This is one of the problems with Node right now. It's practically impossible to track down which request caused an error to be thrown inside a callback.
You're going to have to handle your errors within the callbacks themselves (where you still have a reference to the request and response objects), if possible. The uncaughtException handler will stop the node process from exiting, but the request that caused the exception in the first place will just hang there from the user point of view.
Very good question. I'm dealing with the same problem now. Probably the best way, would be to use uncaughtException. The reference to respone and request objects is not the problem, because you can wrap them into your exception object, that is passed to uncaughtException event. Something like this:
var HttpException = function (request, response, message, code) {
this.request = request;
this.response = response;
this.message = message;
this.code = code || 500;
}
Throw it:
throw new HttpException(request, response, 'File not found', 404);
And handle the response:
process.on('uncaughtException', function (exception) {
exception.response.writeHead(exception.code, {'Content-Type': 'text/html'});
exception.response.end('Error ' + exception.code + ' - ' + exception.message);
});
I haven't test this solution yet, but I don't see the reason why this couldn't work.
I give an answer to my own question... :)
As it seems there is no way around to manually catch errors. I now use a helper function that itself returns a function containing a try/catch block. Additionally, my own web server class checks if either the request handling function calls response.end() or the try/catch helper function waitfor() (raising an exception otherwise). This avoids to a great extent that request are mistakenly left unprotected by the developer. It isn't a 100% error-prone solution but works well enough for me.
handler.waitfor = function(callback) {
var me=this;
// avoid exception because response.end() won't be called immediately:
this.waiting=true;
return function() {
me.waiting=false;
try {
callback.apply(this, arguments);
if (!me.waiting && !me.finished)
throw new Error("Response handler returned and did neither send a "+
"response nor did it call waitfor()");
} catch (e) {
me.handleException(e);
}
}
}
This way I just have to add a inline waitfor() call to be on the safe side.
function handleRequest(request, response, handler) {
fs.read(fd, buffer, 0, 10, null, handler.waitfor(
function(error, bytesRead, buffer) {
buffer.unknownFunction(); // causes exception
response.end(buffer);
}
)); //fs.read
}
The actual checking mechanism is a little more complex, but it should be clear how it works. If someone is interested I can post the full code here.
One idea: You could just use a helper method to create your call backs and make it your standard practice to use it. This does put the burden on the developer still, but at least you can have a "standard" way of handling your callbacks such that the chance of forgetting one is low:
var callWithHttpCatch = function(response, fn) {
try {
fn && fn();
}
catch {
response.writeHead(500, {'Content-Type': 'text/plain'}); //No
}
}
<snipped>
var buffer = new require('buffer').Buffer(10);
fs.read(fd, buffer, 0, 10, null,
function(error, bytesRead, buffer) {
callWithHttpCatch(response, buffer.dontTryThisAtHome()); // causes exception
response.end(buffer);
}); //fs.read
}); //fs.open
I know that probably isn't the answer you were looking for, but one of the nice things about ECMAScript (or functional programming in general) is how easily you can roll your own tooling for things like this.
At the time of this writing, the approach I am seeing is to use "Promises".
http://howtonode.org/promises
https://www.promisejs.org/
These allow code and callbacks to be structured well for error management and also makes it more readable.
It primarily uses the .then() function.
someFunction().then(success_callback_func, failed_callback_func);
Here's a basic example:
var SomeModule = require('someModule');
var success = function (ret) {
console.log('>>>>>>>> Success!');
}
var failed = function (err) {
if (err instanceof SomeModule.errorName) {
// Note: I've often seen the error definitions in SomeModule.errors.ErrorName
console.log("FOUND SPECIFIC ERROR");
}
console.log('>>>>>>>> FAILED!');
}
someFunction().then(success, failed);
console.log("This line with appear instantly, since the last function was asynchronous.");
Two things have really helped me solve this problem in my code.
The 'longjohn' module, which lets you see the full stack trace (across multiple asyncronous callbacks).
A simple closure technique to keep exceptions within the standard callback(err, data) idiom (shown here in CoffeeScript).
ferry_errors = (callback, f) ->
return (a...) ->
try f(a...)
catch err
callback(err)
Now you can wrap unsafe code, and your callbacks all handle errors the same way: by checking the error argument.
I've recently created a simple abstraction named WaitFor to call async functions in sync mode (based on Fibers): https://github.com/luciotato/waitfor
It's too new to be "rock solid".
using wait.for you can use async function as if they were sync, without blocking node's event loop. It's almost the same you're used to:
var wait=require('wait.for');
function handleRequest(request, response) {
//launch fiber, keep node spinning
wait.launchFiber(handleinFiber,request, response);
}
function handleInFiber(request, response) {
try {
if (request.url=="whatever")
handleWhateverRequest(request, response);
else
throw new Error("404 not found");
} catch (e) {
response.writeHead(500, {'Content-Type': 'text/plain'});
response.end("Server error: "+e.message);
}
}
function handleWhateverRequest(request, response, callback) {
if (something)
throw new Error("something bad happened");
Response.end("OK");
}
Since you're in a fiber, you can program sequentially, "blocking the fiber", but not node's event loop.
The other example:
var sys = require('sys'),
fs = require('fs'),
wait = require('wait.for');
require("http").createServer( function(req,res){
wait.launchFiber(handleRequest,req,res) //handle in a fiber
).listen(8124);
function handleRequest(request, response) {
try {
var fd=wait.for(fs.open,"/proc/cpuinfo", "r");
console.log("File open.");
var buffer = new require('buffer').Buffer(10);
var bytesRead=wait.for(fs.read,fd, buffer, 0, 10, null);
buffer.dontTryThisAtHome(); // causes exception
response.end(buffer);
}
catch(err) {
response.end('ERROR: '+err.message);
}
}
As you can see, I used wait.for to call node's async functions in sync mode,
without (visible) callbacks, so I can have all the code inside one try-catch block.
wait.for will throw an exception if any of the async functions returns err!==null
more info at https://github.com/luciotato/waitfor
Also in synchronous multi-threaded programming (e.g. .NET, Java, PHP) you can't return any meaningful information to the client when a custom unkown Exception is caught. You may just return HTTP 500 when you have no info regarding the Exception.
Thus, the 'secret' lies in filling a descriptive Error object, this way your error handler can map from the meaningful error to the right HTTP status + optionally a descriptive result. However you must also catch the exception before it arrives to process.on('uncaughtException'):
Step1: Define a meaningful error object
function appError(errorCode, description, isOperational) {
Error.call(this);
Error.captureStackTrace(this);
this.errorCode = errorCode;
//...other properties assigned here
};
appError.prototype.__proto__ = Error.prototype;
module.exports.appError = appError;
Step2: When throwing an Exception, fill it with properties (see step 1) that allows the handler to convert it to meannigul HTTP result:
throw new appError(errorManagement.commonErrors.resourceNotFound, "further explanation", true)
Step3: When invoking some potentially dangerous code, catch errors and re-throw that error while filling additional contextual properties within the Error object
Step4: You must catch the exception during the request handling. This is easier if you use some leading promises library (BlueBird is great) which allows you to catch async errors. If you can't use promises than any built-in NODE library will return errors in callback.
Step5: Now that your error is caught and contains descriptive information about what happens, you only need to map it to meaningful HTTP response. The nice part here is that you may have a centralized, single error handler that gets all the errors and map these to HTTP response:
//this specific example is using Express framework
res.status(getErrorHTTPCode(error))
function getErrorHTTPCode(error)
{
if(error.errorCode == commonErrors.InvalidInput)
return 400;
else if...
}
You may other related best practices here