I have a small problem with nested callbacks in javascript. Apparently im doing something wrong, but i did my research and tried to follow the tutorials avaialble throughout the web. I know that my code works, since query returns proper data, but i have no idea why my code doesnt "wait" within executeQuery method till the res is fetched from database, it just goes straight to "oh noes" section.
DatabaseConnection.prototype.executeQuery = function(query, executeQueryDone){
var activeConnection;
console.log("YEAAA, executing Query: " + query);
this.pool.getConnection(function (err, connection){
console.log("Got Connection, we are ready to go!");
if (err){
console.log("Error, DAMMNIT! " + err);
executeQueryDone(err);
}
activeConnection = connection;
activeConnection.connect();
activeConnection.query(query, function(error, res){
console.log("Connection from pool is executing Query");
if(error){
console.log("Error during executing query");
executeQueryDone(error);
}
else {
console.log(" OK now release connection (dont be selfish)! ");
activeConnection.release();
executeQueryDone(null, res);
}
});
});
console.log("oh noes! IM AFTER CONNECTION, why dude? WHY???? ");
};
I'd be grateful for any hints since im struggling with that since yesterday.
=====================
PROBLEM SOLVED:
generally all was OK, the "issue" was mistakenly written test:
i made it like that:
describe('testDB2', function () {
it('should return proper STUFF', function (done) {
assert.equal(1, someService.getStuff(function(err, result){
if (err === null){
console.log("err is null, as it should be!");
}
console.log(" result from DB " + result[1].NUMBERS);
}));
});
});
while is should be like that:
describe('testDB2', function () {
it('should return proper STUFF', function (done) {
someService.getStuff(function(err, result){
assert.equal(err, null);
assert.equal(result[1].NUMBERS, 43637654);
done();
});
});
});
as a result (in the incorrect case), i didnt fetch the result the way i wanted as assert couldnt "catch up"
thanks to all for the enlightment ;)
Your console.log call isn't part of a callback, so it will be called as soon as the getConnection call is made. If you want it to be called only after your callback to getConnection fires, you either need to call it at the end of that call, or you need to use some form of promises.
Javascript is single threaded, but it does use a task queue. When the database connection is instantiated and the pool connected to, the response to that is placed into the task queue to be executed or actioned when complete.
Directly after placing that in the task queue the next piece of execution is the log that does the "oh noes" - nice messaging btw lol.
So essentially what happens is the db call gets placed in the task queue for later execution, and then the log occurs, and then the task queue executes at a later time with the db response.
Related
i am trying to save something to a database using mongoose. The thing is i need to make sure the save was completed before i move on in the program and close the connection. Knowing that save is async in mongoose i tried using this code:
saveFunction = function(song){
song.save(function(err, userObj){
if(err){
console.log('Error' + err);
} else{
console.log('saved successfully:', userObj);
}
});
};
database.prototype.add= function(newSong){
mongoose.connect(url);
var song = new songModel({id : newSong.getId(),
title : newSong.getTitle(),
artist : newSong.getArtist,
genre : newSong.getGenre(),
rating : newSong.getRating(),
link : newSong.getLink()});
console.log("before async");
async.parallel([function (callback){
saveFunction(song);
callback();
}],function(){
mongoose.connection.close();
console.log('closed connection');
});
console.log("after async");
nextFreeId++;
};
^songModel is defined globally.
I tried lots of different methods and changed lots of thing but i always get and error of somekind. With this code i get a process.nexttick(function() throw err ) error. I just can't get it to work. Can someone tell me whats wrong or provide me with working code?
I think optimally the console should look like this:
before async
saved successfully
closed connection
after async
Thanks!
EDIT: Open to other alternatives to async aswell. I just want to get this code to work any way possible. I just need to save/find something/remove something and it needs to wait with the rest of the execution of the program until the save/find/removal is done. I'm getting pretty desperate, lost nearly a day on this problem alone on a tight shedule :(
You need to return a callback from your save function.
saveFunction = function(song,callback){
song.save(function(err, userObj){
if(err){
console.log('Error' + err);
return callback(true,err)
} else{
console.log('saved successfully:', userObj);
return callback(null);
}
});
};
Edit
From your comment, the behavior you are expecting will never occur. You are expecting
console.log("before async");
async.parallel -> do your bits
console.log('closed connection');
console.log("after async");
However this will never happen because async.parallel is an asynchronous call, which means that the execution does not wait for it to finish before moving onto the next command. The behavior you are seeing is
console.log("before async");
async.parallel -> starts
console.log("after async");
async.parallel -> console.log('closed connection');
Node is doing the first log, starting the async.parallel, then console.logging "after async". Then when async.parallel gets to its callback function, it prints "closed connection", so it appears after "after async" because it was executed after.
Any logic you want to perform that relies on the result of async.parallel must happen in the callback function. Furthermore, async.parallel is used when you want to run 2 or more functions asynchronously, then execute a callback once they are all finished. Your solution does not require async.parallel. You can replace that with:
saveFunction(song,function(err){
if(err){
//failed to save song
}
mongoose.connection.close(); //you do not need to do this anyway
console.log('closed connection');
nextFreeId++;
//anything else you need to do here
});
I'm using Node.js to write system scripts that run on a server. Due to Node's asynchronous nature, my script is exiting before the database calls have a chance to complete and nothing is ever written to the database.
I'm using Mongoose as an ORM and talking to a MongoDB, if that makes any difference. Node.js offers SYNCHRONOUS method calls for this very reason, for example: https://nodejs.org/api/child_process.html
I guess my questions are:
1) Does mongoose offer a way to block so my scripting process can wait for the database call to return?
2) If not, is there another method I should consider other than something like:
(function wait () {
if (!SOME_EXIT_CONDITION) setTimeout(wait, 1000);
})();
3) Is node not the best tool for the job for writing scripts? I love node for web app development, and can write nested callbacks or work with promises all day long. But what about as a scripting language?
EDIT -----------------------------------------------
Below is an quick example of the script to provide more clarity of the situation:
#!/usr/bin/env node
# Please note the above that this is a bash script
var schema = mongoose.Schema({
// ... attributes ...
});
var model = new (mongoose.model('ModelObject'))();
model['attribute'] = 42;
console.log('This gets printed first');
model.save(function(err) {
console.log('Nothing in the callback gets printed because callback is never called');
if(err) { // Can't check for errors because this is never reached
console.log('This never gets printed to the screen');
console.log('And consequently nothing is ever saved to mongo');
} else {
console.log('This never gets printed either');
}
});
console.log('This gets printed second');
If your model does not get saved, there is a Mongo error. Following MongoDB conventions you have to check for errors:
model.save(function(error, savedItem) {
if(error) {
// nothing is saved
}
});
Otherwise, have you considered using Promises? It useful for chaining events and simpler error handling.
Promise = require('bluebird');
Promise.promisifyAll(mongoose.Query.base);
model.saveAsync().then(function(savedItem) {
// saved
})
.catch(function(error) {
// handle error
});
I think you are looking for this, check below if this help you.
var mongoose = require('mongoose'),
model1 = mongoose.model('model1'),
model2 = mongoose.model('model2');
model1.findOne({"type" : 'Active'}, function err(err, catConfig) {
if(!err.error){
//This will execute once above DB call is done!
model2.findOne(condition).remove(function(err, gAnalysis) {
//Lines of code that you want to execute after second DB call
});
}
});
I don't see you opening a connection to the database so presumably saving a model instance does nothing, not even call the callback with an error...
I've tested the below example:
test.js:
var mongoose = require('mongoose');
var kittySchema = mongoose.Schema({
name: String
});
var Kitten = mongoose.model('Kitten', kittySchema);
mongoose.connect('mongodb://localhost:27017/test', function (err) {
if (err) throw err;
var silence = new Kitten({ name: 'Silence' });
silence.save(function (err, saved) {
if (err) throw err;
console.log('Kitty Silence is saved!');
mongoose.disconnect(function (err) {
if (err) throw err;
console.log('done...');
});
});
});
Running node test.js prints this to the console:
Kitty Silence is saved!
done...
and examining my local test database shows that Silence is indeed saved.
I'm relatively new to Javascript and I am having trouble understanding how to use a MongoDB callback with an ExpressJS get. My problem seems to be if it takes too long for the database search, the process falls out of the app.get() and gives the webpage an "Error code: ERR_EMPTY_RESPONSE".
Currently it works with most values, either finding the value or properly returning a 404 - not found, but there are some cases where it hangs for a few seconds before turning the ERR_EMPTY_RESPONSE. In the debugger, it reaches the end of the app.get(), where it returns ERR_EMPTY_RESPONSE, and after that the findOne callback finishes and goes to the 404, but by then it is too late.
I've tried using async and introducing waits with no success, which makes me feel like I am using app.get and findOne incorrectly.
Here is a general version of my code below:
app.get('/test', function (req, res) {
var value = null;
if (req.query.param)
value = req.query.param;
else
value = defaultValue;
var query = {start: {$lte: value}, end: {$gte: value}};
var data = collection.findOne(query, function (err, data) {
if (err){
res.sendStatus(500);
}
else if (data) {
res.end(data);
}
else{
res.sendStatus(404);
}
});
});
What can I do to have the response wait for the database search to complete? Or is there a better way to return a database document from a request? Thanks for the help!
You should measure how long the db query takes.
If it's slow >5sec and you can't speed it up, than it might be a good idea to decouple it from the request by using some kind of job framework.
Return a redirect the url where the job status/result will be available.
I feel silly about this, but I completely ignored the fact that when using http.createServer(), I had a timeout set of 3000 ms. I misunderstood what this timeout was for and this is what was causing my connection to close prematurely. Increasing this number allowed my most stubborn queries to complete.
I'm aware of the best practice of MongoDB connection pooling in NodeJS of the singleton DB connection type like this
var db = null;
var connection = function getDBConnection(callback) {
if(db) { callback(null, db) } else { MongoClient.connect( .... ) }
}
module.exports = getDBConnection;
However, what I cannot get my head around at the moment is how to handle this in a one-shot script that, say, does some pre-initialization on the documents of a certain db collection:
getDBConnection(function (err, database) {
var collection = database.collection("objects");
var allObjectsArray = collection.find( /* ... */
).toArray(function (err, objects) {
if(err != null) console.log(err);
assert.equal(null, err);
_.each(objects, function (item) {
collection.update(
{ id: item.id},
{ $set: { /* ... */ }},
function (err, result) {
if(err != null) console.log(err);
assert.equal(null, err);
}
);
});
// database.close(); <-- this fails with "MongoError: Connection Closed By Application", thrown by the update callback
});
// database.close(); <-- this fails too, thrown by the toArray callback
});
If I call the script like that, it never terminates, due to the still open connection. If I close the connection at the bottom, it fails because of, well, a closed connection.
Considering that opening a new connection for every update is not really an option, what am I missing? Keeping the connection open may be fine for webapps, but for a one-shot script called from a shell script this really doesn't work out, does it?
Sorry if this question has arisen before, I've given it some research but have not quite been able to come up with a working answer for me...
Thanks!
Julian
As a "pooled connection" there is code running to keep the connection alive and establish more connections in the pool if required under the driver connection. So much like various "server code" methods, event loop handlers have been invoked and the process does not exit at the end of your code until these are de-registered.
Therefore your two choices to call after all your code has executed are either:
Call db.close() or in your code context specifically database.close() once all is done.
Call process.exit() which is a generic call in node.js applications which will shut the whole process down and therefore stop any other current event loop code. This actually gives you an option to throw an error on exit if you want your code to be "shell integrated" somewhere and look for the exit status.
Or call both. db.close() will allow execution to the next line of code and whatever you put there will also run.
But you have to wait until everything is called, so you can cannot use synchronous loops with asynchronous code in the middle:
async.each(objects,function(item,callback) {
collection.update(
{ "_id": item._id },
{
// updates
},
callback
);
},function(err) {
if (err) throw err;
database.close();
});
I just started experimenting with Node.js Express and Mongoose by creating a simple blog website.
I am trying to build some routes that will do some simple db operations, but I'm getting confused with asynchronous functions and if my code will execute properly every time.
This is basically what I have:
app.get('/createUser', function(req, resp) {
var newUser = new User({name: 'abc123', pass: 'password321'});
newUser.save(function(err){ // will this callback always be called correctly?
if(err) resp.send('ERROR!');
resp.send('SUCCESS!');
});
});
So I want the response to be written out as "ERROR!" for any error in saving, and "SUCCESS!" when the save was successful. However, I'm confused about the timing of these functions. Will the get() function ever return before the save() is completed? And if so, will the response not be written out properly?
The get() function will complete before the save function does, but since there is nothing writing a response until the save callback is executed then the response won't be available to the browser until then.
I added a few console.log() calls to your code so you can see the order of execution:
app.get('/createUser', function(req, resp) {
console.log('this will print first');
var newUser = new User({name: 'abc123', pass: 'password321'});
newUser.save(function(err){ // will this callback always be called correctly?
console.log('this will print last');
if(err) {
resp.send('ERROR!');
}
else {
resp.send('SUCCESS!');
}
});
console.log('this will print second');
});