I'm pretty new to node.js and mongodb. I need a function, that will return an array with results from database. When I query 'SELECT * FROM table' in php, it returns array or array of objects. But with node and mongo I have to use lot of code, which seems so unnecessary to me. So I wrote this function:
select: function (table, terms) {
var rows = [];
var find = function (error, db) {
var collection = db.collection(table);
var docs = collection.find(terms);
docs.each(function (error, doc) {
rows.push(doc);
});
};
client.connect(url, find);
return rows;
}
It takes table name [string] and terms [js object] as arguments. And I want to this function return an array like this: [ { doc1 }, { doc2 }, { doc3 }, ...].
When I log variable doc, in each function, it's alright - valid javascript object. But when I log variable rows after pushing all docs, it's empty - []. Also the result of function select() is empty array - [].
Please, is there someone who has an idea about what could be a problem or who uses some similar method to fetch data from MongoDB? Thank you
The connect function is asynchronous, and finishes after your return statement. Put your return statement within the callback function, just after the closing bracket of docs.each(.....); This way you're sure your return statement will be executed after your array has been filled.
Edit for completeness: a return statement doesn't work of course, because it returns from the function in find. Work with a callback function.
Like this:
select: function (table, terms, cb) {
var rows = [];
var find = function (error, db) {
var collection = db.collection(table);
var docs = collection.find(terms);
docs.each(function (error, doc) {
rows.push(doc);
});
cb(error, rows);
};
client.connect(url, find);
}
Edit: Above example (cb function instead of return)
Related
I have MySQL query in Node.js where I'm trying to return "variable" and assign in to getVarible OUTSIDE my query. Is it possible to do in this case? If so, how?
Here is my code :
var sqlQuery = connection.query(result, function(err, rows, fields) {
var count = rows.length;
if(count === 1){
var variable = rows[0].item;
return variable;
}
}
});
var getVariable = variable;
If you declare getVariable before your query call, you should be able to access it once your query return, within your cacllback fuction. You can try something like this:
var getVariable = {};
var sqlQuery = connection.query(result, function(err, rows, fields) {
var count = rows.length;
if(count === 1){
getVariable = rows[0].item;
}
}
});
However keep in mind that query execution is asynchronous, so your variable will be empty until your callback is executed.
So I would suggest you all logic where you use that variable to be executed within callback or use a promise of async framework.
Your connection.query does not return what you think it returns i.e. return variable; statement has no effect.
Checkout the API with the usage of query object.
var query = connection.query('SELECT * FROM posts');
query
.on('error', function(err) {
// Handle error, an 'end' event will be emitted after this as well
})
.on('fields', function(fields) {
// the field packets for the rows to follow
})
.on('result', function(row) {
// Pausing the connnection is useful if your processing involves I/O
connection.pause();
processRow(row, function() {
connection.resume();
});
})
.on('end', function() {
// all rows have been received
});
Alternatively you can have a short hand to handle the success, error by using the second parameter to connection.query. This is a function with signature function (error, results, fields) which does the same thing as above API.
connection.query(sqlQueryStr, function (error, results, fields) {})
The call to connection.query is asynchronous and the function passed as the second argument is executed only after the results are available i.e the query has executed.
I'm pulling some data from a MongoDB collection using Mongoose. I'm getting back my array of objects with just the fields I've selected. All good. Here's my code:
var fieldsToReturn = 'username password';
User.find({username: {$exists: true}}, fieldsToReturn, {sort: {'created.when': 1}}, function(err, data){
if (err) {
return err;
} else {
// clever code needed here to manipulate the data!
return res.json(data);
}
});
What I want to do is iterate through the array of JavaScript objects returned in data and if there is a password (it'll be a text string) replace password with a boolean true, but if it's null then return a boolean false.
I've tried a bit of underscore.js magic:
_.each(data, function(element, index, list){
if (element.password !== null) {element.password = true};
});
But what I get in the JSON that's returned is "password":"true" and not "password":true. Also tried element.password = new Boolean(true). Any suggestions?
You could use map for this.
data = data.map(function(element) {
if (element.password) {
element.password = true;
}
return element;
});
var data = [
{
password: 'secret'
},
{
password: ''
},
{
password: 'swordfish'
}
];
data = data.map(function(element) {
if (element.password) {
element.password = true;
}
return element;
});
document.querySelector('pre').innerText = JSON.stringify(data);
<pre></pre>
Try using asyncjs lybrary. It will call the callback function when finished all the executions for the array objects. Please, also read carefully about asyncronous flow in javascript (nodejs) .
Hope it helps!!
//this should work if placed on your clever code and installed asyncjs
async.map(data, checkPassword, function(err, results){
//when arrived here, all the array has been checked and modified
console.log(results);
}
//Remember This function must be placed outside of the find() method.
function checkPassword(obj, callback)
{
if (obj.password) {
obj.password = true;
}
return callback(null, obj);
}
It's Mongoose. Something is happening before the data is returned which makes it look like straight JSON but if you try and manipulate the element in a _.each loop then the object you're working on (element) isn't a simple object mapping to the document in MongoDB. Any Mongoose experts out there that can shed light on this?
What I did to fix this is use LoDash's _.mapValues:
function renderUserKeys(obj){
var newObj = _.mapValues(obj, function(n){return n});
if (newObj.password !== null) {newObj.password = true};
return newObj;
};
And then in the code:
...
} else {
var newArray = [];
_.each(data, function(element, index, list){
newArray.push(renderUserKeys(element._doc)); // note the ._doc
});
return res.json(newArray);
};
Maybe there is a better function that _.mapValues (maybe _.merge?) to perform the copy, I'll need to look into this, but for now this works and my Mongoose middleware is still running normally (as I'm not using lean()).
Lesson learned: Mongoose !== Duck. As in, it may look like an object but when you try and manipulate it, it doesn't behave as you expect it to!
I have a webserver running in node.js and Express which retrieves data from mongodb . In mongodb collections are getting created dynamically and the name of newly created collection will be stored in one metadata collection “project” . My requirement is to firstly iterate to metadata collection to get the collection name and then get inside the each collection to do multiple query based on some condition . Because my collection metadata is dynamic I have tried to do using for loop .
But it is giving wrong data . It is not executing sequent . Before finishing the loop execution it is returning the value .How to perform sequential execution in node.js using node core modules only (Not other library like async..);
exports.projectCount = function (req, res) {
var mongo = require("mongodb"),
Server = mongo.Server,
Db = mongo.Db;
var server = new Server("localhost", 27017, {
auto_reconnect: true
});
var db = new Db("test", server);
// global JSON object to store manipulated data
var projectDetail = {
projectCount: 0,
projectPercent: 0
};
var totalProject = 0;
db.open(function (err, collection) {
//metadata collection
collection = db.collection("project");
collection.find().toArray(function (err, result) {
// Length of metadata collection
projectDetail.projectCount = result.length;
var count = 0;
//iterate through each of the array which is the name of collection
result.forEach(function (item) {
//change collection object to new collection
collection = db.collection(item.keyParameter.wbsName);
// Perform first query based on some condition
collection.find({
$where: "this.status == 'Created'"
}).toArray(function (err, result) {
// based on result of query one increment the value of count
count += result.lenght;
// Perform second query based on some condition
collection.find({
$where: "this.status=='Completed'"
}).toArray(function (err, result) {
count += result.length;
});
});
});
// it is returning the value without finishing the above manipulation
// not waiting for above callback and value of count is coming zero .
res.render('index', {
projectDetail: projectDetail.projectCount,
count: count
});
});
});
};
When you want to call multiple asynchronous functions in order, you should call the first one, call the next one in it's callback and so on. The code would look like:
asyncFunction1(args, function () {
asyncFunction2(args, function () {
asyncFunction3(args, function () {
// ...
})
})
});
Using this approach, you may end up with an ugly hard-to-maintain piece of code.
There are various ways to achieve the same functionality without nesting callbacks, like using async.js or node-fibers.
Here is how you can do it using node.js EventEmitter:
var events = require('events');
var EventEmitter = events.EventEmitter;
var flowController = new EventEmitter();
flowController.on('start', function (start_args) {
asyncFunction1(args, function () {
flowController.emit('2', next_function_args);
});
});
flowController.on('2', function (args_coming_from_1) {
asyncFunction2(args, function () {
flowController.emit('3', next_function_args);
});
});
flowController.on('3', function (args_coming_from_2) {
asyncFunction3(args, function () {
// ...
});
});
flowController.emit('start', start_args);
For loop simulation example:
var events = require('events');
var EventEmitter = events.EventEmitter;
var flowController = new EventEmitter();
var items = ['1', '2', '3'];
flowController.on('doWork', function (i) {
if (i >= items.length) {
flowController.emit('finished');
return;
}
asyncFunction(item[i], function () {
flowController.emit('doWork', i + 1);
});
});
flowController.on('finished', function () {
console.log('finished');
});
flowController.emit('doWork', 0);
Use callbacks or promises or a flow control library. You cannot program servers in node without understanding at the very least one of these approaches, and honestly all halfway decent node programmers thoroughly understand all three of them (including a handful of different flow control libraries).
This is not a something you are going to just get an answer coded for you by someone else on stackoverflow and then move on. This is a fundamental thing you have to go and study and learn generically as it is only going to come up over and over again on a daily basis.
http://howtonode.org/control-flow
http://callbackhell.com/
Per the resources in the answer above me, nesting the callback when you iterate and only calling it if you are on the last iteration will solve you problem.
I have a for loop that pulls data from a MySQL server. I would like the four values to be put into variables so I can use them later. Here's the code I have; for some reason, it says thev is undefined?
create();
function create(){
for(var i=0;i<4;i++){
var thev=[];
client.query('SELECT curattend FROM table1 WHERE ind=?',[i], function(err,result){
thev[i] = result[0].curattend;
});
}
return thev;
}
console.log(thev[2]);
I would appreciate any advice on this problem.
There are a lot of problems here.
thev is local to create. You don’t assign the return value of create to anything, so it’s still not going to be defined.
var thev = []; should not be inside the for loop. It’ll only end up containing one element. Or it would, but…
The callback to query is not just there for fun; it’s an asynchronous call, and is 100% sure to not have happened by the time you actually return from the function.
I would just do it using the async library:
function range(start, end) {
var result = [];
while(start < end) {
result.push(start);
start++;
}
return result;
}
async.map(range(0, 4), function(i, callback) {
client.query('SELECT curattend FROM table1 WHERE ind = ?', [i], function(err, result) {
if(err) return callback(err);
callback(null, result[0].curattend);
});
}, function(err, thev) {
// Continue
});
I am trying to do exactly what this mongo example is doing but in mongoose. It seems more complex to me in mongoose. Possibly i'm trying to fit a square peg in a round hole?
This example is from http://www.codeproject.com/Articles/521713/Storing-Tree-like-Hierarchy-Structures-With-MongoD (tree structure with parent reference)
I'm trying to build a path.
var path=[];
var item = db.categoriesPCO.findOne({_id:"Nokia"});
while (item.parent !== null) {
item=db.categoriesPCO.findOne({_id:item.parent});
path.push(item._id);
}
path.reverse().join(' / ');
Thanks!
Mongoose is an asynchronous library, so
db.categoriesPCO.findOne({_id:"Nokia"});
doesn't return the answer to the query, it just returns a Query object itself. In order to actually run the query, you'll need to either pass in a callback function to findOne() or run exec() on the Query object returned.
db.categoriesPCO.findOne({_id:"Nokia"}, function (err, item) {
});
However, you can't use the same while loop code to generate the path, so you'll need to use recursion instead. Something like this should work:
var path=[];
function addToPath(id, callback) {
db.categoriesPCO.findOne({_id:id}, function (err, item) {
if (err) {
return callback(err);
}
path.push(item._id);
if (item.parent !== null) {
addToPath(item.parent, callback);
}
else {
callback();
}
});
}
addToPath("Nokia", function (err) {
path.reverse().join(' / ');
});
NB In addition, instead of pushing new items onto the end of the path array and then reversing it, you could use path.unshift() which adds the item to the beginning of the array.