Develop Reference

Javascript/NodeJS: Array empty after pushing values in forEach loop

I got a little bit of a problem. Here is the code:
Situation A:
var foundRiders = [];
riders.forEach(function(rider){
Rider.findOne({_id: rider}, function(err, foundRider){
if(err){
console.log("program tried to look up rider for the forEach loop finalizing the results, but could not find");
} else {
foundRiders.push(foundRider);
console.log(foundRiders);
}
});
});
Situation B
var foundRiders = [];
riders.forEach(function(rider){
Rider.findOne({_id: rider}, function(err, foundRider){
if(err){
console.log("program tried to look up rider for the forEach loop finalizing the results, but could not find");
} else {
foundRiders.push(foundRider);
}
});
});
console.log(foundRiders);
So in Situation A when I console log I get that foundRiders is an array filled with objects. In situation B when I put the console.log outside the loop, my roundRiders array is completely empty...
How come?

As others have said, your database code is asynchronous. That means that the callbacks inside your loop are called sometime later, long after your loop has already finishes. There are a variety of ways to program for an async loop. In your case, it's probably best to move to the promise interface for your database and then start using promises to coordinate your multiple database calls. You can do that like this:
Promise.all(riders.map(rider => {
return Rider.findOne({_id: rider}).exec();
})).then(foundRiders => {
// all found riders here
}).catch(err => {
// error here
});
This uses the .exec() interface to the mongoose database to run your query and return a promise. Then, riders.map() builds and returns an array of these promises. Then,Promise.all()monitors all the promises in the array and calls.then()when they are all done or.catch()` when there's an error.
If you want to ignore any riders that aren't found in the database, rather than abort with an error, then you can do this:
Promise.all(riders.map(rider => {
return Rider.findOne({_id: rider}).exec().catch(err => {
// convert error to null result in resolved array
return null;
});
})).then(foundRiders => {
foundRiders = foundRiders.filter(rider => rider !== null);
console.log(founderRiders);
}).catch(err => {
// handle error here
});
To help illustrate what's going on here, this is a more old fashioned way of monitoring when all the database callbacks are done (with a manual counter):
riders.forEach(function(rider){
let cntr = 0;
Rider.findOne({_id: rider}, function(err, foundRider){
++cntr;
if(err){
console.log("program tried to look up rider for the forEach loop finalizing the results, but could not find");
} else {
foundRiders.push(foundRider);
}
// if all DB requests are done here
if (cntr === riders.length) {
// put code here that wants to process the finished foundRiders
console.log(foundRiders);
}
});
});
The business of maintaining a counter to track multiple async requests is what Promise.all() has built in.
The code above assumes that you want to parallelize your code and to run the queries together to save time. If you want to serialize your queries, then you could use await in ES6 with a for loop to make the loop "wait" for each result (this will probably slow things down). Here's how you would do that:
async function lookForRiders(riders) {
let foundRiders = [];
for (let rider of riders) {
try {
let found = await Rider.findOne({_id: rider}).exec();
foundRiders.push(found);
} catch(e) {
console.log(`did not find rider ${rider} in database`);
}
}
console.log(foundRiders);
return foundRiders;
}
lookForRiders(riders).then(foundRiders => {
// process results here
}).catch(err => {
// process error here
});
Note, that while this looks like it's more synchronous code like you may be used to in other languages, it's still using asynchronous concepts and the lookForRiders() function is still returning a promise who's result you access with .then(). This is a newer feature in Javascript which makes some types of async code easier to write.

Attaching data from callback function inside a for loop Loopback

I´m trying to create a hook where I loop on each result from the response, find a user based on the current object and attach it as another attribute to the final response. However, the async calls are not letting me send the right response:
Board.afterRemote('find',function(context,boards,next){
var users = [];
context.result.admin =[];
var User = app.models.User;
context.result.forEach(function(result){
User.findOne({where:{id:result.adminId}},function(err,user){
result.admin = user;
});
});
console.log("result: "+JSON.stringify(context.result));
next();
});
How can I be able to add the user to each result on the context.result ?

Is User.findOne and asynchronous operation? If so, I would recommend using an async control flow library like async to iterate over the result and perform an asynchronous action on each item. It would look something like this:
Board.afterRemote('find',function(context,boards,next){
async.each(context.result, function(result, callback) {
User.findOne({where:{id:result.adminId}},function(err,user){
result.admin = user;
callback(err) // Done with this iteration
});
}, function(err) { // Called once every item has been iterated over
console.log("result: "+JSON.stringify(context.result));
next();
});
});

Wating for all finished request in a loop with node request

I use the node request ajax package. So, i have an loop, in every iteration it makes an request to my server.
// realItems needs the complete value of items assigned
var realItems;
var items = [];
_.forEach(JSON.parse(body), (value, key) => {
request('myurl/' + id, (error, response, body) => {
items = JSON.parse(body)
});
});
How can i bundle all my requests from request package, so I can assign the value of items variable to the realItems at the end?
// edit:
I use react js, so in this case realItems is an state, and i can't trigger it in every loop iteration, because render triggers on every setState

There are a number of ways to approach this. Here's a brute force method that does not preserve the order of the results:
var items = [];
var cnt = 0;
_.forEach(JSON.parse(body), (value, key) => {
++cnt;
request('myurl/' + value.id, (error, response, body) => {
items.push(JSON.parse(body));
// if all requesets are done
if (--cnt === 0) {
// process items here as all results are done now
}
});
});
Here's a version that uses Bluebird promises:
var Promise = require('bluebird');
var request = Promise.promisify(require("request"));
Promise.promisifyAll(request);
var promises = [];
_.forEach(JSON.parse(body), (value, key) => {
promises.push(request('myurl/' + value.id));
});
Promise.all(promises).then(function(results) {
// all requests are done, data from all requests is in the results array
// and are in the order that the requests were originally made
});
And, here's a little bit simpler Bluebird promises method that uses a Bluebird iterator:
var Promise = require('bluebird');
var request = Promise.promisify(require("request"));
Promise.promisifyAll(request);
Promise.map(JSON.parse(body), function(value) {
return request('myurl/' + value.id);
}).then(function(results) {
// all requests are done, data is in the results array
});

Is it a requirement that you use the request package? I use async which is similar and comes with a parallel method which does exactly what you're asking -
https://github.com/caolan/async#parallel
example:
async.parallel([
function(callback){
setTimeout(function(){
callback(null, 'one');
}, 200);
},
function(callback){
setTimeout(function(){
callback(null, 'two');
}, 100);
}
],
// optional callback
function(err, results){
// the results array will equal ['one','two'] even though
// the second function had a shorter timeout.
});

How to call function after completion of async functions inside loop?

I have a forEach loop in NodeJS, iterating over a series of keys, the values of which are then retrieved asynchronously from Redis. Once the loop and retrieval has complete, I want to return that data set as a response.
My problem at the moment is because the data retrieval is asyncrhonous, my array isn't populated when the response is sent.
How can I use promises or callbacks with my forEach loop to make sure the response is sent WITH the data?
exports.awesomeThings = function(req, res) {
var things = [];
client.lrange("awesomeThings", 0, -1, function(err, awesomeThings) {
awesomeThings.forEach(function(awesomeThing) {
client.hgetall("awesomething:"+awesomeThing, function(err, thing) {
things.push(thing);
})
})
console.log(things);
return res.send(JSON.stringify(things));
})

I use Bluebird promises here. Note how the intent of the code is rather clear and there is no nesting.
First, let's promisify the hgetall call and the client -
var client = Promise.promisifyAll(client);
Now, let's write the code with promises, .then instead of a node callback and aggregation with .map. What .then does is signal an async operation is complete. .map takes an array of things and maps them all to an async operation just like your hgetall call.
Note how Bluebird adds (by default) an Async suffix to promisifed methods.
exports.awesomeThings = function(req, res) {
// make initial request, map the array - each element to a result
return client.lrangeAsync("awesomeThings", 0, -1).map(function(awesomeThing) {
return client.hgetallAsync("awesomething:" + awesomeThing);
}).then(function(things){ // all results ready
console.log(things); // log them
res.send(JSON.stringify(things)); // send them
return things; // so you can use from outside
});
};

No lib is needed. Easy as pie, it's just an async loop. Error handling is omitted. If you need to do a parallel async loop use a counter.
exports.awesomeThings = function(req, res) {
client.lrange("awesomeThings", 0, -1, function(err, awesomeThings) {
var len = awesomeThings.length;
var things = [];
(function again (i){
if (i === len){
//End
res.send(JSON.stringify(things));
}else{
client.hgetall("awesomething:"+awesomeThings[i], function(err, thing) {
things.push(thing);
//Next item
again (i + 1);
})
}
})(0);
});

node.js nested callback, get final results array

I am doing a for loop to find the result from mongodb, and concat the array. But I am not getting the final results array when the loop is finished. I am new to node.js, and I think it's not working like objective-c callback.
app.get('/users/self/feed', function(req, res){
var query = Bill.find({user: req.param('userId')});
query.sort('-createdAt');
query.exec(function(err, bill){
if (bill) {
var arr = bill;
Following.findOne({user: req.param('userId')}, function(err, follow){
if (follow) {
var follows = follow.following; //this is a array of user ids
for (var i = 0; i < follows.length; i++) {
var followId = follows[i];
Bill.find({user: followId}, function(err, result){
arr = arr.concat(result);
// res.send(200, arr);// this is working.
});
}
} else {
res.send(400, err);
}
});
res.send(200, arr); //if put here, i am not getting the final results
} else {
res.send(400, err);
}
})
});

While I'm not entirely familiar with MongoDB, a quick reading of their documentation shows that they provide an asynchronous Node.js interface.
That said, both the findOne and find operations start, but don't necessarily complete by the time you reach res.send(200, arr) meaning arr will still be empty.
Instead, you should send your response back once all asynchronous calls complete meaning you could do something like:
var billsToFind = follows.length;
for (var i = 0; i < follows.length; i++) {
var followId = follows[i];
Bill.find({user: followId}, function(err, result){
arr = arr.concat(result);
billsToFind -= 1;
if(billsToFind === 0){
res.send(200, arr);
}
});
}
The approach uses a counter for all of the inner async calls (I'm ignoring the findOne because we're currently inside its callback anyway). As each Bill.find call completes it decrements the counter and once it reaches 0 it means that all callbacks have fired (this works since Bill.find is called for every item in the array follows) and it sends back the response with the full array.

That's true. Your codes inside for will be executed in parallel at the same time (and with the same value of i I think). If you added console.log inside and after your for loop you will found the outside one will be printed before inside one.
You can wrap the code that inside your for into array of functions and execute them by using async module (https://www.npmjs.org/package/async) in parallel or series, and retrieve the final result from async.parallel or async.series's last parameter.