app.get('/zones/:id/experiences', function(req,res) {
var zone_key = req.params.id;
var recent = [];
var ref = firebase.database().ref('participants/'+zone_key+'/experiences');
ref.on("value", function(snapshot) {
snapshot.forEach((snap) => {
firebase.database().ref('experiences').child(snap.val()).once("value").then((usersnap) => {
recent.push(usersnap.val());
});
});
console.log(recent);
});
res.render('experiences',{key: zone_key, list: recent});
});
In the above code, I am querying a reference point to get a set of "keys". Then for each key, I am querying another reference point to get the object associated to that key. Then for each object returned for those keys, I simply want to push the objects into a list. I then want to pass in this list to the client site to do stuff with the data using the render.
For some reason, the recent [] never gets populated. It remains empty. Is this an issue with my variables not being in scope? I console logged to check what the data the reference points are returning and its all good, I get the data that I want.
P.S is nesting queries like this ok? For loop within another query
As cartant commented: the data is loaded from Firebase asynchronously; by the time you call res.render, the list will still be empty.
An easy way to see this is the 1-2-3 test:
app.get('/zones/:id/experiences', function(req,res) {
var zone_key = req.params.id;
var recent = [];
var ref = firebase.database().ref('participants/'+zone_key+'/experiences');
console.log("1");
ref.on("value", function(snapshot) {
console.log("2");
});
console.log("3");
});
When you run this code, it prints:
1
3
2
You probably expected it to print 1,2,3, but since the on("value" loads data asynchronously, that is not the case.
The solution is to move the code that needs access to the data into the callback, where the data is available. In your code you need both the original value and the joined usersnap values, so it requires a bit of work.
app.get('/zones/:id/experiences', function(req,res) {
var zone_key = req.params.id;
var recent = [];
var ref = firebase.database().ref('participants/'+zone_key+'/experiences');
ref.on("value", function(snapshot) {
var promises = [];
snapshot.forEach((snap) => {
promises.push(firebase.database().ref('experiences').child(snap.val()).once("value"));
Promise.all(promises).then((snapshots) => {
snapshots.forEach((usersnap) => {
recent.push(usersnap.val());
});
res.render('experiences',{key: zone_key, list: recent});
});
});
});
});
In this snippet we use Promise.all to wait for all usersnaps to load.
Related
I'm trying to build a method which reads from firestore an array of elements (object):
I have a service which retrieves the data from firestore, first it gets an array of document references
var data = snapshot.get('elements');
and then it gets all the objects:
getElements(){
return new Promise(res =>{
this.AngularAuth.currentUser
.then( user => {
this.useruid = user.uid;
this.db.firestore.doc(`/users/${this.useruid}`).get().then(snapshot =>{
if(snapshot.exists){
var data = snapshot.get('elements'); //This gets the array of elements
data.forEach(element => {
this.db.firestore.doc(element).get().then(object =>{
if(object.exists){
var elem = object.data() as object;
this.array.push(elem);//I kind of push in the array instances of object
}
else{
console.log("Error. Doc doesn't exist")
}
}).catch(err =>{
console.log(err);
})
});
res(this.array);
}
else{
console.log("Error. Doc doesn't exist")
}
}).catch(function(error) {
// An error happened.
})
})
.catch(function(error) {
// An error happened.
})
});
}
Then in a component I have an async method which calls the service, and tries to push into another array all the names from each object in the first array:
async retrieveArray(){
this.array = await this.service.getElements();
this.array.forEach(element => {
this.names.push(element.name);
});
console.log(this.array);
console.log(this.names);
}
However when I look to the console, the first array (array) gives me indeed an array of objects, but the other array (names) is empty.
I used the method get to retrieve the data because I don't want to listen to it, I might need the value just once.
Personally I find the async/await syntax infinitely more elegant and easier to deal with than a good old .then() callback hell :
async getElements() {
let user;
try{
user = await this.AngularAuth.currentUser();
} catch(err) {
console.log(err);
return;
}
this.useruid = user.uid;
const snapshot = await this.db.firestore.doc(`/users/${this.useruid}`).get();
if (!snapshot.exists) {
console.log("Error. Doc doesn't exist")
return
}
const data = snapshot.get('elements'); //This gets the array of elements
let toReturn = [];
for(let element of data){ // can also use 'await Promise.all()' here instead of for...of
const object = await this.db.firestore.doc(element).get();
toReturn.push(elem);
}
return toReturn;
}
async retrieveArray(){
this.array = await this.service.getElements();
this.names = this.array.map( element => element.name ) // Also use .map() here
console.log(this.array);
console.log(this.names);
}
If you use for...of, all calls will be made one after the other, in order. If you use await Promise.all(), all calls will be made and awaited simultaneously, which is faster but recommended only if you have a small number of calls to make (otherwise this could overload the server you're calling, or even be considered as a DDoS attack.)
I think the issue is in this part of your code:
if(snapshot.exists){
var data = snapshot.get('elements'); //This gets the array of elements
data.forEach(element => {
this.db.firestore.doc(element).get().then(object =>{
if(object.exists){
var elem = object.data() as object;
this.array.push(elem);//I kind of push in the array instances of object
}
else{
console.log("Error. Doc doesn't exist")
}
}).catch(err =>{
console.log(err);
})
});
res(this.nombres);
}
You're looping through the elements and fetching the object from firebase for each one. Each time is an async call, but you're not waiting for each of these calls to finish before calling res(this.nombres).
As for why the console.log(this.array) shows a populated array is that the console can be misleading. It provides the data in a kind of 'live' way (it's a reference to the array), and sometimes by the time the data arrives on the console, it's different to what the data looked like when console.log was called.
To make sure you see the data precisely as it was when console.log was called, try this:
console.log(JSON.parse(JSON.stringify(this.array));
As for the issue with your code, you need to wait for all the elements to have been fetched before you call the resolve function of your promise. Because you don't necessarily know the order in which the responses will come back, one option is to simply have a counter of how many results are remaining (you know how many you are expecting), and once the last response has been received, call the resolve function. This is how I would do it, but obviously I can't test it so it might not work:
if(snapshot.exists){
var data = snapshot.get('elements'); //This gets the array of elements
// *** we remember the number of elements we're fetching ***
let count = data.length;
data.forEach(element => {
this.db.firestore.doc(element).get().then(object =>{
// *** decrement count ***
count--;
if(object.exists){
var elem = object.data() as object;
this.array.push(elem);//I kind of push in the array instances of object
// *** If count has reached zero, now it's time to call the response function
if (count === 0) {
res(this.nombres);
}
}
else{
console.log("Error. Doc doesn't exist")
}
}).catch(err =>{
console.log(err);
})
});
// *** remove this line because it's calling the resolve function before nombres is populated
//res(this.nombres);
}
You might want to add behaviour for when the result of snapshot.get('elements') is empty, but hopefully with this you'll be on your way to a solution.
** EDIT **
I'm keeping this up just because the console.log issue might well be useful for you to know about, but I highly recommend the async/await approach suggested by Jeremy. I agree that's it's much more readable and elegant
Trying to read (and expect) a nested array like this:
var array = [
0: {subArrayy: {...}, title: "Title"},
1: {subArray]: {...}, title: "Title"},
...
However after reading and (!) outputting, the result is fine. My web console shows me the array and everything seems good. BUT doing array.length returns 0. And any iteration returns undefined.
I've tried using ladosh _.toArray thing that I've seen earlier, but it does absolutely nothing.
var locations = []; // Empty array
var ref = db.ref("locations/");
ref.once("value", function(snapshot) {
snapshot.forEach(function(item) {
var itemVal = item.val();
locations.push(itemVal); // Adding new items seems to work at first
});
});
console.log(locations, locations.length);
Output:
chrome output
I expected it to be iterable, in the way I could just use array0 to navigate.
Firebase reads data asynchronously, so that the app isn't blocked while waiting for network traffic. Then once the data is loaded, it calls your callback function.
You can easily see this by placing a few log statements:
console.log("Before starting to load data");
ref.once("value", function(snapshot) {
console.log("Got data");
});
console.log("After starting to load data");
When you run this code the output is:
Before starting to load data
After starting to load data
Got data
This is probably not the order you expected, but it explains exactly why you get a zero length array when you log it. But since the main code continued on straight away, by the time you console.log(locations, locations.length) the data hasn't loaded yet, and you haven't pushed it to the array yet.
The solution is to ensure all code that needs data from the data is either inside the callback, or is called from there.
So this will work:
var locations = []; // Empty array
var ref = db.ref("locations/");
ref.once("value", function(snapshot) {
snapshot.forEach(function(item) {
var itemVal = item.val();
locations.push(itemVal);
});
console.log(locations, locations.length);
});
As will this:
function loadLocations(callback) {
var locations = []; // Empty array
var ref = db.ref("locations/");
ref.once("value", function(snapshot) {
snapshot.forEach(function(item) {
var itemVal = item.val();
locations.push(itemVal);
});
callback(locations);
});
});
loadLocations(function(locations) {
console.log(locations.length);
});
A more modern variant of that last snippet is to return a promise, instead of passing in a callback.
function loadLocations() {
return new Promise(function(resolve, reject) {
var locations = []; // Empty array
var ref = db.ref("locations/");
ref.once("value", function(snapshot) {
snapshot.forEach(function(item) {
var itemVal = item.val();
locations.push(itemVal);
});
resolve(locations);
});
})
});
And you can then call it like this:
loadLocations().then(function(locations) {
console.log(locations.length);
});
Or with modern JavaScript you can use async / await and do:
let locations = await loadLocations()
console.log(locations.length);
Just keep in mind that this last snippet still has the same asynchronous behavior, and the JavaScript runtime (or transpiler) is just hiding it from you.
Placing a sleep function for about 300ms seems to fix the problem. I think it has something to do with syncing, though not entirely sure. It just needs some time to process the query and assign everything, I suppose.
Using await on read functions also seems to help.
What my code should do
I am creating a social media app using react native, and I have run into an problem with my code. I am trying to create a function which grabs all of the posts from all of the user's groups. To do this, I created a loop. The loop repeats once for every group the user is in. Every one loop gets the posts from one group. Every time the loop is called, a new function is called which gets the posts from a new group the user is in, then returns the posts back to the original function, which adds them to a full list of posts.
Problem
The function getting the posts doesn't return the posts. I think that the code is not waiting for the posts to be returned, and moves on. Basically, when I console.log the posts on the function that got them, I get the correct posts, but when I console.log the entire list of posts, I get nothing back.
My question
How can I wait for a value to be returned by a function, and not have the code instantly move on?
My code
runQ(group){
//this function actually gets the posts from the server (from firebase)
var items = [];
firebase.database().ref('posts/'+group).limitToLast(
Math.floor(24/this.state.passGroups.length)*this.state.numOfPosts
).orderByKey().once ('value', (snap) => {
snap.forEach ( (child) => {
items.push({
//post info
});
});
this.setState({passItems: items})
console.log(items); //logs correct items.
}).then(()=>{
if( this.state.passItems.length != 0 ){return this.state.passItems;}
})
}
//gets the user's groups, then sends out a request to each group for the newest posts.
getItems(){
//gets groups...
//...
.then(()=>{
var allItems = [];
//allItems will be the big list of all of the posts.
for (i = 0; i < this.state.passGroups.length; i++) {
//sending request to runQ function to get posts.
allItems.push(this.runQ(this.state.passGroups[i].name)) //allItems logs as allItems: ,,,,
}
})
}
Use async-await to make the for loop wait for each response.
First, you need to return the promise created by your Firebase call (you currently don't return anything from your runQ() function).
Change this line:
firebase.database().ref('posts/'+group).limitToLast(
into:
return firebase.database().ref('posts/'+group).limitToLast(
Then tell your callback to the getItems() call to be an async function and await each response from runQ():
getItems(){
//gets groups...
//...
.then(async () => {
var allItems = [];
for (var i = 0; i < this.state.passGroups.length; i++) {
allItems.push(await this.runQ(this.state.passGroups[i].name))
}
})
}
First you have to return a promise from runQ like this: return firebase.database.....
Then in the for loop you can do like this:
let allPromises = []
for (i = 0; i < this.state.passGroups.length; i++) {
allPromises.push(this.runQ(this.state.passGroups[i].name))
}
Promise.all(allPromises).then(itemsArray => {
allItems = allItems.concat(itemsArray)
})
Make sure that you have allItems declared in the right scope.
In the following code I save each item's key and an email address in one table, and to retrieve the object to fetch from the original table using said key. I can see that the items are being put into the rawList array when I console.log, but the function is returning this.cartList before it has anything in it, so the view doesn't receive any of the data. How can I make it so that this.cartList waits for rawList to be full before it is returned?
ionViewWillEnter() {
var user = firebase.auth().currentUser;
this.cartData.getCart().on('value', snapshot => {
let rawList = [];
snapshot.forEach(snap => {
if (user.email == snap.val().email) {
var desiredItem = this.goodsData.findGoodById(snap.val().key);
desiredItem.once("value")
.then(function(snapshot2) {
rawList.push(snapshot2);
});
return false
}
});
console.log(rawList);
this.cartList = rawList;
});
}
I have tried putting the this.cartList = rawList in a number of different locations (before return false, even inside the .then statement, but that did not solve the problem.
The following function call is asynchronous and you're falling out of scope before rawList has a chance to update because this database call takes a reasonably long time:
desiredItem.once("value").then(function(snapshot2) {
rawList.push(snapshot2);
});
You're also pushing the snapshot directly to this list, when you should be pushing snapshot2.val() to get the raw value.
Here's how I would fix your code:
ionViewWillEnter() {
var user = firebase.auth().currentUser;
this.cartData.getCart().on('value', snapshot => {
// clear the existing `this.cartList`
this.cartList = [];
snapshot.forEach(snap => {
if (user.email == snap.val().email) {
var desiredItem = this.goodsData.findGoodById(snap.val().key);
desiredItem.once("value")
.then(function(snapshot2) {
// push directly to the cartList
this.cartList.push(snapshot2.val());
});
}
return false;
});
});
}
The problem is the Promise (async .once() call to firebase) inside the forEach loop (sync). The forEach Loop is not gonna wait for the then() statement so then on the next iteration the data of the previous iteration is just lost...
let snapshots = [1, 2, 3];
let rawList = [];
snapshots.forEach((snap) => {
console.log(rawList.length)
fbCall = new Promise((resolve, reject) => {
setTimeout(function() {
resolve("Success!");
}, 2500)
});
fbCall.then((result) => {
rawList.push(result);
});
})
You need forEach to push the whole Promise to the rawList and Then wait for them to resolve and do sth with the results.
var snapshots = [1, 2, 3];
var rawList = [];
var counter = 0;
snapshots.forEach((snap) => {
console.log(rawList.length)
var fbCall = new Promise((resolve, reject) => {
setTimeout(function() {
resolve("Success!" + counter++);
}, 1500)
});
rawList.push(fbCall);
})
Promise.all(rawList).then((res) => {
console.log(res[0]);
console.log(res[1]);
console.log(res[2]);
});
The thing is, it is still a bit awkward to assign this.cartList = Promise.all(rawList) as it makes it a Promise. So you might want to rethink your design and make something like a getCartList Service? (dont know what ur app is like :p)
Since you're using angular you should also be using angularfire2, which makes use of Observables which will solve this issue for you. You will still be using the normal SDK for many things but for fetching and binding data it is not recommended to use Firebase alone without angularfire2 as it makes these things less manageable.
The nice things about this approach is that you can leverage any methods on Observable such as filter, first, map etc.
After installing it simply do:
public items$: FirebaseListObservable<any[]>;
this.items$ = this.af.database.list('path/to/data');
And in the view:
{{items$ | async}}
In order to wait for the data to appear.
Use AngularFire2 and RxJS this will save you a lot of time, and you will do it in the proper and maintainable way by using the RxJS operators, you can learn about those operators here learnrxjs
I have a webserver running in node.js and Express which retrieves data from mongodb . In mongodb collections are getting created dynamically and the name of newly created collection will be stored in one metadata collection “project” . My requirement is to firstly iterate to metadata collection to get the collection name and then get inside the each collection to do multiple query based on some condition . Because my collection metadata is dynamic I have tried to do using for loop .
But it is giving wrong data . It is not executing sequent . Before finishing the loop execution it is returning the value .How to perform sequential execution in node.js using node core modules only (Not other library like async..);
exports.projectCount = function (req, res) {
var mongo = require("mongodb"),
Server = mongo.Server,
Db = mongo.Db;
var server = new Server("localhost", 27017, {
auto_reconnect: true
});
var db = new Db("test", server);
// global JSON object to store manipulated data
var projectDetail = {
projectCount: 0,
projectPercent: 0
};
var totalProject = 0;
db.open(function (err, collection) {
//metadata collection
collection = db.collection("project");
collection.find().toArray(function (err, result) {
// Length of metadata collection
projectDetail.projectCount = result.length;
var count = 0;
//iterate through each of the array which is the name of collection
result.forEach(function (item) {
//change collection object to new collection
collection = db.collection(item.keyParameter.wbsName);
// Perform first query based on some condition
collection.find({
$where: "this.status == 'Created'"
}).toArray(function (err, result) {
// based on result of query one increment the value of count
count += result.lenght;
// Perform second query based on some condition
collection.find({
$where: "this.status=='Completed'"
}).toArray(function (err, result) {
count += result.length;
});
});
});
// it is returning the value without finishing the above manipulation
// not waiting for above callback and value of count is coming zero .
res.render('index', {
projectDetail: projectDetail.projectCount,
count: count
});
});
});
};
When you want to call multiple asynchronous functions in order, you should call the first one, call the next one in it's callback and so on. The code would look like:
asyncFunction1(args, function () {
asyncFunction2(args, function () {
asyncFunction3(args, function () {
// ...
})
})
});
Using this approach, you may end up with an ugly hard-to-maintain piece of code.
There are various ways to achieve the same functionality without nesting callbacks, like using async.js or node-fibers.
Here is how you can do it using node.js EventEmitter:
var events = require('events');
var EventEmitter = events.EventEmitter;
var flowController = new EventEmitter();
flowController.on('start', function (start_args) {
asyncFunction1(args, function () {
flowController.emit('2', next_function_args);
});
});
flowController.on('2', function (args_coming_from_1) {
asyncFunction2(args, function () {
flowController.emit('3', next_function_args);
});
});
flowController.on('3', function (args_coming_from_2) {
asyncFunction3(args, function () {
// ...
});
});
flowController.emit('start', start_args);
For loop simulation example:
var events = require('events');
var EventEmitter = events.EventEmitter;
var flowController = new EventEmitter();
var items = ['1', '2', '3'];
flowController.on('doWork', function (i) {
if (i >= items.length) {
flowController.emit('finished');
return;
}
asyncFunction(item[i], function () {
flowController.emit('doWork', i + 1);
});
});
flowController.on('finished', function () {
console.log('finished');
});
flowController.emit('doWork', 0);
Use callbacks or promises or a flow control library. You cannot program servers in node without understanding at the very least one of these approaches, and honestly all halfway decent node programmers thoroughly understand all three of them (including a handful of different flow control libraries).
This is not a something you are going to just get an answer coded for you by someone else on stackoverflow and then move on. This is a fundamental thing you have to go and study and learn generically as it is only going to come up over and over again on a daily basis.
http://howtonode.org/control-flow
http://callbackhell.com/
Per the resources in the answer above me, nesting the callback when you iterate and only calling it if you are on the last iteration will solve you problem.