If I have a database structure like here and I make a query as shown below.Is there a difference on the traffic used to download the snapshot from the database if I access each node with snapshot.forEach(function(childSnapshot) and if I don't access the nodes?
If there is no difference, is there a way to access only the keys in Chats without getting a snapshot data for what each key contains.I'm assuming that this way it will generate less downloaded data
var requests = db.ref("Chats");
requests.on('child_added', function(snapshot) {
var communicationId = snapshot.key;
console.log("Chat id = " + communicationId);
getMessageInfo(
communicationId,
function() {
snapshot.ref.remove();
}
);
When you call requests.on('child_added', ...), you are always going to access all of the data at the requests node. It doesn't matter what you do in the callback function. The entire node is loaded into memory, and cost of the query is paid. What you do with the snapshot in memory doesn't cost anything else.
If you don't want all of the child nodes under requests, you should find some way to filter the query for only the children you need.
As they mentioned in the documentation, either of these methods can be used:
Call a method to get the data.
Set a listener to receive data-change
events.
Traffic depends upon our usage. When your data need not get updated in realtime, you can just call a method to get the data (1) But if you want your data to be updated in realtime, then you should go for (2). When you set a listener, Firebase sends your listener an initial snapshot of the data, and then another snapshot each time the child changes.
(1) - Example
firebase.database().ref('/users/').once('value') // Single Call
(2) - Example
firebase.database().ref('/users/').on('child_added') // Every Update It is Called
And also, I think you cannot get all keys, because when you reference a child and retrieve a data, firebase itself sends it as key-value pairs (DataSnapshot).
Further Reference: https://firebase.google.com/docs/reference/js/firebase.database.DataSnapshot
https://firebase.google.com/docs/database/web/read-and-write
Related
How to retrieve only one recently added child value from firebase without timestamp comparision?
I want to get the same key/value (only one child, not full array of child nodes) on the client side while adding it through my firebase console.
{users:
"id-ue3": "bob",
"id-om4": "john", // <-- recently added
"id-ker": "kevin"
}
firebase.database().ref().on('child_changed', function(snapshot) {
....
console.log(last_name); // output --> "john"
});
There is no way in your JSON to know what data was most recently added. Since the Firebase Realtime Database doesn't keep any metadata about when data was added, you'll need to include the necessary information in your JSON.
Two ways to do that:
Add a timestamp property to your data. You can either use the client-side time for this, or let Firebase generate a server-side timestamp when it writes the data into the database.
Encode the necessary information in the key of the items by adding them with push(). For more on these IDs, see this blog post about push IDs.
Note that you'll likely need to listen to the child_added event. The child_changed event only fires when you update a child node, not when you add a new child node.
I have a users table on Firebase and each user has an email prop.
Structure looks like:
Users -> User UID (looks like n8haBbjgablobA2ranfuabu3aaaga2af) -> User Obj which includes email prop.
I'd like to get an array of all the users' emails (~1m).
How can I most efficiently do this?
Ps.:
I tried:
usersRef.startAt(0).endAt(20).once("value", function(snapshot) {
console.log('FIRST 20');
console.log(snapshot.val()); // null
});
But that fails.
Probably the most efficient approach in terms of data reads would be to denormalize your data. You could store the email addresses both in the individual user nodes and in an emailAddresses node. Then you could just query the emailAddresses node directly for your list of emails.
Still ~1m email address nodes would probably be too much all at once. I'd probably grab it in chunks... I'm guessing.
Update
"Grabbing in chunks" is essentially pagination. I would try to use something off the shelf before trying to roll my own pagination solution.
Pagination libraries to check out:
Firebase Utils Pagination: This is developed by Firebase, but they say it is experimental and not ready for production use. But, it's probably still worth messing around with.
firebase-paginator: this is developed by a community member and it seems pretty solid.
If you want to roll your own pagination, check out:
#kato's response in this StackOverflow answer He makes an interesting point about the potential problem with paginating a real time data set, but then provides some good starter code
Here's a good blog entry that talks about the code that I think is a part of the firebase-paginator library I linked to above
Everybody in their answers said that it was an easy thing, yet had no working solutions. Here's what I came up with:
usersRef.orderByChild('uid').limitToFirst(100).once('value', function (snapshot) {
var users = snapshot.val()
var uids = Object.keys(users);
var lastUid = uids[uids.length - 1];
// could be another property than uid, for example email, or username. Ps.: Consider that the last ID from the previous chunk will be duplicated.
usersRef.orderByChild('uid').startAt(lastUid).limitToFirst(100).once('value', function (snapshot) {
var users = snapshot.val()
var uids = Object.keys(users);
console.log(uids);
var lastUid = uids[uids.length - 1];
// re-run function until done
})
})
Since this is a one-time deal, an option is to simply iterate over each node in the parent 'data' node capturing the child data, stripping out the email address and dumping that to a file.
the event you want is
child_added: retrieve lists of items or listen for additions to a list
of items. This event is triggered once for each existing child and
then again every time a new child is added to the specified path. The
listener is passed a snapshot containing the new child's data.
and the code to iterate all of the child nodes in the data node is
var dataRef = firebase.database().ref('myRootRef/data');
datRef.on('child_added', function(data) {
//data.val() will contain the child data, such as the email address
//append it to a text file here (for example), save to disk etc.
});
The key here is that this event is triggered once for each child, similar to iterating over all of the indexes in an array.
This will retrieve each child and present it to your app, one child at a time, iterating over all the children within the node.
It's going to take a while with that many nodes to chew through.
I have a node in Firebase getting continually updated with information from a logfile. The node is lines/ and each child of lines/ is from a post() so it has a unique ID.
When a client first loads, I want to be able to grab the last X number of entries. I expect I'll do this with once(). From then on, however, I want to use an on() with child_added so that I get all new data. However, child_added gets all data stored in the Firebase and, after the initial setup, only want the new stuff.
I see that I can add a limitToLast() on the on(), but, if I say limitToLast(1) and a flood of entries come in, will my app still get all the new entries? Is there some other way to do this?
You need to include a timestamp property and run a query.
// Get the current timestamp
var now = new Date().getTime();
// Create a query that orders by the timestamp
var query = ref.orderByChild('timestamp').startAt(now);
// Listen for the new children added from that point in time
query.on('child_added', function (snap) {
console.log(snap.val()
});
// When you add this new item it will fire off the query above
ref.push({
title: "hello",
timestamp: Firebase.ServerValue.TIMESTAMP
});
The Firebase SDK has methods for ordering, orderByChild() and methods for creating a range startAt(). When you combine the two you can limit what comes back from Firebase.
I think there is a problem in #David East's solution. He is using the local timestamp which may cause problem if the time is not accurate in client device. Here is my suggested solution (iOS Swift):
Using observeSingleEvent to get the complete data set
Then returned it in reversed order by reversed()
Get the last timestamp by for example data[0].timestamp
Using queryStarting for timestamp
self._dbref.queryOrdered(byChild: "timestamp").queryStarting(atValue: timestamp+1)
.observe(.childAdded, with: {
snapshot in
print(snapshot.value)
})
You have the right idea. child_added should be called only for the new nodes. Without source code it's hard to tell why you get all the data in your child_added event.
You can check the chat demo app to see how they load new chat messages. The use case sounds similar.
https://github.com/firebase/firechat/blob/master/src/js/firechat.js#L347
Here's temporary but quick solution:
// define a boolean
var bool = false;
// fetch the last child nodes from firebase database
ref.limitToLast(1).on("child_added", function(snap) {
if (bool) {
// all the existing child nodes are restricted to enter this area
doSomething(snap.val())
} else {
// set the boolean true to doSomething with newly added child nodes
bool = true;
}
});
Disadvantage: It will load all the child nodes.
Advantage: It will not process existing child nodes but just the newly added child nodes.
limitToLast(1) will do the work.
I have a table called Roster where it stores a User Pointer in one of the column. I am trying set that column in before save method when it is new.So far i have this but i am not sure how to get the user id
Parse.Cloud.beforeSave("Roster",function(request,response){
//Update roster with sender Id
if (request.object.isNew()){
var userPointer = /*NOT SURE HOW TO GET WHO SENT IT*/
request.object.set("User",userPointer);
}
response.success();
});
Any idea?
var userPointer = request.user;
Is the proper method. Todd's answer works on the client side, but not on cloud code, where beforeSave triggers occur.
If you need to access any of the user's information beyond their id, you'll have to first fetch the user, as the entire object is not sent in the request.
Edit - Just wanted to add that before/afterSave triggers have a 3 second timeout. This is enough time to perform a quick query or two, but if you have a lot of objects in your database, or perform many save/fetch/query calls, you may end up exceeding your 3 second limit. If you have a lot of that logic that needs to occur, rather than saving the object from the client, call a cloud code function that handles all of those changes, then saves the object and returns the newly saved object, so that you can set your client side object to the returned, up to date object.
As Jake T pointed out, you will need to use
var user = request.user
Parse.User.current() is not supported in a cloud code environment.
So I'm trying to go through one Firebase database to find entries in the database matching a criteria. Therefore I'm using the deferred object of jQuery to handle the database calls.
Once I get a return value from this first database I want to get the user info from a second database for each of those values in the first db. Then the results are added to a JSON array
so its:
<search for value, find one>
<<<search other db for oher info>>>
<continue search for outer value>
But this only returns one value - although everything else is running fine (and the console logs all the info correct).
Here's the code:
function find(searchLocation, profileID) {
var requestUserData = {
data: []
};
var def = $.Deferred();
//This will be executed as long as there are elements in the database that match the criteria and that haven't been loaded yet (so it's a simple loop)
Ref.orderByChild("location").equalTo(searchLocation).on("child_added", function(snapshot) {
def.ressolve(snapshot.val().ID);
});
return def.promise();
};
I hope you guys have any ideas on what to do or how I could solve this. Thanks in advance!
Edit: upon further testing I discovered that this problem already exists in the outer loop - so only the first value is being returned. I think this is related to the posission of the resolve() method but I didn't find a posibility on how to change this behaviour.
Firebase is a real-time database. The events stream as changes occur at the server. You're attempting to take this real-time model and force it into CRUD strategy and do a GET operation on the data. A better solution would be to simply update the values in real-time as they are modified.
See AngularFire, ReactFire, or BackboneFire for an example of how you can do this with your favorite bindings framework.
To directly answer the question, if you want to retrieve a static snapshot of the data, you want to use once() callback with a value event, not a real-time stream from child_added:
Ref.orderByChild("location").equalTo(searchLocation).once("value", function(snapshot) {
def.resolve(snapshot.val());
});