Develop Reference

How and when to write entity relationships with Firebase Database [duplicate]

I've read the Firebase docs on Stucturing Data. Data storage is cheap, but the user's time is not. We should optimize for get operations, and write in multiple places.
So then I might store a list node and a list-index node, with some duplicated data between the two, at very least the list name.
I'm using ES6 and promises in my javascript app to handle the async flow, mainly of fetching a ref key from firebase after the first data push.
let addIndexPromise = new Promise( (resolve, reject) => {
let newRef = ref.child('list-index').push(newItem);
resolve( newRef.key()); // ignore reject() for brevity
});
addIndexPromise.then( key => {
ref.child('list').child(key).set(newItem);
});
How do I make sure the data stays in sync in all places, knowing my app runs only on the client?
For sanity check, I set a setTimeout in my promise and shut my browser before it resolved, and indeed my database was no longer consistent, with an extra index saved without a corresponding list.
Any advice?

Great question. I know of three approaches to this, which I'll list below.
I'll take a slightly different example for this, mostly because it allows me to use more concrete terms in the explanation.
Say we have a chat application, where we store two entities: messages and users. In the screen where we show the messages, we also show the name of the user. So to minimize the number of reads, we store the name of the user with each chat message too.
users
so:209103
name: "Frank van Puffelen"
location: "San Francisco, CA"
questionCount: 12
so:3648524
name: "legolandbridge"
location: "London, Prague, Barcelona"
questionCount: 4
messages
-Jabhsay3487
message: "How to write denormalized data in Firebase"
user: so:3648524
username: "legolandbridge"
-Jabhsay3591
message: "Great question."
user: so:209103
username: "Frank van Puffelen"
-Jabhsay3595
message: "I know of three approaches, which I'll list below."
user: so:209103
username: "Frank van Puffelen"
So we store the primary copy of the user's profile in the users node. In the message we store the uid (so:209103 and so:3648524) so that we can look up the user. But we also store the user's name in the messages, so that we don't have to look this up for each user when we want to display a list of messages.
So now what happens when I go to the Profile page on the chat service and change my name from "Frank van Puffelen" to just "puf".
Transactional update
Performing a transactional update is the one that probably pops to mind of most developers initially. We always want the username in messages to match the name in the corresponding profile.
Using multipath writes (added on 20150925)
Since Firebase 2.3 (for JavaScript) and 2.4 (for Android and iOS), you can achieve atomic updates quite easily by using a single multi-path update:
function renameUser(ref, uid, name) {
var updates = {}; // all paths to be updated and their new values
updates['users/'+uid+'/name'] = name;
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
updates['messages/'+messageSnapshot.key()+'/username'] = name;
})
ref.update(updates);
});
}
This will send a single update command to Firebase that updates the user's name in their profile and in each message.
Previous atomic approach
So when the user change's the name in their profile:
var ref = new Firebase('https://mychat.firebaseio.com/');
var uid = "so:209103";
var nameInProfileRef = ref.child('users').child(uid).child('name');
nameInProfileRef.transaction(function(currentName) {
return "puf";
}, function(error, committed, snapshot) {
if (error) {
console.log('Transaction failed abnormally!', error);
} else if (!committed) {
console.log('Transaction aborted by our code.');
} else {
console.log('Name updated in profile, now update it in the messages');
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.on('child_added', function(messageSnapshot) {
messageSnapshot.ref().update({ username: "puf" });
});
}
console.log("Wilma's data: ", snapshot.val());
}, false /* don't apply the change locally */);
Pretty involved and the astute reader will notice that I cheat in the handling of the messages. First cheat is that I never call off for the listener, but I also don't use a transaction.
If we want to securely do this type of operation from the client, we'd need:
security rules that ensure the names in both places match. But the rules need to allow enough flexibility for them to temporarily be different while we're changing the name. So this turns into a pretty painful two-phase commit scheme.
change all username fields for messages by so:209103 to null (some magic value)
change the name of user so:209103 to 'puf'
change the username in every message by so:209103 that is null to puf.
that query requires an and of two conditions, which Firebase queries don't support. So we'll end up with an extra property uid_plus_name (with value so:209103_puf) that we can query on.
client-side code that handles all these transitions transactionally.
This type of approach makes my head hurt. And usually that means that I'm doing something wrong. But even if it's the right approach, with a head that hurts I'm way more likely to make coding mistakes. So I prefer to look for a simpler solution.
Eventual consistency
Update (20150925): Firebase released a feature to allow atomic writes to multiple paths. This works similar to approach below, but with a single command. See the updated section above to read how this works.
The second approach depends on splitting the user action ("I want to change my name to 'puf'") from the implications of that action ("We need to update the name in profile so:209103 and in every message that has user = so:209103).
I'd handle the rename in a script that we run on a server. The main method would be something like this:
function renameUser(ref, uid, name) {
ref.child('users').child(uid).update({ name: name });
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
messageSnapshot.update({ username: name });
})
});
}
Once again I take a few shortcuts here, such as using once('value' (which is in general a bad idea for optimal performance with Firebase). But overall the approach is simpler, at the cost of not having all data completely updated at the same time. But eventually the messages will all be updated to match the new value.
Not caring
The third approach is the simplest of all: in many cases you don't really have to update the duplicated data at all. In the example we've used here, you could say that each message recorded the name as I used it at that time. I didn't change my name until just now, so it makes sense that older messages show the name I used at that time. This applies in many cases where the secondary data is transactional in nature. It doesn't apply everywhere of course, but where it applies "not caring" is the simplest approach of all.
Summary
While the above are just broad descriptions of how you could solve this problem and they are definitely not complete, I find that each time I need to fan out duplicate data it comes back to one of these basic approaches.

To add to Franks great reply, I implemented the eventual consistency approach with a set of Firebase Cloud Functions. The functions get triggered whenever a primary value (eg. users name) gets changed, and then propagate the changes to the denormalized fields.
It is not as fast as a transaction, but for many cases it does not need to be.

Best way to manage Chat channels in Firebase

In my main page I have a list of users and i'd like to choose and open a channel to chat with one of them.
I am thinking if use the id is the best way and control an access of a channel like USERID1-USERID2.
But of course, user 2 can open the same channel too, so I'd like to find something more easy to control.
Please, if you want to help me, give me an example in javascript using a firebase url/array.
Thank you!

A common way to handle such 1:1 chat rooms is to generate the room URL based on the user ids. As you already mention, a problem with this is that either user can initiate the chat and in both cases they should end up in the same room.
You can solve this by ordering the user ids lexicographically in the compound key. For example with user names, instead of ids:
var user1 = "Frank"; // UID of user 1
var user2 = "Eusthace"; // UID of user 2
var roomName = 'chat_'+(user1<user2 ? user1+'_'+user2 : user2+'_'+user1);
console.log(user1+', '+user2+' => '+ roomName);
user1 = "Eusthace";
user2 = "Frank";
var roomName = 'chat_'+(user1<user2 ? user1+'_'+user2 : user2+'_'+user1);
console.log(user1+', '+user2+' => '+ roomName);
<script src="https://getfirebug.com/firebug-lite-debug.js"></script>
A common follow-up questions seems to be how to show a list of chat rooms for the current user. The above code does not address that. As is common in NoSQL databases, you need to augment your data model to allow this use-case. If you want to show a list of chat rooms for the current user, you should model your data to allow that. The easiest way to do this is to add a list of chat rooms for each user to the data model:
"userChatrooms" : {
"Frank" : {
"Eusthace_Frank": true
},
"Eusthace" : {
"Eusthace_Frank": true
}
}
If you're worried about the length of the keys, you can consider using a hash codes of the combined UIDs instead of the full UIDs.
This last JSON structure above then also helps to secure access to the room, as you can write your security rules to only allow users access for whom the room is listed under their userChatrooms node:
{
"rules": {
"chatrooms": {
"$chatroomid": {
".read": "
root.child('userChatrooms').child(auth.uid).child(chatroomid).exists()
"
}
}
}
}

In a typical database schema each Channel / ChatGroup has its own node with unique $key (created by Firebase). It shouldn't matter which user opened the channel first but once the node (& corresponding $key) is created, you can just use that as channel id.
Hashing / MD5 strategy of course is other way to do it but then you also have to store that "route" info as well as $key on the same node - which is duplication IMO (unless Im missing something).

We decided on hashing users uid's, which means you can look up any existing conversation,if you know the other persons uid.
Each conversation also stores a list of the uids for their security rules, so even if you can guess the hash, you are protected.

Hashing with js-sha256 module worked for me with directions of Frank van Puffelen and Eduard.
import SHA256 from 'crypto-js/sha256'
let agentId = 312
let userId = 567
let chatHash = SHA256('agent:' + agentId + '_user:' + userId)

Retrieve groupchat history using strophe.js

I am using ejabberd 15.06 version with Strophe.js. Retrieving the one-to-one chat from my backend database works fine. But how can I retrieve the groupchat history from the database??
For example, if I have a "strophe" group. When new users joins in the strophe group, then the chat history done in the group by other users should be displayed.
I am using this code
var pres = $pres({ to: room + "/" + nickname, from: connection.jid });
connection.send( msg.c('x', {xmlns: NS_MUC}));
if(chat_history != null){
var msg_history = msg.c('x', { "xmlns": "http://jabber.org/protocol/muc"}).c("history", chat_history, {maxstanzas: 50});
debugger;
console.log(msg_history);
}
In my console it looks like
h.Builder {nodeTree: presence, node: x}
I am stuck how to fetch the history of groupchat. Please help

Usually, unless the room has been configured not to send any history, send the join presence should be enough to let you receive the latest chat room messages.
Please, note that old messages have a delay tag on them to provide the time at which the original message was send, so make sure your client is not discarding those messages.
If you want control about the history size, you can use the Strophe MUC plugin to join the room and send the max stanzas and time limit as the history_attrs variable. Your server and room must also be configured to provide the history.
conn.muc.join(room, nick, msg_handler_cb, pres_handler_cb, roster_cb, password,{ maxstanzas: 10, seconds: 3600 });

How to write denormalized data in Firebase

I've read the Firebase docs on Stucturing Data. Data storage is cheap, but the user's time is not. We should optimize for get operations, and write in multiple places.
So then I might store a list node and a list-index node, with some duplicated data between the two, at very least the list name.
I'm using ES6 and promises in my javascript app to handle the async flow, mainly of fetching a ref key from firebase after the first data push.
let addIndexPromise = new Promise( (resolve, reject) => {
let newRef = ref.child('list-index').push(newItem);
resolve( newRef.key()); // ignore reject() for brevity
});
addIndexPromise.then( key => {
ref.child('list').child(key).set(newItem);
});
How do I make sure the data stays in sync in all places, knowing my app runs only on the client?
For sanity check, I set a setTimeout in my promise and shut my browser before it resolved, and indeed my database was no longer consistent, with an extra index saved without a corresponding list.
Any advice?

Great question. I know of three approaches to this, which I'll list below.
I'll take a slightly different example for this, mostly because it allows me to use more concrete terms in the explanation.
Say we have a chat application, where we store two entities: messages and users. In the screen where we show the messages, we also show the name of the user. So to minimize the number of reads, we store the name of the user with each chat message too.
users
so:209103
name: "Frank van Puffelen"
location: "San Francisco, CA"
questionCount: 12
so:3648524
name: "legolandbridge"
location: "London, Prague, Barcelona"
questionCount: 4
messages
-Jabhsay3487
message: "How to write denormalized data in Firebase"
user: so:3648524
username: "legolandbridge"
-Jabhsay3591
message: "Great question."
user: so:209103
username: "Frank van Puffelen"
-Jabhsay3595
message: "I know of three approaches, which I'll list below."
user: so:209103
username: "Frank van Puffelen"
So we store the primary copy of the user's profile in the users node. In the message we store the uid (so:209103 and so:3648524) so that we can look up the user. But we also store the user's name in the messages, so that we don't have to look this up for each user when we want to display a list of messages.
So now what happens when I go to the Profile page on the chat service and change my name from "Frank van Puffelen" to just "puf".
Transactional update
Performing a transactional update is the one that probably pops to mind of most developers initially. We always want the username in messages to match the name in the corresponding profile.
Using multipath writes (added on 20150925)
Since Firebase 2.3 (for JavaScript) and 2.4 (for Android and iOS), you can achieve atomic updates quite easily by using a single multi-path update:
function renameUser(ref, uid, name) {
var updates = {}; // all paths to be updated and their new values
updates['users/'+uid+'/name'] = name;
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
updates['messages/'+messageSnapshot.key()+'/username'] = name;
})
ref.update(updates);
});
}
This will send a single update command to Firebase that updates the user's name in their profile and in each message.
Previous atomic approach
So when the user change's the name in their profile:
var ref = new Firebase('https://mychat.firebaseio.com/');
var uid = "so:209103";
var nameInProfileRef = ref.child('users').child(uid).child('name');
nameInProfileRef.transaction(function(currentName) {
return "puf";
}, function(error, committed, snapshot) {
if (error) {
console.log('Transaction failed abnormally!', error);
} else if (!committed) {
console.log('Transaction aborted by our code.');
} else {
console.log('Name updated in profile, now update it in the messages');
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.on('child_added', function(messageSnapshot) {
messageSnapshot.ref().update({ username: "puf" });
});
}
console.log("Wilma's data: ", snapshot.val());
}, false /* don't apply the change locally */);
Pretty involved and the astute reader will notice that I cheat in the handling of the messages. First cheat is that I never call off for the listener, but I also don't use a transaction.
If we want to securely do this type of operation from the client, we'd need:
security rules that ensure the names in both places match. But the rules need to allow enough flexibility for them to temporarily be different while we're changing the name. So this turns into a pretty painful two-phase commit scheme.
change all username fields for messages by so:209103 to null (some magic value)
change the name of user so:209103 to 'puf'
change the username in every message by so:209103 that is null to puf.
that query requires an and of two conditions, which Firebase queries don't support. So we'll end up with an extra property uid_plus_name (with value so:209103_puf) that we can query on.
client-side code that handles all these transitions transactionally.
This type of approach makes my head hurt. And usually that means that I'm doing something wrong. But even if it's the right approach, with a head that hurts I'm way more likely to make coding mistakes. So I prefer to look for a simpler solution.
Eventual consistency
Update (20150925): Firebase released a feature to allow atomic writes to multiple paths. This works similar to approach below, but with a single command. See the updated section above to read how this works.
The second approach depends on splitting the user action ("I want to change my name to 'puf'") from the implications of that action ("We need to update the name in profile so:209103 and in every message that has user = so:209103).
I'd handle the rename in a script that we run on a server. The main method would be something like this:
function renameUser(ref, uid, name) {
ref.child('users').child(uid).update({ name: name });
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
messageSnapshot.update({ username: name });
})
});
}
Once again I take a few shortcuts here, such as using once('value' (which is in general a bad idea for optimal performance with Firebase). But overall the approach is simpler, at the cost of not having all data completely updated at the same time. But eventually the messages will all be updated to match the new value.
Not caring
The third approach is the simplest of all: in many cases you don't really have to update the duplicated data at all. In the example we've used here, you could say that each message recorded the name as I used it at that time. I didn't change my name until just now, so it makes sense that older messages show the name I used at that time. This applies in many cases where the secondary data is transactional in nature. It doesn't apply everywhere of course, but where it applies "not caring" is the simplest approach of all.
Summary
While the above are just broad descriptions of how you could solve this problem and they are definitely not complete, I find that each time I need to fan out duplicate data it comes back to one of these basic approaches.

To add to Franks great reply, I implemented the eventual consistency approach with a set of Firebase Cloud Functions. The functions get triggered whenever a primary value (eg. users name) gets changed, and then propagate the changes to the denormalized fields.
It is not as fast as a transaction, but for many cases it does not need to be.

Sending push notification to specific device knowing phone number, parse.com

I would just to get things clear here or get other suggestions if possible if it is better.
This is how my application works now:
1) Anonymous user is created if its the first time the user open the application
2) Phone verification is needed to be done. If verified, i save the phone number in a custom field in user object ( do i need to make this user a real user after this or can i still go with anonymous user?)( verification is one time only of course)
3) The user will be able to pick a friend from his contact list(ABPeoplePicker) and then send a push notification to that friend's device.
Now i have set up a relationship with the User and the installation object with this code:
PFInstallation *installation = [PFInstallation currentInstallation];
installation[#"user"] = [PFUser currentUser];
[installation saveInBackground];
And this created a pointer to the users ObjectId
So my question is how would i create a query and send a push notification to a number retrieved from that Users friend list?. I am having a hard time to connect how i can get from a phone number to the installation device that i need to send the notification to. If you could provide help in javascript since i read it is safer to send it through cloud code!
Also a subquestion mentioned above if i need to make the anonymous user to a real user.
many thanks!!

I'd recommend subscribing each user to their own channel where the channel name is equal to their phone number (Subscription can't be done in Javascript):
NSString *userPhoneNumber = ...
PFInstallation *currentInstallation = [PFInstallation currentInstallation];
// "x" added to the beginning of the userPhoneNumber since
// Parse doesn't allow channels to begin with a number
[currentInstallation addUniqueObject:[NSString stringWithFormat:#"x%#", userPhoneNumber] forKey:#"channels"];
[currentInstallation saveInBackground];
That way no query is required before pushing the notification:
var friendPhoneNumber = ...
var friendChannel = "x" + friendPhoneNumber;
Parse.Push.send({
channels: [ friendChannel ],
data: {
alert: message
}
}, {
success: function() {
// Push was successful
},
error: function(error) {
// Handle error
}
});