Develop Reference

Issue using Cloud Functions to sync realtime database with Firestore

I'm using an extremely simple Cloud Function to (try to) keep my realtime database in sync with the Firestore.
exports.copyDocument = functions.database.ref('/invoices/{companyId}/{documentId}')
.onWrite((change, context) => {
if (!change.after.exists()) {
return null;
}
return admin.firestore().collection('companies').doc(context.params.companyId).collection('invoices')
.doc(context.params.documentId)
.set(change.after.val());
});
Unfortunately, I am seeing issues where sometimes the Cloud Firestore document does not have the latest copy of the realtime DB data. It's infrequent, but nonetheless impacting my end users.
Why is this happening?
Two ideas I had were -
The Firestore is possibly unavailable to write to in the Cloud Function, and I don't have retries enabled in my Cloud Function, so it could just be erroring out. Enabling retries will solve my problem. However, I have absolutely 0 error logs of this happening.
User makes change A, then user makes change B 2-3 seconds later. In this case, it's possible, I guess, that the Cloud Function trigger for change A hasn't executed, but the trigger for change B quickly executes, then the change A trigger executes and copies 'stale' data. Possible remedies would be to fetch the latest version again from the realtime database in my Cloud Function (not ideal, it's nice using change.after.val()), or perhaps keep some incrementing integer on my document, and instead copy it to the Firestore using a transaction that compares versions instead of a simple set().
The only error message I do see in my Cloud Function error logs is:
The request was aborted because there was no available instance. Additional troubleshooting documentation can be found at: https://cloud.google.com/functions/docs/troubleshooting#scalability
But those docs indicate that this error is always retried, even without explicitly enabling function retries.
I am leaning towards issue 1 - the Firestore just 'blips' sometimes, retries aren't enabled, and I'm not logging the failed promise properly. If this is the case - how can I fix this logging?

What happens when an onCreate event happens again while another Firebase function is already running?

Background
I have a Firebase Cloud Function which sometimes can take a while to complete (20-60 seconds).
It gets triggered by a write to Firebase, which means that it starts processing at the onCreate event.
Problem
I have been doing some scripted testing by creating a new record in Firebase every N seconds, but it seems that if N is less than 20 seconds, the next onCreate trigger just doesn't fire.
In other words I end up in a situation like this:
Firebase:
record1
record2
record3
record4
Results written by the triggered function to another node in Firebase:
result-from-record1
...
record2, record3, record4 does not seem to trigger the function again.
Homework
I have re-checked Firebase documentation, but I cannot seem to find any information that explains this case.
There is some information about quotas for connected users, but it's only about connected users, not about the same triggers firing many times before the previously triggered function completes.
Questions
What is the default behavior of Firebase triggered functions in case it gets triggered while the previously triggered function is still running?
Is there any way to maybe cancel the running function if it gets triggered by a new onWrite?
Is there any queue of those triggered and running functions? (this queue doesn't seem to be the one)

What is the default behavior of Firebase triggered functions in case it gets triggered while the previously triggered function is still running?
There is no guarantee about how functions are invoked - they could happen in sequence on a single server instance, or they could run in run in parallel on multiple server instances. The order of invocation of functions is also not guaranteed.
Is there any way to maybe cancel the running function if it gets triggered by a new onWrite?
No.
Is there any queue of those triggered and running functions? (this queue doesn't seem to be the one)
There is no visible queue. Internally, Cloud Functions is using pubsub to manage the stream of events emitted by the database, but this is an implementation detail, and you have no direct control over how it works.
As for why your function doesn't seem to execute when you expect - there's not enough detail in your question to make a guess. Without seeing actual code, as well as the specific steps to take to reproduce the issue, it's not possible to say.
You might want to watch my video series on how Cloud Functions works in order to better understand its behavior.

Which event is triggered when we use ref.push to write in firebase database javascript

When we are working in firebase using javascript which event is triggered after we insert data using ref.push or ref.set.
I wanted to know if my data is inserted or not
I also wanted to throw an error when user have disconnected from internet while inserting data in firebase
I haven't seen any function or any method in internet which tells me about if data is successfully inserted or not.

This functions Promise-based, so you can use try/catch:
try {
firebase.push(data) // or set
} catch (error) {
console.log(error) // here is error
}

The Firebase Realtime Database doesn't consider a lack of internet connection an error condition. Instead it continues to work to its best ability in the given conditions.
When you perform a write operation (with set, push, update, or remove) while there is no internet connectivity:
The first client fires local events immediately, so that your app can update the UI for the new/updated data.
It then queues the write operation for delivery once the connection is restored.
Once the connection is restored, the client sends any pending write operations it has in the order in which the client performed them.
It then handles the response from the server, which (if the server rejects the operation because of security rules) may lead to firing more local events so that the app can put the UI back into the correct sate.
And it then finally calls any completion listeners, and resolves or rejects the promise for the set(), push(), update(), or remove() method.
You'll note that there is no error raised at any point for a lack of an internet connection.
If you don't want to send any data to the local queue when the app has no internet connection, it's best to detect if the Firebase client is connected to the server. You can do this by listening to the .info/connected pseudo-node. This covers more than just having an internet connection btw, but also cases where the internet connections works but the client can't reach Firebase. The best practice here is to use a "global" listener for this status, and disable the relevant UI elements if the client is not connected.

duplicates in socket.io w/ angular

Using angular and socket.io I am getting duplicate events on the client everytime the server emits. Angular is only included once, there is only one socket.io connection, and there is only one listener per event on the client. Upon receiving an event on the server, data is logged, and this process only ever happens once. Then the data is emitted and the callback is called twice on the client, despite only being in scope once(to my knowledge).
client:
//inside a controller
var thing ='foo';
socket.emit('sentUp',thing)
socket.on('sentDown',function(thing){
console.log(thing)//this happens twice
});
server:
/*
node&express stuff here
*/
socket.on('connection',function(socket){
socket.on('sentUp',function(stuff){
console.log('this happened')//x1
socket.emit('sendDown',stuff);
});
})

Most likely your controllers are being loaded more than once. You can easily check it by logging.
Move out the socket code from the controllers and put them in a service where they're only called once.

I have found in my own socket.io client code that some of the connect events can occur each time the client reconnects. So, if the connected is lost for any reason and then the client automatically reconnects, the client may get a connect event again.
If, like me, you're adding your event handlers in the 'connect' event, then you may be accidentially adding multiple event handlers for the same event and thus you would think you were seeing duplicate data. You don't show that part of your client code so I don't know you're doing it that way, but this is an issue that hit me and it is a natural way to do things.
If that is what is happening to you, there are a couple possible work-arounds:
You can add your event handlers outside the connect event. You don't have to wait for connection to finish before adding event handlers. This way, you'd only ever do them once.
Before adding the event handlers you add upon connection, you can remove any previous event handlers that were installed upon connection to make sure you never get dups.

Firebase Callbacks - what's the underlying trigger?

I understand that in Firebase I can register my page for callbacks with the "on" method.
According to their docs:
on( ) is used to listen for data changes at a particular location.
This is the primary way to read data from Firebase.
firebaseRef.on('value', function(dataSnapshot) {
// code to handle new value.
});
My question is:
How does it work ?
How does it know that something has changed on the serverside ?
(better) How does the server can 'callback' the browser ?
One answer might be that it is "polling". But I have seen no reference about this approach in Firebase documentation or properties to configure polling time ...
Does anybody know ?
Many
Thanks

Firebase uses WebSockets to allow the server to "push" data to the client. Since not all browser versions support WebSockets yet, it also falls back to long polling for those browsers.
The implementation details of how that works on the server are proprietary and sophisticated--enough to write a book about and beyond the scope of a SO question. Logically, works exactly as advertised: The service is designed so that any time a set(), push(), or update() is called (or the REST equivalents), it notifies any listeners of the change.
Regardless of whether the browser uses WebSockets or not, there is no "polling time" as the client is not repeatedly contacting the server. Long polling means waiting for a data change to occur, rather than polling repeatedly to see if a change has occurred. As you can see by trying out the tutorial or any of the real-time examples, data changes are synced to all clients in a matter of milliseconds--nothing to configure.