I have written a cloud function that is working well.
Sometimes this function is being executed more than one time for the same user (I made sure that only one request is being requested from the client android app).
After some debugging I noticed that this problem happens if the connection is bad. I may be or may not be correct.
How to overcome such an issue?
As in the comments I also don't believe the client SDKs would duplicate a Cloud Function call on a bad connection. The expected behaviour would be for them to throw a network-related exception on the client side and not call it again. A problem would arise if the Cloud Function runs successfully and the client is only unable to get the result back.
I can think of the following solutions, with no more details about the Cloud Function itself:
Try to make the function idempotent - Meaning that even if it runs twice the end result is the same, assuming the same input/parameters.
Cache the results and manually throttle the function - This is more complicated, and is only needed if the network problem persists and you can't work around eliminating the side effects of the function. You would have to create another Class to cache the results of the function (maybe keyed by the parameters) and returning the same result with no computation and side effects.
Related
I'm using an extremely simple Cloud Function to (try to) keep my realtime database in sync with the Firestore.
exports.copyDocument = functions.database.ref('/invoices/{companyId}/{documentId}')
.onWrite((change, context) => {
if (!change.after.exists()) {
return null;
}
return admin.firestore().collection('companies').doc(context.params.companyId).collection('invoices')
.doc(context.params.documentId)
.set(change.after.val());
});
Unfortunately, I am seeing issues where sometimes the Cloud Firestore document does not have the latest copy of the realtime DB data. It's infrequent, but nonetheless impacting my end users.
Why is this happening?
Two ideas I had were -
The Firestore is possibly unavailable to write to in the Cloud Function, and I don't have retries enabled in my Cloud Function, so it could just be erroring out. Enabling retries will solve my problem. However, I have absolutely 0 error logs of this happening.
User makes change A, then user makes change B 2-3 seconds later. In this case, it's possible, I guess, that the Cloud Function trigger for change A hasn't executed, but the trigger for change B quickly executes, then the change A trigger executes and copies 'stale' data. Possible remedies would be to fetch the latest version again from the realtime database in my Cloud Function (not ideal, it's nice using change.after.val()), or perhaps keep some incrementing integer on my document, and instead copy it to the Firestore using a transaction that compares versions instead of a simple set().
The only error message I do see in my Cloud Function error logs is:
The request was aborted because there was no available instance. Additional troubleshooting documentation can be found at: https://cloud.google.com/functions/docs/troubleshooting#scalability
But those docs indicate that this error is always retried, even without explicitly enabling function retries.
I am leaning towards issue 1 - the Firestore just 'blips' sometimes, retries aren't enabled, and I'm not logging the failed promise properly. If this is the case - how can I fix this logging?
When we are working in firebase using javascript which event is triggered after we insert data using ref.push or ref.set.
I wanted to know if my data is inserted or not
I also wanted to throw an error when user have disconnected from internet while inserting data in firebase
I haven't seen any function or any method in internet which tells me about if data is successfully inserted or not.
This functions Promise-based, so you can use try/catch:
try {
firebase.push(data) // or set
} catch (error) {
console.log(error) // here is error
}
The Firebase Realtime Database doesn't consider a lack of internet connection an error condition. Instead it continues to work to its best ability in the given conditions.
When you perform a write operation (with set, push, update, or remove) while there is no internet connectivity:
The first client fires local events immediately, so that your app can update the UI for the new/updated data.
It then queues the write operation for delivery once the connection is restored.
Once the connection is restored, the client sends any pending write operations it has in the order in which the client performed them.
It then handles the response from the server, which (if the server rejects the operation because of security rules) may lead to firing more local events so that the app can put the UI back into the correct sate.
And it then finally calls any completion listeners, and resolves or rejects the promise for the set(), push(), update(), or remove() method.
You'll note that there is no error raised at any point for a lack of an internet connection.
If you don't want to send any data to the local queue when the app has no internet connection, it's best to detect if the Firebase client is connected to the server. You can do this by listening to the .info/connected pseudo-node. This covers more than just having an internet connection btw, but also cases where the internet connections works but the client can't reach Firebase. The best practice here is to use a "global" listener for this status, and disable the relevant UI elements if the client is not connected.
I have a JavaScript client which connects to my local Parse server just fine. Login works, Parse.Query works, however calling any Cloud Code function with Parse.Cloud.run does not work -- instead I get the error 100/XMLHttpRequest failed. Calling CC functions from commandline using curl does work, indicating that the overall server setup is correct.
It's obviously a client configuration problem on my part, but I just can't figure out what the problem might be. Client setup is simple enough:
Parse.initialize('myappid');
Parse.serverURL = 'http://localhost:1337/parse';
Any ideas?
The problem was that my CC funtion requires a Date argument but I passed in a moment object... now I converted it to Date and all is well again. Silly me, but 100/XMLHttpRequest failed/ConnectionFailed is also a very misleading error message for this!
Keep in mind, I'm running an old version of AngularJS (1.0?), so things may have changed, but I have code that looks like:
promise = $http.get(urlFormattedString).success(function (data) {
data.forEach(function (result) {
//do something with result and $scope});
promises.push(promise);
$q.all(promises).then(function (data) {
//do something good when everything works!
};
When no errors are thrown, everything "works", but my question is what happens when one of the promises throws an error (say 1 out of 20)? Let's make this more interesting (and closer to my application) and assume that each promise is requesting data from a database (MongoDB in my case).
If I have to re-run everything, does that mean necessarily that all the data needs to be fetched again? Am I relying on the database to cache the data so the repeated requests run much faster? Or maybe the server (NodeJS in my case) caches the data? Along these lines, when are the data actually sent to the client from the server? Is it only upon success of all promises or is the data returned to the client from each promise separately? And if so, does Angular do the caching?
Just laying this out here makes me realize this is pretty complex and lots of scenarios to consider. Would appreciate any help or pointers to reading/documentation on this subject. Thanks!
Suggest you familiarize yourself with the network tab of your browser dev tools.
You can see every request made for all resources from html to images to scripts as well as ajax requests. This will give you a far better feel for how the app works in general
As for errors ... unless you implement error handling your app will simply fail for that request with no indication given to user that anything went wrong.
As for caching ... your ajax requests won't be cached client side and unless you have implemented caching mechaanisms on server they won't be cached there either
I have breezeJs running in an angular app on mobile device (cordova), which talks to .Net WebApi.
Everything works great, except once in a while the device will get PrimaryKey violations (from my SQL Server).
I think I narrowed it down to only happening when data connection is shakey on the device.
The only way I can figure these primary key violations are happening is somehow the server is Saving Changes, but the mobile connection drops out before the response can come back from server that everything saved OK.
What is supposed to happen when BreezeJS doesn't hear back from server after calling SaveChanges?
Anyone familiar with BreezeJS know of a way to handle this scenario?
I've had to handle the same scenario in my project. The approach I took was two part:
Add automatic retries to failed ajax requests. I'm using breeze with jQuery, so I googled "jQuery retry ajax". There's many different implementations, mine is somewhat custom, all center around hijacking the onerror callback as well as the deferred's fail handler to inject retry logic. I'm sure Angular will have similar means of retrying dropped requests.
In the saveChanges fail handler, add logic like this:
...
function isConcurrencyException(reason: any) {
return reason && reason.message && /Store update, insert, or delete statement affected an unexpected number of rows/.test(reason.message);
}
function isConnectionFailure(reason: any): boolean {
return reason && reason.hasOwnProperty('status') && reason.status === 0
}
entityManager.saveChanges()
.then(... yay ...)
.fail(function(reason) {
if (isConnectionFailure(reason)) {
// retry attempts failed to reach server.
// notify user and save to local storage....
return;
}
if (isConcurrencyException(reason)) {
// EF is not letting me save the entities again because my previous save (or another user's save) moved the concurrency stamps on the record. There's also the possibility that a record I'm try to save was deleted by another user.
// recover... in my case I kept it simple and simply attempt to reload the entity. If nothing is returned I know the entity was deleted. Otherwise I now have the latest version. In either case a message is shown to the user.
return;
}
if (reason.entityErrors) {
// We have an "entityErrors" property... this means the saved failed due to server-side validation errors.
// do whatever you do to handle validation errors...
return;
}
// an unexpected exception. let it bubble up.
throw reason;
})
.done(); // terminate the promise chain (may not be an equivalent in Angular, not sure).
One of the ways you can test spotty connections is to use Fiddler's AutoResponder tab. Set up a *.drop rule with a regex that matches your breeze route and check the "Enable Automatic Responses" box when you want to simulate dropped requests.
This is a somewhat messy problem to solve- no one size fits all answer, hope this helps.
NOTE
Ward makes a good point in the comments below. This approach is not suitable in situations where the entity's primary key is generated on the server (which would be the case if your db uses identity columns for PKs) because the retry logic could cause duplicate inserts.