Firebase synchronisation of locally-modified data: handling errors & global status - javascript

I have two related questions regarding the Firebase web platform's
synchronisation of locally-modified data to the server:
Every client sharing a Firebase database maintains its own internal version of any active data.
When data is updated or saved, it is written to this local version of the database.
The Firebase client then synchronizes that data with the Firebase servers and with other clients on a 'best-effort' basis.
1. Handling sync errors
The data-modification methods
(set(),
remove(), etc)
can take an onComplete callback parameter:
A callback function that will be called when synchronization to the Firebase servers
has completed. The callback will be passed an Error object on failure; else null.
var onComplete = function(error) {
if (error) {
console.log('Synchronization failed');
} else {
console.log('Synchronization succeeded');
}
};
fredRef.remove(onComplete);
In the example above, what kind of errors should the fredRef.remove() callback expect to receive?
Temporary errors?
Client is offline (network connection lost) ?
Firebase server is temporarily overloaded or down for maintenance, but will be available again soon?
Permanent errors?
Permission denied (due to security rules) ?
Database location does not exist?
Is there a way to distinguish between temporary and permanent errors?
How should we handle / recover from these errors?
For temporary errors, do we need to call fredRef.remove() again after a short period of time, to retry the operation?
2. Global sync status
I realise that each call to set() and remove() will receive an individual sync success/failure
result in the onComplete callback.  But I'm looking for a way to determine the
global sync status of the whole Firebase client.
I'd like to use a beforeunload event listener
to warn the user when they attempt to leave the page before all modified data has been synced to the server,
and I'm looking for some function like firebase.isAllModifiedDataSynced().  Something like this:
window.addEventListener('beforeunload', function (event) {
if (!firebase.isAllModifiedDataSynced()) {
event.returnValue = 'Some changes have not yet been saved. If you ' +
'leave this page, your changes will be lost.';
}
});
Here's an example of the same functionality in Google Drive:
I'm aware of the special /.info/connected location:
it is useful for a client to know when it is online or offline.
Firebase clients provide a special location at /.info/connected which is updated every time the client's connection state changes.
Here is an example:
var connectedRef = new Firebase("https://<YOUR-FIREBASE-APP>.firebaseio.com/.info/connected");
connectedRef.on("value", function(snap) {
if (snap.val() === true) {
alert("connected");
} else {
alert("not connected");
}
});
The special /.info/connected location can be connected to a beforeunload event listener like this:
var connectedRef = new Firebase('https://myapp.firebaseio.com/.info/connected');
var isConnected = true;
connectedRef.on('value', function (snap) {
isConnected = snap.val();
});
window.addEventListener('beforeunload', function (event) {
if (!isConnected) {
event.returnValue = 'Some changes have not yet been saved. If you ' +
'leave this page, your changes will be lost.';
}
});
My question is:
If isConnected is true, does this also mean that all modified data has been synced to the server?
i.e. Does "connected" also mean "synced"?
If not, how can the app determine the global sync status of the whole Firebase client?
Is there a special /.info/synchronized location?
Does the app need to manually keep track of the sync success/failure result of every onComplete callback?

In the example above, what kind of errors should the fredRef.remove() callback expect to receive?
Client is offline (network connection lost) ?
No, this will not cause an error to be passed to the completion listener. It will simply cause the completion listener to not be called (yet).
Firebase server is temporarily overloaded or down for maintenance, but will be available again soon?
No. This is essentially the same as being without a network connection.
Permission denied (due to security rules) ?
Yes, this is will indeed cause an error to be passed to the completion handler.
Database location does not exist?
No, this will not cause an error to be caused to the completion listener.
If isConnected is true, does this also mean that all modified data has been synced to the server? i.e. Does "connected" also mean "synced"?
No it does not. .info/connected will fire with true when a connection is made to the database.
If not, how can the app determine the global sync status of the whole Firebase client?
There is currently no way to determine whether your local data is up to date with the server.
Is there a special /.info/synchronized location?
No, such a location doesn't exist.
Does the app need to manually keep track of the sync success/failure result of every onComplete callback?
That depends on the use-case. But if you want to simply know when all your writes are executed, push a dummy value and wait for that to complete. Since Firebase executes the writes in order, you can be certain at that stage that you've gotten the other events.

Related

Azure Functions - Handle Error while Writing to Table Output Bindings

I have an Azure function setup which. Its trigger is set to a Service Bus Queue message. When Function is triggered the message is processed in some way. If it encounters any exception while processing I handle that error and save the message in Azure Table Storage set as output Binding for logging purposes.
I want to handle any errors thrown when writing something to the output binding. How would I go about doing that?
Detail of the Error
One of the errors I encountered during writing to output binding (table storage) is that value of some key cannot be saved. For e.g. one of the key in the message is time with value of type number. When writing this data, I got exception that the key's value cannot fit in Int32 number. I think table storage tried to convert the number to Int32 by default but failed. To workaround this I converted all keys to have string values. Now I can save.
But still I want to handle any unforeseeable errors while writing to table storage output binding.
Handling error is important in this case, because if the Service Bus Message is not properly consumed, then message is not deleted in Service Bus Queue, and after completion of Azure Function execution, that message again triggers the Function, and Function enters an indefinite loop.
Following is a sample of the Azure function
module.exports = async function (context, mySbMsg) {
try{
// process service bus message
await processMsg(mySbMsg);
} catch(e) {
// if processing fails, save the message in Azure table
try{
// my own try to handle errors, but was unsuccessful
await new Promise(function (resolve, reject) {
context.bindings.tableBinding = [];
context.bindings.tableBinding.push({
PartitionKey: mySbMsg.id|| "unknown",
RowKey: time + "",
errorMessage: e.message || "",
...mySbMsg
})
resolve();
});
} catch (e) {
// do something HERE;
// exception on output binding didn't brought me here
}
}
context.done();
}
The input and output binding execute outside of the scope of the function, which is why you can't enter catch blocks within the function. If you want to catch the error you can handle the write to storage yourself instead of using an output binding.
However, your Service Bus queue will not trigger the function in an infinite loop if you fail to write to storage. Service Bus limits the number of retries and once that limit is hit, the message is moved to a seperate dead letter queue to avoid infinite loops. The messages in there don't expire- they will sit there until you are ready to process them or delete them.
The default maximum delivery count is 10 and you can configure this on the queue's properties blade.
As far as dealing with the messages in the dead letter queue, I've found Service Bus Explorer to be very helpful. It makes it fairly straightforward to take a look at the messages in the DLQ, and then either push them back into the main queue or delete them.
You can also access the DLQ progromatically and build automated systems to deal with these messages. However, since there is nowhere for the messages to go from there, you can get into a loop if your DLQ handler fails to resolve the messages.

pusher api: how can I check if the connection is established?

I'm using the pusher interface, and I would like to write a fallback for environments where the pusher service is unavailable. I can't find a way to check if the pusher subscription is ok.
I tried this
$scope.channel.bind('pusher:error', function() {
console.log("pusher:error");
});
and also pusher:subscription_error but it does nothing.
any help will be appreciated.
It's important to highlight the difference between connection and subscription.
A connection is a persistent connection to Pusher over which all communication takes place.
A subscription is a request for data. In Pusher these are represented by channels. Subscriptions and associated data use the established connection and multiple subscriptions are multiplexed over a single connection.
To determine if the Pusher service is reachable or not you should check the connection state.
However, if you ever see this I'd also recommend contacting Pusher support since this shouldn't happen.
Detecting & Querying Connection State
It's possible to detect connection state by binding to events on the connection object.
pusher.connection.bind('state_change', function(states) {
var prevState = states.previous;
var currState = states.current;
});
You can additional get the state right now.
var currentState = pusher.connection.state;
Full documentation on this can be found here:
https://pusher.com/docs/client_api_guide/client_connect#connection-states
The example in the questions appears to use Angular so you'll need to get reference to the connection object from the $scope. If you're using pusher-angular then the API should be the same as the normal Pusher library.
Subscription Status
You can bind to two events to determine the result of a subscription:
pusher:subscription_succeeded
pusher:subscription_error
The code to use these looks as follows:
var channel = pusher.subscribe('my-channel');
channel.bind('pusher:subscription_succeeded', function() {
// Yipee!!
});
channel.bind('pusher:subscription_error', function() {
// Oh nooooos!
});
Documentation on the success event can be found here:
https://pusher.com/docs/client_api_guide/client_events#subscription_succeeded
Docs on the error event can be found here:
https://pusher.com/docs/client_api_guide/client_events#subscription_error

resending peerconnection offer

I'm currently trying to rebroadcast my local stream to all my peer connections. options I tried:
1) Loop trough all my peer connection and recreate them with the new local stream. Problem that I encounter here is the fact that createOffer is asynchronous.
2) create 1 sdp and send it to all peers. Problem: no video
Would anyone have a way to resend an offer to a list of peers?
Each PC needs to recreate an offer (as bwrent said).
as you obviously are using a p2p multiparty (multiple peer connections) you might want to pass on the peerID to the createOffer success callback every time, then you don't have to worry about it being asynchronous. You need to make the full handshake (offer, answer, candidate) peerID dependent.
(Simplified) Example from our SDK
Skyway.prototype._doCall = function (targetMid) {
var pc = this._peerConnections[targetMid]; // this is thread / asynchronous safe
pc.createOffer(
function (offer) {
self._setLocalAndSendMessage(targetMid, offer); // pass the targetID down the callback chain
},
function (error) {this._onOfferOrAnswerError(targetMid, error);},
constraints
);
};
Skyway.prototype._setLocalAndSendMessage = function (targetMid, sessionDescription) {
var pc = this._peerConnections[targetMid]; // this is thread / asynchronous safe
pc.setLocalDescription(
sessionDescription,
self._sendMessage({ target: targetMid, ... }), // success callback
function () {} // error callback
);
};
If you mean async in a way that when a callback fires it has the wrong variable of who to send it to as the loop has ended and the variable contains the last 'person'? You could scope it to solve the asynchronous problem:
For(var i=0;i<peerConnections.length;i++){
(function(id){
//inside here you have the right id. Even if the loop ended and the i variable has changed to something else, the I'd variable still is the same.
})(i);
}
This is a bit like Alex' answer, as his anwer also describes an example of scoping the variable inside the function executing the .createOffer
Another way to handle this correctly is to use renegotiation. Whenever you change a stream, the on onnegotiation event handler is automatically fired. Inside this function you create a new offer and send that to the other person. As you mentioned you have multiple peer connect ions listening to the stream, you need to know whom to send the sdp to. If you would add the persons id to the rtc object, you can then get it back inside the onnegotioation event by calling this.id.

Picking up meteor.js user logout

Is there any way to pick up when a user logs out of the website? I need to do some clean up when they do so. Using the built-in meteor.js user accounts.
I'll be doing some validation using it, so I need a solution that cannot be trigger on behalf of other users on the client side - preferably something completely server side.
You may use Deps.autorun to setup a custom handler observing Meteor.userId() reactive variable changes.
Meteor.userId() (and Meteor.user()) are reactive variables returning respectively the currently logged in userId (null if none) and the corresponding user document (record) in the Meteor.users collection.
As a consequence one can track signing in/out of a Meteor application by reacting to the modification of those reactive data sources.
client/main.js :
var lastUser=null;
Meteor.startup(function(){
Deps.autorun(function(){
var userId=Meteor.userId();
if(userId){
console.log(userId+" connected");
// do something with Meteor.user()
}
else if(lastUser){
console.log(lastUser._id+" disconnected");
// can't use Meteor.user() anymore
// do something with lastUser (read-only !)
Meteor.call("userDisconnected",lastUser._id);
}
lastUser=Meteor.user();
});
});
In this code sample, I'm setting up a source file local variable (lastUser) to keep track of the last user that was logged in the application.
Then in Meteor.startup, I use Deps.autorun to setup a reactive context (code that will get re-executed whenever one of the reactive data sources accessed is modified).
This reactive context tracks Meteor.userId() variation and reacts accordingly.
In the deconnection code, you can't use Meteor.user() but if you want to access the last user document you can use the lastUser variable.
You can call a server method with the lastUser._id as argument if you want to modify the document after logging out.
server/server.js
Meteor.methods({
userDisconnected:function(userId){
check(userId,String);
var user=Meteor.users.findOne(userId);
// do something with user (read-write)
}
});
Be aware though that malicious clients can call this server method with anyone userId, so you shouldn't do anything critical unless you setup some verification code.
Use the user-status package that I've created: https://github.com/mizzao/meteor-user-status. This is completely server-side.
See the docs for usage, but you can attach an event handler to a session logout:
UserStatus.events.on "connectionLogout", (fields) ->
console.log(fields.userId + " with connection " + fields.connectionId + " logged out")
Note that a user can be logged in from different places at once with multiple sessions. This smart package detects all of them as well as whether the user is online at all. For more information or to implement your own method, check out the code.
Currently the package doesn't distinguish between browser window closes and logouts, and treats them as the same.
We had a similar, though not exact requirement. We wanted to do a bit of clean up on the client when they signed out. We did it by hijacking Meteor.logout:
if (Meteor.isClient) {
var _logout = Meteor.logout;
Meteor.logout = function customLogout() {
// Do your thing here
_logout.apply(Meteor, arguments);
}
}
The answer provided by #saimeunt looks about right, but it is a bit fluffy for what I needed. Instead I went with a very simple approach like this:
if (Meteor.isClient) {
Deps.autorun(function () {
if(!Meteor.userId())
{
Session.set('store', null);
}
});
}
This is however triggered during a page load if the user has not yet logged in, which might be undesirable. So you could go with something like this instead:
if (Meteor.isClient) {
var userWasLoggedIn = false;
Deps.autorun(function (c) {
if(!Meteor.userId())
{
if(userWasLoggedIn)
{
console.log('Clean up');
Session.set('store', null);
}
}
else
{
userWasLoggedIn = true;
}
});
}
None of the solutions worked for me, since they all suffered from the problem of not being able to distinguish between manual logout by the user vs. browser page reload/close.
I'm now going with a hack, but at least it works (as long as you don't provide any other means of logging out than the default accounts-ui buttons):
Template._loginButtons.events({
'click #login-buttons-logout': function(ev) {
console.log("manual log out");
// do stuff
}
});
You can use the following Meteor.logout - http://docs.meteor.com/#meteor_logout

Handle websocket send during handshake without message loss

I'd like to achieve the following behavior. Suppose there is a button on the client side which triggers a function that sends a message via a websocket. The first time this function is called, a WebSocket instance is created and then used in the future calls.
Creating a WebSocket instance is non-blocking. The constructor immediately returns a WebSocket object, starting the handshake in the background. A successful connection triggers the onopen callback.
A problem shows up when the second call comes in and the websocket is still performing the handshake (e.g. user makes double click on the button). All the messages need to be queued and sent when the websocket completes the handshake.
The following piece of code uses jQuery and a custom event to gather all messages received during the handshake.
var ws = null;
function sendMessage(message) {
if (!ws) {
ws = new WebSocket("ws://example.com");
ws.onopen = function() {
$(document).trigger('handshakeComplete');
}
}
if (ws.readyState == WebSocket.CONNECTING) { // *
$(document).bind('handshakeComplete', message, function(event) {
ws.send(event.data);
});
} else if (ws.readyState == WebSocket.OPEN) {
// use the persistent connection
ws.send(message);
}
}
It is possible the condition at the starred line to be evaluated as true and in that moment the websocket passes in the OPEN state, the onopen callback is executed and only after that the current message is added in the queue waiting for the handshakeComplete event. This would result in the loss of messages.
I would like to avoid this and I would appreciate any ideas, comments or suggestions.
I think you're worried that handshakecomplete could fire between the test (== CONNECTING) and when the bind() happens, and the message for the handshakecomplete would never be sent.
In theory if all that existed was JS, this would be wrong, since the event wouldn't fire until the current code finished. However, in reality the event may be generated on another thread in the background, and merely be run in the foreground after we reach a stable state.
The answer is to not embed the message in the handshake event handler. I suggest adding the message to a queue/array, and in the onopen handler process the array. The onopen event can fire (get queued) while we're executing, but it won't run until we hit a stable state (where all the messages are in the queue/array).

Categories

Resources