I have a simple web app that uses Firestore to receive realtime updates. For simplicity, I can demonstrate the issue with the example from the documentation:
const unsub = onSnapshot(doc(db, "cities", "SF"), (doc) => {
console.log("Current data: ", doc.data());
});
I am NOT using offline persistence and am using v 9.6.8.
Under normal conditions, everything works as expected. When I lose connectivity, however, I am not notified of changes that happen while disconnected even after connectivity is restored (I simulate this by going on 'airplane mode').
One option I've considered, but not yet tried, is to subscribe to connectivity events from the device (online and offline windows events). Before pursuing this option, I was wondering if there is a simpler or better solution.
Firestore synchronizes the state of the documents, and not necessarily all the changes in that state.
So if multiple changes happened to the SF document while you were offline, and some of the later changes erased changes that came earlier, you will not see those earlier changes. You will only see the correct current state when the connection is reestablished.
If you want to see all state changes, consider storing those actual state changes in the database (e.g. in an updates subcollection), so that none of them are overwritten by later changes to the same data.
Related
I'm trying to get a handle on what is best practice when it comes to updating/refreshing your Redux state with data from an API.
Think of this scenario:
You have a To Do app where a number of users are potentially updating the same To Do's as you at any given point in time. Obviously you'll want your local Redux store to reflect these changes from the other users eventually.
Would you:
Fetch the updated set of To Do's on route change?
Fetch an updated version of that To Do when/if the user makes changes to the To Do or interacts with it?
Both?
Never - you keep the state in the store once fetched and it becomes the source of truth for that session (obviously not ideal)
Something else?
I'm not interested in pushing changes to the client right now nor polling an API (ugh!). I'm simply interested in gaining some sort of consensus as to when most developers refresh API derived data that is stored in their Redux state.
If you don't want to push nor poll, then fetch data when the user request it. Never fetching may be well suited for edge case application but definitly not a common use case.
This is what I got from your question
Same space in your app is updated by multiple user.
Any user who is on the app must get the changes done by other user.
You want to know when to fetch data or refresh your store.
I'm not interested in pushing changes to the client right now nor
polling an API (ugh!).
I am not getting it what is this for. You may not be updating the app but to see the changes your app should be updated by one or other.
Apart from this I told you to subscribe the fetching of data wherever you are fetching it. I don't know what you understood by subscription, according to me subscription is to keep on doing fetching at regular interval to get updates. Subscription is async in nature.
I am giving a simple example of implementing subscription.
console.log("Started");
let subscription = false;
const getData = () => {
// call your database and fetch the data and once
// fetching done call the getData again
// Think setTimeout as database call.
// upon resolving the data I called it again
// It neither stops other call back from running run
setTimeout(() => {
console.log("Fetching");
if(subscription) {
console.log("shuting");
return;
}
getData();
}, 1000);
}
getData();
setTimeout(() => {
console.log("hello");
}, 5000);
setTimeout(() => {
subscription = true;
}, 8000);
componentWillUpdate() {
//Axios GET method for getting all the trades
//Adding data of db into mobx store/or syncing data of db and store
axios.get(`http://localhost:8091/trade`)
.then(res => {
this.props.store.arr = res.data;
})
}
This piece of code causing my browser to crash, my laptop to not responding.
Actually, whenever i was trying to delete the row of table containing trade by click of button then the trade was deleted but it took the need of refresh to see that the trade is deleted.
This was because my mobx store and db were not in sync.So as soon i refresh the (REST api) controller updates data in my mobx store.After this i can see that trade is deleted.
So in order to remove the need of refresh i thought to use component will update method.Within that method i tried to sync mobx store with controller data (db data).It worked but it caused the browser to take more than 2.5 gb of memory & at this point all the running applications starts getting crashed also.
So what is the good way to achieve the desired result?
Note i don't know why component will update is getting called too many times.
But i can verify the it because i can see the selection statements(of database) in spring (my server which is sending data to controller ).
Putting the above code inside component did mount is not removing the need of refresh but it is not causing the browser to crash also.
You should not be doing this type of operation in this lifecycle hook. You should use componentDidMount instead for any remote calls that need to happen. However since you are using mobx, you really should not be having these problems as they handle these type of problems for you with the observer pattern. Please read: https://mobx.js.org/getting-started.html to get up to speed and you should have no issues at that point.
Something in your component's props or state is causing it to update often, which is causing a lot of calls to the api. Your best bet would be to find out what is causing those updates and use nextState and nextProps arguments supplied to componentWillUpdate to check and send an api call only when needed. Something like:
componentWillUpdate(nextProps,nextState) {
if (nextProps.needToGetApi !== this.props.needToGetApi) {
//Axios GET here, so that unrelated prop/state change does not cause this to run
}
}
Hint: Add a breakpoint in componentWillUpdate and see what props or state mutations are happening on each call.
I have very large and complicated objects like userInfo, chatInfo, and etc as in objects & arrays with very large and nested information. The thing is in my react app every time I refresh my page the redux state gets reset and I have to call all those API's again.
I did some research on this topic. I checked Dan Abramov's egghead tutorial on redux. What he does is maintain the redux state in localStorage of the browser and updated the localStorage after every 100 or 500 ms. I feel as if this is a code smell.
Continuously watching the localStorage state and updating it, wouldn't it effect the performance of the browser. I mean wasn't this on of the reasons Angular 1 failed because it continuously kept on watching state variables and after a while if the site was kept live in the browser it just slowed down. Because our script continuously kept on checking the state of the variables. i feel as if we are doing the same thing here.
If maintaining the redux state in localStorage is the right approach can someone tell me why so? And if not is there a better approach?
This is not a duplicate of How can I persist redux state tree on refresh? because I am asking for advice whether persisting state in local storage is a code smell or not
I think using localStorage is your best option here, since it seems the data you are storing there is needed on the client. If the data is not changing, you shouldn't need to repeatedly query, or watch, the localStorage.
Another thing you can do is wrap a closure around your localStorage, so that you are not always hitting disk when retrieving your "large" data. Every browser implements localStorage differently, so there are no guarantees on consistent behaviour or I/O performance.
This also adds a simple layer of abstraction which hides the implementation, and controls everything related to your user data in one place.
Here is a simple example of user profile data closure:
// UserProfile.js
var UserProfile = (function() {
var userData = {};
var getUserData = function() {
if (!userData) {
userData = JSON.parse(localStorage.getItem("userData"));
}
return userData;
};
var setUserData = function(userData) {
localStorage.setItem("userData", JSON.stringify(userData));
userData = JSON.parse(localStorage.getItem("userData"));
};
return {
getUserData: getUserData,
setUserData: setUserData
}
})();
export default UserProfile;
Set the user data object: This will overwrite the localStorage, and set the local variable inside the closure.
import UserProfile from '/UserProfile';
UserProfile.setUserData(newUserData);
Get the user data object: This will get the data from the local variable inside the closure, or else go get it from localStorage if it is not set.
import UserProfile from '/UserProfile';
var userData = UserProfile.getUserData();
The idea here is to load data into memory, from localStorage, the first time, when your app loads, or on the first API call. Until such a time when the user profile data changes, (i.e. a user updates their profile, for example), then you would query the API again, and update the data again via the UserProfile.setUserData(..) call.
The question is at what point you need to achieve persistent. I feel that the answer in your case is on a page reload. So if you are worry about performance I'll say:
* Update the localStorage only when the state changes. Inside your reducer when you update the state.
* Read from localStorage when you boot the app.
(This way you write only when the state changes and you read only once)
P.S.
I'll recommend https://github.com/rt2zz/redux-persist package for achieving persistent in Redux apps.
I would only do this as a weak cache and would not rely on it. Local storage is limited (5mb on Chrome e.g.), and may not be available. You'd have to carefully verify that your data was written.
As others have said, you wouldn't be watching localStorage you'd be periodically flushing the store. But I would agree that it seems like a rather coarse hack to blindly assume that all of your state was appropriate to persist in local storage. As with all caching solutions, you need to carefully consider the implications (freshness, expiration, etc.). It sounds like you might want to do this incrementally - pick off a few low-hanging pieces of fruit that are appropriate for caching and consider the implications of caching that state in local storage.
Try redux-persist you can do optimistic persistence, agnostic of the underlying platform (web/mobile).
If you still find performance is a bottleneck. You can do either of the following in steps.
Cache Middleware
Create a middleware that listen's in on changes and records them but only flushes them out every 5 seconds.
Attach an event handler to window.beforeunload to detect user navigating away or closing the browser to flush changes
Persistence Strategy
To persist the data you can use a either of the two strategies below.
Store it in localStorage if there is no performance bottle neck.
Send a JSON blob to server like file-upload. Fetch last JSON state when app loads and persist locally.
I'd suggest you start of with using redux-persist. If there is still a performance bottle neck then use a cache middleware along with one of the two persistence strategies.
I think that in most cases you want to persist your state in localStorage after certain action(s) happens. At least that's always been the case in the projects where I've needed to persist it.
So, if you are using redux-saga, redux-observable or redux-cycles for orchestrating your side-effects you can easily make that side-effect (persisting the state into localStorage) happen whenever one of those actions take place. I think that's a much better approach than "randomly" persisting the data based on a time-interval.
It seems that your understanding of watching state is that you have some kind interval which will keep checking your state and updating it in localStorage. However, I think you can achieve this same thing by updating your localStorage in the react lifecycle method componentDidUpdate. This method will fire every time your component updates, so you can take advantage of it and update your localStorage every time it fires without incurring any performance hits.
One option is to load the data in INITIAL_STATE
window.__INITIAL_STATE__ = { ...state... }
and load the reducer:
window.__APP_STATE__.__REDUCERS__.push(reducer)
It is completely ok to persist state in large redux+react applications.
Regarding angular1, watchers and digest cycle though on every state change even if you rehydrate state not all components are rendered because of redux connect API and vrtualDOM of react.
You can check:
https://github.com/redux-offline/redux-offline
https://github.com/rt2zz/redux-persist
Performance should not be your primary concern. If it comes to that normalise your state and only persist important information like drafts etc(Less info-fewer rehydrations).
The bigger problem with this kind of setup is usually if you have any background sync or socket updates in the app. Having multiple tabs of browser causes async writes to local DB causing to overwrite with previous states.
Here is a thin wrapper on top of it for cross tab sync redux-persist-crosstab
You can check some of these implementations in mattermost webapp and how they use it for realtime app.
They go through some hoops for extra stability and performance - link to store file in mattermost webapp
I think you can use npm module called redux-storage. It provides Persistence layer for redux with flexible backends. You can combine redux-storage with any redux-storage-engine you want. And in my view you can combine this with redux-storage-engine-sessionstorage assuming you will only want to share and save information till browser is open. No need to bloat browser with localStorage. You will need extra javascript support for this to get it clear.
Redux-storage will trigger load and save actions after every state changes. Providing all more flexibility about what to do in case of load and save. Also if you don't to save some state changes then you define array of state which you want to filter out.
We're writing an app that monitors online presence. There are multiple scenarios where we need the user to have more than one browser window open. We're running into a problem where the user, after opening and running the firebase js code in a secondary browser window, will close that secondary window. This sets their presence to offline in the primary window because the onDisconnect event fires in the secondary window.
Is there a workaround for this scenario? Is this where the special /.info/connected location could be used?
The .info/connected presence data only tells a given client if they are linked up to the Firebase server, so it won't help you in this case. You could try one of these:
Reset the variable if it goes offline
If you have multiple clients monitoring the same variable (multiple windows falls into this category), it's natural to expect them to conflict about presence values. Have each one monitor the variable for changes and correct it as necessary.
var ref = firebaseRef.child(MY_ID).child('status');
ref.onDisconnect().remove();
ref.on('value', function(ss) {
if( ss.val() !== 'online' ) {
// another window went offline, so mark me still online
ss.ref().set('online');
}
});
This is fast and easy to implement, but may be annoying since my status might flicker to offline and then back online for other clients. That could, of course, be solved by putting a short delay on any change event before it triggers a UI update.
Give each connection its own path
Since the purpose of the onDisconnect is to tell you if a particular client goes offline, then we should naturally give each client its own connection:
var ref = firebaseRef.child(MY_ID).child('status').push(1);
ref.onDisconnect().remove();
With this use case, each client that connects adds a new child under status. If the path is not null, then there is at least one client connected.
It's actually fairly simple to check for online presence by just looking for a truthy value at status:
firebaseRef.child(USER_ID).child('status').on('value', function(ss) {
var isOnline = ss.val() !== null;
});
A downside of this is that you really only have a thruthy value for status (you can't set it to something like "idle" vs "online"), although this too could be worked around with just a little ingenuity.
Transactions don't work in this case
My first thought was to try a transaction, so you could increment/decrement a counter when each window is opened/closed, but there doesn't seem to be a transaction method off the onDisconnect event. So that's how I came up with the multi-path presence value.
Another simple solution (Using angularfire2, but using pure firebase is similar):
const objRef = this.af.database.list('/users/' + user_id);
const elRef = objRef.push(1);
elRef.onDisconnect().remove();
Each time a new tab is opened a new element is added to the array. The reference for "onDisconnect" is with the new element, and only it is excluded.
The user will be offline when this array is empty.
Overview:
I am trying to avoid a race condition with accessing an IndexedDB from both a webpage and a web-worker.
Setup:
Webpage that is saving items to the local IndexedDB as the user works with the site. Whenever a user saves data to the local DB the record is marked as "Unsent".
Web-worker background thread that is pulling data from the IndexedDB, sending it to the server and once the server receives it, marking the data in the IndexedDB as "Sent".
Problem:
Since access to the IndexedDB is asynchronous, I can not be guaranteed that the user won't update a record at the same time the web-worker is sending it to the server. The timeline is shown below:
Web-worker gets data from DB and sends it to the server
While the transfer is happening, the user updates the data saving it to the DB.
The web-worker gets the response from the server and then updates the DB to "Sent"
There is now data in DB that hasn't been sent to the server but marked as "Sent"
Failed Solution:
After getting the response from the server, I can recheck to row to see if anything has been changed. However I am still left with a small window where data can be written to the DB and it will never be sent to the server.
Example:
After server says data is saved, then:
IndexedDB.HasDataChanged(
function(changed) {
// Since this is async, this changed boolean could be lying.
// The data might have been updated after I checked and before I was called.
if (!changed){
IndexedDB.UpdateToSent() }
});
Other notes:
There is a sync api according to the W3 spec, but no one has implemented it yet so it can not be used (http://www.w3.org/TR/IndexedDB/#sync-database). The sync api was designed to be used by web-workers, to avoid this exact situation I would assume.
Any thoughts on this would be greatly appreciated. Have been working on it for about a week and haven't been able to come up with anything that will work.
I think I found a work around for this for now. Not really as clean as I would like, but it seems to be thread safe.
I start by storing the datetime into a LastEdit field, whenever I update the data.
From the web-worker, I am posting a message to the browser.
self.postMessage('UpdateDataSent#' + data.ID + '#' + data.LastEdit);
Then in the browser I am updating my sent flag, as long as the last edit date hasn't changed.
// Get the data from the DB in a transaction
if (data.LastEdit == lastEdit)
{
data.Sent = true;
var saveStore = trans.objectStore("Data");
var saveRequest = saveStore.put(data);
console.log('Data updated to Sent');
}
Since this is all done in a transaction in the browser side, it seems to work fine. Once the browsers support the Sync API I can throw it all away anyway.
Can you use a transaction?
https://developer.mozilla.org/en/IndexedDB/IDBTransaction
Old thread but the use of a transaction would solve the Failed Solution approach. I.e. the transaction only needs to span the check that the data in the IndexedDB hasn't change after the send and marking it as sent if there was no change. If there was a change, the transaction ends without writing.