I am creating a web application, and need to store some user data (like their favorites history in the app). I had considered using IndexedDB for this purpose, as it would be a little easier to implement than on the server side where I am using MySQL. But, I do want to run analytics against this user data, such as to ask what are the most popular pieces of content that users are saving to their favorites list. If I use IndexedDB, I could use some Javascript on the client side to occasionally forward user data from each IndexedDB store so that it could be analyzed. My question is: would this be scalable for > 25,000 users? Or, am I just asking for trouble with this kind of an approach, in which case I should just store all the user data in MySQL on the server side to make analytics easier?
Those are two different concepts as i understand your question.
indexDB is store locally on the user browser and it's subject to local user behaviors - clean cach , cookies refresh and browser options that are enable.
You should use the indexDB if your site/app load some data to the user browser - like a table/graphdata/friendlist etc. and you want to allow quick navigation with no server calls for some queries sending back to the server.
with this in mind you can store different user sessions and build a small key/value data base and send this when you want to you Server MySQL DB and save this on your side and allow you user to navigate this data or to use the JS API in order to load the right content for your user.
MySQL DB on your server side can do the same but you got some more latency parameters in here - user location vs Server Location , DNS ,network latency , server CPU , Disk , Table structure and Table size that can slow you down.
You should decide what you need in order to learn more on your users on the server side and which data your users need for quick and easy use with your APP . Those 2 data flows are not tied up and you can run them with no code blocking using asynchronies JS commands.
Regarding the scale Issue , IndexDB is good for 1-2MB of data and you should think on a cleansing process or counting main features on the App if you will have 1 key of favorite site . then you just need to add +1 for this key value - sitename_favorite - 5
this data is per user only and the load will be on your MySql server that will get updates from time to time for the users. the IndexDB will be managed by the local machines of the users so 25K users is not relevant for the indexDB part, the load will be on the MySQL server once users updates arrive.
Related
I have a node js application which displays graphs based on the selected parameters, these parameters can vary based on a users interest. I would like to collect information about a users activity , so that the next time the same user uses the application, it should display relevant feeds based on the graphs visualised before.
Can I do this without having to register a user in the db? How will the application remember every unique user? Can we do this via cookies?
If you do not want to store user information server side, you could try cookies or browser storages like LocalStorage and then requests that attached with parameters can be recognized what the user's preferences are.
I am building applications that are used on a touch screen in an educational environment. The applications gather data from user input. The data is then send to a server. There are multiple units, and whilst exact synchronisation is not paramount, the gathered data (along with other data collection from another source) will be combined and distributed back to the touch screen applications.
The applications are being build in Backbone with initial data loaded from a single JSON document. The JSON document is parsed from a remote MySQL database, which is downloaded (along with assets) on initialisation.
Whilst when possible the app should send new data back to the remote mySQL DB as soon as it is gathered, this may not always be possible and I need to collect the data so as to send it when I can.
My first thoughts are that storing everything in localstorage and syncing whenever possible (clearing the localstorage each time a successful sync takes place) is the way to go.
Over the bank holiday weekend, I have been playing meteor.js, and I think that maybe if I write my localstorage solution I will be reinventing the wheel, and a tricky wheel at that. It seems that Meteor.js has a way of mimicking a database offline, in order to fake instant updating.
My question is: How can I use a similar technique to add some offline protection? Is there a JS framework, or backbone plugin I can utilise, or a technique I can tap into?
You can use Backbone.localStorage to save your models and collections to the local storage, while the connection is offline.
Detecting if your user is offline is as easy as noticing that your xhr requests are failing. (https://stackoverflow.com/a/189443/1448860e).
To combine these two.
When you suspect the user is offline (an ajax request to your backend gets no response), switch Backbone.localStorage and store everything there. Inform the user!
When the user gets Internet connectivity again, save any changes from localStorage to the server. Inform the user again!
Voilà!
I am working on a commission calculator and would like to know how I take sales the users would input and save them to return MTD/YTD commission amounts?
I am currently working on learning the html/javascript for the input form, but now I am wondering how I can link the sales data to a nice calendar.
You have two main directions to go:
You can save sales data to a central server so that the user can login and view their info from any computer and so that it is securely saved.
You can save sales data on the local computer in Local Storage (modern browsers only). Local Storage is generally persistent, but not guaranteed to last forever and it is ONLY on that particular local computer.
If this is an important business application, I would generally think you would save the data to a central server in which case you would need a server and a server-side language (perhaps PHP) and database (perhaps mySQL) that you could receive the form data and store securely with a server-side web application. You would probably also need to create a login system so that users got access only to their own data.
I need to ask for some advice regarding offline web applications and database sync.
Offline Scenario
We have a web site (HTML5) that needs to operate in an offline mode for extended periods of time with complex data, the product owner does not want the data put into local storage.
We have two options as I see it;
Use javascript to detect if we are offlline and if so point the urls to a local web server that replicates the stack at the data center and writes to an offline db
◦Biggest stumbling block is how, on the first load if you are offline do you get the location (URL) of the local web server? i.e. user goes to www.xyz.com, but you are offline so
Question 1: how to redirect him to localhost.xyz.com via javascript for that first call
Point all calls at the client , offline or not to a wcf service that checks offline status at the NIC and redirects every web and service call to the correct place
◦seems like a big job,
Question 2: is there any product/ opensource project you guys know off that does this?
Sync Scenario
•They want to use MS sync framework
◦But they have many clients syncing to different database, so you either need 1 sync service per client or some way to identify who the client syncing is and point them to the correct place
◦Need to minimize locking during sync as other clients are using the same tables during sync
Question 3: can the sync framework be extended to even do this
Question 4: What other options exist for database sync on MS platform?
Thanks
If the user puts the URL into their browser, they are going to go that URL. There is no javascript at that point. You would only have Javascript once a page is loaded. You will have to search for a better solution.
Here is an idea: Users ALWAYS go to the local website, and if the remote site is up, then you redirect them.
In terms of MS sync I do not know what it is, so I cannot help you there.
Re What other options exist for database sync on MS platform: there is also SQL Azure Data Sync, a windows azure web service. It is actually built upon the Microsoft Sync Framework you refer to.
There is an example in the book "Programming Microsoft SQL Server 2012" by Leonard Lobel & Andrew Brust (MS Press) - chapter 13 covers building occasionally connected systems that incorporate SQL Azure Data Sync, Windows Azure and the Windows Phone 7 development platform. In the sample solution, on the back end, an on-premise SQL Server database is kept synchronized with a public facing SQL Azure database in the cloud using SQL Azure Data Sync. The cloud database is exposed using WCF Data Services (also hosted in the cloud by deploying to Windows Azure) and consumed via OData by a mobile client application running on a Windows Phone 7 device. The sample solution detailed in the chapter demonstrates how these technologies work to keep data in sync across on-premise SQL Server, SQL Azure databases in the cloud, and local storage on Windows Phone 7 devices.
Sync Framework do not lock tables when synching.
depending on what client database you want to use on the client, you can either use Sync Framework itself which works with MS databases (SQL CE, SQL Express,LocalDB,SQL Server, SQL Azure) or you can use the Sync Framework Toolkit
whichever platform you choose, i would suggest simply writing to the local store and synching it rather than dynamically choosing which store to use when.
for example, if you went offline and you wrote to the local store. then your network monitor detects you are back online and redirects you to the online service, what would you do with the data you stored locally? or you transacted online and you suddenly went offline, how recent/updated is the local store for you to actually starting working agaisnt it?
You could use Service-Workers to make the website work while users are offline. see: Making PWAs work offline with Service workers. This allows your website to work for the users if they are offline (they need to have internet at least once every 24 hours).
Service-Workers also allows you to detect when your user is offline or online, and you can for example use the IndexedDB to store your offline changes and then synchronize them when the user is online again.
I don't know about MS Sync.
I have a backend where people may take some time filling out the form. The form is written to a temporary table every 5 minutes to store the data. The problem I have is that some peoples internet connections are not strong, so they drop at times without the person knowing and when they go to submit, the form can't because of loss of connection. Every 5 minutes I was going to add if the ajax fails, then prompt the user, but I want to go further and possible allow them to store the data offline and then reconnect to submit it.
The problem is, how would I start storing the data to a file on their local machine? As far as I know, client side scripting can't create files and we couldn't use ajax to call a remote file that saves to a local file. I suppose I could prompt to save the file at the beginning locally, but the local machine still would need to support the language. We are using JSP with MySQL.
Does anyone know of how I would accomplish saving data offline when a connection drops?
Implementing offline capabilities can become a tough exercise. Your question is too broad to provide a final answer, but you should have a look at HTML5 capabilities, an overview is available here. Using this feature requires your users to use a modern browser version. You have to think of synchronization/replication as well.
If the connection only drops for a limited amount of time, it may be enough to just use a rich-client and store the data in memory using JavaScript.
Something like this should do the work :
First way:
Use HTML5 localStorage/session storage, or something like Gear to
save the data on the client side
Make an ajax loop to ping the connection, idealy the server should be configured to reply with a http code 100 (I presume)
Send the data when the connection is on
Second way:
Use HTML5 offline capability or a substitue (see modernizr and polyfill)