I need to build a PHP/MySQL app that allows offline access on iPads (for travelling salesmen often in deep country without internet connection). I have to manage a daily (or on demand when the device is online) sync between the local data and my remote server. I have a large database, which cannot fit in the 5M localStorage limitation. IndexedDB or Filesystem API are not available on Safari (according to http://caniuse.com/).
Is there other ways that would be appropriate to get it done?
Can't you use WebSQL? ( see question What is the maximum size of a Web SQL DB in iOS (Safari) on the iPad? How about LocalStorage? ).
If you coded it for LocalStorage you could easily use that LocalStorage API to access WebSQL on iOS devices...
Perhaps a jump to PhoneGap would solve some of your problems?
Do you need to store ALL data from your database?
I have a library for doing Syncrhonization, it's LocalStorage at the moment but the next update will allow me to use nearly any storage mechanism as it only needs one index. Purging data that is no longer required is on the list too... It's located at https://github.com/forbesmyester/SyncIt and you can even see a presentation courtesy of SkillsMatter / LondonAJAX.
Related
Is there any local storage limit in Apache Cordova like in ordinary browser? I want to store large amount of data, and need to know if it has any limit or not... I don't have any sensitive data to store in it.
The documentation says of local storage:
Limited total amount of storage (typically around 5MB).
It also lists other options, like WebSQL, IndexedDB, and plug-ins.
Apache Cordova has a local storage limit of 5 mbs.
There are some other restrictions on the type of data you can store.
As per the below documentation, you can have different storages linked to Apache Cordova as per your requirement. For instance -
LocalStorage :- Have storage limit of 5 mbs
WebSql :- To store structured data (It also has a limitation of 5 mbs)
IndexedDB :- Though the documentation is not very much clear here, but if you are searching for better indexing and having a large database that can provide features of localStorage and WebSql, I would suggest going for IndexedDB. As IndexedDB is a low-level API for client-side storage of significant amounts of structured data, including files/blobs.
Please refer to the Cordova documentation for storage from here:-
Apache Cordova Doc
Below is the link to explore more about IndexedDB:-
Indexed DB
I am writing a mobile website that needs to store some data offline, and to sync that data when there is an internet connection. It is possible to use HTML5 Web SQL Databases, but there's a space limit of 5MB. I am wondering:
Can I create multiple databases in HTML5 Web SQL Databases, so that the total size can exceed 5MB?
If not, is it possible to store offline data into a local database app (if there is any) installed on smartphones?
Thanks.
Web browsers will eventually support file access using a sandbox. But for now you are limited to webStorage, Web SQL, or indexedDB along with a data limit per domain.
Here is information on webStorage, maybe you can put some data there.
Here is some information on web SQL., but it looks like web SQL is being deprecated in favor of indexedDB. So you might want to re-consider your direction.
This would likely be your best bet - indexedDB. It is a form of no-SQL storage. Check the spec. for limits. SO won't let me post the link, but it is in caniuse as well.
Introduction:
App must be able to run completely offline, store data locally and post it online via AJAX whenever there is an internet connection available - this may be some days later.
Question:
How to store data using Javascript?
Additional notes:
I don't want to use any server-side technology.
It must be secure like a database. I've read about cookies and html5 storage but none of them sound convincing.
If you are supporting modern browsers, you can make use of HTML5 Local Storage.
Persistent local storage is one of the areas where native client applications have held an advantage over web applications. For native applications, the operating system typically provides an abstraction layer for storing and retrieving application-specific data like preferences or runtime state. These values may be stored in the registry, INI files, XML files, or some other place according to platform convention. If your native client application needs local storage beyond key/value pairs, you can embed your own database, invent your own file format, or any number of other solutions.
Example
// Save data to a the current session's store
sessionStorage.setItem("username", "John");
// Access some stored data
alert( "username = " + sessionStorage.getItem("username"));
// Get the text field that we're going to track
var field = document.getElementById("field");
// See if we have an autosave value
// (this will only happen if the page is accidentally refreshed)
if ( sessionStorage.getItem("autosave")) {
// Restore the contents of the text field
field.value = sessionStorage.getItem("autosave");
}
// Check the contents of the text field every second
setInterval(function(){
// And save the results into the session storage object
sessionStorage.setItem("autosave", field.value);
}, 1000);
Browser Compatibility
Older Browsers
Use Polyfill.
Depending on how complex your data structures are that you want to store you could look at indexedDB. It's availability is still pretty bleeding edge but with a polyfil you can target the majority of modern desktop and mobile browsers.
The data stored is no more secure than any other client storage model since it's meant to be read with JavaScript.
The API itself is pretty complex to dive straight into using so you might want to look at wrapper APIs such as PouchDB which syncs with CouchDB or if you want something much simpler there's db.js.
You can use HTML5 Local Storage
Use polyfill for older browser
https://github.com/Modernizr/Modernizr/wiki/HTML5-Cross-Browser-Polyfills#web-storage-localstorage-and-sessionstorage
Exactly what you want:
You can set up a CouchDB instance on IrisCouch to store your data. CouchDB is a database that acts as a webserver, so it can serve html pages based on its own data -- this use of the CouchDB (to serve pages) is commonly called CouchApp.
So you learn about CouchDB and write a HTML/Javascript/CouchDB-flavored app to serve your page. There are tools that facilitate this.
After that, you only need to send the data to your CouchDB database and it will be on your web page. You'll manage the client side stuff with PouchDB, a implementation of CouchDB that runs on your browser and saves your data locally, so you never lose it, and automatically updates your local data on the CouchDB server and vice-versa. It's the bleeding edge of the offline storages on the internet.
To ensure that the clients will not send bad data to the server, you can set up authentication (so to connect Pouch with Couch you will need to provide a password) or you can set up validation functions (so the server will only accept data storage requests that match certain parameters you define). These two approaches are well explained in the guide I linked before (here), but you will certainly run into all of this during your CouchDB learning process.
A lot of stuff, but a cool solution enough for the trouble. Also, this CouchDB thing is so easy I can bet you'll read and learn everything in one or two days.
I need to ask for some advice regarding offline web applications and database sync.
Offline Scenario
We have a web site (HTML5) that needs to operate in an offline mode for extended periods of time with complex data, the product owner does not want the data put into local storage.
We have two options as I see it;
Use javascript to detect if we are offlline and if so point the urls to a local web server that replicates the stack at the data center and writes to an offline db
◦Biggest stumbling block is how, on the first load if you are offline do you get the location (URL) of the local web server? i.e. user goes to www.xyz.com, but you are offline so
Question 1: how to redirect him to localhost.xyz.com via javascript for that first call
Point all calls at the client , offline or not to a wcf service that checks offline status at the NIC and redirects every web and service call to the correct place
◦seems like a big job,
Question 2: is there any product/ opensource project you guys know off that does this?
Sync Scenario
•They want to use MS sync framework
◦But they have many clients syncing to different database, so you either need 1 sync service per client or some way to identify who the client syncing is and point them to the correct place
◦Need to minimize locking during sync as other clients are using the same tables during sync
Question 3: can the sync framework be extended to even do this
Question 4: What other options exist for database sync on MS platform?
Thanks
If the user puts the URL into their browser, they are going to go that URL. There is no javascript at that point. You would only have Javascript once a page is loaded. You will have to search for a better solution.
Here is an idea: Users ALWAYS go to the local website, and if the remote site is up, then you redirect them.
In terms of MS sync I do not know what it is, so I cannot help you there.
Re What other options exist for database sync on MS platform: there is also SQL Azure Data Sync, a windows azure web service. It is actually built upon the Microsoft Sync Framework you refer to.
There is an example in the book "Programming Microsoft SQL Server 2012" by Leonard Lobel & Andrew Brust (MS Press) - chapter 13 covers building occasionally connected systems that incorporate SQL Azure Data Sync, Windows Azure and the Windows Phone 7 development platform. In the sample solution, on the back end, an on-premise SQL Server database is kept synchronized with a public facing SQL Azure database in the cloud using SQL Azure Data Sync. The cloud database is exposed using WCF Data Services (also hosted in the cloud by deploying to Windows Azure) and consumed via OData by a mobile client application running on a Windows Phone 7 device. The sample solution detailed in the chapter demonstrates how these technologies work to keep data in sync across on-premise SQL Server, SQL Azure databases in the cloud, and local storage on Windows Phone 7 devices.
Sync Framework do not lock tables when synching.
depending on what client database you want to use on the client, you can either use Sync Framework itself which works with MS databases (SQL CE, SQL Express,LocalDB,SQL Server, SQL Azure) or you can use the Sync Framework Toolkit
whichever platform you choose, i would suggest simply writing to the local store and synching it rather than dynamically choosing which store to use when.
for example, if you went offline and you wrote to the local store. then your network monitor detects you are back online and redirects you to the online service, what would you do with the data you stored locally? or you transacted online and you suddenly went offline, how recent/updated is the local store for you to actually starting working agaisnt it?
You could use Service-Workers to make the website work while users are offline. see: Making PWAs work offline with Service workers. This allows your website to work for the users if they are offline (they need to have internet at least once every 24 hours).
Service-Workers also allows you to detect when your user is offline or online, and you can for example use the IndexedDB to store your offline changes and then synchronize them when the user is online again.
I don't know about MS Sync.
Is there anyway to have a client side small database that syncs with server side database whenever there is a change in data?
So I am looking at writing a javascript program to store a bunch of student application forms. But the internet connection is gonna be unstable as the personnel using it will be moving around campus to collect form data on his tablet.
I have looked at localstorage, but it does not have any database features.
I am really looking for technologies that can do local database entries and make asynchronous syncing easy (like what dropBox did was awesome except that it is not a web application)
I hope my question is clear
Thanks
It depends upon what kind of support you want.
Check out http://diveintohtml5.ep.io/storage.html#future where they talk about the Web SQL Database specification and IndexedDB.
It may work (Web SQL Database that is), since it sounds like a controlled environment.
following storage mechanisms when available:
Standard HTTP Cookies
Local Shared Objects (Flash Cookies)
Silverlight Isolated Storage
Storing cookies in RGB values of auto-generated, force-cached
PNGs using HTML5 Canvas tag to read pixels (cookies) back out
Storing cookies in HTTP ETags
Storing cookies in Web cache
Internet Explorer userData storage
HTML5 Session Storage
HTML5 Local Storage
HTML5 Global Storage
HTML5 Database Storage via SQLite
Copypasta from evercookie description. Several items were removed because too short lifespan and/or too few space. Do not borrow his code, tho, it uses jquery and is clumsy in the other ways.