I need to ask for some advice regarding offline web applications and database sync.
Offline Scenario
We have a web site (HTML5) that needs to operate in an offline mode for extended periods of time with complex data, the product owner does not want the data put into local storage.
We have two options as I see it;
Use javascript to detect if we are offlline and if so point the urls to a local web server that replicates the stack at the data center and writes to an offline db
◦Biggest stumbling block is how, on the first load if you are offline do you get the location (URL) of the local web server? i.e. user goes to www.xyz.com, but you are offline so
Question 1: how to redirect him to localhost.xyz.com via javascript for that first call
Point all calls at the client , offline or not to a wcf service that checks offline status at the NIC and redirects every web and service call to the correct place
◦seems like a big job,
Question 2: is there any product/ opensource project you guys know off that does this?
Sync Scenario
•They want to use MS sync framework
◦But they have many clients syncing to different database, so you either need 1 sync service per client or some way to identify who the client syncing is and point them to the correct place
◦Need to minimize locking during sync as other clients are using the same tables during sync
Question 3: can the sync framework be extended to even do this
Question 4: What other options exist for database sync on MS platform?
Thanks
If the user puts the URL into their browser, they are going to go that URL. There is no javascript at that point. You would only have Javascript once a page is loaded. You will have to search for a better solution.
Here is an idea: Users ALWAYS go to the local website, and if the remote site is up, then you redirect them.
In terms of MS sync I do not know what it is, so I cannot help you there.
Re What other options exist for database sync on MS platform: there is also SQL Azure Data Sync, a windows azure web service. It is actually built upon the Microsoft Sync Framework you refer to.
There is an example in the book "Programming Microsoft SQL Server 2012" by Leonard Lobel & Andrew Brust (MS Press) - chapter 13 covers building occasionally connected systems that incorporate SQL Azure Data Sync, Windows Azure and the Windows Phone 7 development platform. In the sample solution, on the back end, an on-premise SQL Server database is kept synchronized with a public facing SQL Azure database in the cloud using SQL Azure Data Sync. The cloud database is exposed using WCF Data Services (also hosted in the cloud by deploying to Windows Azure) and consumed via OData by a mobile client application running on a Windows Phone 7 device. The sample solution detailed in the chapter demonstrates how these technologies work to keep data in sync across on-premise SQL Server, SQL Azure databases in the cloud, and local storage on Windows Phone 7 devices.
Sync Framework do not lock tables when synching.
depending on what client database you want to use on the client, you can either use Sync Framework itself which works with MS databases (SQL CE, SQL Express,LocalDB,SQL Server, SQL Azure) or you can use the Sync Framework Toolkit
whichever platform you choose, i would suggest simply writing to the local store and synching it rather than dynamically choosing which store to use when.
for example, if you went offline and you wrote to the local store. then your network monitor detects you are back online and redirects you to the online service, what would you do with the data you stored locally? or you transacted online and you suddenly went offline, how recent/updated is the local store for you to actually starting working agaisnt it?
You could use Service-Workers to make the website work while users are offline. see: Making PWAs work offline with Service workers. This allows your website to work for the users if they are offline (they need to have internet at least once every 24 hours).
Service-Workers also allows you to detect when your user is offline or online, and you can for example use the IndexedDB to store your offline changes and then synchronize them when the user is online again.
I don't know about MS Sync.
Related
I am playing around with the idea of creating a website for cryptocurrencies, where a user can sign up on my website, enter his API details for one of the exchange markets that I will support, which allows him to trade on that exchange, but using my “more user friendly” web interface.
My main goal is to create a more user friendly interface than what most exchange websites offer. I am not hooking directly into any cryptocurrencies or wallets, all I do is use the API of existing exchange markets, relay the information to my website, where I have a more user friendly interface.
Since this is a very sensitive subject in regards to security, I am trying to figure out, what the best way would be to store the API details of the users.
In general I don’t like the idea of storing the API details on my database server, nor on my server in general. The thought of having my website hacked and all the API details being exposed is terrifying. Of course each exchange website that supports APIs has their own security built in, such as API sessions with 2FA, IP restrictions, weekly generations of new API secret keys, daily trading limits via API, and not allowing withdrawals of wallets via API. But damage can still be done if those API details get stolen.
I would prefer if there would be a way where I would not need to store the API details on my server at all, but rather have the user save them locally on his PC. That way he is in charge of keeping the API details secure.
This thought brought me then to the idea of creating a desktop app using electron (https://electron.atom.io/). That way I can still create the website the way I want, but it’s wrapped into electron, so it always run locally. Before I pursue this idea, I would like to keep investigating my previous idea of a regular website, as I prefer to have my website cloud based, SaaS, to prevent piracy.
So I wonder, storing API details of a user, without saving them on the server, what other options would I have?
Cookies? Probably not secure.
What about localStorage? https://developer.mozilla.org/en-US/docs/Web/API/Web_Storage_API
Are there other options or am I too paranoid about this? Is it generally accepted to store sensitive API details on a database server along with the rest of the users details?
I think saving data in to users computers is wrong way, because when you will save user's personal data in to your server, you will be able to control security of your server, when it will be saved on user compputer the security of your server will be depended from users. Today we know many methods how to deceve users and I think, that the programmers must take care of his users. when you will save data in server db you can switch many methods, like email verification or verification by phone you can send message with some verification code, switch ssl service, also you can avoid on sql injection using a modern framework like Laravel or Yii 2, in any case if you will save user data in you server the security of your application will be depended of you.
if you will save user data in local computer, today hackers uses many methods to steal users cookies or methods to get a controll on pc, for example you can read this post
https://krebsonsecurity.com/2011/09/right-to-left-override-aids-email-attacks/
today hackers using this method, creates an exe file which extension on first look is docx or other some extension for example pdf and so on ...
but in real it is an exe file and it is runnable, user can download it, and run... I think you understood what can do hacker with users computers by this way, today so many viruses which even very professional users cant recognize.
I am creating a web application, and need to store some user data (like their favorites history in the app). I had considered using IndexedDB for this purpose, as it would be a little easier to implement than on the server side where I am using MySQL. But, I do want to run analytics against this user data, such as to ask what are the most popular pieces of content that users are saving to their favorites list. If I use IndexedDB, I could use some Javascript on the client side to occasionally forward user data from each IndexedDB store so that it could be analyzed. My question is: would this be scalable for > 25,000 users? Or, am I just asking for trouble with this kind of an approach, in which case I should just store all the user data in MySQL on the server side to make analytics easier?
Those are two different concepts as i understand your question.
indexDB is store locally on the user browser and it's subject to local user behaviors - clean cach , cookies refresh and browser options that are enable.
You should use the indexDB if your site/app load some data to the user browser - like a table/graphdata/friendlist etc. and you want to allow quick navigation with no server calls for some queries sending back to the server.
with this in mind you can store different user sessions and build a small key/value data base and send this when you want to you Server MySQL DB and save this on your side and allow you user to navigate this data or to use the JS API in order to load the right content for your user.
MySQL DB on your server side can do the same but you got some more latency parameters in here - user location vs Server Location , DNS ,network latency , server CPU , Disk , Table structure and Table size that can slow you down.
You should decide what you need in order to learn more on your users on the server side and which data your users need for quick and easy use with your APP . Those 2 data flows are not tied up and you can run them with no code blocking using asynchronies JS commands.
Regarding the scale Issue , IndexDB is good for 1-2MB of data and you should think on a cleansing process or counting main features on the App if you will have 1 key of favorite site . then you just need to add +1 for this key value - sitename_favorite - 5
this data is per user only and the load will be on your MySql server that will get updates from time to time for the users. the IndexDB will be managed by the local machines of the users so 25K users is not relevant for the indexDB part, the load will be on the MySQL server once users updates arrive.
I need to build a PHP/MySQL app that allows offline access on iPads (for travelling salesmen often in deep country without internet connection). I have to manage a daily (or on demand when the device is online) sync between the local data and my remote server. I have a large database, which cannot fit in the 5M localStorage limitation. IndexedDB or Filesystem API are not available on Safari (according to http://caniuse.com/).
Is there other ways that would be appropriate to get it done?
Can't you use WebSQL? ( see question What is the maximum size of a Web SQL DB in iOS (Safari) on the iPad? How about LocalStorage? ).
If you coded it for LocalStorage you could easily use that LocalStorage API to access WebSQL on iOS devices...
Perhaps a jump to PhoneGap would solve some of your problems?
Do you need to store ALL data from your database?
I have a library for doing Syncrhonization, it's LocalStorage at the moment but the next update will allow me to use nearly any storage mechanism as it only needs one index. Purging data that is no longer required is on the list too... It's located at https://github.com/forbesmyester/SyncIt and you can even see a presentation courtesy of SkillsMatter / LondonAJAX.
I am building applications that are used on a touch screen in an educational environment. The applications gather data from user input. The data is then send to a server. There are multiple units, and whilst exact synchronisation is not paramount, the gathered data (along with other data collection from another source) will be combined and distributed back to the touch screen applications.
The applications are being build in Backbone with initial data loaded from a single JSON document. The JSON document is parsed from a remote MySQL database, which is downloaded (along with assets) on initialisation.
Whilst when possible the app should send new data back to the remote mySQL DB as soon as it is gathered, this may not always be possible and I need to collect the data so as to send it when I can.
My first thoughts are that storing everything in localstorage and syncing whenever possible (clearing the localstorage each time a successful sync takes place) is the way to go.
Over the bank holiday weekend, I have been playing meteor.js, and I think that maybe if I write my localstorage solution I will be reinventing the wheel, and a tricky wheel at that. It seems that Meteor.js has a way of mimicking a database offline, in order to fake instant updating.
My question is: How can I use a similar technique to add some offline protection? Is there a JS framework, or backbone plugin I can utilise, or a technique I can tap into?
You can use Backbone.localStorage to save your models and collections to the local storage, while the connection is offline.
Detecting if your user is offline is as easy as noticing that your xhr requests are failing. (https://stackoverflow.com/a/189443/1448860e).
To combine these two.
When you suspect the user is offline (an ajax request to your backend gets no response), switch Backbone.localStorage and store everything there. Inform the user!
When the user gets Internet connectivity again, save any changes from localStorage to the server. Inform the user again!
Voilà!
I have to create a Windows Phone 7 app as part of a research project. The app needs to:
Allow the user to login by connecting to an sql server database (stored locally for the purpose of this project)
If login successful, return a list of products associated with the user (i.e. in product table where UserID=x)
User can click on an item in the list, and then add photos for that item. This can be done offline as well (using HTML5 offline storage)
When an internet connection is available again, user can click 'Upload' (or automatically synced, but not necessary) and the images are all uploaded to the Images table with the correct ProductID as Foreign key
Since this is the first time I am doing any mobile programming, I am not sure what is the best approach. I am especially unsure of how I am going to connect to the database. I'm not really interested in learning Silverlight, so the app should use mainly HTML5 and javascript, I also am looking into JQueryMobile.
I have already installed PhoneGap and am trying it out, but my main concern is how to connect the app to the database as I am having trouble finding the 'proper' way of doing this e.g. using some kind of web service or directly through javascript (read it can be done but is not recommended). If anyone could recommend or suggest a good approach of doing this that would be great!
SqlServer is only available on the phone via LinqToSql which you need to expose via a service or a DAL. I dont think that its possible to make calls directly from Javascript to LinqToSql on the phone.
If you're not set on the idea that the DB needs to be on the phone, you can just make service calls via ajax to a server and access the db that way. I recently did a blog post exposing a db via a WCF Data service and consuming it on the phone. It doesn't cover calling it via ajax but should help get you started.
http://www.ritzcovan.com/2012/02/building-a-simple-windows-phone-apppart-4/
If you want to store data within the WP7 database, use the PhoneGap storage APIs.