I'm working on a web based form builder that uses a mix of Jquery and PHP server side interaction. While the user is building the form I'm trying to determine the best method to store each of one of the form items before all the data is sent to the server. I've looked at the following methods
Javascript arrays
XML document
Send each form item to the server side to be stored in a session
The good, the bad and the ugly
Depends on your application functionality and requirements, but Javascript would probably be the best way. You can use either arrays or objects or whatever in javascript. It's server independent and it will preserve data over a long period of time as long as client session stays present (browser window doesn't close for whatever reason) but this can be quite easily avoided (check my last paragraph).
Using XML documents would be the worst solution because XML is not as well supported on the client side as you might think.
Server side sessions are good and bad. They are fine if you store intermediate results from time to time, so if client session ends because of whatever reason, user doesn't loose all data. But the problem is that it may as well expire on the server.
If I was you, I'd use Javascript storage and if needed occasionally send JSON serialized results to server and persist them there as well (based on business process storig this data somewhere else than session could be a better solution). I'd do the second part (with sever side combination) only if I would know that user will most probably build forms in multiple stages over a longer period of time and multiple client sessions. but can be used for failure preventions as well. Anyway. Javascript is your best bet with possible server-side interaction.
Preserving data between pages on the client
Be aware that it's also possible to preseve data between pages on the client side. Check sessvars library for this. So even if the page gets refreshed or redirected and then returned all this can be stored on the client side between these events like magic. Marvelous any rather tiny library that made my life several times. And lessened application complexity considerably that would otherwise have to be implemented with something more complex.
I used TaffyDB to store data, and it's just wonderfully easy to implement.
Hope this helps you
You may want to check out PersistJS, which exposes a cross-browser persistent storage object. Of course, being persistent, data stored with this library survives sessions, not just page changes.
The latest version (0.2.0) is here – note the version in the above linked post is 0.1.0.
A combination of #1 (although I'd use objects, not arrays necessarily) and #3 would seem like a good approach. Storing the data locally in the browser (#1) makes it immediately accessible. Backing that up with session-based server-side storage defends you from the page being refreshed; you can magically restore the page just as it was.
Related
I have a 40-50MB JSON object that I need to persist across to a different page.
This only needs to happen once (one transition) but I'm still way over HTML5 LocalStorage limits, what other options do I have?
Unfortunately, that is too much data to store for most browsers. Even combining sessionStorage and localStorage both will not get us even close.
There are a few options you can try though:
You can store the data on your own server. This will depend on what web server/environment you are using.
You can use someone else's server to store the data. For example, you could use Google Drive's API. This does mean that your user needs a google account. You could also pay for a service like Amazon S3 to store it.
You could create a 'container' page, which loads and displays the pages, but keeps the session going. How exactly this works depends again on your environment.
40-50m is too huge for a browser, the worse part is if mobile is involved, what you can do is split the data into chunks, keep some in sessionStorage, localStorage and the remaining on your server, so that the part on the server will be fast enough to load, You will have to join them once all is loaded and done. I wouldn't recommend this method though.
When is it appropriate to use the many different ways that modern day AJAX based applications are storing data? I'm hoping for some specific guidelines that I can give developers. Here's what I'm seeing so far, and it's getting messy.
PHP Server Side Session: PHP Session data is probably the oldest way to store session based information. I'm often passing in parameters through various AJAX calls from JavaScript/jQuery objects - to store in PHP Session. I'm also returning objects of data (some session information) back through as a response/result to JavaScript/jQuery methods.
Browser based Local Storage: This is often used to store data that needs to persist on the front end, yet I'm uncertain at times when to use it. One good use was to store geolocation information from navigator.geolocation. I've been storing a lot of information here, but I am not certain that is wise. It never seems to expire, but can be deleted from Resources.
JavaScript Object with config parameter(s): I've been building JavaScipts objects with an init method that sets-up a 'settings' parameter. This is very useful as I usually build it from data passed in from PHP. With jQuery Mobile this data can even persist from page to page and change with AJAX request responses.
So, what guidelines would you give on usage of each?
PHP Session data is NOT Permanent Data storage as when you destroy the browsers session you will loose the Data. This is useful if you dont
want to permanently store data.
Browsers Local Storage is Permanent unless you delete the data yourself or you clear the browsers cache. Some users clear the cache from time to time so this can be a problem.
Any other method Such as Objects is not permanent Data storage.
Other Browser related Permanent storage are COOKIES (if you don't
expire them at session close), IndexedDb (Check here for current browser support http://caniuse.com/#feat=indexeddb).
So depending on your Website or App you need to decide what data needs to be
stored for a short time, or for long time or forever until you deleted it manually.
As an example, you will use LocalStorage if you were storing
Bookmarks, and if you were Storing Geolocation points you use Cookies
and expire them after you close the browser or the App.
If you were Logging in to an Account using PHP then best practice is to create a PHP
Session, and even change the session timeout when the user clicks
(Remember me).
These are just a couple of examples from thousands of possible needs.
I've seen several other questions on SO that are similar to this, but none of them are really what I'm looking for, so hopefully this won't be seen as a duplicate.
I have a client-side Javascript/HTML5 web application built with jQuery Mobile. I am finding that performance can be quite slow and it was suggested that having too much going on in the DOM could be the cause. My app does have several data-role="page" divs that could be bulking up the DOM in a single html page. I'm trying to split my app into several html pages to improve performance, but I want the experience to be seamless for the user. This means I will need to pass Javascript variables between the physical html pages within my app.
So far I've seen the following options in my searching:
Use a query string in the url going to the other pages. - I'm not sure I want my users seeing a what could be a rather large and confusing query string in the address bar.
Use server side code like ASP.Net or PHP to handle postback data. - I'm open to this, but I'm not really sure how it would work. I don't want to convert my html pages to aspx or php files. Could I have a simple server side script that could embed the postback data into a regular html file?
Use Cookies to store relevant data. - I'm not to sure of this one either because the majority of my users are in enterprise environments that may limit cookie usage.
Are there any other methods for accomplishing this? At this point, I'm leaning toward some sort of server side processing. If that is the best method, could someone point me in the right direction for figuring out how to do that?
Try out Local Storage or Session Storage http://www.w3schools.com/html/html5_webstorage.asp
Local Storage would be a way to go if you are HTML5 compliant. It will store values, reduce the calls to any server until you are actually ready to update all the info and the info will be present even when the browser is closed; use session storage or JS like this
window.onbeforeunload = function() {
localStorage.removeItem(key);
return '';
};
if you need to clear local storage of sensitive info on closing the browser.
Remember that anything you pass into local storage will come out as a string so you will need to convert it to the appropriate data type when you get the info out of storage.
You'll also be limited to storing 5 megs of data (I believe that is standard) but you probably have other issues if your form requires that much info. :)
Check these out for more info
http://msdn.microsoft.com/en-us/library/bg142799(v=vs.85).aspx
http://dev.w3.org/html5/webstorage/
You could use a POST instead of GET if you're only concern with the GET is the user seeing lengthy querystrings.
Use localStorage. localStorage lets you store values in the browser.
I'm working on a new web app where a large amount of content (text, images, meta-data) is requested via an Ajax request.
No auth or login required for a user to access this.
My concern is that you could easily lookup the data source URL and hit it directly outside the app to get large data. In some ways, if you can do this you could probably scrape the static HTML pages elsewhere that also have this content.
Are there any suggestions on methods to obfuscate, hide, or otherwise make it very difficult to access the data directly?
Example: web app HTML page contains a key that is republished every 30 min. On the server side the data is obfuscated based on this key. In order to get the data outside the app you'd need to figure out the data source but also the extra step of scraping the page for a key every 30 min.
I realize there is no 100% way to stop someone, but I'm talking more about deterrence.
Use sessions in your webapp. Make a note (e.g. database entry or some other mechanism which your server-side code can access) when a valid request for the first page is received and include code in the second page to exclude the data when processing a request without a corresponding session entry.
Obviously the specifics on how to do this will vary between languages, but most robust web platforms will support sessions, largely for this type of reason.
If you are wanting to display real-time data and are concerned about scrapers...if this is a big enough concern, then I suggest doing it with flash instead of JS (AJAX). Have the data display withing a flash object. Flash can make real-time send/receive requests to the server just like AJAX. But the benefit of Flash is that the whole stage, data, code, etc.. are within a flash object, which cannot be scraped. Flash object makes the request, you output the stuff as a crypted string of code. Decrypt it within flash and display from there.
"Are there any suggestions on methods to obfuscate, hide, or otherwise make it very difficult to access the data directly?"
Answers your own question because if the data is worth getting it will be obtained because you are obfuscating is merely making it harder to find.
You could in the server side script processing the ajax and returning the data check where the request came from.
For my web application, I need to store form inputs spanning across multiple pages, until I finally process/manipulate them to produce some results (its mostly formatting the data entered and presenting it in some layout). The options I think I have are -
Keep sending user's inputs to the server, store it there in some database, do the final manipulation there only, and show the result.
Store the inputs in browser's storage as the user fills the forms, and finally use this stored data to manipulate and show results.
I very much want to use the second method, and perhaps a possible way is using cookies, but I'm afraid I might just hit some upper limit of cookie data storage. I'm also open to understanding the merits of the first method, or any third method.
thanks.
Use webstorage (you can client-side store around 5MB of text or binary data)
Firefox demo: http://codebase.es/test/webstorage.html
DOM Storage is supported in these web browsers:
Internet Explorer 8
Firefox 2 for sessionStorage, 3.5 for localStorage
Safari 4
Just google for sessionStorage and localStorage objects.
Also modern webkit browsers supports client-side sql.
Edit:
I'm not sure about what you want to do but using AJAX you can store everything in javascript variables and serverside databases or sessions are a good choice.
Hitting the storage limit of the cookie could indicate you are trying to store too much on the client side. It might be prudent to store it serverside, in something like a session. The key to the session could then be stored in a cookie.
An alternative method is to not have the requests span multiple pages, and just store the data on the client side, not as a cookie, but as different form fields and/or text fields (they could be hidden). The merit of such a method is it doesnt hit the cookie limit as you have. It also makes your serverside code easier/cleaner, since it doesn't have to keep track of state (something you'd always have to do if spanning across pages, and thus the reason you are hitting the cookie limit in the first place).
You could use a small Flash Movie to store some data via Flash's Shared Memory Api or have a look at Google Gears.
Maybe also consider, that every byte you store in the cookie have to be transmitted everytime you website makes a request to the server.
Generally cookies have a max size of 4k so you could store quite a bit of data in there.
Be careful with validating all information that lives cookies - all the information resides on a client browser and can easily be manipulated by users of the site at any time.
You didn't say which platform you use. Spring Webflow does exactly the kind of form processing that you want:
http://www.springsource.org/webflow
Even if you don't use Java you could use some of the principles.
Edit: One more drawback of big/complex persistent cookies is that you have to make sure that any new code you deploy is backwards compatible with all the cookies that are out in the wild.
I would suggest storing the data in a session variable until you get to the final step rather than a cookie. I think this would be safer for your data as the user does not have direct access to the data, so you can validate as you go.