Develop Reference

Local Storage Limits - How do I know if I've reached it?

I've got an application where i'd like to share data across windows. The data will almost always be less than 5MB but in some cases it may be 7MB or more. In order to share this data across windows I need to write the data to local storage in chunks, and once the data has been collected, replace it with the next chunk to be collected.
My question is. How do I calculate this with javascript? Not only do I need to calculate the byte size of the data, but I also need to calculate whatever head I will give the "packet" of data, so when it's received it can be concatenated to the right (incomplete) "packet" in the other windows. In local storage how do I figure out,
A. How much data is available?
and
B. How much space will my data take up?

There is no standard size in the spec. This StackOverflow answer shows two quotes, the defaults appear to be 5mb for most browsers, and 10mb for Internet Explorer. The second quote is important, as it says that each browser gives the user the ability to customize this limit. There is no way detect if you are approaching the limit.
Per the W3C spec the setItem method will throw a QuotaExceededError if you exceed the browser's quota.
Regarding the size of the data stored, what you are storing is strings. You can get to 5mb, or 10mb, sure, but that's a huge amount of data. To get perspective, look at a text file that is 5mb large. Although that may not be an accurate depiction of how much text you could store using localStorage, it would still be massive! To either end, when implementing code that uses setItem, be sure to check if the exception is thrown.

Using a JSON object vs. localStorage/sessionStorage/IndexedDB/WebSQL/etc.?

I've got a web app which gets a couple dozen items at boot. All these items are JSON and are smaller then 1kb.
Now there are a number of storage options as seen in the Question.
I was thinking of just storing these objects inside a variable in the browser JS. I don't really see why I would want to use any of these browser storages?
So what would be reasons to use any of the browser-based storage instead of a variable inside JS.
Could be that from a certain data size it is preferable to use browser storage, e.g. from 100kb onwards it's better to not use a JS variable.
var myModel = {}
NOTE
Every time the user enters the app he will get fresh content from the server. The content is too realtime for caching.
`

localStorage , globalStorage and sessionStorage:
These features are ready in browsers that have implemented the "Web Storage", they all refer to a kind of HashMap, a map between string keys and string values. but the life is different. once the active page is closed sessionStorage would be cleaned but the localStorage is permanent.(MDN DOM Storage guide)
There is a point about globalStorage, which is its being obsolete since Gecko 1.9.1 (Firefox 3.5) and unsupported since Gecko 13 (Firefox 13), since then we should use localStorage. the difference between these 2 was just the HTML5 scope support(scheme + hostname + non-standard port).
These could be useful for you to:
-Share your objects between your different pages, in your site.
-Offline programming.
-Caching large object
-Or whenever you need to a local persistent storage.
IndexedDB:
IndexedDB is useful for applications that store a large amount of data (for example, a catalog of DVDs in a lending library) and applications that don't need persistent internet connectivity to work (for example, mail clients, to-do lists, and notepads)
based on this quote from MDN you can easily find your answer out, regarding using IndexedDB, if you don't know whether IndexedDB is useful for you or not, just answer these questions:
Do you store a large amount of data on client? if yes, so consider using it.
Does your app need to be offline enabled? if yes, so consider using IndexedDB.
Does your app need to a persistent internet connectivity? If yes, it stays still an option, based on the other factors.
So other than working offline as far as you don't need it, I guess, because as you said:
The content is too realtime for caching.
These have some features like sharing objects, and managing large amount of data, which you should be the one to decide.

localStorage and sessionStorage are solving a caching problem; think of them as cookies. You've said you don't want caching, so you can ignore them.
JavaScript objects behave basically like O(1) lookup tables (see How is a JavaScript hash map implemented?, and make sure you read both the top two answers, as both have something useful to say), and there is no maximum memory limit that I am aware of, or a point where another solution becomes a better choice
The only reason I can think of that you should bother with the extra step of inserting the data in an IndexedDB is if you need O(1) lookups on a field that is not the object key you are using.

How long variables and array will retained in browser memory?

Inorder to reduce server load, I would like to bring many task into client side. There is no clear information about browser memory limit and its recycling ... Just like PHP session is there any clean function or any limit of storage size for browsers?

Basic javascript variables will last only for the duration of the page
For more persistent variables, use either sessionStorage (these will last until the user closes their browser/tab) or localStorage (these will last "forever"/until the user clears their personal data)
https://developer.mozilla.org/en-US/docs/Web/Guide/API/DOM/Storage
Note that the above are available in all modern browsers, but notably not in IE<=7, in which case you would need to use cookies as a storage mechanism. However, if you need that, you can probably find a library that will take care of the nitty gritty details

localStorage store large size data

I want to use localStorage to store large amount of data(like 800GB), according to http://arty.name/localstorage.html and also I'm using Firefox, I changed my localStorage size and also the cache size. So the size isn't a problem. However, I write some jquery like the following:
$("a[href]").each(function(){
$(this).click(function(event){
localStorage.test += "somenewinformation";
...
If this localStorage.test already have large amount of data like 400GB, so the storing information step would be extremely slow. When I click on a link,will the jquery wait for me to finish the appending new information to localStorage.test or it will just go to the next page and information in localStorage.test will all lost or localStorage.test will just remain the old value? What I dont understand is whether a new thread will be generated to do this storing in background or not and closing browser in the middle will affect it or not.
Sorry about the messy description and thanks in advance!

You can't! The usual limit is 5 MB.
Some browsers such as Opera allow you to adjust the size, but this is purely dependent on the browser and is a user-initiated action, not a programmable one.
And even if you could remember that localStorage can only store strings, so anything else need to be stringified first. That together with this being an key-value storage array you will run into pretty poor performance at the end.
If you need large storage capacity, look into File API instead. This is made to work with large files (Blobs) and you can store anything as a Blob.
800 Gb is a large size and File API can only work as fast as the file system (at best, more likely a bit slower as it is sandboxed, ie. not a pure file system).
More about File System API (Note: discontinued as of 4/2014):
http://www.w3.org/TR/file-system-api/
Tutorial:
http://www.html5rocks.com/en/tutorials/file/dndfiles/
Blob:
https://developer.mozilla.org/en-US/docs/Web/API/Blob
Update As the Filesystem API has been discontinued there are essentially only two options left (besides from storing data to the server):
Indexed Database API aka Indexed DB (recommended)
Web SQL (deprecated but still supported in browsers such as Safari)
Also these are initially limited in size by the browser. It is possible to request a larger quote which the user is asked to accept.
See more details about this API here:
http://www.w3.org/TR/IndexedDB/

There is LargeLocalStorage which provides a cross-browser way to storage large amounts of data locally. You can store binary data or strings.
LargeLocalStorage uses the FilesystemAPI on Chrome, IndexedDB in IE and Firefox and WebSQL in Safari.

Does HTML5 localStorage maximum size include key names?

HTML5's localStorage WebStorage has a maximum size of 5MB.
Does this include the key names?
For instance, if I were to use key names "quite-a-long-key-name-and-this-is-only-1" instead of "key1", would I run out of space sooner?
On a slightly related topic; is there any de-facto convention for naming localStorage keys?
How are namespace collisions prevented when using third party JS scripts?

Does this include the key names?
Yes those do, they become part of data eg they identify data you store and later retrieve so that got to be saved as well.
How are namespace collisions prevented when using third party JS
scripts?
That's a good question, I generally prefix localStorage with application name. Though a better approach would be to create a hash eg some algorithim that accepts a string like application name, etc and later when reading you use them again.

First note that this is implementation dependent as the norm doesn't give a limit. So you shouldn't rely on the size.
Secondly, yes, the limit in today's browsers includes the names : this is the size of the storage space ("disk space").
In order to avoid collisions, I use namespace (eg myplugin.mypart.myval). 5 MB is already big for a storage that can be deleted or not available any more at any time so I never thought about reducing the size of the keys...