I've got a web app which gets a couple dozen items at boot. All these items are JSON and are smaller then 1kb.
Now there are a number of storage options as seen in the Question.
I was thinking of just storing these objects inside a variable in the browser JS. I don't really see why I would want to use any of these browser storages?
So what would be reasons to use any of the browser-based storage instead of a variable inside JS.
Could be that from a certain data size it is preferable to use browser storage, e.g. from 100kb onwards it's better to not use a JS variable.
var myModel = {}
NOTE
Every time the user enters the app he will get fresh content from the server. The content is too realtime for caching.
`
localStorage , globalStorage and sessionStorage:
These features are ready in browsers that have implemented the "Web Storage", they all refer to a kind of HashMap, a map between string keys and string values. but the life is different. once the active page is closed sessionStorage would be cleaned but the localStorage is permanent.(MDN DOM Storage guide)
There is a point about globalStorage, which is its being obsolete since Gecko 1.9.1 (Firefox 3.5) and unsupported since Gecko 13 (Firefox 13), since then we should use localStorage. the difference between these 2 was just the HTML5 scope support(scheme + hostname + non-standard port).
These could be useful for you to:
-Share your objects between your different pages, in your site.
-Offline programming.
-Caching large object
-Or whenever you need to a local persistent storage.
IndexedDB:
IndexedDB is useful for applications that store a large amount of data (for example, a catalog of DVDs in a lending library) and applications that don't need persistent internet connectivity to work (for example, mail clients, to-do lists, and notepads)
based on this quote from MDN you can easily find your answer out, regarding using IndexedDB, if you don't know whether IndexedDB is useful for you or not, just answer these questions:
Do you store a large amount of data on client? if yes, so consider using it.
Does your app need to be offline enabled? if yes, so consider using IndexedDB.
Does your app need to a persistent internet connectivity? If yes, it stays still an option, based on the other factors.
So other than working offline as far as you don't need it, I guess, because as you said:
The content is too realtime for caching.
These have some features like sharing objects, and managing large amount of data, which you should be the one to decide.
localStorage and sessionStorage are solving a caching problem; think of them as cookies. You've said you don't want caching, so you can ignore them.
JavaScript objects behave basically like O(1) lookup tables (see How is a JavaScript hash map implemented?, and make sure you read both the top two answers, as both have something useful to say), and there is no maximum memory limit that I am aware of, or a point where another solution becomes a better choice
The only reason I can think of that you should bother with the extra step of inserting the data in an IndexedDB is if you need O(1) lookups on a field that is not the object key you are using.
Related
I'm looking for a solution that allows me to cache a large object over a page reload. More specifically I have a large crossfilter.js object that I'd like to retain, since creating it takes a while. I couldn't find any native way to persist a crossfilter.js instance.
I know about the following generic options:
Local storage/session storage. Problem: Apparently no objects bigger
than 5 or 10 MB can be stored, and the ones I'm trying to cache are upwards of 10 MB.
Abusing window.name for a serialized version of my object.
Problem: While it works for classes I've written myself, trying to
serialize and de-serialize the crossfilter.js instance or its
groups/dimensions leads to exceptions; i. e. the internal state of the
crossfilter.js instance is not maintained. I'm using https://github.com/hunterloftis/cryo for serialization.
IndexedDB. Problem: Same as with window.name - I'd have to
serialize my data, which I haven't found a feasible approach for yet. Also a bit of an overkill for my needs, I guess.
Summarized: I want to keep a particular complex object in memory after a page reload. Possible solutions would allow to
store complex objects/class instances (with methods) in the
local/session storage with increased or no memory limits,
using a hack like dumping the object in window.name, but accepting a
complex object without the need for serialization as necessary for
window.name or
using native crossfilter.js functionality to dump/cache one of its instances.
Any tips? Although a browser-independent version is preferred, a Chrome-specific solution will be accepted as well.
Thanks!
I have a large 40,000 words array loading from a database into a JavaScript/HTML array on every page of our web application... What would be the best way/technology to optimize it? In order to avoid this unnecessary downloads.
Somehow keep the array in a cookie and read from there?
Use ajax to load the array dynamically only parts that are needed?
What is the common practice?
On modern browsers you can use sessionStorage to have it persist during the current session, or localStorage to have it hang around between sessions.
NB: both only permit storage of strings - you'll have to serialise the array (e.g. into JSON) and deserialise it on retrieval.
If you want to actually use the word list as a local database with efficient lookup you might also want to investigate indexedDB
you can place the data in session and retrieve it, the same can be used in every page with out fetching the same every time.
Thanks & best regards.
If you need all the 40k words in all pages then you can use localStorage or sessionStorage. Just keep in mind sessionStorage will delete saved data when the tab/window is closed so the whole array will be downloaded again when the website is opened in new windows/tabs.
If you need only specific parts of the array in different pages I would tidy the array's elements into taxonomy/categories (if you are able to), so that you can download only the needed for a specific section of your application.
This depends on the composition of your array, if it is formed only by words or complex objects. This will help to avoid slow load of your website when it's visited the first time.
If the array is always the same (there is no need to update it), I'd create a js file and then I'd add it to every html page. The browser's cache would do the rest to avoid unnecessary re-loading. Something like:
big-array.js file:
var myBigArray=[...]
In each html file
<html>
... whatever you need
<script src="/my-path/big-array.js"></script>
...my other scripts here
</html>
It's a bit difficult to answer this question properly as to do so would require more information about your hosting environment and what you have access to. If you have a server side language available, such as PHP, you could look at caching which is generally the most efficient way to handle data that is used repeatedly across pages. Perhaps you could post more info about what technologies you have available to you?
Inorder to reduce server load, I would like to bring many task into client side. There is no clear information about browser memory limit and its recycling ... Just like PHP session is there any clean function or any limit of storage size for browsers?
Basic javascript variables will last only for the duration of the page
For more persistent variables, use either sessionStorage (these will last until the user closes their browser/tab) or localStorage (these will last "forever"/until the user clears their personal data)
https://developer.mozilla.org/en-US/docs/Web/Guide/API/DOM/Storage
Note that the above are available in all modern browsers, but notably not in IE<=7, in which case you would need to use cookies as a storage mechanism. However, if you need that, you can probably find a library that will take care of the nitty gritty details
I want to use localStorage to store large amount of data(like 800GB), according to http://arty.name/localstorage.html and also I'm using Firefox, I changed my localStorage size and also the cache size. So the size isn't a problem. However, I write some jquery like the following:
$("a[href]").each(function(){
$(this).click(function(event){
localStorage.test += "somenewinformation";
...
If this localStorage.test already have large amount of data like 400GB, so the storing information step would be extremely slow. When I click on a link,will the jquery wait for me to finish the appending new information to localStorage.test or it will just go to the next page and information in localStorage.test will all lost or localStorage.test will just remain the old value? What I dont understand is whether a new thread will be generated to do this storing in background or not and closing browser in the middle will affect it or not.
Sorry about the messy description and thanks in advance!
You can't! The usual limit is 5 MB.
Some browsers such as Opera allow you to adjust the size, but this is purely dependent on the browser and is a user-initiated action, not a programmable one.
And even if you could remember that localStorage can only store strings, so anything else need to be stringified first. That together with this being an key-value storage array you will run into pretty poor performance at the end.
If you need large storage capacity, look into File API instead. This is made to work with large files (Blobs) and you can store anything as a Blob.
800 Gb is a large size and File API can only work as fast as the file system (at best, more likely a bit slower as it is sandboxed, ie. not a pure file system).
More about File System API (Note: discontinued as of 4/2014):
http://www.w3.org/TR/file-system-api/
Tutorial:
http://www.html5rocks.com/en/tutorials/file/dndfiles/
Blob:
https://developer.mozilla.org/en-US/docs/Web/API/Blob
Update As the Filesystem API has been discontinued there are essentially only two options left (besides from storing data to the server):
Indexed Database API aka Indexed DB (recommended)
Web SQL (deprecated but still supported in browsers such as Safari)
Also these are initially limited in size by the browser. It is possible to request a larger quote which the user is asked to accept.
See more details about this API here:
http://www.w3.org/TR/IndexedDB/
There is LargeLocalStorage which provides a cross-browser way to storage large amounts of data locally. You can store binary data or strings.
LargeLocalStorage uses the FilesystemAPI on Chrome, IndexedDB in IE and Firefox and WebSQL in Safari.
I'm making a to-do list application with HTML, CSS, and JavaScript, and I think the best way for me to store the data would be a local database. I know how to use localStorage and sessionStorage, and I also know how to use an online MySQL database. However, this application must be able to run offline and should store its data offline.
Is there a way I could do this with just HTML and JavaScript?
Responding to comments:
"You said you know how to use localStorage... so what seems to be the problem?"
#Lior All I know about localStorage is that you can store a single result, as a variable whereas I wish to store a row with different columns containing diffenent data about the object. However, can localStorage hold an object and if so is it referenced with the usual object notation?
Any implementation will probably depend on what browser(s) your users prefer to use.
#paul I think chrome will be most popular.
Okay, I would like to clarify that what I was asking was indeed How can I do this with JavaScript and HTML rather than Is there a way I could do this with just HTML and JavaScript?. Basically, I wanted a type of SQL database that would save its contents on the user's machine instead of online.
What solved my problem was using WebDB or WEBSQL (I think it was called something like that that).
I'm about 3 years late in answering this, but considering that there was no actual discussion on the available options at the time, and that the database that OP ended up choosing is now deprecated, I figured i'd throw in my two cents on the matter.
First, one needs to consider whether one actually needs a client-side database. More specifically...
Do you need explicit or implicit relationships between your data items?
How about the ability to query over said items?
Or more than 5 MB in space?
If you answered "no" to all of the above, go with localStorage and save yourself from the headaches that are the WebSQL and IndexedDB APIs. Well, maybe just the latter headache, since the former has, as previously mentioned , been deprecated.
Otherwise, IndexedDB is the only option as far as native client-side databases go, given it is the only one that remains on the W3C standards track.
Check out BakedGoods if you want to utilize any of these facilities, and more, without having to write low-level storage operation code. With it, placing data in the first encountered native database which is supported on a client, for example, is as simple as:
bakedGoods.set({
data: [{key: "key1", value: "val1"}, {key: "key2", value: "val2"}],
storageTypes: ["indexedDB", "webSQL"],
//Will be polyfilled with defaults for equivalent database structures
optionsObj: {conductDisjointly: false},
complete: function(byStorageTypeStoredKeysObj, byStorageTypeErrorObj){}
});
Oh, and for the sake of complete transparency, BakedGoods is maintained by this guy right here :) .