HTML5's localStorage WebStorage has a maximum size of 5MB.
Does this include the key names?
For instance, if I were to use key names "quite-a-long-key-name-and-this-is-only-1" instead of "key1", would I run out of space sooner?
On a slightly related topic; is there any de-facto convention for naming localStorage keys?
How are namespace collisions prevented when using third party JS scripts?
Does this include the key names?
Yes those do, they become part of data eg they identify data you store and later retrieve so that got to be saved as well.
How are namespace collisions prevented when using third party JS
scripts?
That's a good question, I generally prefix localStorage with application name. Though a better approach would be to create a hash eg some algorithim that accepts a string like application name, etc and later when reading you use them again.
First note that this is implementation dependent as the norm doesn't give a limit. So you shouldn't rely on the size.
Secondly, yes, the limit in today's browsers includes the names : this is the size of the storage space ("disk space").
In order to avoid collisions, I use namespace (eg myplugin.mypart.myval). 5 MB is already big for a storage that can be deleted or not available any more at any time so I never thought about reducing the size of the keys...
Related
I'm looking for a solution that allows me to cache a large object over a page reload. More specifically I have a large crossfilter.js object that I'd like to retain, since creating it takes a while. I couldn't find any native way to persist a crossfilter.js instance.
I know about the following generic options:
Local storage/session storage. Problem: Apparently no objects bigger
than 5 or 10 MB can be stored, and the ones I'm trying to cache are upwards of 10 MB.
Abusing window.name for a serialized version of my object.
Problem: While it works for classes I've written myself, trying to
serialize and de-serialize the crossfilter.js instance or its
groups/dimensions leads to exceptions; i. e. the internal state of the
crossfilter.js instance is not maintained. I'm using https://github.com/hunterloftis/cryo for serialization.
IndexedDB. Problem: Same as with window.name - I'd have to
serialize my data, which I haven't found a feasible approach for yet. Also a bit of an overkill for my needs, I guess.
Summarized: I want to keep a particular complex object in memory after a page reload. Possible solutions would allow to
store complex objects/class instances (with methods) in the
local/session storage with increased or no memory limits,
using a hack like dumping the object in window.name, but accepting a
complex object without the need for serialization as necessary for
window.name or
using native crossfilter.js functionality to dump/cache one of its instances.
Any tips? Although a browser-independent version is preferred, a Chrome-specific solution will be accepted as well.
Thanks!
I was asking myself this question. Is Set a hashed collection in JavaScript?
For example, Set.prototype.has will iterate the entire Set or do its implementations use an internal hash table to locate an item within the collection?
The ECMAScript 2015 specification says that:
Set objects must be implemented using either hash tables or other mechanisms that, on average, provide access times that are sublinear on the number of elements in the collection.
Obviously they can't force a particular JS engine to actually do that, but in practice JS engines will do the right thing.
The ES6 specification does not require a specific implementation, but it does indicate that it should be better than O(n) (so better than a linear lookup). And, since the purpose of the Set object is to be efficient at looking up items in the Set, it most surely uses some sort of efficient lookup system like a hash.
If you want to know for sure how it works, you'd have to look at the open source code for the Firefox or Chrome implementations.
You could also benchmark it to prove that the lookup speed is not O(n), but something more efficient than that.
I've got a web app which gets a couple dozen items at boot. All these items are JSON and are smaller then 1kb.
Now there are a number of storage options as seen in the Question.
I was thinking of just storing these objects inside a variable in the browser JS. I don't really see why I would want to use any of these browser storages?
So what would be reasons to use any of the browser-based storage instead of a variable inside JS.
Could be that from a certain data size it is preferable to use browser storage, e.g. from 100kb onwards it's better to not use a JS variable.
var myModel = {}
NOTE
Every time the user enters the app he will get fresh content from the server. The content is too realtime for caching.
`
localStorage , globalStorage and sessionStorage:
These features are ready in browsers that have implemented the "Web Storage", they all refer to a kind of HashMap, a map between string keys and string values. but the life is different. once the active page is closed sessionStorage would be cleaned but the localStorage is permanent.(MDN DOM Storage guide)
There is a point about globalStorage, which is its being obsolete since Gecko 1.9.1 (Firefox 3.5) and unsupported since Gecko 13 (Firefox 13), since then we should use localStorage. the difference between these 2 was just the HTML5 scope support(scheme + hostname + non-standard port).
These could be useful for you to:
-Share your objects between your different pages, in your site.
-Offline programming.
-Caching large object
-Or whenever you need to a local persistent storage.
IndexedDB:
IndexedDB is useful for applications that store a large amount of data (for example, a catalog of DVDs in a lending library) and applications that don't need persistent internet connectivity to work (for example, mail clients, to-do lists, and notepads)
based on this quote from MDN you can easily find your answer out, regarding using IndexedDB, if you don't know whether IndexedDB is useful for you or not, just answer these questions:
Do you store a large amount of data on client? if yes, so consider using it.
Does your app need to be offline enabled? if yes, so consider using IndexedDB.
Does your app need to a persistent internet connectivity? If yes, it stays still an option, based on the other factors.
So other than working offline as far as you don't need it, I guess, because as you said:
The content is too realtime for caching.
These have some features like sharing objects, and managing large amount of data, which you should be the one to decide.
localStorage and sessionStorage are solving a caching problem; think of them as cookies. You've said you don't want caching, so you can ignore them.
JavaScript objects behave basically like O(1) lookup tables (see How is a JavaScript hash map implemented?, and make sure you read both the top two answers, as both have something useful to say), and there is no maximum memory limit that I am aware of, or a point where another solution becomes a better choice
The only reason I can think of that you should bother with the extra step of inserting the data in an IndexedDB is if you need O(1) lookups on a field that is not the object key you are using.
I was asked this during an interview. My immediate answer was for every read and write. The interviewer then asked, "Are you sure the hash isn't cached in the table somewhere?"
This made me second guess myself. In the end, I stuck to my original answer, but out of curiosity, I figured I'd as the question here.
Also note that this interview was for a JavaScript position but the question wasn't necessarily specific to JavaScript.
So, in general, is a key's hash computed once or for every read/write? What about specific to JavaScript?
Of course it depends on the implementation, and even if you ask about JS there are several implementations (V8, SpiderMonkey, MSFT etc.).
It also should depend on the application. If your application is one that more frequently use the last item put into the hashtable then it should make sense to somehow cache the hash. In some cases this would be preferable.
I guess the interviewer just tried to see how you handle second-guessing...
It depends on the hash table and the key types, and on whether we're talking about the key used to read/write or the keys already in the table. The hash values of the former can and sometimes is cached in the object (example: strings in Python). The hash values of the latter can and sometimes are cached in the table - instead of key, value pairs you store hash, key, value triples.
In both cases, the decision depends on the kind of keys: Are they large and expensive to hash? Is it worth the extra space and memory traffic? For example, it's probably a clear win for strings larger than a couple dozen characters, and probably useless or harmful for 2D points. Also note that the hash values can be used to avoid comparisons, which might be useful but doesn't seem as important.
I need to write a script to manage conflicting versions of a document in local storage. I'm currently thinking about two alternatives and have trouble deciding which one will be more scalable.
My documents will be stored as JSON and will have an id and a revision to identify the version.
Currently I'm creating a path in localstorage like so:
PATH/TO/DOCUMENT/id
At this path the document is stored as JSON
{"id":"abc","version":"1-a","content":"foo"}
I'm using POST,PUT(update),GET and REMOVE. POST/PUT require id and version while GET/REMOVE only require id.
To allow for conflicting versions to exist in local storage I'm not sure whether to
a) store at existing path and add version as 2nd JSON string like so:
PATH/TO/DOCUMENT/id {"id":"abc","version":"1-a","content":"foo"},
{"id":"abc","version":"2-b","content":"foodforthought"}
b) store at path.id and keep a "single files"
PATH/TO/DOCUMENT/id.1-a {"id":"abc","version":"1-a","content":"foo"}
PATH/TO/DOCUMENT/id.2-b {"id":"abc","version":"2-b","content":"foodforthought"}
Question:
Which one makes more sense in terms of scalability and a lot of different versions to exist?
Which one makes more sense in terms of scalability and a lot of different versions to exist?
Option B. Here you can read / write single document versions without loading the whole JSON string (and then object, when it comes to manipulation) into the memory. Especially when it comes to many and huge documents, this will save you some time.
However, if you have lots of versions of one huge document that do not differ much from each other, it might be better in terms of performance and memory-usage to store them with incremental or differential versioning, which would make more sense in one single "JSON file".
Choose Option A: If each JSON entry is small, having only the id, not the whole document. This is easy to manage and cleanup.
Choose Option B: If each JSON entry is large. I agree with #Bergi
If I understand correctly it is a question about best architecture.
Since you may have "a lot of different versions", you need a quick lookup for localStorage therefore the option b is better. Because in option a, whenever you want to find a specifiv version of the document, you have to go through all items that their id is abc and then iterate through them (worst case: linear search) trying to find the version you are looking for.
However, using . (dot) as the separator isn't probably the best idea. The separator should be a character that is not used in the version number and file name. I would suggest something like /. So the keys will be like:
PATH/TO/DOCUMENT/id/1-a {"id":"abc","version":"1-a","content":"foo"}
PATH/TO/DOCUMENT/id/2-b {"id":"abc","version":"2-b","content":"foodforthought"}