Check if object fits into browser cache - javascript

I have a single page application, which loads a JSON data file from a server and displays it on the client. The data file only changes once per day, so after fetching it, I want to keep it cached on the client. So far, I have been using jStorage and am happy with the overall result. One thing is problematic though: according to the jStorage Browser support page, the cache size varies from browser to browser.
So my question: can I somehow find out if jStorage will be able to store all my data? The jStorage website itself does not give any clues to that.
Alternatively, since I'm only storing one big object, I could use a plug-in like sizeOf to check the object size before saving it, but I don't know if that is a reliable approach.

I am not sure how jStorage implements, you say 5mb so I guess it's using localstorage. Have you tried indexedDB which is browser native and theoretically limit to your hard drive capacity.
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API

Related

Writing and downloading files clientside crossbrowser

I have a program where the user does some actions (i.e. clicking on several buttons). I want to record their clicks and the buttons that they click to allow the user to then download a text file with a record of their clicks when they click a separate "download" button. I looked at the File-system APIs for HTML 5, but they seemed to not have cross-browser support. I would ideally like to have this entire file generation and download scheme be entirely client-side, but I am open to server-side ideas as well.
TL;DR: Essentially I'm looking for an equivalent to Java's FileWriter, FileReader, ObjectOutputStream, and ObjectInputStream within Vanilla JS or jQuery (would like to stay away from php, but I'll use it as a last option).
Also, why don't all browsers support the filesystem api? (I'm guessing that it would make MSWord and Pages go out of business with all the open source clientside text editors that could come out.)
Unfortunately the HTML5-File-system is no longer a part of the spec, long story short FF refused to implement because they claimed everything you could do in the File-System API was doable in the HTML5 Indexeddb (which was mostly true). Please see this blog post for more on why FF didn't implement. I do not know IE's story. (I may have exagerated why FireFox didn't implement, I'm still bummed because you cannot actually do everything in indexeddb that you can do in the noew "Chrome File-system API")
Typically if two of those three browsers implement a spec, it stays in the spec. Otherwise that spec gets orphaned. However, I'm fairly certain a large reason the file-system api didn't take off is because of the IndexedDB API (caniuse IndexedDB) really took off when both specs were introduced. If you want cross browser support, check this api out.
That all said if you are still set on the file-system api some developers wrote a nice wrapper around the IndexedDB, the File-system api wouldn't actually supply you with a stream anyway. You would have to keep appending events to a given file given a fileWriter object. you'd then have to read the entire file and send to the server via an ajax request and then downloaded from the server once successfully uploaded.
The better route would be to use the IndexedDB apiwhich as stated on developer.mozilla
Open a database.
Create an object store in upgrading database.
Start a transaction and make a request to do some database operation, like adding or retrieving data.
Wait for the operation to complete by listening to the right kind of DOM event.
Do something
with the results (which can be found on the request object).
Here are a couple tutorials on the IndexedDB.
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB
http://www.html5rocks.com/en/tutorials/indexeddb/todo/
As for giving the user that file, as mentioned briefly before you would have to upload the file to the server and download upon the "download" request. Unfortunately you have to trick the user into giving them the data already on their machine. Anyway, hope this all helps.

File API, Persistent link to a local file

I'm currently working on some web script based on a game (to port a game to the web).
The scripts download data from my web host so the loading is slow (have to download each files : maps, models, textures, ...).
To correct this, I added an option that allow users to select their local game data in their computer (using the File API - drag and drop) to parse content directly from local and avoid downloading multiple megas from the web, the result is incredibly fast.
Here the problem : each time they reload the browser, they have to re-select their files, again and again. It's not user-friendly.
So, is there a way to keep a reference from this game archive to avoid the user to re-do the drag and drop each time ? I know about security concern, just want to know if there is something like a persistent URL.createObjectURL().
Note: the game data is about ~2Go, so it's not possible to store it in the FileSystem API (and I don't want to copy it, it's waste space to copy data when you can just keep a reference to it).
Thank you :)
You have to have an input from the user
It is not possible to access the files on a client's computer without the user confirming it. Once the user chooses a file (You can listen to it with the change event), you can then use the FileReader API for example to read the file.
document.getElementById("input").addEventListener("change", function() {
const fs = new FileReader();
fs.readAsText(this.files[0]);
fs.onloadend = function() {
document.getElementById("output").innerText = fs.result;
}
});
<input id="input" type="file" />
<div id="output"></div>
Using localStorage to store a fraction of your files
You could use the localStorage API to store some of your files, but the capacity is very limited, especially for a project like yours (maximum of 5 to 10 MB on current most popular browsers).
This would make your coding much harder, as you would have to check every time what was stored or not, and load what is not saved.
Caching
By using caching, you basically fall into the same problem as you are with localStorage: each browser has its own maximum capacity.
The advantage of this method is that you do not have to worry about what has been loaded or not, as the browser will do this by itself.
Using Flash
Now if you really do not care at all about security, you could use a Flash plugin to store and load the files, and then use ExternalInterface to load the data in your JavaScript code.
ExternalInterface.call("loaded", filename, data);
// And then in JavaScript:
// function loaded(filename, data)
// ...
You could use SharedObject to save and load your data.
I am not an AS3 expert, please excuse any clumsiness.
A Desktop application?
Last option would be converting and bundling your game into a desktop application, for example by bundling it through electron, and then using for example NeDB, which is currently the suggested tool by electron for persistant storage.
You may want to consider using IndexedDB. It is supported in recent browsers including Chrome, Firefox, and Safari (macOS and iOS). IndexedDB allows you to save Blob, File or ArrayBuffer as values in an IndexedDB object store.
Check this IndexedDB: Store file as File or Blob or ArrayBuffer. What is the best option?.

How can I generate and download a large file clientside using Javascript?

I have an application that displays a large set of data using Slickgrid. The dataset may be 30-50MB in size. I would like users to be able to download the current view of the data displayed. What is the best way to set this up?
I have considered the approach described here using data URIs, but the maximum size of a URI is much too small.
I have also considered the approach described here where the client posts arbitrary data to the server, which the server echos back as a download. I worry that the documents may exceed the maximum POST size.
Since you want to do this on client side and If HTML5 is an option why not HTML5 File System API
One option is to use the HTML5 FileWriter API. As of today it's only supported in Chrome (and the BlackBerry browser of all things).

Web client cache

We are in process of building conceptual model of web-based audio editor. And the first trouble we met is client-side caching system. In my opinion as server-side programmer having huge cache on client side is perfect idea, because in many cases it takes of server load by excepting multiple loading of the same data. Furthermore such cache could be good candidate for buffer for providing per-track operations, like filtering.
Our flex programmer says that this is a great trouble and it is impossible in almost any cases. But I am in great doubt, cause I know that actual Google Chrome browser version can simple keep up to 2 Gb in localStorage. Moreover I've found this example of online track-editor and looks like its caching mechanism working pretty good.
Is it possible to cache some data (smth about 100-200mb) on the client side using flash and js?
You can use SharedObject to store the data.
I am pretty sure that default size limit is too low for your needs, so your app will need to ask user to accept your new limit:
http://www.macromedia.com/support/documentation/en/flashplayer/help/help06.html
SharedObject is more reliable than the browser cache, and you control it from your app.
If you are using html5 then you can store large data on client side using html5 inbuilt database.
also refer this link
What we did when writing a video editor. Well, actually, in Flash you can save files to the user's machine, with the restriction that it must be transparent to the user (i.e. the user initiates the action, goes through the OS dialog and saves the file as they would normally save anything they download), similarly, you can load in a file from a user's computer, with the restriction that the user must initiate the action (as in by clicking with a pointing device or pressing a key).
This has certain advantages over different local storage strategies, which are mostly opaque to users (people don't usually know how to erase cookies, SharedObjects or web storage that comes with more modern browsers, but they are pretty much capable of saving and deleting the files on their system). Furthermore, all other opaque local storages may have restrictions that less savvy users might not know how to overcome / may not be possible to overcome in general - these would be size, location and ownership.
This will still be a bit of hindrance for your audience, because every time they need to save a file, they have to go through the OS's dialog, instead of doing Ctrl+S / Cmd+S / C-x C-s... But given all other options, this, IMO, leaves the user with the most choices / delivers best experience.
Another suggestion - you could, in principle, come up with a browser-based "enhanced" version of your application, which users would install as a browser plugin (if that's an editor they are using on a regular basis - why not?), in which case you wouldn't be limited to the clumsy options provided by web technologies. Chrome and Mozilla-based browsers encourage such development, however it's not standardized. Still, since these two browsers run on virtually any OS, that doesn't sound particularly as locking in your users into a certain platform...

How to Extend localStorage

I am using the localStorage in this demo here,
http://help.arcgis.com/en/webapi/javascript/arcgis/demos/exp/exp_webstorage.html
Basically it is a mapping application which caches map tiles in the localStorage.
I quite quickly reach the 5MB limit and from then onwards I get errors, QUOTA_EXCEEDED_ERR.
How can I extend the localStorage? Or what other options do I have to store data on the client side in HTML5, has anybody used the indexdDB, does it work in chrome?
http://www.w3.org/TR/IndexedDB/
And of course the web database specification has been deprecated so I would like to avoid that,
http://www.w3.org/TR/webdatabase/
My understanding is that the user can extend localstorage but the website can't (by design). You simply need to catch the error in Javascript and show the user a dialog requesting they increase their storage limit - preferably providing some instructions for major browsers.
EDIT: Perhaps not so simple. It seems some browsers don't allow the user to increase the storage size. Google seems convinced the localStorage API doesn't scale well to large files and developers should consider IndexedDB instead.

Categories

Resources