I'm currently working on some web script based on a game (to port a game to the web).
The scripts download data from my web host so the loading is slow (have to download each files : maps, models, textures, ...).
To correct this, I added an option that allow users to select their local game data in their computer (using the File API - drag and drop) to parse content directly from local and avoid downloading multiple megas from the web, the result is incredibly fast.
Here the problem : each time they reload the browser, they have to re-select their files, again and again. It's not user-friendly.
So, is there a way to keep a reference from this game archive to avoid the user to re-do the drag and drop each time ? I know about security concern, just want to know if there is something like a persistent URL.createObjectURL().
Note: the game data is about ~2Go, so it's not possible to store it in the FileSystem API (and I don't want to copy it, it's waste space to copy data when you can just keep a reference to it).
Thank you :)
You have to have an input from the user
It is not possible to access the files on a client's computer without the user confirming it. Once the user chooses a file (You can listen to it with the change event), you can then use the FileReader API for example to read the file.
document.getElementById("input").addEventListener("change", function() {
const fs = new FileReader();
fs.readAsText(this.files[0]);
fs.onloadend = function() {
document.getElementById("output").innerText = fs.result;
}
});
<input id="input" type="file" />
<div id="output"></div>
Using localStorage to store a fraction of your files
You could use the localStorage API to store some of your files, but the capacity is very limited, especially for a project like yours (maximum of 5 to 10 MB on current most popular browsers).
This would make your coding much harder, as you would have to check every time what was stored or not, and load what is not saved.
Caching
By using caching, you basically fall into the same problem as you are with localStorage: each browser has its own maximum capacity.
The advantage of this method is that you do not have to worry about what has been loaded or not, as the browser will do this by itself.
Using Flash
Now if you really do not care at all about security, you could use a Flash plugin to store and load the files, and then use ExternalInterface to load the data in your JavaScript code.
ExternalInterface.call("loaded", filename, data);
// And then in JavaScript:
// function loaded(filename, data)
// ...
You could use SharedObject to save and load your data.
I am not an AS3 expert, please excuse any clumsiness.
A Desktop application?
Last option would be converting and bundling your game into a desktop application, for example by bundling it through electron, and then using for example NeDB, which is currently the suggested tool by electron for persistant storage.
You may want to consider using IndexedDB. It is supported in recent browsers including Chrome, Firefox, and Safari (macOS and iOS). IndexedDB allows you to save Blob, File or ArrayBuffer as values in an IndexedDB object store.
Check this IndexedDB: Store file as File or Blob or ArrayBuffer. What is the best option?.
Related
I have a single page application, which loads a JSON data file from a server and displays it on the client. The data file only changes once per day, so after fetching it, I want to keep it cached on the client. So far, I have been using jStorage and am happy with the overall result. One thing is problematic though: according to the jStorage Browser support page, the cache size varies from browser to browser.
So my question: can I somehow find out if jStorage will be able to store all my data? The jStorage website itself does not give any clues to that.
Alternatively, since I'm only storing one big object, I could use a plug-in like sizeOf to check the object size before saving it, but I don't know if that is a reliable approach.
I am not sure how jStorage implements, you say 5mb so I guess it's using localstorage. Have you tried indexedDB which is browser native and theoretically limit to your hard drive capacity.
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API
I'm working on a Chrome extension to be used as an internal tool. Its required behavior is:
As a page action, enable an address bar icon when looking at certain intranet pages.
when the user clicks the icon, identify all files of a certain media type (say, .jpg) on the page, and
silently save them all to a directory on the user's local drive.
This question has been asked before, but the answer then was "use NPAPI", and NPAPI is now derelict.
So, what is the currently available way to achieve this? The ones I've looked at are:
The chrome.FileSystem API --- but this does not save files in any user-accessible location. Instead the stored files are hidden behind obfuscated names in an undocumented directory. User requires that the files be stored under their original names in an accessible directory.
The HTML5 download attribute, by creating a data: URL and programmatically clicking it. This pops up a "save as..." dialog for each file, which is unacceptable when there are a hundred assets on a single page. User requires that the files be downloaded without further interaction beyond the single icon click.
The Chrome Download API, but that is only available in the beta and dev channels. User requires this extension work with mainstream Chrome.
Use the Native Messaging API by creating a small .exe that simply saves a file to disk, and then pass the .jpg as a blob to it. This seems very cumbersome and I am not even sure how to reliably pass large blobs to EXEs like that.
Is there another approach I can try?
You've done quite a lot of research. Indeed, regular web pages cannot write to the user's filesystem without any plugins or extensions. Also, the HTML5 Filesystem API only provides access to a virtual filesystem, as you've observed.
However, you are confusing the chrome.fileSystem API with the HTML5 FileSystem API. Unlike the HTML FileSystem API, Chrome's fileSystem (app) API can directly write to the user's filesystem (e.g. ~/Documents or %USERPROFILE%\Documents), specified by the user.
This API is only available to Chrome apps, not extensions. This is not a problem, especially since you're developing an internal tool, because you can install the app and extension, and use message passing to communicate between the extension (page action) and app (file system access) (example).
About chrome.downloads: Since your extension is internal, you can probably force users to get on the beta/dev channel in order to use this API. The only limitation of this API is that the files will be saved in (a subdirectory of) the user-defined Downloads folder.
EDIT: The chrome.downloads API is now avaiable in all channels, including the stable branch (since Chrome 31).
I am afraid that you have done your homework, meaning you looked at all possible alternatives.
The best way to achieve exactly what you want, would be (as you mentioned) using a supporting native app and communicating through Native Messaging. BTW, since bandwidth is rarely a problem on intranets, you might find it simpler to pass the resources (e.g. images) URLs and have the app download and save them.
(Yes, it will be more cumbersome than simply developing an extension, but one's got to do what they've got to do, right ?)
On the other hand, if you are willing to sacrifice a little bit of user's experience over simplicity of development, I suggest combining the HTML5 goodies (that allow you to create and download a file locally) with a JS zipping library (e.g. JSZip), so the user only has to download a single zip file (and is only prompted once). BTW, if the user wishes, he/she can choose to always download the files without prompting (but you knew that already).
Use the Native Messaging App idea.
The native app is cumbersome and a pain to write because documentation is poor, and unless you get the JSON formatting exactly correct on both ends you will not see anything in a console because stdin and stdout are taken over.
But, you will be happier when it is done because you can use standard tools (e.g., Windows Explorer, a hex editor, TeamViewer...) to view, move, and delete files, and otherwise see what is going on. Chrome's sand-boxed file system works, but seems to now be a dead-end (no other browsers have picked it up). No one is likely to develop third-party tools for it. Of course you probably don't need tools once everything is working, but until then, debugging is a nightmare because you need to write code (and quite a lot of code) just to track what files are in what directories, file versions, remaining disk space...
Another solution for internal (or may be non-internal) usage is to connect to a websocket server, local or remote.
You can put it in both background.js or content.js (use wss:// for https://)
var ws = new WebSocket('ws://echo.websocket.org');
// var ws = new WebSocket('ws://127.0.0.1:9000');
ws.onmessage = function(res) {
console.log('received data:', res.data);
};
ws.onopen = function() {
ws.send('hello');
};
The set-up in question:
I have a stand alone, offline, kiosk mode instance of Chrome running on a Windows machine. I have full access to the system and any admin rights. I can start Chrome with any flags set or unset.
The task:
I have been asked to create a log file which tracks user activity within the offline app I am coding. It's a simple form of analytics which will append each event to the end of the file separated with a comma. The file will then be sent to a server once a day via a scheduled task. (None of this is my idea so please don't troll me)
Ruled out:
Any server side code - I have lobbied for Node, PHP etc but as this will be distributed to many different installations so we cannot guarantee this will be installed.
Flash/ActiveX/Java - ideally would like to avoid this (even though these will be installed by default)
Possible solutions:
File API - I have looked at this but AFAIK if opens dialogue boxes to save the data to each file and this needs to happen in the background.
Security - I have read in other SO Questions that this can be achieved if the security settings are reduced but no-one goes on to explain which ones. Is there a simple flag which allows it?
How to enable local file system read write access from Google chrome? - similar question!
Ideal result: (something akin to PHP)
$file = 'log.txt';
$current = file_get_contents($file);
$current .= ",clicked:link";
file_put_contents($file, $current);
Possible ideal side result: proving this isn't possible and forcing PHP/Node/Java to be used ;)
In reply to those suggesting local storage : I'm not storing unique key/value pairs and that is very much like setting a cookie. Similarily there are file size limits.
To those suggesting web SQL such as SQLite in chrome - there are file size limits if it's not a chrome extension. The only way I see that working is if I were to find the location of the file in the windows directory (C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Default\databases) and upload that from the schedules task. Perfectly feasible but it is not a desirable answer.
You could use HTML5?
http://diveintohtml5.info/storage.html
var foo = localStorage.getItem("bar");
// ...
localStorage.setItem("bar", foo);
You can use the Chrome Apps File API, you will need to grant access to the file via a user action once, but after that you can get access the file again by using restoreEntry
You can use localStorage to save offline data. It's impossible to access other files using Javascript since it's a violation of the sandbox.
From http://en.wikipedia.org/wiki/JavaScript#Security:
JavaScript and the DOM provide the potential for malicious authors to deliver scripts to run on a client computer via the web. Browser authors contain this risk using two restrictions. First, scripts run in a sandbox in which they can only perform web-related actions, not general-purpose programming tasks like creating files.
You may want to look into Local Storage which is part of the HTML5 spec.
This will only be supported in modern browsers though.
If you need to use older browsers then may still be able to achieve what you're after using dojox.storage
Use HTML5 features like Web Storage or Web SQL database to store your logs.
Whenever needed read logs from the client side storage and send it back to the server & delete the client storage.
Refer http://www.html5rocks.com/en/features/storage.
Say I have a webapp which executes in its entirety on the client-side. Its purpose is to act as a file conversion utility, for example converting a user's local stored word document into a PDF.
So with the user's permission, the app can read a specified local file and process it, in memory, into PDF format.
How can I get the user to 'download' the result? since the data is held in the browser's memory anyway, I do not wish to upload it to some server.
[edit]
No flash based solutions
Expected file sizes to be up to 15mb
The solution for my case will be to use the HTML5 FileSaver API.
Perhaps this question should just be closed as it is effectively a duplicate of
Using HTML5/Javascript to generate and save a file
Thanks to aefxx
I want to allow a web site users to be able to download files from my site, but with the help of a client-side downloader with an ability to continue an interrupted download.
For example, I want to sent a person a file with a size of 30+ Meg. I want the user to have the best downloading experience, so I can't afford him downloading 25 Meg and then getting the download dropped due to the network problems on his side.
Therefore, I want to have a javascript downloader rendered on a download page, that will show the actual client-side file delivery, and when it is downloaded, to give an ability to a user to save the file.
Or is it not possible due to the fact that javascript won't be able to open a save file dialog and save to a file system?
I'm afraid that is not possible with JavaScript and that's why:
To continue downloading from the certain point you should send to the server the position number to start downloading from. And as JavaScript has no access to local file system, you can't get that position.
UPD: it seems that I was too hurrying with the reply.
The file size can be gotten using the HTML5 File API and after getting the file size you can pass it to the server which should support the partial downloading.
But anyway, after downloading another part of the file you should sew two pieces together in some way; standard web browser dialog will only suggest to overwrite the file.
UPD2: to work with files in some Internet Explorers you can use FileSystemObject:
var fso;
fso = new ActiveXObject("Scripting.FileSystemObject");
I'd look into making a plugin or extension. Like those DownloadThemAll extensions for firefox and Google chrome. Another alternative would be to use Flash, either alone or integrating it with javascript like hinted here: http://www.communitymx.com/content/article.cfm?cid=0922A