I have a GitHub pages site where I'm hosting a project. It allows users to export their data from localStorage as stringified JSON object into a .txt file. They can then import their data back from that .txt file, which will store its contents into localStorage.
Having them paste the file contents into a text box is an option, but since I'm intending this to be a single-page application for use on mobile devices, that's not an ideal user experience.
How do I make sure they aren't uploading malicious, incorrect, or unusable data?
As far as security goes, I'm not sure how much of a risk this even is, since GitHub Pages only hosts static pages, and I'm not dealing with any sensitive data in any way. Still, it feels like I should be doing something other than just accepting plaintext files.
The first thing that comes to mind for validating the data is to use regex or another formulaic way to check object contents. The data is organized as an object of objects; all child objects will have the same keys with different values, and the number of objects can vary. I also plan to build in a way to handle empty file uploads, where it defaults to setting localStorage to {}.
Related
I'm writing a web app which fetches a list of files from the server and displays it. The user can click on a folder to descend into it (without actually leaving the page). Retrieving the list can take a while (~10ms per file, which is a lot when you have 2000 files), so I want to cache the results when possible to avoid having to re-fetch it if the user goes into a subdirectory and then back out.
However if I just store the results in some global variable, they'll quickly fill up all the user's memory. Is there some way to tell the browser "feel free to delete this object if you're low on memory", or to be notified when memory is low?
If you'd like to store those objects on the users computer, to prevent requesting from the server again you'd probably want to use something like LocalStorage to do so.
Store.js provides a nice API around local storage-solutions.
The hard part for you now will be to check which files belong to a certain folder so you can store it. Something like a tree data-structure might be nice to give shape to these folders, paired with an ID you might be able to map them to a place in localstorage.
I have a research project which is focusing on understanding user's curiosity and connecting to Free WIFI including remote areas where internet is poor here.
To do this, we developed simple webpages with the first as a Visitor's form where we need a user/visitor to enter their data and let it be retrieved, saved or viewed later as user logs. We want to do this by simply having a Router and a Flash disk where the webfiles and data storing system will be...No server!
Is this very possible with Javascript, xml and or any other languages anyone has ever done this? That is except Javascript's LocalStorage which in this say, the user will be the one to have his/her own data.
This is an unusual request really. I agree a relational DB would be the best choice. Of course you can just save to text file (xml or not) and later use that file in Excel or so. That would be my choice.
You can probably store data with LocalStorage, and allow user to save data by exporting a .JSON or .CSV or something like that.
You can give the user a file to download by doing a window.open() on a Data URI containing the text of the JSON or CSV:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs
In my rails app, I have a page that accesses data from a JSON file. However, I want to prevent the JSON file from being accessed directly with its URL without the user being authenticated.
I've tried putting the file in the assets pipeline, but it's still accessible from a URL without authentication, just the URL has an additional string of random characters inside of it. I'd also rather not paste the entirety of the JSON into the javascript file, because at later stages the project will be integrating multiple JSON files some of which will reach sizes of 50 MB and which may also be updated periodically.
Is there a way I can keep the JSON file totally secure from outside access while still allowing Javascript to use it? If not, are there any alternatives that would be relatively easy to implement?
My question is about IndexedDB vs. WebSQL vs. SQLite. There is no need to explain that they are different, what I would like to know is:
Do those three "Database Solutions" allow for storing all its Data
to a file?
(and of course to do the reverse, initialize all its data given a backup file?)
.
Background
Since I already have done some research, which partly answers this question, allow me to provide this background info to the question:
SQLite
(yes it does allow storage and retrievel of the Database to and from a file)
I have already done some work with SQLite. For this I know that there it actually right away starts the database via a reference to the file. Backup is simple copying the file. Restoring is rewriting the file.
IndexedDB and WebSQL
??? Are to my understanding Database Solutions which "life their life in the Browser's Javascript land" and there we do not deal much with files. Here is part of where the question lies. If I wanted to export the data from either of the two solution to a flat file or lets say a one string variable representation, would that be possible?
This are some SO question I think that relate to it:
SO Question: Exporting WebSQL Data
SO Question: Import and Export Indexeddb data
which indicate that there is no easy toString() (Store Database) method and FromSting() in IndexedDB nor in WebSQL.
It indeed true (and affirmed in an answer here) and there is not easy backup and retrieval for those Database, this would be very sad, and I think a gap. Databases without backup function, really?!
At Present, there is no way to back up and restore Browser databases. The only way you can achieve this is by continuously syncing your back-end database with the browser database and thus keep track of changes in data generated on browser.
Old guy will do in old way, whether appropriate or not. How on earth, browser database need a backup. Data in the client is just cache copy of a slice of server data. There is no need back. If you think IndexedDB (or web sql database) data is durable, you will be glad to know, IndexedDB data belong to a temporary class of browser data, meaning that UA can delete data at their discretion without prompting to user or app.
If your app treat browser data more than cache copy, you are doing wrong.
I'm working on a new web app where a large amount of content (text, images, meta-data) is requested via an Ajax request.
No auth or login required for a user to access this.
My concern is that you could easily lookup the data source URL and hit it directly outside the app to get large data. In some ways, if you can do this you could probably scrape the static HTML pages elsewhere that also have this content.
Are there any suggestions on methods to obfuscate, hide, or otherwise make it very difficult to access the data directly?
Example: web app HTML page contains a key that is republished every 30 min. On the server side the data is obfuscated based on this key. In order to get the data outside the app you'd need to figure out the data source but also the extra step of scraping the page for a key every 30 min.
I realize there is no 100% way to stop someone, but I'm talking more about deterrence.
Use sessions in your webapp. Make a note (e.g. database entry or some other mechanism which your server-side code can access) when a valid request for the first page is received and include code in the second page to exclude the data when processing a request without a corresponding session entry.
Obviously the specifics on how to do this will vary between languages, but most robust web platforms will support sessions, largely for this type of reason.
If you are wanting to display real-time data and are concerned about scrapers...if this is a big enough concern, then I suggest doing it with flash instead of JS (AJAX). Have the data display withing a flash object. Flash can make real-time send/receive requests to the server just like AJAX. But the benefit of Flash is that the whole stage, data, code, etc.. are within a flash object, which cannot be scraped. Flash object makes the request, you output the stuff as a crypted string of code. Decrypt it within flash and display from there.
"Are there any suggestions on methods to obfuscate, hide, or otherwise make it very difficult to access the data directly?"
Answers your own question because if the data is worth getting it will be obtained because you are obfuscating is merely making it harder to find.
You could in the server side script processing the ajax and returning the data check where the request came from.