I'm trying to find a way to upload large files without having to read them all into memory. I need to be able to post multiple files along with a data payload in a single request. I'm able to accomplish it with standard HTML inputs, something like this:
<input id="myFile1" type="file" />
<input id="myFile2" type="file" />
<input id="myData" type="text" />
And javascript code like this:
var file1 = document.getElementById('myFile1').files[0];
var file2 = document.getElementById('myFile2').files[0];
var data = document.getElementById('myData').value;
var formData = new FormData();
formData.append('myFile1', file1, file1.name);
formData.append('myFile2', file2, file2.name);
formData.append('myData', data);
Even when selecting a large file this is very fast. Keep in mind I'm only referring to the client side performance, as the actual upload portion is going to be bottlenecked by the users network speed.
So now comes my problem: I need to use the Ionic camera plugin to allow a user to either record video or select video files on their phone. But when a user picks a file, you aren't immediately given a File object to work with; a user picks a file, then you are given a file URL and have to convert that into a Blob in order to post it to a server. However, as you can see from this stackoverflow answer, you have to read the entire file contents into memory to create the Blob object. This is slow (and very inefficient) when selecting a large file.
Does anyone know of a way to get a Blob or File object without reading everything into memory? I looked at the File and Blob API documentation but from what I see there isn't a way, unless I'm missing something. I'm wondering how the File API works behind the scenes when using a standard HTML input and choosing a file, because as shown by the example above, you're immediately given access to a File object (which is also a Blob) and it's very fast even for large files.
Related
In javascript, when opening a file via a button returns a Blob object (e.g. blob1).
I can then get the actual data of the blob via blob1ArrayBuffer = blob1.arrayBuffer();
When the Blob object (e.g. blob1) is created, does it load all the bytes into memory?
Or does it just returns the address so that later the actual bytes can be read via blob1.arrayBuffer() ?
No, all the file isn't read in memory (you can try to open a file of a few TB, that should still work).
However note that the OS will still need to read some of that file to produce the metadata of the file. This may take some times in some conditions (e.g when selecting folders with many files, or when selecting a file from a network disk etc.)
Even when doing blob1.arraybuffer() the full file isn't necessarily put in memory, since the specs ask that all the consumers of the Blob use a ReadableStream to get the data from it. But obviously in this case, the full data will be copied in the resulting ArrayBuffer that will most probably live in memory.
I have the next input on a webpage
<input accept="image/jpeg" class="class1" type="file">
And I'm trying to set it's file location via chrome console , I've tried to do the next thing
var Files = ['C:/Users/user/Desktop/dir/toUpload/file.jpg'];
var upload=document.getElementsByClassName("class1");
upload.files = Files;
But it seems to have no effect on the page at all.
How can I achieve to upload a file that way? (There is no submit button just file input)
I'm 90% sure that this isn't possible from JavaScript. The browser creates a File object after a user gesture. Allowing the page to create files without user gestures would enable random pages to read local files on user's computers.
File objects are generally retrieved from a FileList object returned as a result of a user selecting files using the element, from a drag and drop operation's DataTransfer object, or from the mozGetAsFile() API on an HTMLCanvasElement. In Gecko, privileged code can create File objects representing any local file without user interaction (see Implementation notes for more information.)
https://developer.mozilla.org/en-US/docs/Web/API/File
I want to create plugin mechanizm. It is, you can load js file on my website and run your js "plugin" (function) when this plugin is set to run (toggled as running).
All this I want to do without any server.
I mean, I want to keep in localstorage js files or path to this files.
It looks to be hard to do because js can't easy access files path.
I handle file by <input type="file"/>
And I react on onchange event. I get event where I can find selected file by event.srcElement.files[0]
With that I can create URL of that object by : URL.createObjectURL(event.srcElement.files[0])
And I tried to store that URL in localstorage but this URL is temporary.
Also I tried to store whole event or just file (event.srcElement.files[0]).
But I need to create string from that if I want to put it to the function .setItem :
localStorage.setItem("functionURL", JSON.stringify(this.functionURL));
.toString() creates [Object Event/File]
JSON.stringify() creates {} from [Object Event/File]
So, maybe is there a way to somehow remember file which we can use as a function without any server ?
So, maybe is there a way to somehow remember file which we can use as a function without any server ?
Basically, no. :-) Web storage only stores strings. You can't use a string to access a file on the user's local filesystem from your web page, for obvious security reasons.
You could, instead:
Make it possible for them to "upload" the file into your page (without a server) by having them identify the file in an input[type=file], reading its text (via the File API), and then storing that text in local storage
On page load, if local storage has code to run, run it
Offer the user a way to delete or update the code they've uploaded to the page
Since all of that happens in the browser, you don't need a server.
Web storage does have size limits, though they're pretty generous, (around 2.5-5MB) and per-origin, so you have that largely to yourself. But if you run into those limits, you could take it further by caching those files via a service worker, but the complexity goes up markedly. I'd start with web storage and only move on if you really need to support massive files.
#1 (reading the script file the user identifies via an input[type=file]) is really simple on modern browsers:
var file = input.files[0];
var fr = new FileReader();
fr.onload = function() {
// Use `fr.result` here, it's a string containing the text
};
fr.readAsText(file);
I have been struggling with this for a while and I am going to provide you with as much information as possible (some maybe irrelevant) because I am completely stuck. I am using Ionic and I would like to be able to take a picture with my phone and upload it to an AWS S3 bucket. I used Cordova camera to accomplish this.
As far as I know; these pictures come out in a large base64 string and I have to convert it to a Blob, convert it to a File object then upload that file object into AWS. However, when I do this it always uploads it as something other than an image. Whenever I open it I get an error saying:
"Not a valid bitmap file. its format is not currently supported."
https://s3.amazonaws.com/mng-moment/moment/PA/40.008446_-75.26046_1502414224619.jpg
Here is an example of a WORKING one (This used to work it somehow broke):
https://s3.amazonaws.com/mng-moment/bestMoments/40.008446_-75.26046_1499659199473.jpg
I tried to open each one in a text editor to see what is going on. For the first one (The broken one) I get this:
When I try to open the working one in a text editor I get this:
Now it seems like a conversion problem but I think I am converting it correctly.
Here is the code I am using to upload (You can see the console.logs later on the post):
core.js
awsServices.js
If you look at the comments in the code I labeled some of the console logs. I will display them here for more information:
A - (uploadToAWS):
B - (awsServices.upload):
This is how I convert the dataURI to a Blob (Called in uplpoadToAWS - The first screenshot):
This is what gets passed into the 'dataURI' parameter in the code right above:
If there is any more information please let me know. I've been scratching my head at this for a while. Any help is appreciated. Thanks!
As stated in MDN File API:
A File object is a specific kind of a Blob, and can be used in any context that a Blob can. In particular, FileReader, URL.createObjectURL(), createImageBitmap(), and XMLHttpRequest.send() accept both Blobs and Files.
So, i think your problem reside in your uploadToAws function because you first create a Blob and then use the Blob to create a File, when I think you simply should initialize the File object with the byte array returned by dataURItoBlob since the File object is in fact already a Blob object.
I am making a browser based audio player. So for making a playlist from the local directory I am using :
<input type="file" id="getFile" />
Then I am using a button to confirm the playlist.On clicking the button I am calling a javascript function to change the src of the audio tag to play the new audio file selected in the playlist. I want the exact path of the file from the input file to run in the HTML5 audio player but it starts taking the path as C://Fakepath/filename.mp3. Can someone help me with this.
This is a security feature, by design. You should not be able to read the original file path of a file input into a browser form. File input is for reading file contents only, not metadata like path on the user's file system.
The good news is that you don't need the original file path. You can use FileReader's readAsDataURL to convert the file contents into a base64-encoded data URL and use that as the audio src. To read from #myUploadInput and output through #myAudioElement (also available as a working fiddle):
var reader = new FileReader();
reader.onload = function (event) {
document.getElementById("myAudioElement").src = event.target.result;
};
reader.readAsDataURL(document.getElementById("myUploadInput").files[0]);
if the user is 'building' / creating the playlist based on files they have locally you could do a 'browse' field (s) where they select the local audio files, then take the contents of the field (that Should include the paths to those images), build an array of the count/id, filename.mp3, and path... then, based on what is 'chosen' to play, just reassemble the full local path and play that file.
that would be an approach I would take anyway to see if it would work. the necessary piece here is getting the user to disclose the paths to the audio files... but Im still not 100% sure it would work given the security feature that the earlier commenter posted a link to.
if this were included in an application the user approved for local installation you could just refer to it using the 'application directory' and copy the file to that 'safe location' but since its web based it just really opens up a whole can of worms in terms of a potentially unapproved / authorized web function knowing your local directory structure. good luck, let me know if you find a solution.