Every question I found told me that the only way to get back the object is to fetch it with an ajax request using the blob:https://www.example.com/0ea6c8a8-732f-42c7-9530-4805c4e785f5 as the destination url. Are blobs not saved in my browsers memory and should therefore be immediately accessible? The way I understand it, it has nothing to do with the remote server/website. Some JS file created the blob object, generated the blob URL and saved it in memory.
I tried using let blob = await fetch(url).then(r => r.blob()); on several websites, always running in cors limitations. Perhaps only the script that created it (different domain) is allowed to access it within its context, which is very unfortunate considering the blob content is literally saved in my browser's memory.
I know other methods of accessing the resource the blob points to or contains by observing network requests. That is not what I am asking here. I wish to understand, how to unpack the blob URL to access Blob object inside browser console to see how the information was saved in the first place. When it comes to videos, Blob simply can't contain the actual video, because of size and bandwidth constrains, so what does it contain then? Manifest file itself?
See this answer of mine for a way to retrieve the Blob from a blob: URI (you need to run the script there before the Blob is created by the page). Fetching only creates a copy.
The blob: URL is linked to the remote server in that it shares the same origin. So yes, the Blob (binary data) is on your computer, but the URL is only accessible to scripts running from the same origin than the one it was generated from.
And yes, the Blob does contain all the video data, but the string you have is a blob: URL, which is only a pointer to that Blob, itself stored in the browser's memory.
Related
In javascript, when opening a file via a button returns a Blob object (e.g. blob1).
I can then get the actual data of the blob via blob1ArrayBuffer = blob1.arrayBuffer();
When the Blob object (e.g. blob1) is created, does it load all the bytes into memory?
Or does it just returns the address so that later the actual bytes can be read via blob1.arrayBuffer() ?
No, all the file isn't read in memory (you can try to open a file of a few TB, that should still work).
However note that the OS will still need to read some of that file to produce the metadata of the file. This may take some times in some conditions (e.g when selecting folders with many files, or when selecting a file from a network disk etc.)
Even when doing blob1.arraybuffer() the full file isn't necessarily put in memory, since the specs ask that all the consumers of the Blob use a ReadableStream to get the data from it. But obviously in this case, the full data will be copied in the resulting ArrayBuffer that will most probably live in memory.
I have an issue similar to this one where I am successfully downloading a blob generated from a backend via HTTP GET but the file is being saved to browser memory before the download begins.
There's no problem when downloading small files but it doesn't immediately download 100mb+ files.
Subscribing to the GET itself is causing the delay of saving the large files.
I'm using Angular 6 with an object store backend. Here's the download function:
finalDownload(url: string) {
let headers = new HttpHeaders();
headers = headers.append('X-Auth-Token', token);
return this.http.get(url, { headers, responseType: 'blob' })
.subscribe(response => {
saveAs(response);
})
}
Here's the process:
User hits the download button
GET request with headers is fired to back end
As soon as I subscribe for the response, the blob is stored in browser memory.
When the blob is completely stored in browser, the saveAs/download begins
Step 3 is where the issue is.
This devtools screenshot with 108 MB transferred accumulates to the file size (I downloaded a 100 mb file) before the download itself to filesystem begins.
You can try to use URL.createObjectURL:
URL.createObjectURL() can be used to construct and parse URLs. URL.createObjectURL() specifically, can be used to create a reference to a File or a Blob. As opposed to a base64-encoded data URL, it doesn’t contain the actual data of the object – instead it holds a reference.
The nice thing about this is that it’s really fast. Previously, we’ve had to instantiate a FileReader instance and read the whole file as a base64 data URL, which takes time and a lot of memory. With createObjectURL(), the result is available straight away, allowing us to do things like reading image data to a canvas.
Use the following code as reference
const blob = new Blob([data], { type: 'application/octet-stream' });
this.fileUrl = this.sanitizer.bypassSecurityTrustResourceUrl(window.URL.createObjectURL(blob));
I try to write an extension caching some large media files used on my website so you can locally cache those files when the extension is installed:
I pass the URLs via chrome.runtime.sendMessage to the extension (works)
fetch the media file via XMLHttpRequest in the background page (works)
store the file using FileSystem API (works)
get a File object and convert it to a URL using URL.createObjectURL (works)
return the URL to the webpage (error)
Unfortunately the URL can not be used on the webpage. I get the following error:
Not allowed to load local resource: blob:chrome-extension%3A//hlcoamoijhlmhjjxxxbl/e66a4ebc-1787-47e9-aaaa-f4236b710bda
What is the best way to pass a large file object from an extension to the webpage?
You're almost there.
After creating the blob:-URL on the background page and passing it to the content script, don't forward it to the web page. Instead, retrieve the blob using XMLHttpRequest, create a new blob:-URL, then send it to the web page.
// assuming that you've got a valid blob:chrome-extension-URL...
var blobchromeextensionurlhere = 'blob:chrome-extension....';
var x = new XMLHttpRequest();
x.open('GET', blobchromeextensionurlhere);
x.responseType = 'blob';
x.onload = function() {
var url = URL.createObjectURL(x.response);
// Example: blob:http%3A//example.com/17e9d36c-f5cd-48e6-b6b9-589890de1d23
// Now pass url to the page, e.g. using postMessage
};
x.send();
If your current setup does not use content scripts, but e.g. the webRequest API to redirect request to the cached result, then another option is to use data-URIs (a File or Blob can be converted to a data-URI using <FileReader>.readAsDataURL. Data-URIs cannot be read using XMLHttpRequest, but this will be possible in future versions of Chrome (http://crbug.com/308768).
Two possibilities I can think of.
1) Employ externally_connectable.
This method is described in the docs here.
The essence of it: you can declare that such and such webpage can pass messages to your extension, and then chrome.runtime.connect and chrome.runtime.sendMessage will be exposed to the webpage.
You can then probably make the webpage open a port to your extension and use it for data. Note that only the webpage can initiate the connection.
2) Use window.PostMessage.
The method is mentioned in the docs (note the obsolete mention of window.webkitPostMessage) and described in more detail here.
You can, as far as I can tell from documentation of the method (from various places), pass any object with it, including blobs.
I have two audio files which have to be played one after the other.
In order to do this, I downloaded the two files using XHR
var xhr = new XMLHttpRequest();
xhr.open('GET', fileURL, true);
xhr.responseType = 'arraybuffer';
xhr.onload = function () {
data = new Uint8Array(xhr.response);
//... other handling code
};
and constructed a Blob from them
var blob = new Blob([data], {type: 'video/mp4'})
These I used to construct Blob URLs
var url = URL.createObjectURL(blob)
which are then injected into <audio> tags. (audioDOM.src = url;)
This procedure works in Chrome and Firefox. However IE11 sometimes gives me a problem, it displays following notice:
One or more blob URLs were revoked by closing the blob for which they
were created. These URLs will no longer resolve as the data backing
the URL has been freed.
However, the weirdest part is that it does work (in IE) for the first file, but not for the second one. They both use the same code for the entire procedure, which is simply called using a different fileURL. Both files exist, are downloaded properly and have been logged to console for verification.
I attempted copying the data before constructing the blobs, but it does not seem to matter: the error remains.
Does anyone have an idea what causes the problem and how it could be fixed?
EDIT after sbgoran:
The entire script runs on a single document, which is not being reloaded. I am still confused why it is the case. I did manage to create a workaround by looping and creating a new URL when the audio fails to load, but this is not a workable solution. The weird part is that the method described above fails at random: sometimes the URL is available when loaded into the <audio> tag, sometimes it isn't.
Apart from sbgoran's answer, I have another possible cause - as discussed on WhatWG mailing list, there's a bug in IE that causes a Blob to be garbage collected when the variable that points to it goes out of scope - even when there are still object URLs pointing to the Blob that haven't been revoked.
A workaround could rely on holding on to the original Blob object in a variable as long as the object URL is needed, then nullifying the variable and revoking its object URL afterwards.
Based on Remarks section of MSDN createObjectURL method page it could be plenty of things that IE might complain about. I'm not sure if you read this MSDN page before but maybe it can help in some way.
I would especially check note about blob urls origin policy
Blob urls are subject to an origin policy. This means that they can only be used in documents that have the same site-of-origin as the document running the script that created the url. If you need to use the blob object from an that is running in a different domain, you must use the postMessage API to send the blob data to the frame and then create the blob: url there.
and
URL returned by createObjectURL is valid for the lifetime of the creating document,
which are then injected into <audio> tags.
Can you please give this code ?
Maybe IE has some kind of garbage collector, which frees the blob url as soon as there is no DOMString instance containing the blob url.
With this theory, creating your audio tag with innerHTML (and maybe setAttribute ?), I mean by concatenation of the url (foo.innerHTML = '<audio src="' + url + '" />';) implies that the only DOMString containing the blob url (the one returned by URL.createObjectURL) will be garbage collected some time after the end of your function.
i have a blob url like blob:blahblah that points to a file. I want to write the file behind this blob to local filesystem. The writer.write() documentation says it accepts a file object (from input-type-file) and a blob. But it throws a type mismatch error when try this
fileEntry.createWriter(function(writer) {
writer.write(blob); //blob is a var with the value set to the blob url
i know the problem is that the blob does not get accepted but i would like to know how can i store a blob to the filesystem. i created the said blob earlier in the script from input-type-file and stored it's value in a var.
EDIT
Ok so i think i should have given more code in the first place.
first i create a blob url and store it in a var like this
files[i]['blob'] = window.webkitURL.createObjectURL(files[i]);
files is from an input-type-file html tag and i is looped for number of files. you know the gig.
then the variable goes through a number of mediums, first through chrome's message passing api to another page and then from that page to a worker via postMessage and then finally back to the parent page via postMessage again.
on the final page i intend to use it to store the blob's file to local file system via file system api like this..
//loop code
fileSystem.root.getFile(files[i]['name'], {create: true}, function(fileEntry) {
fileEntry.createWriter(function(writer) {
writer.write(files[i]['blob']);
});
});
//loop code
but the writer.write throws Uncaught Error: TYPE_MISMATCH_ERR: DOM File Exception 11
i believe this error is because the variable supplied to writer.write is a text and not a blob object from something like createObjectUrl (directly and not after passing through multiple pages/scopes) or not a window.WebKitBlobBuilder. So how can a blob's url be used to store a file?
From your edited code snippet and description, it sounds like you're writing the blobURL to the filesystem rather than the File itself (e.g. files[i]['name'] is a URL). Instead, pass around the File object between main page -> other page -> worker -> main page. As of recent (in Chrome at least), your round trip is now possible. File objects can be passed to window.postMessage(), whereas before, the browser serialized the argument into a string.
You 'fashion' a handler/reference to a Blob with createObjectURL(). There's not really a way to go from blobURL back to a Blob. So in short, no need to create createObjectURL(). Just pass around files[i] directly.