I need to be able to take the raw data from any file and generate a response object within a service worker.
Short explanation:
I have a website that takes file names, paths, mime types and raw text and stores it in cache. If I make a request to a file with that path and name, a service worker responds with that raw data.
Here is the very basic service worker response code:
self.addEventListener("fetch", async event =>
event.respondWith(caches.match(event.request))
);
This system works fine for HTML, CSS, JS and probably other files, but not for PNGs. I keep getting the image not found image:
I have checked that the correct mime type is being sent and the correct data is stored in cache. I have tried putting the data in cache with the text I find in notepad after opening the PNG, and the text result of a fetch request to an actual PNG file, using the .text() method.
Here are partial images of both:
Notepad++:
Fetch:
This data in the images is put into cache with this code:
cache.put(
rawFileText,
new Response(
"filePath/example.html",
{
status: 200,
headers: new Headers({
"content-type": "image/png" + "; charset=utf-8",
}),
},
),
);
More background info:
This is for a web code editor I am working on. When I want to run code in the editor:
website1 will create an iframe with website2 as source
website1 post messages all the file names, paths, mime types and raw text containing code, images, etc. (by raw data I mean whatever you would see if you opened the file with notepad)
then the iframe (website2) stores the file names and data as cache
when a request is made to any file stored in website2's cache, the service worker responds with whatever data is under the file name
The reason I use 2 different websites is to avoid conflicts with the editor website's files, local storage, and everything else. There could still be conflict with the second website's cache and service worker but that isn't a problem in my case.
Related
Explanation: So, basically I am trying to make a express.js website where I can upload files and it gets saved in the database and users can download them later on. So, my idea was to convert the file (.jar , .txt and .json) into buffer data and store it in MongoDB. And then fetch the buffer data from MongoDB and download that file. So, I read through the docs and couldn't find anything that does this.
Code: There is nothing really to show, I have a variable which has buffer data and I am using "express.js#4.17.1".
For testing, I am currently uploading the file using multer and once I click "upload", I am trying to download the same file to save time(instead of saving and fetching from database), so the buffer data is currently stored in "req.file.buffer". When I log "req.file" , I get this
console.log(req.file)
{
fieldname: 'fileupload',
originalname: 'jg.txt',
encoding: '7bit',
mimetype: 'text/plain',
buffer: <Buffer >,
size: 0
}
Conclusion: I want to convert the buffer data (req.file.buffer) into a downloadable file which the user can download(later on, make it such that it can be sent as a email attachment but both of them requires a file path and name, so once I figure out how to do the express.js thing, I can do the mail thing too). I have read many articles which first saves the buffer data into a file and then uploads, but I want to do without that
res.set({
"Content-Type": <content type of your file>,
"Content-Disposition": "attachment; filename=<name of your file>"
});
res.end(req.file.buffer);
This first sets the headers that cause the browser to download the file, then sends the file contents.
But your req.file.buffer seems to be an empty buffer, so the upload has perhaps not worked?
I show images uploaded on another platform on my site, but I want to protect my users from downloading extremely large files (50MB gifs for example). I dont have any control over the upload process and cant limit files during upload.
Is there any option to limit the file size of an image on a webpage?
Something like <img src="..." maxSize="5MB" /> and the browser would cancel the download if it detects that the file is bigger than 5MB ?
(This answer comes with the assumption that you have permission from the other platform to use images. This kind of activity is fraught with difficulty over things like copyright. Caveat venditor.)
Use a call to your own site as a proxy
Rather than hotlinking to the remote site directly from HTML, send requests to your own server.
From there, stream the image from the remote source. If you detect that the image is too large, you can produce an error response.
Even assuming you have permission to use images from the remote site, be sure to set the proper cache headers. You might also want to store a copy of the image on your own server.
Before rendering the images on your site, you could perform a HEAD-request and inspect the Content-Length header of the response.
HEAD
The HTTP HEAD method requests the headers that would be returned if the HEAD request's URL was instead requested with the HTTP GET method. For example, if a URL might produce a large download, a HEAD request could read its Content-Length header to check the filesize without actually downloading the file.
Source: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD
As an example, you could use the fetch api:
// This URL should point to a very large image, but currently it points to an avatar
const imageUrl = 'https://www.gravatar.com/avatar/eb03f51fd573efda46453d9b928de5e2?s=48&d=identicon&r=PG';
// Example
fetch(imageUrl, { method: 'HEAD' })
.then(response => {
const imageSizeInBytes = response.headers.get('content-length');
console.log(`Image size: ${imageSizeInBytes} bytes.`);
});
I have a webview that opens from a messenger bot.
From the webview I want to send image data to the conversation (no URL - data coming from the canvas).
I tried to use Messenger SDK beginShareFlow with file data attachment:
function uploadImage(data) {
let message = {
"attachment": {
"type": "image",
"payload": {
"is_reusable": true
},
"filedata": data
}
};
MessengerExtensions.beginShareFlow(function (share_response) {
// User dismissed without error
if (share_response.is_sent) {
// The user actually did share.
//close the webview
MessengerExtensions.requestCloseBrowser(function success() {
// webview closed
}, function error(err) {
console.log(err);
});
}
},
function (errorCode, errorMessage) {
// An error occurred in the process
console.log(errorMessage);
},
message, "current_thread");
}
But I get an error:
Messenger Extensions unexpected error.
Would appreciate help =]
EDIT:
I found out that filedata is used to transfer a file location (which I do not have).
So I tried other solutions:
I created from my cavas blob, and tried to pass it in filedata - did not work
I created a blob file (by adding name and date) and tried to move the location - did not work
I created a url from blob and tried to move it as a url (not as filedata) - and got an error:
Invalid image URL provided in message content
When I go to the blob url from the browser I see the image =[
Per the SDK's section on sending attachments:
There are three ways to attach an asset to a message:
URL
File
attachment_id
The attachment_id refers to previously uploaded URL/File attachments. Sending raw file data is not an option. You must either upload the image to a URL or save it to a file. Blob URLs do not work because they refer only to data stored in the local system's memory. You need to move that data to an image or file on a server.
Upload the image to a URL
Your first option is to upload the image to a URL. Depending on how private the contents of the image are, you could use a public image hosting service like imgur, or you could upload the image to a public location on your server. If you want to keep the image hidden, you could save the image at a URL containing a randomly generated hash and delete that file as soon as the attachment has been uploaded to Messenger. However, you could keep the image totally private with the second option:
Upload the image from a (temp) file
Your second option is to upload the image according to a file location. By uploading the image to a file on your server, you could avoid the image ever becoming visible to the public. To avoid filling up server space, you could manually delete the file once the attachment has uploaded, or you could use a temp file. In fact, the SDK's example for sending a file demonstrates sending a temporary file saved in the /tmp folder.
I have an issue similar to this one where I am successfully downloading a blob generated from a backend via HTTP GET but the file is being saved to browser memory before the download begins.
There's no problem when downloading small files but it doesn't immediately download 100mb+ files.
Subscribing to the GET itself is causing the delay of saving the large files.
I'm using Angular 6 with an object store backend. Here's the download function:
finalDownload(url: string) {
let headers = new HttpHeaders();
headers = headers.append('X-Auth-Token', token);
return this.http.get(url, { headers, responseType: 'blob' })
.subscribe(response => {
saveAs(response);
})
}
Here's the process:
User hits the download button
GET request with headers is fired to back end
As soon as I subscribe for the response, the blob is stored in browser memory.
When the blob is completely stored in browser, the saveAs/download begins
Step 3 is where the issue is.
This devtools screenshot with 108 MB transferred accumulates to the file size (I downloaded a 100 mb file) before the download itself to filesystem begins.
You can try to use URL.createObjectURL:
URL.createObjectURL() can be used to construct and parse URLs. URL.createObjectURL() specifically, can be used to create a reference to a File or a Blob. As opposed to a base64-encoded data URL, it doesn’t contain the actual data of the object – instead it holds a reference.
The nice thing about this is that it’s really fast. Previously, we’ve had to instantiate a FileReader instance and read the whole file as a base64 data URL, which takes time and a lot of memory. With createObjectURL(), the result is available straight away, allowing us to do things like reading image data to a canvas.
Use the following code as reference
const blob = new Blob([data], { type: 'application/octet-stream' });
this.fileUrl = this.sanitizer.bypassSecurityTrustResourceUrl(window.URL.createObjectURL(blob));
I develop a javascript application which display data from xml with charts and lists.
For now I put some sample files onto the server's directory that I load with :
$.ajax({ type: 'GET', url: 'data/default.xml', dataType: 'xml', ...})
The Xml files can be very heavy so when one of them is loaded I put the data in an IndexedDB.
In a second time I would like to let the visitor loads its own xml file by giving the filepath of the xml (f.i. : /home/user/sample.xml). I do not want to upload this file onto the server because I do not need it and it could be too big. But I do want to load those data in the IndexedDB and let the app displays data without any call to the server.
I do not know if browsers could work this way?
If they could, how can I do such a trick?
You can't use Ajax to get data from a file on the client system, but in sufficiently modern browsers you can use the File API. MDN has a guide to the File API that is friendlier then the specification.