Write image file to Firebase Storage from HTTP function - javascript

I am trying to write an image Firebase Storage via a Cloud Function (for more suitable write access).
My current attempt is to read the file object on the client, send it (the data) to an http firebase function, and then save it to storage from there. After saving the file successfully, I try using the download url as an img src value, but the file does not display. I also see an error in the Storage console (Error loading preview) when attempting to view the file.
If I save the data in Storage as base64, I can copy the contents of the file into the img src attribute, and it displays fine. However, I'd like to simply use the download URL as I could do if I just uploaded the image via the client SDK or directly via the console.
In the client, I'm simply using FileReader to read the uploaded file for sending. I've tried all the ways of reading it (readAsText,readAsBinaryString, readAsDataURL, readAsArrayBuffer), but none seem to solve the issue.
Here is how I am uploading the file via the Firebase Function:
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
import * as path from 'path';
import * as os from 'os';
import * as fs from 'fs-extra';
export default functions.https.onCall(async(req, context) => {
const filename = req.filename;
const bucket = admin.storage().bucket(environment.bucket)
const temp_filename = filename;
const temp_filepath = path.join(os.tmpdir(), temp_filename);
await fs.outputFile(temp_filepath, req.data, {});
// Upload.
await bucket.upload(temp_filepath, {destination: 'logos'})
.then((val) => {})
.catch((err) => {});
});
This uploads the file successfully, however, the Download URL does not work when used as the img src attribute.
One thing I have noticed is that when using the client SDK to send a file (via AngularFireStorage), the payload is the raw png contents. E.g. a snippet of the file:
PNG
IHDRÈÈ­X®¤IDATx^í]
Eµ¾·{&1,!dù»*yVQ#PTEDPA>ÊâC\P"ÈÄ"
F}òIW÷üCL#BÉL÷}
....
However, reading the file as text does not yield this encoding. I have tried several other encodings.
Any help would be immensely appreciated.
Edit
Here is what I mean about using the download URL:
<img alt='logo' src='https://firebasestorage.googleapis.com/v0/b/y<project-name>/o/logos%2FAnM65PlBGluoIzdgN9F5%2Fuser.png?alt=media&token=<token>' />
The above src url is the one provided in the Firebase Storage console when clicking on the file. It is labeled as 'Download URL' (I believe this is the one retrieved by calling getDownloadUrl() via the sdk).
When using AngularFireStorage to put the file in storage, the Download URL will work. When I say it 'will work', I mean the image will display properly. When using FileReader to pass the data to an http cloud function to upload (as seen above), the image will not display. In other words, after uploading the file via the backend, the download url does in fact provide what was uploaded, it's just not in a format that an img tag can display.
One possible issue may be that I am not getting the encoding correct when using FileReader readAsText. Here is what I am doing with FileReader:
const reader = new FileReader();
reader.onloadend = () => {
firebase.functions().httpsCallable('http_put_logo')(reader.result);
};
// Have tried various encodings here, as well as all reader methods.
reader.readAsText(file);
Edit 2
All of the discussion on this question so far seems to be around correctly getting the download URL. I'm not sure if Firebase docs have this information, but the download URL is available in the Storage console. I'm simply copying and pasting that URL to for testing purposes at the moment.
The reason why I am doing this is because I plan to save these image URLs in the DB since they are going to be frequently used and publicly readable. So, I'm not going to use the getDownLoadURL() method to fetch these images, I'm simply just going to link to them directly in img tags.
Here is an image of my console to see what I mean (bottom right):
You just have to click it and copy it. You can then open it in a browser tab, download it, use it as a src value, etc.
Edit 3
Here is an image of what the request payload looks like when using the client sdk:
Here is when I read the file as text and send to backend for upload:
Notice there are differences in the payloads. That's why I'm uncertain if I'm properly reading the file or encoding it incorrectly.

What part of your code is taking care of getting the URL? I recently used a similar approach to uploading images to Firebase Storage using cloud functions. What worked best for me was to execute a different function to get the URL after the upload is complete. Something like this:
const bucket = admin.storage().bucket(environment.bucket)
const temp_filename = filename;
const temp_filepath = path.join(os.tmpdir(), temp_filename);
await fs.outputFile(temp_filepath, req.data, {});
// Upload.
await bucket.upload(temp_filepath, {destination: 'images'})
.then((val) => {retrieveUrl(temp_filename)})
.catch((err) => {});
retrieveUrl = (imageName) => {
const storage = firebase.storage();
storage.ref(`/images/${imageName}.jpg`).getDownloadURL()
.then( url => {
/*Save the url to a variable or attach it directly to the src of your image, depending on the structure of your project*/
})
.catch(err => console.log(err));
}
Keep in mind that you need to install firebase in your project in order to call firebase.storage.

Related

Display file.pdf As a Blob

I am working on different ways of displaying a PDF to get better on a project at work. I am able to insert a url to a PDF into an iframe element and it displays the PDF fine. Sometimes we have a use case where the front end receives a pdf as application/pdf instead of a url in a json object.
For this, I turn the application/pdf into a blob and create a url for the blob. This worked great on everything except Android, so I am testing out some methods with iFrame.
I would like to take this sample pdf http://www.pdf995.com/samples/pdf.pdf, turn it into a blob, and insert the blob url in the src of an iframe element for the purposes of testing blobs as iframe sources on Android Chrome browsers.
function App() {
const samplePdf = "http://www.pdf995.com/samples/pdf.pdf"
const blob = new Blob([samplePdf], { type: 'application/pdf' });
const url = URL.createObjectURL(blob)
return (
<>
<h1>iFrame Rendering of PDF Blob</h1>
<iframe title="pdf" src={url} style={{ height: '1250px', width: '100%' }}></iframe>
</>
);
}
export default App;
This is what renders in the React app
What am I missing to get the content of the pdf to display? React is not a requirement, just seemed an easy way to start a quick practice project.
You need the data to construct the blob while the URL just points to the data you need. Let's go & get it:
const getLocalPdfUrl = async () => {
const url = 'http://www.pdf995.com/samples/pdf.pdf';
const response = await fetch(url);
const blob = await response.blob();
return URL.createObjectURL(blob);
};
This function returns a Promise that will (hopefully) resolve with the URL you can use to construct the iframe. It's async, so, don't forget to wait for the promise to resolve.
Testing note
Fetching external resources from the frontend is restricted by CORS, so, pdf995.com's link will not work. It's also not a trivial task to find a dummy PDF document that would allow fetching itself from the FE.
To test if it works, I'd propose to place the PDF file in the /public folder (or similar) & serve it on the same locslhost as the app.

How to display PDFs fetched from S3 using JavaScript?

I need to fetch a PDF file from s3.amazonaws.com and when I query it using Postman (or paste directly into the browser), it loads fine. However when I try to generate the file path for it (to pass to a viewer later), it didn't work:
fetch(<S3URL>).then(res => res.blob()).then(blob => {
// THIS STEP DOES NOT WORK
let myBlob = new Blob(blob, {type: 'application/pdf'});
// expect something like 'www.mysite.com/my-file.pdf'
let PDFLink = window.URL.createObjectURL(myBlob);
return PDFLink;
}
I'm using Autodesk's Forge PDF viewer and it works perfectly fine for local PDF files:
let myPDFLink = 'public/my-file.pdf';
Autodesk.Viewing.Initializer(options, () => {
viewer = new Autodesk.Viewing.Private.GuiViewer3D(document.getElementById('forgeViewer'));
viewer.start();
viewer.loadExtension('Autodesk.PDF').then( () => {
viewer.loadModel(myPDFLink, viewer); // <-- works fine here
});
});
// from https://github.com/wallabyway/offline-pdf-markup
So, how do I go from the S3 URL (e.g. s3.amazonaws.com/com.autodesk.oss-persistent/0d/ff/c4/2dfd1860d1...) to something the PDF viewer can understand (i.e. has .pdf extension in the URL)?
I know for JSON files I need to do res.json() to extract the JSON content, but for PDFs, what should I do with the res object?
Note: I don't have control over the S3 URL. Autodesk generates a temporary S3 link whenever I want to download documents from their BIM360 portal.
I tried a lot of options and the only way I could display a PDF fetched via API calls is by using an object element:
<object data='<PDF link>' type='application/pdf'>
Converting the downloaded blob to base64 doesn't work. Putting the PDF link in an iframe doesn't work either (it still downloads instead of displaying). All the options I have read only work if the PDFs are part of the frontend application (i.e. local files, not something fetched from a remote server).

How to convert Base64 string to PNG (currently using Jimp)?

I'm currently creating a real-time chat application. This is a web application that uses node.js for the backend and uses socket.io to connect back and forth.
Currently, I'm working on creating user profiles with profile pictures. These profile pictures will be stored in a folder called images/profiles/. The file will be named by the user's id. For example: user with the id 1 will have their profile pictures stored in images/profiles/1.png. Very self-explanatory.
When the user submits the form to change their profile picture, the browser JavaScript will get the image, and send it to the server:
form.addEventListener('submit', handleForm)
function handleForm(event) {
event.preventDefault(); // stop page from reloading
let profilePicture; // set variable for profile picture
let profilePictureInput = document.getElementById('profilePictureInput'); // get image input
const files = profilePictureInput.files[0]; // get input's files
if (files) {
const fileReader = new FileReader(); // initialize file reader
fileReader.readAsDataURL(files);
fileReader.onload = function () {
profilePicture = this.result; // put result into variable
socket.emit("request-name", {
profilePicture: profilePicture,
id: userID,
}); // send result, along with user id, to server
}
}
I've commented most of the code so it's easy to follow. The server then gets this information. With this information, the server is supposed to convert the sent image to a png format (I can do whatever format, but it has to be the same format for all images). I am currently using the jimp library to do this task, but it doesn't seem to work.
const jimp = require('jimp'); // initialize Jimp
socket.on('request-name', (data) => { // when request has been received
// read the buffer from image (I'm not 100% sure what Buffer.from() does, but I saw this online)
jimp.read(Buffer.from(data.profilePicture), function (error, image) {
if (error) throw error; // throw error if there is one
image.write(`images/profiles/${data.id}.png`); // write image to designated place
}
});
The error I get:
Error: Could not find MIME for Buffer <null>
I've scoured the internet for answers but was unable to find any. I am available to use another library if this helps. I can also change the file format (.png to .jpg or .jpeg, if needed; it just needs to be consistent with all files). The only things I cannot change are the use of JavaScript/Node.js and socket.io to send the information to the server.
Thank you in advance. Any and all help is appreciated.
If you're just getting the data URI as a string, then you can construct a buffer with it and then use the built in fs to write the file. Make sure the relative path is accurate.
socket.on('request-name', data => {
const imgBuffer = Buffer.from(data.profilePicture, 'base64');
fs.writeFile(`images/profiles/${data.id}.png`, imgBuffer);
}

Blob name issue with new tab in chrome and firefox [duplicate]

In my Vue app I receive a PDF as a blob, and want to display it using the browser's PDF viewer.
I convert it to a file, and generate an object url:
const blobFile = new File([blob], `my-file-name.pdf`, { type: 'application/pdf' })
this.invoiceUrl = window.URL.createObjectURL(blobFile)
Then I display it by setting that URL as the data attribute of an object element.
<object
:data="invoiceUrl"
type="application/pdf"
width="100%"
style="height: 100vh;">
</object>
The browser then displays the PDF using the PDF viewer. However, in Chrome, the file name that I provide (here, my-file-name.pdf) is not used: I see a hash in the title bar of the PDF viewer, and when I download the file using either 'right click -> Save as...' or the viewer's controls, it saves the file with the blob's hash (cda675a6-10af-42f3-aa68-8795aa8c377d or similar).
The viewer and file name work as I'd hoped in Firefox; it's only Chrome in which the file name is not used.
Is there any way, using native Javascript (including ES6, but no 3rd party dependencies other than Vue), to set the filename for a blob / object element in Chrome?
[edit] If it helps, the response has the following relevant headers:
Content-Type: application/pdf; charset=utf-8
Transfer-Encoding: chunked
Content-Disposition: attachment; filename*=utf-8''Invoice%2016246.pdf;
Content-Description: File Transfer
Content-Encoding: gzip
Chrome's extension seems to rely on the resource name set in the URI, i.e the file.ext in protocol://domain/path/file.ext.
So if your original URI contains that filename, the easiest might be to simply make your <object>'s data to the URI you fetched the pdf from directly, instead of going the Blob's way.
Now, there are cases it can't be done, and for these, there is a convoluted way, which might not work in future versions of Chrome, and probably not in other browsers, requiring to set up a Service Worker.
As we first said, Chrome parses the URI in search of a filename, so what we have to do, is to have an URI, with this filename, pointing to our blob:// URI.
To do so, we can use the Cache API, store our File as Request in there using our URL, and then retrieve that File from the Cache in the ServiceWorker.
Or in code,
From the main page
// register our ServiceWorker
navigator.serviceWorker.register('/sw.js')
.then(...
...
async function displayRenamedPDF(file, filename) {
// we use an hard-coded fake path
// to not interfere with legit requests
const reg_path = "/name-forcer/";
const url = reg_path + filename;
// store our File in the Cache
const store = await caches.open( "name-forcer" );
await store.put( url, new Response( file ) );
const frame = document.createElement( "iframe" );
frame.width = 400
frame.height = 500;
document.body.append( frame );
// makes the request to the File we just cached
frame.src = url;
// not needed anymore
frame.onload = (evt) => store.delete( url );
}
In the ServiceWorker sw.js
self.addEventListener('fetch', (event) => {
event.respondWith( (async () => {
const store = await caches.open("name-forcer");
const req = event.request;
const cached = await store.match( req );
return cached || fetch( req );
})() );
});
Live example (source)
Edit: This actually doesn't work in Chrome...
While it does set correctly the filename in the dialog, they seem to be unable to retrieve the file when saving it to the disk...
They don't seem to perform a Network request (and thus our SW isn't catching anything), and I don't really know where to look now.
Still this may be a good ground for future work on this.
And an other solution, I didn't took the time to check by myself, would be to run your own pdf viewer.
Mozilla has made its js based plugin pdf.js available, so from there we should be able to set the filename (even though once again I didn't dug there yet).
And as final note, Firefox is able to use the name property of a File Object a blobURI points to.
So even though it's not what OP asked for, in FF all it requires is
const file = new File([blob], filename);
const url = URL.createObjectURL(file);
object.data = url;
In Chrome, the filename is derived from the URL, so as long as you are using a blob URL, the short answer is "No, you cannot set the filename of a PDF object displayed in Chrome." You have no control over the UUID assigned to the blob URL and no way to override that as the name of the page using the object element. It is possible that inside the PDF a title is specified, and that will appear in the PDF viewer as the document name, but you still get the hash name when downloading.
This appears to be a security precaution, but I cannot say for sure.
Of course, if you have control over the URL, you can easily set the PDF filename by changing the URL.
I believe Kaiido's answer expresses, briefly, the best solution here:
"if your original URI contains that filename, the easiest might be to simply make your object's data to the URI you fetched the pdf from directly"
Especially for those coming from this similar question, it would have helped me to have more description of a specific implementation (working for pdfs) that allows the best user experience, especially when serving files that are generated on the fly.
The trick here is using a two-step process that perfectly mimics a normal link or button click. The client must (step 1) request the file be generated and stored server-side long enough for the client to (step 2) request the file itself. This requires you have some mechanism supporting unique identification of the file on disk or in a cache.
Without this process, the user will just see a blank tab while file-generation is in-progress and if it fails, then they'll just get the browser's ERR_TIMED_OUT page. Even if it succeeds, they'll have a hash in the title bar of the PDF viewer tab, and the save dialog will have the same hash as the suggested filename.
Here's the play-by-play to do better:
You can use an anchor tag or a button for the "download" or "view in browser" elements
Step 1 of 2 on the client: that element's click event can make a request for the file to be generated only (not transmitted).
Step 1 of 2 on the server: generate the file and hold on to it. Return only the filename to the client.
Step 2 of 2 on the client:
If viewing the file in the browser, use the filename returned from the generate request to then invoke window.open('view_file/<filename>?fileId=1'). That is the only way to indirectly control the name of the file as shown in the tab title and in any subsequent save dialog.
If downloading, just invoke window.open('download_file?fileId=1').
Step 2 of 2 on the server:
view_file(filename, fileId) handler just needs to serve the file using the fileId and ignore the filename parameter. In .NET, you can use a FileContentResult like File(bytes, contentType);
download_file(fileId) must set the filename via the Content-Disposition header as shown here. In .NET, that's return File(bytes, contentType, desiredFilename);
client-side download example:
download_link_clicked() {
// show spinner
ajaxGet(generate_file_url,
{},
(response) => {
// success!
// the server-side is responsible for setting the name
// of the file when it is being downloaded
window.open('download_file?fileId=1', "_blank");
// hide spinner
},
() => { // failure
// hide spinner
// proglem, notify pattern
},
null
);
client-side view example:
view_link_clicked() {
// show spinner
ajaxGet(generate_file_url,
{},
(response) => {
// success!
let filename = response.filename;
// simplest, reliable method I know of for controlling
// the filename of the PDF when viewed in the browser
window.open('view_file/'+filename+'?fileId=1')
// hide spinner
},
() => { // failure
// hide spinner
// proglem, notify pattern
},
null
);
I'm using the library pdf-lib, you can click here to learn more about the library.
I solved part of this problem by using api Document.setTitle("Some title text you want"),
Browser displayed my title correctly, but when click the download button, file name is still previous UUID. Perhaps there is other api in the library that allows you to modify download file name.

Edit a file to be uploaded

I would like to be able to edit a file that has been selected for upload. I want to search and replace text in case absolute files should be made relative...
I notice in the File API I can do some of it, but I get a little stuck:
document.getElementById('exampleInputFile').onchange = function(event) {
var fileToLoad = event.target.files[0];
if (fileToLoad) {
var reader = new FileReader();
reader.onload = function(fileLoadedEvent) {
var textFromFileLoaded = fileLoadedEvent.target.result;
//Use logic to remove absolute files
//Upload S3
};
reader.readAsText(fileToLoad, 'UTF-8');
}
};
I am trying to figure out how now to convert that text to a proper File so that I can upload it to S3 using an existing api that expects something returned by: event.target.files[0] code above.
I do not want the server to handle any heavy lifting here if I can avoid it (files can easily be a few megabytes since they can be 3D models).
Assuming you know the url of the file when it lands in the S3 bucket, you can retrieve the file using a http.get, which will give you the contents of the (I assume plain text file). You can then parse that file and do whatever modification you need to do on the contents. If the file has changed, you can then write it back to the S3 bucket to replace the original file.
On AWS you can use Lambda to execute NodeJS code when an event is triggered (for example an upload to a specified bucket).

Categories

Resources