Develop Reference

firebase functions upload file from web via stream only partially working

I am trying to create a firebase function that downloads a photo off the web via URL and uploads it to firebase storage.
Using Axios to stream the file to the bucket. it appears the file gets uploaded but i cant download or view it.
This is my code:
let fileURL = 'https://www.example.file.path.png'
let myFile = await axios.get(fileURL, { responseType: 'stream' })
let destFile = bucket.file(photoId).createWriteStream({ contentType: myFile.headers['content-type']})
myFile.data.pipe(destFile)
And here is the storage console from firebase:
I have messed around with the storage api and attempted using the upload and save functions. using axios get and write streams is the closest that I'v got to getting this to work.
Reading the example in the docs only aids in my confusion because the file is never reference in the upload function.. just the file name??
Feel like i'm almost there considering the file or rather the name of the file is there and the size and the type.. just not the content?

This problem is related with Firebase console only. There is no problem with downloading with command: gsutil cp gs://<your bucket>/yourfile . (doc) and as well it is working in Google Cloud Console in Storage browser (direct link).
However indeed file uploaded like this is not possible to be downloaded or previewed with Firebase console.
After many tries I have realized that this is related with custom metadata Access token, which underneath is calledfirebaseStorageDownloadTokens. It appears automatically if ex. you download the file directly in Firebase console.
Anyway I noticed that value of the metadata is irrelevant. According to my test if you change your createWriteStream method parameters to:
let destFile = bucket.file(photoId)
.createWriteStream({
metadata: {
contentType: myFile.headers['content-type'],
metadata: {
firebaseStorageDownloadTokens: 'canBeAnyThing'
}
}
});
The problem disappears...
A file already downloaded to Firebase Storage and affected by the issue can be fixed by adding the same metadata. In the screenshot you have provided you can see "File Location" if you open you will see link "Create new access token" or as well you can add it in GCP Storage Browser adding it manually in "Edit metadata" for such object (remember to refresh FB console in browser).
At this point we can think of why it's looks like this. I have found interesting information in github here.

Why am I not able to get the download URL of my Firebase storage file?

To summarize the problem, users are uploading lots of PDF files to a storage bucket. After upload, the users have a button they can press that gets the download URL of the file they have selected and opens it in a new window.
const storageRef = Storage.ref(`Inbound_Forms/${selectedInbound.id}/${row.FileName}`);
storageRef.getDownloadURL().then(url => {
window.open(url, '__blank');
});
The opening of the URL works about 95% of the time, but every once in a while when the user clicks the button to open the URL, this error gets thrown:
This would lead me to believe that the file does not exist, but when I open up the bucket in the Firebase console, the file exists, and I can get download it via it's link in the console:
The path included: /Inbound_Forms/5eE2Oytwrpc7FTkmH4gy
The solution for the time being is to have the users send me an email with a link to the page where the file is. I then track the original down in the bucket, download the file, and upload the exact same file with suffixed with 'copy', as seen in the image above. The new file works without issue, where the original continues throwing the same error. I just inform the user a copy has been uploaded, and they access it and continue on with their work.
There have been a few solutions I have looked into, but those worth mentioning are:
1) The file names have spaces. This does not seem to cause an issue, as even the copy I upload works without problems. And as stated before, this process works 95% of the time, many cases with spaces in the file name.
2) Storage rules. The same user has been able to open many other files with the same rules, including the newly uploaded copy.
Currently, I am unable to recreate this bug, as it happens so rarely. I thank anyone in advanced for and leads on what is going on here.

firebase cloud functions onFinalize trigger with not predefined bucket?

Is it possible to write a firebase cloud function that triggers when a new fila was created in firebase storage - onFinalize (but we don't know the exact bucket in advance) ?
Inside my firebase storage a have a folder 'loads' and inside I have folders named with load id like:
/loads/-Lw1UySdunYyFMrrV6tl
/loads/-LwisA5hkNl_uxw3k36f
/loads/-LwkPUm-q7wNv-wZ49Un
https://ibb.co/NnQkTyC here's a screenshot of storage
I want to trigger cloud function when new file has been created inside one of these folders. I don't know in andvance where the file will be created. I don't know if this is even possible. That's why I need an advice.
My main goal is to merge 2 pdf files in one within cloud functions. In my app (TMS written with vuejs) on frontend I create confirmationOrder.pdf using html2canvas/jsPDF and then save it to storage/loads/${loadID}. And later on user can manually upload POD.pdf on the same bucket. When it happens I want my cloud function to merge these two pdfs in one new file (in same storage bucket). But again I don't know the bucket in advance.
Here's how I upload PDFs in frontend:
async uploadConfPDF({ commit }, payload) {
const PDF = payload.PDF;
const loadID = payload.loadID;
const fileName = payload.fileName;
const fileData = await firebase
.storage()
.ref(`loads/${loadID}/${fileName}.pdf`)
.put(PDF);
const confOrderURL = await fileData.ref.getDownloadURL();
return confOrderURL;
},
Any help is highly appreciated. Sorry if my explanation could seem not clear enough. English is not my native language.

EDIT FOLLOWING YOUR QUESTION RE-WORKING
Based on your code and on the print screen of your Cloud Storage console, you are working in the default bucket of your project, which location's URL is gs://alex-logistics.appspot.com.
As we can see on the print screen of your Cloud Storage console, the files in your bucket are presented in a hierarchical structure, just like the file system on your local hard disk. However, this is just a way of presenting the files: there aren't genuine folders/directories in a bucket, the Cloud Storage console just uses the different part of the files paths to "simulate" a folder structure.
So, based on the above paragraphs, I think that your question can be re-phrased to "In a Cloud Function, how can I extract the different parts of the path of a file that is uploaded to the default bucket of my Firebase Project?".
Answer:
In a Cloud Function that is triggered when a file is added to the default bucket, you can get the file path as follows:
exports.yourCloudFunction = functions.storage.object().onFinalize(async (object) => {
const filePath = object.name; // File path in the bucket.
//...
});
Since we use an onFinalize event handler, you get the path of this new file by using the name property of the object Object, which is of type ObjectMetadata.
You can then use some String methods to, for example, extract from this path the ${loadID} you refer to in your question.

Ionic - Read file from URI

Users of my application can select and crop images using Ionic Native - Crop. After they have cropped their image, I will have the URI of that image such as:
file:///storage/emulated/0/Android/data/com.myApp/cache/1535369478970-cropped.jpg?1535369482232
I want to use Ionic's File API, since it has a method readAsDataURL(path, file) which will convert the file to a base64 encoded data url, which is what I exactly need.
However, how would I properly separate the path and file from the URI of the file I have above so that readAsDataURL(path, file) is satisfied?
I also do not know what these numbers behind the .jpg?1535369482232 mean and I do not know what the name of the file would be or if it has a different directory on iOS since the URI above is provided from a test using Android Emulator.
P.S. I have tried calling the method with just the path above and no file name passed as second argument, but got the following error:
{"code":13,"message":"input is not a directory"}
How can I achieve the result I want for both iOS and Android file paths?

Your path refers 'cacheDirectory' this.file.cacheDirectory.
To get the file name.
const pathsplit = "file:///storage/emulated/0/Android/data/com.myApp/cache/1535369478970-cropped.jpg".split('/');
filename = pathsplit[pathsplit.length-1];

Firebase getDownloadURL() is really slow at getting image URLs

I have a page which displays multiple (12+) objects in components, each of which has an image. This image has a src attribute, which I'm setting via a call to my Firebase storage:
this.props.storage.child('Land.jpg').getDownloadURL().then(function(url) {
var img = document.getElementById('imageGoesHere'+this.state.currentId);
img.src = url;
}.bind(this))
The storage prop is generated and passed down by a parent component:
firebase.initializeApp(config);
var storage = firebase.storage();
var storageRef = storage.ref("");
The storageRef var is passed as a prop to each child component that calls the getDownloadURL() function.
This works, but the getDownloadURL function seems to be really slow. The page renders, and then each image takes at least a second to be shown/rendered. I know it's not the raw URL's issue, since I can get the raw URL and paste it into the src and it loads fine (i.e. nearly instantly); I look at the Network tab in chrome devtools and it shows at least a second between the getDownloadURL call and the response. Is this the proper way of getting images to be shown in web pages? The Firebase Storage docs don't mention any other way.
edit: Two things; firstly I only have 5 test images in the storage, so it's not like I'm syncing to a whole bunch with the root storage ref.
Second thing; chrome devtools shows me that it's "stalling" for 1-4 seconds, and the actual request only takes 100ms tops. What can cause stalling?

This can happen because you are downloading every single image you have every time this code runs. It can happen if you sync the instance to the entire image hierarchy.
You could also figure out the reason by looking at Chrome's Network tab, and see if you have tons of files downloading, like this:

you should use the firebase public url when the image is uploaded (if you doing that way). Something like this:
uploadTask.then(function(snapshot) { return snapshot.ref.getDownloadURL() })
And save it to the DB inmediatly. Just create an "images" table and create a column named "url" or "link". That url/link you will use every time you need that image.
Now if you are uploading manually , select the image (in firebase console) and in the "file location" section, go to "access token".