I have a Google Cloud Function hosted on Firebase which takes an image URL from Twilio and, using pipe(), uploads the image to a bucket on GCS. This works, however I want to call another function once the image has been successfully uploaded. I have a .on('finish') to trigger another function, however in looking at logs, it appears the finish is being called almost a minute after the file has actually finished uploading. I can physically see the image in my GCS console well before the finish function is called.
What might be the cause of this delay?
const mediaUrl = request.body.MediaUrl0;
const contentType = request.body.MediaContentType0;
const extension = mime.extension(contentType);
global.filename = Date.now() + "." + extension;
console.log("Filename: " + filename);
//Save image to gcs bucket
const gc = new Storage({
projectID: 'testProject-rgjdxr'
});
const bucketName = 'msg-images';
const bucket = gc.bucket(bucketName);
const file = bucket.file(filename);
const w = file.createWriteStream({
metadata:{
contentType: contentType
}
});
pollRequest(mediaUrl).auth(accountSid, authToken).pipe(w).on('finish', () => {
global.gcsUrl = `https://storage.cloud.google.com/${bucketName}/${filename}`;
console.log(gcsUrl);
//I can see this log, however the timestamp is about 60sec
//after the file was uploaded successfully to GCS bucket.
analyzeImage(gcsUrl); //passing image url to another function
});
Related
Im having problem uploading image to firebase storage, it keeps uploading 9B file to storage even if selected file is a 100mb file. It is showing the progress as NaN%, once i successfully uploaded a image to firebase storage but now im failing 😠here is the code,
const app = initializeApp(firebaseConfig);
const analytics = getAnalytics(app);
const storage = getStorage();
var picker = document.getElementById('img');
picker.onchange = function(){
var file = window.URL.createObjectURL(picker.files[0]);
var filename = picker.files[0].name;
const storageRef = ref(storage, 'icons/' + filename);
// 'file' comes from the Blob or File API
uploadBytes(storageRef, file).then((snapshot) => {
console.log('Uploaded a blob or file!');
});
}
I tried many options i doesn't know why it is not working, i want to upload image & get download url.
You have to pass the actual File object to uploadBytes and not the object URL. Try:
picker.onchange = function() {
const file = picker.files[0];
if (!file) {
alert("No file selected")
return
}
const storageRef = ref(storage, 'icons/' + file.name);
uploadBytes(storageRef, file).then((snapshot) => {
console.log('Uploaded a blob or file!');
}).catch(e => console.log(e));
}
It seems you are providing a url to the image blob/file instead of passing the file itself. Try changing line 8 to just var file = picker.files[0].
If that doesn’t work, try logging fileafter it is initialized to make sure it exists.
The file does get uploaded but instead of it being 100s of Kbs or few MBs its just a couple of Bytes, and when trying to open it shows blank or "file not exist" error. same issue with text files and images.
I believe the problem is with the stream not waiting to read to the whole file before uploading it to bucket.
Code sample is the one provided by google from their "Google Cloud Storage: Node.js Client" Documentation
function main(
bucketName = 'myBucket',
destFileName = 'MyUploadedFile',
contents = 'testFile.pdf'
) {
// Imports the Google Cloud client library
const {Storage} = require('#google-cloud/storage');
// Import Node.js stream
const stream = require('stream');
// Creates a client
const storage = new Storage();
// Get a reference to the bucket
const myBucket = storage.bucket(bucketName);
// Create a reference to a file object
const file = myBucket.file(destFileName);
const passthroughStream = new stream.PassThrough();
passthroughStream.write(contents);
//console.log(passthroughStream.write(contents))
passthroughStream.end();
async function streamFileUpload() {
passthroughStream.pipe(file.createWriteStream({resumable:true,gzip:true})).on('finish', () => {
// The file upload is complete
});
console.log(`${destFileName} uploaded to ${bucketName}`);
}
streamFileUpload().catch(console.error);
// [END storage_stream_file_upload]
}
main(...process.argv.slice(2));
I am trying to download the image and save it in my server from the url address. So for example I make a POST request with URL of the image. I download the image and I save it in my server. The problem comes when I need to figure our the extension of the image. Right now it works staticaly only for jpg files, but it should work for png aswell. How can I find out the extension of the file before saving it?
One way would be to get the extension from the url itself, but not all urls will have the extension , for example: https://media.istockphoto.com/photos/winter-in-the-sequoias-picture-id1292624259
This is the code that I have made right now. It works, however how I said, its static and only working for jpg:
var config = {
responseType: 'stream'
};
async function getImage(url) {
let time = Math.floor(Date.now() / 1000)
let resp = await axios.get(url, config)
resp.data.pipe(fs.createWriteStream(time+'.jpg')) // here I need to get the image extension isntead of static '.jpg'
}
You can use response headers for that. The Content-Type header should tell you the type of the file and with Content-Disposition you can get the filename with extension.
In your code you can access these headers like this
resp.headers['content-type'];
resp.headers['content-disposition'];
I'd suggest using a module such as mime to get the extension from the content-type.
Complete example:
const axios = require('axios');
const fs = require('fs');
const mime = require('mime');
var config = {
responseType: 'stream'
};
async function getImage(url) {
let time = Math.floor(Date.now() / 1000)
let resp = await axios.get(url, config)
const contentLength = resp.headers['content-length'];
const contentType = resp.headers['content-type'];
const extension = mime.extension(contentType);
console.log(`Content type: ${contentType}`);
console.log(`Extension: ${extension}`);
const fileName = time + "." + extension;
console.log(`Writing ${contentLength} bytes to file ${fileName}`);
resp.data.pipe(fs.createWriteStream(fileName));
}
const url = 'https://media.istockphoto.com/photos/winter-in-the-sequoias-picture-id1292624259';
getImage(url)
This will give an output somewhat like:
Content type: image/jpeg
Extension: jpeg
Writing 544353 bytes to file 1638867349.jpeg
This is what I am trying to achieve, implement the firebase's resize image extension, upload an image, then when the resize is completed, add that dowloadUrl's thumbs to a Cloud Firestore document. This question helps me, but still can not identify the thumbs and get the download URL, this is what am have been trying so far.
Note: I set my thumbnail to be at root/thumbs
const functions = require('firebase-functions');
const { Storage } = require('#google-cloud/storage');
const storage = new Storage();
exports.thumbsUrl = functions.storage.object().onFinalize(async object => {
const fileBucket = object.bucket;
const filePath = object.name;
const contentType = object.contentType;
if (fileBucket && filePath && contentType) {
console.log('Complete data');
if (!contentType.startsWith('thumbs/')) {
console.log('This is not a thumbnails');
return true;
}
console.log('This is a thumbnails');
} else {
console.log('Incomplete data');
return null;
}
});
Method 1 : Client Side
Don't change the access token when creating the thumbnail.
Edit the function from gcloud cloud function console
Go to the function code by clicking detailed usage stats
Then click on code
Edit the following lines
Redeploy the function again
// If the original image has a download token, add a
// new token to the image being resized #323
if (metadata.metadata.firebaseStorageDownloadTokens) {
// metadata.metadata.firebaseStorageDownloadTokens = uuidv4_1.uuid();
}
Fetch the uploaded image using getDownloadURLfunction
https://firebasestorage.googleapis.com/v0/b/<project_id>/o/<FolderName>%2F<Filename>.jpg?alt=media&token=xxxxxx-xxx-xxx-xxx-xxxxxxxxxxxxx
Because the access token will be similar
https://firebasestorage.googleapis.com/v0/b/<project_id>/o/<FolderName>%2Fthumbnails%2F<Filename>_300x300.jpg?alt=media&token=xxxxxx-xxx-xxx-xxx-xxxxxxxxxxxxx
Method 2: Server Side
Call this function after thumbnail is created
var storage = firebase.storage();
var pathReference = storage.ref('users/' + userId + '/avatar.jpg');
pathReference.getDownloadURL().then(function (url) {
$("#large-avatar").attr('src', url);
}).catch(function (error) {
// Handle any errors
});
you need to use filePath for checking the thumbs
if(filePath.startswith('thumbs/'){...}
contentType has the metadata of files like type of image and etc.
FilePath will have the full path.
I am currently attempting to upload a photo to my Firebase app's storage in my Apache Cordova app. I currently get the photo's URI with the following code:
function getPhotoFromAlbum() {
navigator.camera.getPicture(onPhotoURISuccess, onFail, {
quality: 50,
sourceType: navigator.camera.PictureSourceType.SAVEDPHOTOALBUM,
destinationType: navigator.camera.DestinationType.FILE_URI
});
}
function onPhotoURISuccess(imageURI) {
var image = document.getElementById('image');
image.style.display = 'block';
image.src = imageURI;
getFileEntry(imageURI);
}
And then am attempting to convert the image into a file and push it to my Firebase storage with the following function:
function getFileEntry(imgUri) {
window.resolveLocalFileSystemURL(imgUri, function success(fileEntry) {
console.log("got file: " + fileEntry.fullPath);
var filename = "test.jpg";
var storageRef = firebase.storage().ref('/images/' + filename);
var uploadTask = storageRef.put(fileEntry);
}, function () {
// If don't get the FileEntry (which may happen when testing
// on some emulators), copy to a new FileEntry.
createNewFileEntry(imgUri);
});
}
I have both the file and the camera cordova plugins installed, the only errors I get when I attempt to do this is
Error in Success callbackId: File1733312835 : [object Object]
Which is just an error message from cordova.js
I also know I have my Firebase storage set up correctly because I have tested it through an emulator by adding a file input and successfully uploading whatever file the user added, to the Firebase storage.
Is it possible to upload a file to Firebase storage using this method of converting an image to a file through its URI, and then uploading it? If so, what is the correct way to do so / what is wrong with the way i'm doing it?
I was able to accomplish uploading an image by using a data url. Below is my code:
var filename = "test.jpg";
var storageRef = firebase.storage().ref('/images/' + filename);
var message = 'data:image/jpg;base64,' + imageUri;
storageRef.putString(message, 'data_url').then(function (snapshot) {
console.log('Uploaded a data_url string!');
});
Is it possible to upload a file to Firebase storage using this method of converting an image to a file through its URI, and then uploading it? If so, what is the correct way to do so / what is wrong with the way i'm doing it?
Yes it is possible to upload a file on firebase through its URI. However you have to follow the correct way.
1. You have to store the data in firebase after file reading operation is completed.you can use FileReader.onloadend for this.
2. By using a data_url you can store to firebase.
Here is the snippet for more clarity:
function getFileEntry(imgUri) {
window.resolveLocalFileSystemURL(imgUri, function onSuccess(fileEntry) {
fileEntry.file(function(file) {
var reader = new FileReader();
reader.onloadend = function() {
filename = "test.jpg";
var storageRef = firebase.storage().ref('/images/' + filename);
var data = 'data:image/jpg;base64,' + imgUri;
storageRef.putString(data, 'data_url').then(function (snapshot) {
console.log('Image is uploaded by base64 format...');
});
};
reader.readAsArrayBuffer(file);
});
},
function onError(err) {
console.log(err);
createNewFileEntry(imgUri);
});
}