This question already has answers here:
Get Download URL from file uploaded with Cloud Functions for Firebase
(25 answers)
Closed 4 years ago.
I have a cloud function that generates a set of resized images for every image uploaded. This is triggered with the onFinalize() hook.
Cloud Function to resize an uploaded image:
export const onImageUpload = functions
.runWith({
timeoutSeconds: 120,
memory: '1GB'
})
.storage
.object()
.onFinalize(async object => {
const bucket = admin.storage().bucket(object.bucket)
const filePath = object.name
const fileName = filePath.split('/').pop()
const bucketDir = dirname(filePath)
const workingDir = join(tmpdir(), 'resizes')
const tmpFilePath = join(workingDir, fileName)
if (fileName.includes('resize#') || !object.contentType.includes('image')) {
return false
}
await fs.ensureDir(workingDir)
await bucket.file(filePath).download({
destination: tmpFilePath
})
const sizes = [
500,
1000
]
const uploadPromises = sizes.map(async size => {
const resizeName = `resize#${size}_${fileName}`
const resizePath = join(workingDir, resizeName)
await sharp(tmpFilePath)
.resize(size, null)
.toFile(resizePath)
return bucket.upload(resizePath, {
destination: join(bucketDir, resizeName)
})
})
// I need to now update my Firestore database with the public URL.
// ...but how do I get that here?
await Promise.all(uploadPromises)
return fs.remove(workingDir)
})
That's all well and good and it works, but I also need to somehow retrieve the public URL for each of these images, in order to write the values into my Firestore.
I can do this on the frontend using getDownloadURL(), but I'm not sure how to do it from within a Cloud Function from the newly generated images.
As I see it, this needs to happen on the backend anyway, as my frontend has no way of knowing when the images have been processed.
Only works on the client:
const storageRef = firebase.storage().ref()
const url = await storageRef.child(`images/${image.name}`).getDownloadURL()
Any ideas?
Answer (with caveats):
This question was technically answered correctly by #sergio below, but I just wanted to point out some additional things that need doing before it can work.
It appears that the 'expires' parameter of getSignedUrl() has to be a number according to TypeScript. So, to make it work I had to pass a future date represented as an epoch (milliseconds) like 3589660800000.
I needed to pass credentials to admin.initializeApp() in order to use this method. You need to generate a service account key in your Firebase admin. See here: https://firebase.google.com/docs/admin/setup?authuser=1
Hope this helps someone else out too.
I believe the promises returned from bucket upload contain a reference to the File, which then you can use to obtain a signed URL.
Something like (not tested):
const data = await bucket.upload(resizePath, { destination: join(bucketDir, resizeName) });
const file = data[0];
const signedUrlData = await file.getSignedUrl({ action: 'read', expires: '03-17-2025'});
const url = signedUrlData[0];
Related
I have two questions here, an Expo MediaLibrary question, and a linked Javascript/scoping question.
I have a ReactNative Expo app that loads recent images from a users media library. For this I need to use the MediaLibrary.getAssetsAsync() method.
Annoyingly, .getAssetsAsync() does not return localUri needed to display the image. It returns uri property which looks like this: "uri": "ph://8A1262C3-23F7-4BD3-8943-C01128DCEEB1"
Unless I'm mistaken, I can't use an asset file uri like this to display images in react, we need localUri. So for each photo I am calling the MediaLibrary.getAssetInfoAsync() method - which returns localUri. Here's how I am accomplishing that:
const selectAlbum = async (albumName) => {
const foundAlbum = await MediaLibrary.getAssetsAsync({album: albumName, mediaType: 'photo', first: 20})
const assets = foundAlbum['assets'] //array of photos or 'assets'
const assetsWithInfo = []
//for each selected photo
for(let i=0; i<assets.length; i++){
const assetWithInfo = getAssetInfo(assets[i].id)
//console.log(assetWithInfo) //doesnt work, null etc - WHY?!
assetsWithInfo.push(assetWithInfo)
}
}
const getAssetInfo = async (id) =>{
let res = await MediaLibrary.getAssetInfoAsync(id)
let asset={
creationTime: res['creationTime'],
isFavorite: res['isFavorite'],
localUri: res['localUri'],
}
//console.log(asset) //works, object correctly displays
return asset
}
My questions are:
Is there a more efficient way to do this i.e. display images from MediaLibrary without having to call so many functions. This feels messy and more complicated than it should be.
In the getAssetInfo async, the asset object is correctly displayed (using the console.log) with all properties. But when I return this object to the selectAlbum function and console.log the result, I get null etc. Why?
For #2, the issue is you're not awaiting the result of the call.
const assetWithInfo = await getAssetInfo(assets[i].id); should fix it.
I wanted to delete the file from storage when a data node is deleted from the realtime database. the name of the file to be deleted is taken before deleted. The name of the file is saved in the node in child named "imageTitle". The code works fine before implementing the file delete code. I mean the nodes get deleted perfectly.
When I implement the file delete code the rest doesn't work, but there is no any errors. The code after file delete doesn't work. I dunno why.
This is for an academic final project.
There's a folder named images in the bucket, and the file I need to delete is in there. The file name is taken from the child in the node which is to be deleted in the realtime database named imageTitle:
'use strict';
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
exports.changetime = functions.database.ref('/posts/{pushId}')
.onCreate((snapshot, context) => {
const editDate = Date.now()
datas = snapshot.val();
return snapshot.ref.update({editDate})
})
const CUT_OFF_TIME = 1 * 60 * 1000;
/**
* This database triggered function will check for child nodes that are older than the
* cut-off time. Each child needs to have a `timestamp` attribute.
*/
exports.deleteOldItems = functions.database.ref('/posts/{pushId}').onWrite(async (change,
context) => {
const ref = change.after.ref.parent; // reference to the parent
const now = Date.now();
const id = context.params.pushId;
const cutoff = now - CUT_OFF_TIME;
const oldItemsQuery = ref.orderByChild('createdDate').endAt(cutoff);
const snapshot = await oldItemsQuery.once('value');
const getImageTitle = admin.database().ref(`/posts/${id}/imageTitle`).once('value');
return getImageTitle.then(imageTitle => {
console.log('post author id', imageTitle.val());
const imageName = imageTitle.val();
const filePath = 'images/' + imageName;
const path = `images/${imageName}`;
const bucket = app.storage().bucket();
return bucket.file(path).delete().then(() =>{
console.log(`File deleted successfully in path: ${imageName}`)
/* The problem is here. the code doesn't work after file.delete function. no errors. but doesn't work
if I remove that piece of code the other codes work fine. I mean the realtime database nodes get deleted and updated perfectly */
const updates = {};
snapshot.forEach(child => {
updates[child.key] = null;
if(updates[child.key] == null){
admin.database().ref(/post-likes/+id).remove();
}
})
return ref.update(updates);
});
});
[my storage looks like this][1]});
There's a folder named images in the bucket, and the file I need to delete is in there. The file name is taken from the child in the node which id to be deleted in the realtime database imageTitle:
enter image description here
I think this is the problem:
const filePath = 'images/' + imageName;
const path = `images/${imageName}`;
You should be using filePath not path in your bucket reference. const path = `images/${imageName}`; is wild-card syntax used when querying, not when assigning variables....think you stayed in 'Dart' mode here:-). Not sure what path contains, therefore, but console.log it to see.
Here is some code that I use to delete images from my storage bucket that works perfectly. First, a couple of things to check (best done using console.log, never assume you know what is in the variable), and do:
Ensure imageName, filePath, path and bucket contain what they should contain.
Include the .catch block in your code to check if there actually is an error
I am not clear if console.log(`File deleted successfully in path: ${imageName}`) is executing but if it isn't then your file.delete must be throwing an error which the catch block should trap.
The code snippet:
const oldProfileImagePath = "profileImages/" + authenticatedUserId + "/" + oldProfileImageName;
return bucket.file(oldProfileImagePath).delete()
.then(function () {
console.info("Old Profile Image: " + oldProfileImagePath + " deleted!");
})
.catch(function (error) {
console.error("Remove Old Profile Image: " + oldProfileImagePath +
" failed with " + error.message)
});
I am stuck in what I thought was a very simple use case: I have a list of client ids in an array. All I want to do is fetch all those clients and "watch" them (using the .onSnapshot).
To fetch the client objects, it is nice and simple, I simply go through the array and get each client by their id. The code looks something like this:
const accessibleClients = ['client1', 'client2', 'client3']
const clients = await Promise.all(
accessibleClients.map(async clientId => {
return db
.collection('clients')
.doc(clientId)
.get()
})
)
If I just needed the list of clients, it would be fine, but I need to perform the .onSnapshot on it to see changes of the clients I am displaying. Is this possible to do? How can I get around this issue?
I am working with AngularFire so it is a bit different. But i also had the problem that i need to listen to unrelated documents which can not be queried.
I solved this with an object which contains all the snapshot listeners. This allows you to unsubscribe from individual client snapshots or from all snapshot if you do not need it anymore.
const accessibleClients = ['client1', 'client2', 'client3'];
const clientSnapshotObject = {};
const clientDataArray = [];
accessibleClients.forEach(clientId => {
clientSnapshotArray[clientId] = {
db.collection('clients').doc(clientId).onSnapshot(doc => {
const client = clientDataArray.find(client => doc.id === client.clientId);
if (client) {
const index = clientDataArray.findIndex(client => doc.id === client.clientId);
clientDataArray.splice(index, 1 , doc.data())
} else {
clientDataArray.push(doc.data());
}
})
};
})
With the clientIds of the accessibleClients array, i create an object of DocumentSnapshots with the clientId as property key.
The snapshot callback function pushes the specific client data into the clientDataArray. If a snapshot changes the callback function replaces the old data with the new data.
I do not know your exact data model but i hope this code helps with your problem.
I'm looking for a way to check if two files/documents (PDF, JPG, PNG) are the same.
If a user selects one or more files, I convert the File Object to a javascript object. I'm keeping the size, type, filename and I create a blob so I can store the object in my redux store.
When a user selects another file I want to compare this file with the files that already has been added (so I can set the same blobURL).
I can check if two files has the same name, type and size but there is a change that all these properties match and the files aren't the same so I would like to check the file path. Unfortunately, that property isn't provided in the File Object. Is there a way to get this or another solution to make sure both files are (not) the same?
No there is no way to get the real path, but that doesn't matter.
All you have access to is a FakePath, in the form C:\fakepath\yourfilename.ext (from input.value), and sometimes a bit more if you gained access to a directory.
But anyway you probably don't want to check that two files came from the same place on the hard-disk, this has no importance whatsoever, since they could very well have been modified since first access.
What you can and probably want to do however, is to check if their content
are the same.
For this, you can compare their byte content:
inp1.onchange = inp2.onchange = e => {
const file1 = inp1.files[0];
const file2 = inp2.files[0];
if(!file1 || !file2) return;
compare(file1, file2)
.then(res => console.log('are same ? ', res));
};
function compare(file1, file2) {
// they don't have the same size, they are different
if(file1.size !== file2.size)
return Promise.resolve(false);
// load both as ArrayBuffers
return Promise.all([
readAsArrayBuffer(file1),
readAsArrayBuffer(file2)
]).then(([buf1, buf2]) => {
// create views over our ArrayBuffers
const arr1 = new Uint8Array(buf1);
const arr2 = new Uint8Array(buf2);
return !arr1.some((val, i) =>
arr2[i] !== val // search for diffs
);
});
}
function readAsArrayBuffer(file) {
// we could also have used a FileReader,
// but Response is conveniently already Promise based
return new Response(file).arrayBuffer();
}
<input type="file" id="inp1">
<input type="file" id="inp2">
Now, you say that you don't have access to the original Files anymore, and that you can only store serializable data. In this case, one less performant solution is to generate a hash from your Files.
This can be done on front-end, thanks to the SubtleCrypto API,
but this operation being quite slow for big files, you may want to do it systematically from server instead of doing it on front, and to only do it on front when the sizes are the same:
// a fake storage object like OP has
const store = [
{ /* an utf-8 text file whose content is `hello world`*/
name: "helloworld.txt",
size: 11,
hash: "b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9" // generated from server
}
];
// the smae file as the one we fakely stored
const sameFile = new File(['hello world'], 'same-file.txt');
// a file the same size as the one we stored (needs deep check)
const sameSizeButDifferentContent = new File(['random text'], 'differentcontent.txt');
inp.onchange = e => tryToStore(inp.files[0]);
tryToStore(sameFile); // false
tryToStore(sameSizeButDifferentContent);
// hash: "a4e082f56a58e0855a6abbf2f4ebd08895ff85ea80e634e02b210def84b557dd"
function tryToStore(file) {
checkShouldStore(file)
.then(result => {
console.log('should store', file.name, result)
if(result)
store.push(result);
// this is just for demo, in your case you would do it on the server
if(!result.hash)
generateHash(file).then(h => result.hash = h);
});
}
async function checkShouldStore(file) {
const {name, size} = file;
const toStore = {name, size, file}; // create a wrapper object
// first check against the sizes (fast checking)
const sameSizes = store.filter(obj => obj.size === file.size);
// only if some files have the same size
if(sameSizes.length) {
// then we generate a hash directly
const hash = await generateHash(file);
if(sameSizes.some(obj => obj.hash === hash)) {
return false; // is already in our store
}
toStore.hash = hash; // save the hash so we don't have to generate it on server
}
return toStore;
}
async function generateHash(file) {
// read as ArrayBuffer
const buf = await new Response(file).arrayBuffer();
// generate SHA-256 hash using crypto API
const hash_buf = await crypto.subtle.digest("SHA-256", buf);
// convert to Hex
const hash_arr = [...new Uint8Array(hash_buf)]
.map(v => v.toString(16).padStart(2, "0"));
return hash_arr.join('');
}
<input type="file" id="inp">
I am writing a Firebase function where I am following the code from the sample code in Github provided by Firebase
However, I am consistently getting the error
Function returned undefined, expected Promise or value in my Firebase functions log.
I have pretty much modified my code to be exactly the same and yet no respite. Has anyone tried this code? Is it error free? Why am I getting the error? Same code is also in the Firebase guide
Sample code producing the error is below
exports.imageToJPG = functions.storage.object().onChange(event => {
const object = event.data;
const filePath = object.name;
const baseFileName = path.basename(filePath, path.extname(filePath));
const fileDir = path.dirname(filePath);
const JPEGFilePath = path.normalize(path.format({dir: fileDir, name: baseFileName, ext: JPEG_EXTENSION}));
const tempLocalFile = path.join(os.tmpdir(), filePath);
const tempLocalDir = path.dirname(tempLocalFile);
const tempLocalJPEGFile = path.join(os.tmpdir(), JPEGFilePath);
// Exit if this is triggered on a file that is not an image.
if (!object.contentType.startsWith('image/')) {
console.log('This is not an image.');
return;
}
// Exit if the image is already a JPEG.
if (object.contentType.startsWith('image/jpeg')) {
console.log('Already a JPEG.');
return;
}
// Exit if this is a move or deletion event.
if (object.resourceState === 'not_exists') {
console.log('This is a deletion event.');
return;
}
const bucket = gcs.bucket(object.bucket);
// Create the temp directory where the storage file will be downloaded.
return mkdirp(tempLocalDir).then(() => {
// Download file from bucket.
return bucket.file(filePath).download({destination: tempLocalFile});
}).then(() => {
console.log('The file has been downloaded to',
tempLocalFile);
// Convert the image to JPEG using ImageMagick.
return spawn('convert', [tempLocalFile, tempLocalJPEGFile]);
}).then(() => {
console.log('JPEG image created at', tempLocalJPEGFile);
// Uploading the JPEG image.
return bucket.upload(tempLocalJPEGFile, {destination: JPEGFilePath});
}).then(() => {
console.log('JPEG image uploaded to Storage at', JPEGFilePath);
// Once the image has been converted delete the local files to free up disk space.
fs.unlinkSync(tempLocalJPEGFile);
fs.unlinkSync(tempLocalFile);
});
});
Any pointers?
It seems that recently Firebase has updated their SDK due to which their sample code and documentation is little out of date. You must return with a boolean value even if you are just trying to exit the function. So it must be return true for each of the return statements in the code above where there is no Promise being returned.
I will delete this question and answer once Firebase has updated their sample code and documentation. Till then leaving it here for those who may still stumble upon this issue without knowing why.