Develop Reference

Delete same value from multiple locations Firebase Functions

I have a firebase function that deletes old messages after 24 hours as in my old question here. I now have just the messageIds stored in an array under the user such that the path is: /User/objectId/myMessages and then an array of all the messageIds under myMessages. All of the messages get deleted after 24 hours, but the iDs under the user's profile stay there. Is there a way to continue the function so that it also deletes the messageIds from the array under the user's account?
I'm new to Firebase functions and javascript so I'm not sure how to do this. All help is appreciated!

Building upon #frank-van-puffelen's accepted answer on the old question, this will now delete the message IDs from their sender's user data as part of the same atomic delete operation without firing off a Cloud Function for every message deleted.
Method 1: Restructure for concurrency
Before being able to use this method, you must restructure how you store entries in /User/someUserId/myMessages to follow best practices for concurrent arrays to the following:
{
"/User/someUserId/myMessages": {
"-Lfq460_5tm6x7dchhOn": true,
"-Lfq483gGzmpB_Jt6Wg5": true,
...
}
}
This allows you to modify the previous function to:
// Cut off time. Child nodes older than this will be deleted.
const CUT_OFF_TIME = 24 * 60 * 60 * 1000; // 2 Hours in milliseconds.
exports.deleteOldMessages = functions.database.ref('/Message/{chatRoomId}').onWrite(async (change) => {
const rootRef = admin.database().ref(); // needed top level reference for multi-path update
const now = Date.now();
const cutoff = (now - CUT_OFF_TIME) / 1000; // convert to seconds
const oldItemsQuery = ref.orderByChild('seconds').endAt(cutoff);
const snapshot = await oldItemsQuery.once('value');
// create a map with all children that need to be removed
const updates = {};
snapshot.forEach(messageSnapshot => {
let senderId = messageSnapshot.child('senderId').val();
updates['Message/' + messageSnapshot.key] = null; // to delete message
updates['User/' + senderId + '/myMessages/' + messageSnapshot.key] = null; // to delete entry in user data
});
// execute all updates in one go and return the result to end the function
return rootRef.update(updates);
});
Method 2: Use an array
Warning: This method falls prey to concurrency issues. If a user was to post a new message during the delete operation, it's ID could be removed while evaluating the deletion. Use method 1 where possible to avoid this.
This method assumes your /User/someUserId/myMessages object looks like this (a plain array):
{
"/User/someUserId/myMessages": {
"0": "-Lfq460_5tm6x7dchhOn",
"1": "-Lfq483gGzmpB_Jt6Wg5",
...
}
}
The leanest, most cost-effective, anti-collision function I can come up for this data structure is the following:
// Cut off time. Child nodes older than this will be deleted.
const CUT_OFF_TIME = 24 * 60 * 60 * 1000; // 2 Hours in milliseconds.
exports.deleteOldMessages = functions.database.ref('/Message/{chatRoomId}').onWrite(async (change) => {
const rootRef = admin.database().ref(); // needed top level reference for multi-path update
const now = Date.now();
const cutoff = (now - CUT_OFF_TIME) / 1000; // convert to seconds
const oldItemsQuery = ref.orderByChild('seconds').endAt(cutoff);
const snapshot = await oldItemsQuery.once('value');
// create a map with all children that need to be removed
const updates = {};
const messagesByUser = {};
snapshot.forEach(messageSnapshot => {
updates['Message/' + messageSnapshot.key] = null; // to delete message
// cache message IDs by user for next step
let senderId = messageSnapshot.child('senderId').val();
if (!messagesByUser[senderId]) { messagesByUser[senderId] = []; }
messagesByUser[senderId].push(messageSnapshot.key);
});
// Get each user's list of message IDs and remove those that were deleted.
let pendingOperations = [];
for (let [senderId, messageIdsToRemove] of Object.entries(messagesByUser)) {
pendingOperations.push(admin.database.ref('User/' + senderId + '/myMessages').once('value')
.then((messageArraySnapshot) => {
let messageIds = messageArraySnapshot.val();
messageIds.filter((id) => !messageIdsToRemove.includes(id));
updates['User/' + senderId + '/myMessages'] = messageIds; // to update array with non-deleted values
}));
}
// wait for each user's new /myMessages value to be added to the pending updates
await Promise.all(pendingOperations);
// execute all updates in one go and return the result to end the function
return ref.update(updates);
});

Update: DO NOT USE THIS ANSWER (I will leave it as it may still be handy for detecting a delete operation for some other need, but do not use for the purpose of cleaning up an array in another document)
Thanks to #samthecodingman for providing an atomic and concurrency safe answer.
If using Firebase Realtime Database you can add an onChange event listener:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
exports.onDeletedMessage = functions.database.ref('Message/{messageId}').onChange(async event => {
// Exit if this item exists... if so it was not deleted!
if (event.data.exists()) {
return;
}
const userId = event.data.userId; //hopefully you have this in the message document
const messageId = event.data.messageId;
//once('value') useful for data that only needs to be loaded once and isn't expected to change frequently or require active listening
const myMessages = await functions.database.ref('/users/' + userId).once('value').snapshot.val().myMessages;
if(!myMessages || !myMessages.length) {
//nothing to do, myMessages array is undefined or empty
return;
}
var index = myMessages.indexOf(messageId);
if (index === -1) {
//nothing to delete, messageId is not in myMessages
return;
}
//removeAt returns the element removed which we do not need
myMessages.removeAt(index);
const vals = {
'myMessages': myMessages;
}
await admin.database.ref('/users/' + userId).update(vals);
});
If using Cloud Firestore can add an event listener on the document being deleted to handle cleanup in your user document:
exports.onDeletedMessage = functions.firestore.document('Message/{messageId}').onDelete(async event => {
const data = event.data();
if (!data) {
return;
}
const userId = data.userId; //hopefully you have this in the message document
const messageId = data.messageId;
//now you can do clean up for the /user/{userId} document like removing the messageId from myMessages property
const userSnapShot = await admin.firestore().collection('users').doc(userId).get().data();
if(!userSnapShot.myMessages || !userSnapShot.myMessages.length) {
//nothing to do, myMessages array is undefined or empty
return;
}
var index = userSnapShot.myMessages.indexOf(messageId);
if (index === -1) {
//nothing to delete, messageId is not in myMessages
return;
}
//removeAt returns the element removed which we do not need
userSnapShot.myMessages.removeAt(index);
const vals = {
'myMessages': userSnapShot.myMessages;
}
//To update some fields of a document without overwriting the entire document, use the update() method
await admin.firestore().collection('users').doc(userId).update(vals);
});

Any way to ensure I get the return value from a sequence of async functions

I'm working with repos and trying to get the pull requests, issues and commits for a repo. I have the following code:
const axios = require('axios');
var gitPullApiLink = "https://api.github.com/repos/alirezadir/Production-Level-Deep-Learning/pulls"
var listOfCommits = [];
var listOfSHAs = [];
var mapOfInfoObjects = new Map();
var mapPullRequestNumberToCommits = new Map();
var mapPRNumbersToCommitObjects = new Map();
var listOfPrObjects = [];
var setOfFileObjects = new Set();
var listOfNumbersOfTargetedIssues = [];
var mapPRnumberToCloseOpenDateObjects = new Map();
class PullRequestParser {
async getListOfPullRequests(pullrequestLink) {
const message = await axios.get(pullrequestLink);
//console.log(message);
listOfPrObjects = message['data'];
}
async getCommitsForEachPullRequestAndPRinformation() {
var listOfPrNumbers = [];
var k;
// this loop will just make a list of Pull Request Numbers
for (k = 0; k < listOfPrObjects.length; k++){
var currPrNumber = listOfPrObjects[k]['number'];
listOfPrNumbers.push(currPrNumber);
}
// I created a separate list just because... I did it this way because on the github API website it seems
// like the pull request has the same number as the issue it affects. I explain how you can see this down below
listOfNumbersOfTargetedIssues = listOfPrNumbers;
// next loop will make objects that contain information about each pull request.
var n;
for (n = 0; n < listOfPrNumbers; n++){
var ApiLinkForEachPullRequest = gitPullApiLink + "/" + listOfPrNumbers[n];
const mes = await axios.get(ApiLinkForEachPullRequest);
var temp = {OpeningDate: mes['data']['created_at'],
ClosingDate: mes['data']['closed_at'],
IssueLink: mes['data']['_links']['issue']['href']};
//mapPRnumberToCloseOpenDateObjects will be a map where the key is the pull request number and the value
// is the object that stores the open date, close date, and issue link for that pull request. The reason
// why I said I think the pull request number is the same as the number of the issue it affects is because
// if you take any object from the map, say you do mapPRnumberToCloseOpenDateObjects.get(10). You'll
// get an object with a pull request number 10. Now if you take this object and look at it's "IssueLink"
// field, the very last part of the link will have the number 10, and if you look at the github API
// it says for a single issue, you do: /repos/:owner/:repo/issues/:issue_number <---- As you can see,
// the IssueLink field will have this structure and in place of the issue_number, the field will be 10
// for our example object.
mapPRnumberToCloseOpenDateObjects.set(listOfPrNumbers[n], temp);
}
//up to this point, we have the pull request numbers. we will now start getting the commits associated with
//each pull request
var j;
for (j = 0; j < listOfPrNumbers.length; j++){
var currentApiLink = gitPullApiLink + "/" + listOfPrNumbers[j] + "/commits";
const res = await axios.get(currentApiLink);
//here we map a single pull request to the information containing the commits. I'll just warn you in
// advance: there's another object called mapPRNumbersToCommitObjects. THIS MAP IS DIFFERENT! I know it's
// subtle, but I hope the language can make the distinction: mapPullRequestNumberToCommits will just
// map a pull request number to some data about the commits it's linked to. In contrast,
// mapPRNumbersToCommitObjects will be the map that actually maps pull request numbers to objects
// containing information about the commits a pull request is associated with!
mapPullRequestNumberToCommits.set(listOfPrNumbers[j], res['data']);
}
// console.log("hewoihoiewa");
}
async createCommitObjects(){
var x;
// the initial loop using x will loop over all pull requests and get the associated commits
for (x = 0; x < listOfPrObjects.length; x++){
//here we will get the commits
var currCommitObjects = mapPullRequestNumberToCommits.get(listOfPrObjects[x]['number']);
//console.log('dhsiu');
// the loop using y will iterate over all commits that we get from a single pull request
var y;
for (y = 0; y < currCommitObjects.length; y++){
var currentSHA = currCommitObjects[y]['sha'];
listOfSHAs.push(currentSHA);
var currApiLink = "https://api.github.com/repos/alirezadir/Production-Level-Deep-Learning/commits/" + currentSHA;
const response = await axios.get(currApiLink);
//console.log("up to here");
// here we start extracting some information from a single commit
var currentAuthorName = response['data']['commit']['committer']['name'];
var currentDate = response['data']['commit']['committer']['date'];
var currentFiles = response['data']['files'];
// this loop will iterate over all changed files for a single commit. Remember, every commit has a list
// of changed files, so this loop will iterate over all those files, get the necessary information
// from those files.
var z;
// we create this temporary list of file objects because for every file, we want to make an object
// that will store the necessary information for that one file. after we store all the objects for
// each file, we will add this list of file objects as a field for our bigger commit object (see down below)
var tempListOfFileObjects = [];
for (z = 0; z < currentFiles.length; z++){
var fileInConsideration = currentFiles[z];
var nameOfFile = fileInConsideration['filename'];
var numberOfAdditions = fileInConsideration['additions'];
var numberOfDeletions = fileInConsideration['deletions'];
var totalNumberOfChangesToFile = fileInConsideration['changes'];
//console.log("with file");
var tempFileObject = {fileName: nameOfFile, totalAdditions: numberOfAdditions,
totalDeletions: numberOfDeletions, numberOfChanges: totalNumberOfChangesToFile};
// we add the same file objects to both a temporary, local list and a global set. Don't be tripped
// up by this; they're doing the same thing!
setOfFileObjects.add(tempFileObject);
tempListOfFileObjects.push(tempFileObject);
}
// here we make an object that stores information for a single commit. sha, authorName, date are single
// values, but files will be a list of file objects and these file objects will store further information
// for each file.
var tempObj = {sha: currentSHA, authorName: currentAuthorName, date: currentDate, files: tempListOfFileObjects};
var currPrNumber = listOfPrObjects[x]['number'];
console.log(currPrNumber);
// here we will make a single pull request number to an object that will contain all the information for
// every single commit associated with that pull request. So for every pull request, it will map to a list
// of objects where each object stores information about a commit associated with the pull request.
mapPRNumbersToCommitObjects.set(currPrNumber, tempObj);
}
}
return mapPRNumbersToCommitObjects;
}
startParsingPullRequests() {
this.getListOfPullRequests(gitPullApiLink + "?state=all").then(() => {
this.getCommitsForEachPullRequestAndPRinformation().then(() => {
this.createCommitObjects().then((response) => {
console.log("functions were successful");
return mapPRNumbersToCommitObjects;
}).catch((error) => {
console.log("printing first error");
console.log(error);
})
}).catch((error2) => {
console.log("printing the second error");
console.log(error2);
})
}).catch((error3) => {
console.log("printing the third error");
console.log(error3);
});
}
//adding some getter methods so they can be used to work with whatever information people may need.
//I start all of them with the this.startParsingPullRequests() method because by calling that method it gets all
// the information for the global variables.
async getSetOfFileObjects(){
var dummyMap = this.startParsingPullRequests();
return setOfFileObjects;
}
async OpenCloseDateObjects(){
var dummyMap = this.startParsingPullRequests();
return mapPRnumberToCloseOpenDateObjects;
}
async getNumbersOfTargetedIssues(){
var dummyMap = this.startParsingPullRequests();
return listOfNumbersOfTargetedIssues;
}
}
I'm trying to make a "dummy map" that gets the return value of calling startParsingPullRequests() in the following way:
var dummy = new PullRequestParser();
var dummyMap = dummy.startParsingPullRequests();
console.log(dummyMap);
But I end up with the following output:
undefined
3
1
functions were successful
I understand dummyMap is undefined because startParsingPullRequests() makes a series of async calls, but I'm wondering how do I go about making sure dummyMap gets its value and then prints. Thanks!

Since you already have code using async/await, why are you changing to .then? When you are using .then/.catch, it can be a lot harder to tell where things are waiting and what they are waiting on.
Here is your function converted to use async/await. With this change, you can await this function (though in that case, you might want to not catch the error and let the caller handle it, otherwise you need to check the return for undefined to "detect" the error). I've also simplified the error handling (but again, unless the code here can "fix" the problem, no reason to handle it here)
async startParsingPullRequests() {
try {
await this.getListOfPullRequests(gitPullApiLink + "?state=all");
await this.getCommitsForEachPullRequestAndPRinformation();
const response = await this.createCommitObjects();
console.log("functions were successful");
return mapPRNumbersToCommitObjects;
} catch (error) {
console.log("printing error");
console.log(error);
}
}

Function returned undefined, expected Promise or value and unable to delete old data from firebase database using cloud functions

I'm trying to delete multiple nodes on my database that are older than 12hrs. I"m using a pub/sub function to trigger this event. I don't know if my code is actually looping through all nodes as I'm not using the onWrite, onCreate database triggers on specific. Here is the image sample of the database
this is the pub/sub code
exports.deletejob = functions.pubsub.topic('Oldtask').onPublish(() => {
deleteOldItem();
})
and the deleteOldItem function
function deleteOldItem(){
const CUT_OFF_TIME = 12 * 60 * 1000; // 12 Hours in milliseconds.
//var ref = admin.database().ref(`/articles/${id}`);
const ref = admin.database().ref(`/articles`);
const updates = {};
ref.orderByChild('id').limitToLast(100).on('value', function (response) {
var index = 0;
response.forEach(function (child) {
var element = child.val();
const datetime = element.timestamp;
const now = Date.now();
const cutoff = now - datetime;
if (CUT_OFF_TIME < cutoff){
updates[element.key] = null;
}
});
//This is supposed to be the returened promise
return ref.child(response.key).update(updates);
});
If there's something I'm doing wrong, I'll like to know. The pub/sub is triggered with a JobScheduler already setup on google cloud scheduler

You had several problems in your code that were giving you trouble.
The handling of promises wasn't correct. In particular, your top level function never actually returned a promise, it just called deleteOldItems().
You should use the promise form of once() instead of calling on() with a callback since you don't want to install a listener in this case, you just need the result a single time, and you want to handle it as part of a promise chain.
To delete nodes, you should call remove() on a reference to that node. It also generates a promise for you to use here.
You didn't calculate 12 hours in milliseconds properly, you calculated 12 minutes in milliseconds :)
Here's what I came up with. It uses an http function instead of a pubsub function as well as adding a log statement for my testing, but the modification you need should be trivial/obvious (just change the prototype and remove the response after deleteOldItems, but do make sure you keep returning the result of deleteOldItems()):
const functions = require('firebase-functions');
const admin = require('firebase-admin');
function deleteOldItems() {
const CUT_OFF_TIME = 12 * 60 * 60 * 1000; // 12 Hours in milliseconds.
const ref = admin.database().ref('/articles');
return ref.orderByChild('id').limitToLast(100).once('value')
.then((response) => {
const updatePromises = [];
const now = Date.now();
response.forEach((child) => {
const datetime = child.val().timestamp;
const cutoff = now - datetime;
console.log(`processing ${datetime} my cutoff is ${CUT_OFF_TIME} and ${cutoff}`);
if (CUT_OFF_TIME < cutoff){
updatePromises.push(child.ref.remove())
}
});
return Promise.all(updatePromises);
});
}
exports.doIt = functions.https.onRequest((request, response) => {
return deleteOldItems().then(() => { return response.send('ok') });
}
While I have not tested it, I'm pretty sure this will work to include inside your original function call for cloud scheduler:
exports.deletejob = functions.pubsub.topic('Oldtask').onPublish(() => {
return deleteOldItems();
})
Of course, this is still more complicated than you need, since ordering by id doesn't really gain you anything here. Instead, why not just use the query to return the earliest items before the cut off time (e.g. exactly the ones you want to remove)? I've also switched to limitToFirst to ensure the earliest entries get thrown out, which seems more natural and ensures fairness:
function deleteOldItems() {
const cutOffTime = Date.now() - (12 * 60 * 60 * 1000); // 12 Hours earlier in milliseconds.
const ref = admin.database().ref('/articles');
return ref.orderByChild('timestamp').endAt(cutOffTime).limitToFirst(100).once('value')
.then((response) => {
const updatePromises = [];
response.forEach((child) => {
updatePromises.push(child.ref.remove())
});
return Promise.all(updatePromises);
});
}
If you do this on more than a few items, of course, you probably want to add an index on the timestamp field so the range query is more efficient.

Get the ChildrenCount of a child in Firebase using JavaScript

I have been doing this for an hour. I simply want to get the number of children in the child "Success" in the database below. The answers in similar stackoverflow questions are not working. I am new in Javascript Programming.
So far I have tried this
var children = firebase.database().ref('Success/').onWrite(event => {
return event.data.ref.parent.once("value", (snapshot) => {
const count = snapshot.numChildren();
console.log(count);
})
})
and also this
var children = firebase.database().ref('Success/').onWrite(event => {
return event.data.ref.parent.once("value", (snapshot) => {
const count = snapshot.numChildren();
console.log(count);
})
})
Where might I be going wrong.

As explained in the doc, you have to use the numChildren() method, as follows:
var ref = firebase.database().ref("Success");
ref.once("value")
.then(function(snapshot) {
console.log(snapshot.numChildren());
});
If you want to use this method in a Cloud Function, you can do as follows:
exports.children = functions.database
.ref('/Success')
.onWrite((change, context) => {
console.log(change.after.numChildren());
return null;
});
Note that:
The new syntax for Cloud Functions version > 1.0 is used, see https://firebase.google.com/docs/functions/beta-v1-diff?authuser=0
You should not forget to return a promise or a value to indicate to the platform that the Cloud Function execution is completed (for more details on this point, you may watch the 3 videos about "JavaScript Promises" from the Firebase video series: https://firebase.google.com/docs/functions/video-series/).

const db = getDatabase(app)
const questionsRef = ref(db, 'questions')
const mathematicalLiteracy = child(questionsRef, 'mathematicalLiteracy')
onValue(mathematicalLiteracy, (snapshot) => {
const data = snapshot.val()
const lenML = data.length - 1
console.log(lenML)
})
This method worked for me. I wanted to get the children's count of the mathematicalLiteracy node in my database tree. If I get its value using .val() it returns an array that contains that node's children and an extra empty item. So, I subtracted that one empty item's count. Finally, I get my needed children's count.

Firebase Cloud function update all entries in database

i have bunch of comments in Firebase database and i want to do some updates to the comments via Cloud Function ( this is simplified example, i will be doing some logic which does require Cloud Function ).
What i need to do is go through all the comments in the database, adjust its rating node and then update the database with adjusted comments.
I spent a lot of time researching this, but i am completely new to Cloud Functions, so i have realy hard time figuring this out.
I am assuming i want to store all the changes to all the comments (there can be thousands of them) in the array or object and then do the update at one time instead of for each comment separately ?
Btw this code is not working, i am assuming the array and return is completely wrong.
exports.increaseRating = functions.database.ref('/comments/')
.onUpdate((snapshot) => {
var updates = [];
snapshot.before.forEach((element) => {
var comment = element.val();
comment.rating += 1000;
updates.push(comment);
});
return updates;
})
Code i am using to update one entry. I need to do the same thing for all the comments at one time.
exports.increaseRating = functions.database.ref('/comments/{commentId}')
.onUpdate((snapshot, context) => {
const comment = snapshot.before.val();
const newRating = comment.rating += 1000;
const now = new Date().getTime();
if (comment.lastUpdate) {
if (comment.lastUpdate > now - (30 * 1000)) {
return null;
}
}
return admin.database().ref(`/comments/${context.params.commentId}`).update({
"rating": newRating,
"lastUpdate": now
})
})

If you want to update all child nodes, you can do something like this:
var ref = firebase.database().ref("comments"); // or admin.database().ref("comments")
ref.once("value").then((snapshot) => {
var updates = {};
snapshot.forEach((commentSnapshot => {
var comment = commentSnapshot.val();
var newRating = comment.rating + 1000;
updates[commentSnapshot.key+"/rating"] = newRating;
});
ref.update(updates);
})
This performs a single multi-location update for all comments. Note that the performance benefit over performing separate updates is quite small, since Firebase pipelines the multiple requests over a single connection.
Also note that you should not put this in a Cloud Functions trigger on /comments, since that will lead to an endless loop: every time the comments get written, your function triggers, which updates the comments, which triggers the function again.
If you need this in Cloud Functions, you'll want to use a HTTP-triggered function, which is triggered by HTTP calls instead of database writes.
exports.updateCommentRatings = functions.https.onRequest((req, res) => {
var ref = admin.database().ref("comments")
ref.once("value").then((snapshot) => {
var updates = {};
snapshot.forEach((commentSnapshot => {
var comment = commentSnapshot.val();
var newRating = comment.rating + 1000;
updates[commentSnapshot.key+"/rating"] = newRating;
});
ref.update(updates).then(() => {
res.status(200).send("Comment ratings updated");
});
})
})
You can then periodically call this URL/function with a service like cron-job.org. For more on this see Cloud Functions for Firebase trigger on time?.