Develop Reference

Discord,js: How to get every user who reacted to an old message

I am trying to get the every user, that reacted with a specific emoji to a specific message. So far, I've tried getting the message, then the emoji and then fetiching all the users who reacted with that emoji:
let messages = await client.channels.cache
.get('920031186871545916')
.messages.fetch('1074327140138483835');
const reaction = messages.reactions.cache.get('✅')
const users = reaction.users.fetch();
The result however is a collection with only 100 users, while over 500 users have reacted to the message. I know that the fetch method is limited to only 100 elements, which is why I tried using the "before" property in combination with a loop, as it was suggested on stackoverflow before, when fetching messages.
const users = [];
const usersToFetch = 520;
while (users.length < usersToFetch) {
if (!users.length) {
const user = await reaction.users.fetch();
users.push(...user);
continue;
}
const user = await reaction.users.fetch({ limit: 100, before: users[0].id});
users.push(...user):
}
That however didn't work as I desired. The excact same collection is alwas added to the "users" array in each iteration. I tried a different approach that I've seen on stackoverflow, but the result is excactly the same:
let users = await lots_of_users_getter(reaction, 520);
async function lots_of_users_getter(reaction, limit) {
const sum_users = [];
let last_id;
while (true) {
const options = { limit: 100 };
if (last_id) {
options.before = last_id;
}
const user = await reaction.users.fetch(options);
sum_users.push(...user);
last_id = user.last().id;
if (user.size != 100 || sum_users.length >= limit) {
break;
}
}
return sum_users;
}
I am starting to believe that the "before" property is meant for fetching messages only. Is there any way I can get the ID of each user that reacted to the message?

Nested Javascript promises - fetching data from firestore

I have been stuck with this error for the past 3 days and i have tried literally everything, tried to structure the promises in a 1000 ways but nothing seems to work. Maybe I am losing the "big picture" so hopefully new eyes will help. Thanks for reading:
I have a scheduled function running in Firebase Cloud Functions. What the code tries to accomplish is
checking if a document expired and changing it to "inactive" >> this part works
if a document was set to inactive, i want to see if i have any other docs in the firestore database of the same "type". if there is no other document of the same type, then i want to remove that type from my document "types".
In my latest attempt (copied below) I check if there is a document in the snapshot (which would mean that there is another document of the same type, therefore the doc doesnt have to be deleted). Then if res!== true, I would delete the document.
The problem is that for some reason, res is never true.... maybe the "res" promise resolves before the "snapshot" promise?
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
exports.scheduledFunction = functions.pubsub
.schedule('0 23 * * *').timeZone('Europe/Madrid')
.onRun( async (context) => {
const expiredDocs = await admin.firestore().collection('PROMOTIONS_INFO')
.where('active','==',true)
.where('expiration', '<=', new Date())
.get()
.then(async (snapshot) => {
await Promise.all(snapshot.docs.map( async (doc) => {
doc.ref.update({active: false})
const type = doc.data().business.type
const id = doc.data().id
const exists = await admin.firestore().collection('PROMOTIONS_INFO')
.where('active','==',true)
.where('business.type','==', type)
.where('id', '!=', id)
.limit(1)
.get()
.then((snapshot) => {
snapshot.docs.map((doc)=>{return true})
}).then(async (res) => {
res===true ? null :
(await admin.firestore().collection('PROMOTIONS_INFO').doc('types')
.update('types', admin.firestore.FieldValue.arrayRemove(type)))
})
}))
});
});

To achieve your desired result, you might want to consider making use of Batched Writes and splitting your code into distinct steps.
One possible set of steps is:
Get all expired documents that are still active
No expired documents? Log result & end function.
For each expired document:
Update it to inactive
Store it's type to check later
For each type to check, check if an active document with that type exists and if it doesn't, store that type to remove later.
No types to remove? Log result & end function.
Remove all the types that need to be removed.
Log result & end function.
In the above steps, step 3 can make use of Batched Writes and step 6 can make use of the arrayRemove() field transform which can remove multiple elements at once to ease the burden on your database.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
exports.scheduledFunction = functions.pubsub
.schedule('0 23 * * *').timeZone('Europe/Madrid')
.onRun( async (context) => {
// get instance of Firestore to use below
const db = admin.firestore();
// this is reused often, so initialize it once.
const promotionsInfoColRef = db.collection('PROMOTIONS_INFO');
// find all documents that are active and have expired.
const expiredDocsQuerySnapshot = await promotionsInfoColRef
.where('active','==',true)
.where('expiration', '<=', new Date())
.get();
if (expiredDocsQuerySnapshot.empty) {
// no expired documents, log the result
console.log(`No documents have expired recently.`);
return; // done
}
// initialize an object to store all the types to be checked
// this helps ensure each type is checked only once
const typesToCheckObj = {};
// initialize a batched write to make changes all at once, rather than call out to Firestore multiple times
// note: batches are limited to 500 read/write operations in a single batch
const makeDocsInactiveBatch = db.batch();
// for each snapshot, add their type to typesToCheckObj and update them to inactive
expiredDocsQuerySnapshot.forEach(doc => {
const type = doc.get("business.type"); // rather than use data(), parse only the property you need.
typesToCheckObj[type] = true; // add this type to the ones to check
makeDocsInactiveBatch.update(doc.ref, { active: false }); // add the "update to inactive" operation to the batch
});
// update database for all the now inactive documents all at once.
// we update these documents first, so that the type check are done against actual "active" documents.
await makeDocsInactiveBatch.commit();
// this is a unique array of the types encountered above
// this can now be used to check each type ONCE, instead of multiple times
const typesToCheckArray = Object.keys(typesToCheckObj);
// check each type and return types that have no active promotions
const typesToRemoveArray = (await Promise.all(
typesToCheckArray.map((type) => {
return promotionsInfoColRef
.where('active','==',true)
.where('business.type','==', type)
.limit(1)
.get()
.then((querySnapshot) => querySnapshot.empty ? type : null) // if empty, include the type for removal
})
))
.filter((type) => type !== null); // filter out the null values that represent types that don't need removal
// typesToRemoveArray is now a unique list of strings, containing each type that needs to be removed
if (typesToRemoveArray.length == 0) {
// no types need removing, log the result
console.log(`Updated ${expiredDocsQuerySnapshot.size} expired documents to "inactive" and none of the ${typesToCheckArray.length} unique types encountered needed to be removed.`);
return; // done
}
// get the types document reference
const typesDocRef = promotionsInfoColRef.doc('types');
// use the arrayRemove field transform to remove all the given types at once
await typesDocRef.update({types: admin.firestore.FieldValue.arrayRemove(...typesToRemoveArray) });
// log the result
console.log(`Updated ${expiredDocsQuerySnapshot.size} expired documents to "inactive" and ${typesToRemoveArray.length}/${typesToCheckArray.length} unique types encountered needed to be removed.\n\nThe types removed: ${typesToRemoveArray.sort().join(", ")}`);
Note: Error checking is omitted and should be implemented.
Batch Limits
If you ever expect to hit the 500 operations per batch limit, you can add a wrapper around batches so they automatically get split up as required. One possible wrapper is included here:
class MultiBatch {
constructor(dbRef) {
this.dbRef = dbRef;
this.batchOperations = [];
this.batches = [this.dbRef.batch()];
this.currentBatch = this.batches[0];
this.currentBatchOpCount = 0;
this.committed = false;
}
/** Used when for basic update operations */
update(ref, changesObj) {
if (this.committed) throw new Error('MultiBatch already committed.');
if (this.currentBatchOpCount + 1 > 500) {
// operation limit exceeded, start a new batch
this.currentBatch = this.dbRef.batch();
this.currentBatchOpCount = 0;
this.batches.push(this.currentBatch);
}
this.currentBatch.update(ref, changesObj);
this.currentBatchOpCount++;
}
/** Used when an update contains serverTimestamp, arrayUnion, arrayRemove, increment or decrement (which all need to be counted as 2 operations) */
transformUpdate(ref, changesObj) {
if (this.committed) throw new Error('MultiBatch already committed.');
if (this.currentBatchOpCount + 2 > 500) {
// operation limit exceeded, start a new batch
this.currentBatch = this.dbRef.batch();
this.currentBatchOpCount = 0;
this.batches.push(this.currentBatch);
}
this.currentBatch.update(ref, changesObj);
this.currentBatchOpCount += 2;
}
commit() {
this.committed = true;
return Promise.all(this.batches.map(batch => batch.commit()));
}
}
To use this, swap out db.batch() in the original code for new MultiBatch(db). If an update in the batch (like someBatch.update(ref, { ... })) contains a field transform (such as FieldValue.arrayRemove()), make sure to use someMultiBatch.transformUpdate(ref, { ... }) instead so that single update is correctly counted as 2 operations (a read and a write).

I'm assuming in this case that res is undefined and is evaluating as falsey.
Before your .then promise with the res parameter, you have a previous .then promise that returns void:
//...
.then((snapshot) => {
snapshot.docs.map((doc)=>{return true}) // <--- This is not actually returning a resolved value
}).then(async (res) => {
res===true ? null :
(await admin.firestore().collection('PROMOTIONS_INFO').doc('types')
.update('types', admin.firestore.FieldValue.arrayRemove(type)))
})
//...
Depending on your intent, you'll need to return a value in this previous promise. It looks like you're creating an array of boolean values that is the length of the number of snapshot.docs that you have, so if you put a return statement simply in the previous .then clause, res would be something like, [true, true, true, true, /* ... */]
//...
.then((snapshot) => {
return snapshot.docs.map((doc)=>{return true})
}).then(async (res) => {
res===true ? null :
(await admin.firestore().collection('PROMOTIONS_INFO').doc('types')
.update('types', admin.firestore.FieldValue.arrayRemove(type)))
})
//...

snapshot.docs.map((doc)=>{return true}) returns Array like [true, false] not boolean like true.
So .then( async (res) => { res===true ? null : await admin.firestore(... cannot work.
and
Maybe you should modify like the following.
.then((snapshot) =>
snapshot.docs.length > 0 ? null :
await admin.firestore(...

Firestore: Unable to fetch document from query - doc.data is not a function

here usersList contains the list of username of people and androidNotificationToken is the field in the document of the user which contains the token for sending the notification.
const registrationTokens=[];
const indexOfSender=usersList.indexOf(senderUsername); // to remove the name of person sending message
let removedUsername=usersList.splice(indexOfSender,1);
usersList.forEach(async(element)=>{
const userRef = admin.firestore().collection('users').where("username","==",element);
const doc = await userRef.get();
registrationTokens.push(doc.data().androidNotificationToken);
});
The error on running the cloud computing am receiving is :-
TypeError: doc.data is not a function
at usersList.forEach (/workspace/index.js:191:37)
at process._tickCallback (internal/process/next_tick.js:68:7)

userRef.get() is going to return a QuerySnapshot (not a DocumentSnapshot) object that can contain 0 or more documents matched from the query. Use the API provided by QuerySnapshot to find the resulting documents, even if you are expecting only one document.
If you are expecting just one document from the query, you should at least verify that you got one before indexing into the result set.
const query = admin.firestore().collection('users').where("username","==",element);
const qsnapshot = await query.get();
if (qsnapshot.docs.length > 0) {
const doc = qsnapshot.docs[0];
const data = doc.data();
}
else {
// decide what you want to do if the query returns nothing
}

Add user ID to Array to be banned upon return to guild?

As stated in the title I'm wanting to extend the current code I have for a ban function (included below).
If a user has left the guild/server how would I add their userID into an Array? This way I can loop through it every time someone joins the guild and check if they were previously banned: if so, they will be properly banned.
So the process would be:
<prefix>ban <userid>
If the user is not in the guild then add the ID to an Array and display a ban message
When a new member joins, check them against the Array and if ID exists then ban them and remove them from Array.
switch (args[0]) {
case 'ban':
if (!message.content.startsWith(PREFIX)) return;
if (!message.member.roles.cache.find(r => r.name === 'Moderator') && !message.member.roles.cache.find(r => r.name === 'Staff')) return message.channel.send('You dont not have the required permissions').then(msg => {
msg.delete({ timeout: 5000 })
})
var user1 = message.mentions.users.first();
if (member) {
// There's a valid user, you can do your stuff as usual
member.ban().then(() => {
message.channel.send('User has been banned')
})
} else {
let user = message.mentions.users.first(),
// The ID will be either picked from the mention (if there's one) or from the argument
// (I used args, I don't know how your variable's called)
userID = user ? user.id : args[1]
if (userID) {
// Read the existing banned ids. You can choose your own path for the JSON file
let bannedIDs = require('./bannedIDs.json').ids || []
// If this is a new ID, add it to the array
if (!bannedIDs.includes(userID)) bannedIDs.push(userID)
// Save the file
fs.writeFileSync('./bannedIDs.json', JSON.stringify({ ids: bannedIDs }))
// You can then send any message you want
message.channel.send('User added to the list')
} else {
message.channel.send('No ID was entered')
}
}
}
}
);
bot.on('guildMemberAdd', member => {
let banned = require('./bannedIDs.json').ids || []
if (banned.includes(member.id)) {
// The user should be properly banned
member.ban({
reason: 'Previously banned by a moderator.'
})
// You can also remove them from the array
let newFile = {
ids: banned.filter(id => id != member.id)
}
fs.writeFileSync('./bannedIDs.json', JSON.stringify(newFile))
}
})

This can be done, but you'll need to grab the user ID manually since you can't mention someone that's not in the guild (you technically can, but you would need to grab the ID anyway).
If there's no user mention or if there's one, but the user isn't in the guild, you'll need to grab their ID and push it to an array. You may want to save that array in a JSON file so it's kept between reloads: you can use fs.writeFileSync() for that.
Here's an example:
let member = message.mentions.members.first()
if (member) {
// There's a valid user, you can do your stuff as usual
member.ban().then(() => {
message.channel.send('User has been banned')
})
} else {
let user = message.mentions.users.first(),
// The ID will be either picked from the mention (if there's one) or from the argument
// (I used args, I don't know how your variable's called)
userID = user ? user.id : args[0]
if (userID) {
// Read the existing banned ids. You can choose your own path for the JSON file
let bannedIDs = require('./bannedIDs.json').ids || [] // WRONG: read the edit below
// If this is a new ID, add it to the array
if (!bannedIDs.includes(userID)) bannedIDs.push(userID)
// Save the file
fs.writeFileSync('./bannedIDs.json', JSON.stringify({ ids: bannedIDs }))
// You can then send any message you want
message.channel.send('User added to the list')
} else {
message.channel.send('No ID was entered')
}
}
Remember that you have to import the fs module, which is included with Node:
const fs = require('fs')
Also, create a JSON file that looks like this:
{
ids: []
}
When a new user joins, you can check their ID:
client.on('guildMemberAdd', member => {
let banned = require('./bannedIDs.json').ids || [] // WRONG: read the edit below
if (banned.includes(member.id)) {
// The user should be properly banned
member.ban({
reason: 'Previously banned by a moderator.'
})
// You can also remove them from the array
let newFile = {
ids: banned.filter(id => id != member.id)
}
fs.writeFileSync('./bannedIDs.json', JSON.stringify(newFile))
}
})
Edit:
How to solve the problem with the array being cached:
You load the array only once, when starting the bot, and store it in a variable. Every time you need to read or write to the array you use that variable, and periodically you update the JSON file.
As you mentioned in your comment, you can replace the require() with fs.readFileSync(). You can read the file with this fs method, and then you can parse the JSON object with the JSON.parse() function.
Here's how you can do it:
// Wrong way to do it:
let bannedIDs = require('./bannedIDs.json').ids || []
// Better way to do it:
let file = fs.readFileSync('./bannedIDs.json')
let bannedIDs = JSON.parse(file).ids || []

Delete same value from multiple locations Firebase Functions

I have a firebase function that deletes old messages after 24 hours as in my old question here. I now have just the messageIds stored in an array under the user such that the path is: /User/objectId/myMessages and then an array of all the messageIds under myMessages. All of the messages get deleted after 24 hours, but the iDs under the user's profile stay there. Is there a way to continue the function so that it also deletes the messageIds from the array under the user's account?
I'm new to Firebase functions and javascript so I'm not sure how to do this. All help is appreciated!

Building upon #frank-van-puffelen's accepted answer on the old question, this will now delete the message IDs from their sender's user data as part of the same atomic delete operation without firing off a Cloud Function for every message deleted.
Method 1: Restructure for concurrency
Before being able to use this method, you must restructure how you store entries in /User/someUserId/myMessages to follow best practices for concurrent arrays to the following:
{
"/User/someUserId/myMessages": {
"-Lfq460_5tm6x7dchhOn": true,
"-Lfq483gGzmpB_Jt6Wg5": true,
...
}
}
This allows you to modify the previous function to:
// Cut off time. Child nodes older than this will be deleted.
const CUT_OFF_TIME = 24 * 60 * 60 * 1000; // 2 Hours in milliseconds.
exports.deleteOldMessages = functions.database.ref('/Message/{chatRoomId}').onWrite(async (change) => {
const rootRef = admin.database().ref(); // needed top level reference for multi-path update
const now = Date.now();
const cutoff = (now - CUT_OFF_TIME) / 1000; // convert to seconds
const oldItemsQuery = ref.orderByChild('seconds').endAt(cutoff);
const snapshot = await oldItemsQuery.once('value');
// create a map with all children that need to be removed
const updates = {};
snapshot.forEach(messageSnapshot => {
let senderId = messageSnapshot.child('senderId').val();
updates['Message/' + messageSnapshot.key] = null; // to delete message
updates['User/' + senderId + '/myMessages/' + messageSnapshot.key] = null; // to delete entry in user data
});
// execute all updates in one go and return the result to end the function
return rootRef.update(updates);
});
Method 2: Use an array
Warning: This method falls prey to concurrency issues. If a user was to post a new message during the delete operation, it's ID could be removed while evaluating the deletion. Use method 1 where possible to avoid this.
This method assumes your /User/someUserId/myMessages object looks like this (a plain array):
{
"/User/someUserId/myMessages": {
"0": "-Lfq460_5tm6x7dchhOn",
"1": "-Lfq483gGzmpB_Jt6Wg5",
...
}
}
The leanest, most cost-effective, anti-collision function I can come up for this data structure is the following:
// Cut off time. Child nodes older than this will be deleted.
const CUT_OFF_TIME = 24 * 60 * 60 * 1000; // 2 Hours in milliseconds.
exports.deleteOldMessages = functions.database.ref('/Message/{chatRoomId}').onWrite(async (change) => {
const rootRef = admin.database().ref(); // needed top level reference for multi-path update
const now = Date.now();
const cutoff = (now - CUT_OFF_TIME) / 1000; // convert to seconds
const oldItemsQuery = ref.orderByChild('seconds').endAt(cutoff);
const snapshot = await oldItemsQuery.once('value');
// create a map with all children that need to be removed
const updates = {};
const messagesByUser = {};
snapshot.forEach(messageSnapshot => {
updates['Message/' + messageSnapshot.key] = null; // to delete message
// cache message IDs by user for next step
let senderId = messageSnapshot.child('senderId').val();
if (!messagesByUser[senderId]) { messagesByUser[senderId] = []; }
messagesByUser[senderId].push(messageSnapshot.key);
});
// Get each user's list of message IDs and remove those that were deleted.
let pendingOperations = [];
for (let [senderId, messageIdsToRemove] of Object.entries(messagesByUser)) {
pendingOperations.push(admin.database.ref('User/' + senderId + '/myMessages').once('value')
.then((messageArraySnapshot) => {
let messageIds = messageArraySnapshot.val();
messageIds.filter((id) => !messageIdsToRemove.includes(id));
updates['User/' + senderId + '/myMessages'] = messageIds; // to update array with non-deleted values
}));
}
// wait for each user's new /myMessages value to be added to the pending updates
await Promise.all(pendingOperations);
// execute all updates in one go and return the result to end the function
return ref.update(updates);
});

Update: DO NOT USE THIS ANSWER (I will leave it as it may still be handy for detecting a delete operation for some other need, but do not use for the purpose of cleaning up an array in another document)
Thanks to #samthecodingman for providing an atomic and concurrency safe answer.
If using Firebase Realtime Database you can add an onChange event listener:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
exports.onDeletedMessage = functions.database.ref('Message/{messageId}').onChange(async event => {
// Exit if this item exists... if so it was not deleted!
if (event.data.exists()) {
return;
}
const userId = event.data.userId; //hopefully you have this in the message document
const messageId = event.data.messageId;
//once('value') useful for data that only needs to be loaded once and isn't expected to change frequently or require active listening
const myMessages = await functions.database.ref('/users/' + userId).once('value').snapshot.val().myMessages;
if(!myMessages || !myMessages.length) {
//nothing to do, myMessages array is undefined or empty
return;
}
var index = myMessages.indexOf(messageId);
if (index === -1) {
//nothing to delete, messageId is not in myMessages
return;
}
//removeAt returns the element removed which we do not need
myMessages.removeAt(index);
const vals = {
'myMessages': myMessages;
}
await admin.database.ref('/users/' + userId).update(vals);
});
If using Cloud Firestore can add an event listener on the document being deleted to handle cleanup in your user document:
exports.onDeletedMessage = functions.firestore.document('Message/{messageId}').onDelete(async event => {
const data = event.data();
if (!data) {
return;
}
const userId = data.userId; //hopefully you have this in the message document
const messageId = data.messageId;
//now you can do clean up for the /user/{userId} document like removing the messageId from myMessages property
const userSnapShot = await admin.firestore().collection('users').doc(userId).get().data();
if(!userSnapShot.myMessages || !userSnapShot.myMessages.length) {
//nothing to do, myMessages array is undefined or empty
return;
}
var index = userSnapShot.myMessages.indexOf(messageId);
if (index === -1) {
//nothing to delete, messageId is not in myMessages
return;
}
//removeAt returns the element removed which we do not need
userSnapShot.myMessages.removeAt(index);
const vals = {
'myMessages': userSnapShot.myMessages;
}
//To update some fields of a document without overwriting the entire document, use the update() method
await admin.firestore().collection('users').doc(userId).update(vals);
});