Develop Reference

Firestore transactions showing unexpected behaviour when used in cloud functions

I am writing an app that features an inventory in which users can reserve products. I want to ensure that 2 users cannot simultaneously reserve a product at the same time, for this, I intend on using transactions. When using transactions from the Firebase SDK, everything works as intended, but I am getting unexpected behavior when using transactions from a callable cloud function. To simulate the use case where 2 users happen to reserve the same product, I use setTimeout in my cloud function to halt the function for 3 seconds. I am launching this function from 2 different clients with different user contexts.
export const reserveProduct = functions.https.onCall(async (data,context) => {
function testTimeout(){
return new Promise((resolve,reject) => {
setTimeout(()=> {
return resolve(true)
},3000)
})
}
if(!context.auth){
return {
error: `You must be logged in to reserve products`
}
}else{
const productRef = admin.firestore().collection('products').doc(data.productID)
const userRef = admin.firestore().collection('users').doc(context.auth.uid)
return admin.firestore().runTransaction((transaction) => {
return transaction.get(productRef).then(async(doc) => {
if(doc.get('status') == 'reserved'){
throw "Document already reserved!"
}else{
console.log("Product not reserved, reserving now!")
}
await testTimeout()
transaction.update(productRef, {status: 'reserved'});
transaction.update(userRef, {reserved: admin.firestore.FieldValue.arrayUnion(data.productID)})
})
}).then(() => {
console.log("Transaction Successfully committed !")
}).catch((error) => {
throw "Transaction failed, product already reserved"
})
}
After running this function call from 2 different clients simultaneously, The function call from my first client returns successfully as expected, but only after roughly 35s (which is way too long for the simplicity of the transaction). However, the second function call times out without returning any value. I have not seen any documentation explicitly stating the use of transactions in callable cloud functions, nor should it be affected when used within the emulator.
I am expecting to simply get a return value for whichever function call is able to modify the data first, and catch the error from the function which has retried and validated the reserved state.
Any help would be appreciated, thanks!

One major difference between the two places is in the way the SDKs used handle transactions:
The client-side SDKs use an optimistic compare-and-set approach for transactions, meaning that they pass the values you read in the transaction with the data you're writing. The server then only writes the new data if the documents you read haven't been updated.
The server-side SDKs (used in your Cloud Function) use a more traditional pessimistic approach for transactions, and place a lock on each document that you read in the transaction.
You can read more about database contention in the SDKs in the documentation.
While I'm not exactly certain how this is affecting your code, I suspect it is relevant to the difference in behavior you're seeing between the client-side and server-side implementations.

Would giving response to client while letting asynchronous operation continue to run a good idea?

So I need to implement an "expensive" API endpoint. Basically, the user/client would need to be able to create a "group" of existing users.
So this "create group" API would need to check that each users fulfill the criteria, i.e. all users in the same group would need to be from the same region, same gender, within an age group etc. This operation can be quite expensive, especially since there are no limit on how many users in one group, so its possible that the client requests group of 1000 users for example.
My idea is that the endpoint will just create entry in database and mark the "group" as pending, while the checking process is still happening, then after its completed, it will update the group status to "completed" or "error" with error message, then the client would need to periodically fetch the status if its still pending.
My implementation idea is something along this line
const createGroup = async (req, res) => {
const { ownerUserId, userIds } = req.body;
// This will create database entry of group with "pending" status and return the primary key
const groupId = await insertGroup(ownerUserId, 'pending');
// This is an expensive function which will do checking over the network, and would take 0.5s per user id for example
// I would like this to keep running after this API endpoint send the response to client
checkUser(userIds)
.then((isUserIdsValid) => {
if (isUserIdsValid) {
updateGroup(groupId, 'success');
} else {
updateGroup(groupId, 'error');
}
})
.catch((err) => {
console.error(err);
updateGroup(groupId, 'error');
});
// The client will receive a groupId to check periodically whether its ready via separate API
res.status(200).json({ groupId });
};
My question is, is it a good idea to do this? Do I missing something important that I should consider?

Yes, this is the standard approach to long-running operations. Instead of offering a createGroup API that creates and returns a group, think of it as having an addGroupCreationJob API that creates and returns a job.
Instead of polling (periodically fetching the status to check whether it's still pending), you can use a notification API (events via websocket, SSE, webhooks etc) and even subscribe to the progress of processing. But sure, a check-status API (via GET request on the job identifier) is the lowest common denominator that all kinds of clients will be able to use.
Did I not consider something important?
Failure handling is getting much more complicated. Since you no longer create the group in a single transaction, you might find your application left in some intermediate state, e.g. when the service crashed (due to unrelated things) during the checkUser() call. You'll need something to ensure that there are no pending groups in your database for which no actual creation process is running. You'll need to give users the ability to retry a job - will insertGroup work if there already is a group with the same identifier in the error state? If you separate the group and the jobs into independent entities, do you need to ensure that no two pending jobs are trying to create the same group? Last but not least you might want to allow users to cancel a currently running job.

How can I set data from firestore to a global variable

I am trying to fetch data from firestore and set it to a global variable but I'm getting undefined. Here is what I have;
let myScore;
auth.onAuthStateChanged((user) => {
let docRef = db.collection("players");
let user_email = user.email;
docRef.onSnapshot((players) => {
players.forEach((doc) => {
data = doc.data();
if (user_email == data.email) {
myScore = data.endless_score;
console.log(myScore); //This log the right score
}
});
});
});
console.log(myScore); // This logs undefined
How can I get myScore i.e. the second console log to output the score from firestore?

Okay, from what I see you are trying to get the current users when the user logs in but I think attaching a snapshot listener is not the optimal solution. Because first of it would count as a read for every player document there is. What I suggest you to do is this :
const myUser = (await (await firebase.firestore().collection("players").where("email","==",user.email).limit(1).get()).docs[0].data()
This Will Give You Your Current User's data then you can use it however you want by doing myUser.endless_score. It will also reduce the cost because you are only getting a single document where email is equal to current users email
also don't forget to make your arrow function async in order to use await :)

Calling onSnapshot creates a background-call to firestore. It run independently of the rest of your code. So your second console log is practically called immediatly after let myScore, meaning it's still undefined.
As well, onAuthChanged is an observer. It can be triggered at random, depending on when you sign in/out of firestore.
To the functionality you want, sort of, you'd need to rewrite the code to use async/await.
Defining a function() as 'async' and then calling 'await function()' in your code will make the code literally wait for the online call to finish before continuing.
read:
https://javascript.info/async-await

(Slack API Bolt project) Using Context object to pass property from one method to another

I am building a Slack app using the JavaScript Bolt framework. The concept of the app is just listening to specific message keywords in channels and then forwarding those messages to the users of the app.
What I am trying to achieve is including a permalink in the forwarded message. I am trying to use the chat.getPermalink method to get the url and then include that in my chat.postMessage method. I am trying to leverage Bolt's 'Context' in order to pass the property in chat.getPermalink to chat.postMessage. I am asking for help here because I cannot get the Context to work..
const app = new App({
token: process.env.SLACK_BOT_TOKEN,
signingSecret: process.env.SLACK_SIGNING_SECRET
});
let token = process.env.SLACK_BOT_TOKEN,
web = new WebClient(token);
let jira_text = "jira";
let rdu_qa = '#rdu_qa';
//Get permalink
async function PermaLinks({payload, context, next}) {
let perm = app.client.chat.getPermalink({
token: context.botToken,
channel: "C0109KMQCFQ",
message_ts: payload.ts
});
context.permalink = perm.permalink;
await next();
}
app.event('message', PermaLinks, async ({ payload, message, context}) => {
let userzArray = ["D010Q34TQL9", "UVBBD8989"];
//if channel is general and incldues the text 'Jira' or 'rdu_qa'
if (payload.channel === "C0109KMQCFQ") {
if (payload.text.includes(jira_text) || payload.text.includes(rdu_qa)) {
try {
// Call the chat.postMessage to each of the users
let oneUser = await userzArray.forEach(userId => { app.client.chat.postMessage({
token: context.botToken,
bot_id: "USLACKBOT",
channel: userId,
blocks: [
{
type: "section",
text: {
text: payload.text,
type: "mrkdwn"
},
fields: [
{
type: "mrkdwn",
text: `posted by <#${message.user}>`
},
{
type:"mrkdwn",
text: "in General channel" //channel.name//getChannelNameGeneral
},
{
type:"mrkdwn",
text: context.permalink // Permalink should be right here
}
]
},
{
"type": "divider"
},
] // End of block of Jira notification stuff
});
});
// console.log(result);
} catch (error) {
console.error(error);
}
} // If text sent to General channel includes keyword 'Jira' or 'rdu_qa'
} //end of if message was posted in General channel

There are a couple problems I can see in the example code, but I think the main issue regarding the context is that you're storing a Promise as context.permalink, not the actual result of the method call. In order to store the result, you should use the await keyword before calling the method (app.client.chat.getPermalink(...)).
I've revised the code you shared here, and I'll explain the modifications below.
const { App } = require('#slack/bolt');
const token = process.env.SLACK_BOT_TOKEN
const app = new App({
signingSecret: process.env.SLACK_SIGNING_SECRET,
token,
});
// Users who should be notified when certain messages are heard
let userzArray = ["D010Q34TQL9", "UVBBD8989"];
// Conversation IDs corresponding to the users in the array above. This variable will be set automatically when the app starts.
let conversationsToNotify;
// Match messages that include the text 'jira' or '#rdu_qa'
app.message(/jira|#rdu_qa/, async ({ message, client }) => {
// Match the messages that are in the specified channel
if (message.channel === 'C0109KMQCFQ') {
try {
// Get a permalink to this message
const permalinkResult = await client.chat.getPermalink({
channel: message.channel,
message_ts: message.ts,
});
// Send a message to each user containing the permalink for this message
await Promise.all(conversationsToNotify.map((conversationId) => {
return client.chat.postMessage({
channel: conversationId,
blocks: [
{
type: 'section',
text: {
type: 'mrkdwn',
text: `>>> ${payload.text}`,
},
fields: [
{
type: 'mrkdwn',
text: `posted by <#${message.user}>`,
},
{
type: 'mrkdwn',
text: `in <#${message.channel}>`,
},
{
type:'mrkdwn',
text: `<Original|${permalinkResult.permalink}>`,
},
],
},
{
type: 'divider'
},
],
});
}));
} catch (error) {
console.error(error);
}
}
});
async function convertUsersToConversations(input) {
return Promise.all(input.map((id) => {
// For all IDs that seem like user IDs, convert them to a DM conversation ID
if (id.startsWith('U')) {
return app.client.conversations.open({
token,
users: id,
})
.then((result) => result.channel.id);
}
// For all IDs that don't seem to belong to a user, return them as is
return id;
}));
});
(async () => {
// Start the app
conversationsToNotify = await convertUsersToConversations(userzArray);
await app.start(process.env.PORT || 3000);
console.log('⚡️ Bolt app is running!');
})();
I've removed the initialization of a new WebClient object. In Bolt v1.6.0 and later, there is a client argument available in listeners and middleware which you can use to call Web API methods instead. The advantage of using the client argument is that you don't need to read the token from the context and pass it as an argument for each method call on your own, Bolt will find the right token for you.
Instead of using the app.event('message', ...) method to listen for message events, I've changed to using app.message(...). The latter works mostly the same, but has one more advantage: you can pass a pattern to match the text of a message as the first argument (before the listener function): app.message(pattern, ...). That helps remove some of the conditions inside the listener. Instead of using just the two string variables jira_text and #rdu_qa, I've combined them in a single regular expression that matches when either of those values is seen in the text: /jira|#rdu_qa/.
Instead of using middleware to find the permalink of a message, I've moved that code into the listener. Middleware should be used to reuse code across multiple listeners (or global middleware to reuse code across all listeners). In your example, it doesn't seem like the code to find the permalink is being reused, but if you do use this in many listeners, it should be relatively easy to extract. Another advantage is now the logic only runs after the pattern was matched, so you're not making these calls for every single message that the bot sees in all channels that it is a member of (this is much better for performance).
Use Promise.all() to collect the Promises of each call to chat.postMessage into one promise. Currently, you're using userzArray.forEach(...), which doesn't return anything. So then using await on that value will immediately resolve, and doesn't really do anything useful. What we need to do is collect each of the Promises and wait for them to all complete. This is what Promise.all() does. We just need an array of Promises to pass in, which we can get by simply changing userzArray.forEach() to userzArray.map().
There's a problem with the way you're calling chat.postMessage. You're trying to use Slackbot to send those messages, but that's not recommended because users are less likely to understand where that message is coming from. Instead, you should send this message as a DM from your bot user. In order to do that, you need a conversation ID, not a user ID, for each user you want to send this notification to. One of the items in userzArray is already a DM conversation ID (it starts with a D), but the other is not. In order to make this work consistently, I've created the conversationsToNotify array which contains the conversation IDs for each user after calling conversations.open to create a DM. Now in the code, you'll see conversationsToNotify.map() instead of userzArray.map(). Your Slack app will now need the im:write and chat:write permission scopes (don't forget to reinstall once you add scopes). Looking up the conversation IDs will slow down your app from starting up if the number of users in the array gets larger. My recommendation would be to save the conversation IDs in your code (or in a database) instead of the user IDs. This will ensure a consistently fast start up time for your app.
There's an opportunity to do even better. What happens when the first call to chat.postMessage fails? The way I've written the code above, the error would be logged to the console, but later if the second call fails, there's no way to know. That's because Promise.all() returns a promise that will reject as soon as any one of the promises rejects, and then ignores what happens afterwards. If you're using Node v12.9.0 or greater, I would recommend using Promise.allSettled() instead (which would require a few changes in your catch clause as well).
General cleanup:
Use message argument in the listener everywhere instead of payload argument. These are actually the same value when dealing with message events. payload is mostly only useful in middleware that handle several kinds of events (action, event, view, shortcut, etc) so that there's one way to refer to all of their payloads.
Move userzArray outside the listener, and make it a constant. There's no point in redeclaring it inside the listener each time it runs, and it doesn't change.
I added a function to convert from user IDs to conversation IDs (convertUsersToConversations). This function is called before the app is started to avoid a race condition where the an incoming message is handled before the app knows which channels to notify.
Formatted the text content of the message as quoted text, formatted the channel mention, and formatted the permalink. One improvement I'd also recommend is to use a context block to show the message author's name and avatar image.

How to query firebase for many to many relationship?

It is my first time developing a SPA, and I am not using JS frameworks like React, Vue or Angular. My project just uses the firebase sdk and jquery to access the DOM elements.
In my app, the users can be associated with projects. Since that, I have a user-projects and project-users paths to represent that relationship.
When a user logs in my app I request users/uid to get the user data. After that I have to fetch the projects associated with the user. I will take the ids of the associated projects to finally request the data of each project.
I'm trying to use promises as described here, but I get nothing in the console.
function loadUserProjects() {
// Authenticated user
var user = firebase.auth().currentUser;
// General reference to the real time db
var ref = firebase.database().ref();
// Request the user data
ref.child('users/'+user.uid).on('value').then(function(snapshot) {
var user_data = snapshot.val(); console.log(user_data);
// Global variable to store the id of the selected project
project_selected_key = user_data.project_selected;
// Get the list of associated projects
return ref.child('user-projects/'+user.uid).on('value').then(function(snapshot) {
console.log(snapshot);
return snapshot;
});
}).then(function (projectsSnapshot) {
console.log(projectsSnapshot);
// List associated projects
var project_options = '';
projectsSnapshot.forEach(function (e) {
project_options += '<option value="'+e.key+'">'+e.val()+'</option>';
});
if (! project_options) {
project_options = '<option disabled selected value>- Ningún proyecto -</option>';
}
$('#project_selected').html(project_options);
}, function(error) {
// Something went wrong.
console.error(error);
});
}
I know that I have to use one additional request, because at this point the <select>will be populated with truevalues (the additional request have to query the full data of each project). But I am not getting messages in the console.
Thanks in advance.
After that, I need to define different levels of privilege in each project, and associate a level when a project is assigned to a specific user. Initially I was very excited about the real time, but it seems that firebase is getting more complicated than I supposed.

A Firebase on() listener can respond to multiple events. A promise can only resolve once, that's why it's only available when you use Firebase's once() operation.
return ref.child('user-projects/'+user.uid).once('value');