Develop Reference

Firestore transactions showing unexpected behaviour when used in cloud functions

I am writing an app that features an inventory in which users can reserve products. I want to ensure that 2 users cannot simultaneously reserve a product at the same time, for this, I intend on using transactions. When using transactions from the Firebase SDK, everything works as intended, but I am getting unexpected behavior when using transactions from a callable cloud function. To simulate the use case where 2 users happen to reserve the same product, I use setTimeout in my cloud function to halt the function for 3 seconds. I am launching this function from 2 different clients with different user contexts.
export const reserveProduct = functions.https.onCall(async (data,context) => {
function testTimeout(){
return new Promise((resolve,reject) => {
setTimeout(()=> {
return resolve(true)
},3000)
})
}
if(!context.auth){
return {
error: `You must be logged in to reserve products`
}
}else{
const productRef = admin.firestore().collection('products').doc(data.productID)
const userRef = admin.firestore().collection('users').doc(context.auth.uid)
return admin.firestore().runTransaction((transaction) => {
return transaction.get(productRef).then(async(doc) => {
if(doc.get('status') == 'reserved'){
throw "Document already reserved!"
}else{
console.log("Product not reserved, reserving now!")
}
await testTimeout()
transaction.update(productRef, {status: 'reserved'});
transaction.update(userRef, {reserved: admin.firestore.FieldValue.arrayUnion(data.productID)})
})
}).then(() => {
console.log("Transaction Successfully committed !")
}).catch((error) => {
throw "Transaction failed, product already reserved"
})
}
After running this function call from 2 different clients simultaneously, The function call from my first client returns successfully as expected, but only after roughly 35s (which is way too long for the simplicity of the transaction). However, the second function call times out without returning any value. I have not seen any documentation explicitly stating the use of transactions in callable cloud functions, nor should it be affected when used within the emulator.
I am expecting to simply get a return value for whichever function call is able to modify the data first, and catch the error from the function which has retried and validated the reserved state.
Any help would be appreciated, thanks!

One major difference between the two places is in the way the SDKs used handle transactions:
The client-side SDKs use an optimistic compare-and-set approach for transactions, meaning that they pass the values you read in the transaction with the data you're writing. The server then only writes the new data if the documents you read haven't been updated.
The server-side SDKs (used in your Cloud Function) use a more traditional pessimistic approach for transactions, and place a lock on each document that you read in the transaction.
You can read more about database contention in the SDKs in the documentation.
While I'm not exactly certain how this is affecting your code, I suspect it is relevant to the difference in behavior you're seeing between the client-side and server-side implementations.

matrix-js-sdk setup and configuration

I am having some issues trying to connect to a matrix server using the matrix-js-sdk in a react app.
I have provided a simple code example below, and made sure that credentials are valid (login works) and that the environment variable containing the URL for the matrix client is set. I have signed into element in a browser and created two rooms for testing purposes, and was expecting these two rooms would be returned from matrixClient.getRooms(). However, this simply returns an empty array. With some further testing it seems like the asynchronous functions provided for fetching room, member and group ID's only, works as expected.
According to https://matrix.org/docs/guides/usage-of-the-matrix-js-sd these should be valid steps for setting up the matrix-js-sdk, however the sync is never executed either.
const matrixClient = sdk.createClient(
process.env.REACT_APP_MATRIX_CLIENT_URL!
);
await matrixClient.long("m.login.password", credentials);
matrixClient.once('sync', () => {
debugger; // Never hit
}
for (const room of matrixClient.getRooms()) {
debugger; // Never hit
}
I did manage to use the roomId's returned from await matrixClient.roomInitialSync(roomId, limit, callback), however this lead me to another issue where I can't figure out how to decrypt messages, as the events containing the messages sent in the room seems to be of type 'm.room.encrypted' instead of 'm.room.message'.
Does anyone have any good examples of working implementations for the matrix-js-sdk, or any other good resources for properly understanding how to put this all together? I need to be able to load rooms, persons, messages etc. and display these respectively in a ReactJS application.

It turns out I simply forgot to run startClient on the matrix client, resulting in it not fetching any data.

How to throw a custom message using Dialogflow after three times of fallback

I am developing a chatbot using Dialogflow, I would like to throw a message to user when the chatbot doesn't understand the user input for three times in a row and for the forth time respond with a custom message (not the one of the options declared on the dialogflow interface)
One idea that I have is to make a counter within the input unknown action like this:
var counter = 1;
// The default fallback intent has been matched, try to recover (https://dialogflow.com/docs/intents#fallback_intents)
'input.unknown': () => {
// Use the Actions on Google lib to respond to Google requests; for other requests use JSON
if (requestSource === googleAssistantRequest) {
sendGoogleResponse('I\'m having trouble, can you try that again?'); // Send simple response to user
} else {
if (counter == 3) {
counter = 1;
sendResponse('Custom message');
} else {
counter++;
sendResponse('I\'m having trouble, can you try that again?'); // Send simple response to user
}
}
},
This would work, but idk if this will work for multiple user at the same time, I was thinking to create a storage for storing requests attached by a unique id and have a different counter for each request!
Do you have any better idea of achieving such thing in Dialogflow?

This will not work the way you've designed it. Not quite for the reason you think, but close.
You don't show the rest of your code (that's ok), but the counter variable is probably in a function that gets called each time it processes a message. When that function is finished, the counter variable goes out of scope - it is lost. Having multiple calls at the same time won't really be an issue since each call gets a different scope (I'm glossing over some technical details, but this should be good enough).
One solution is that you could store the variable in a global context - but then you do have the issue of multiple users ending up with the same counter. That is very very bad.
Your solution about keeping a counter in a database, keyed against the user, does make sense. But for this need, it is overkill. It is useful for saving data between conversations, but there are better ways to save information during the same conversation.
The easiest solution would be to use a Dialogflow Context. Contexts let you save state in between calls to your webhook fulfillment during the same conversation and for a specific number of messages received from the user (the lifespan).
In this case, it would be best if you created a context named something like unknown_counter with a lifespan of 1. In the parameters, you might set val to 1.
The lifespan of 1 would mean that you'll only see this context the next time your webhook is called. If they handle it through some other Intent (ie - you understood them), then the context would just vanish after your fulfillment runs.
But if your input.unknown handler is called again, then you would see the context was there and what the value is. If it doesn't meet the threshold, send the context again (with a lifespan of 1 again), but with the value being incremented by 1. If it did meet the threshold - you'd reply with some other answer and close the connection.
By "send the context", I mean that the context would be included as part of the reply. So instead of sending just a string to sendGoogleResponse() or sendResponse() you would send an object that included a speech property and an outputContexts property. Something like this:
var outputContexts = [
{
name: 'unknown_counter',
lifespan: 1,
parameters: {
'val': counterValue,
}
}
];
sendResponse({
speech: "I'm confused. What did you say?",
outputContexts: outputContexts
});

Firebase synchronisation of locally-modified data: handling errors & global status

I have two related questions regarding the Firebase web platform's
synchronisation of locally-modified data to the server:
Every client sharing a Firebase database maintains its own internal version of any active data.
When data is updated or saved, it is written to this local version of the database.
The Firebase client then synchronizes that data with the Firebase servers and with other clients on a 'best-effort' basis.
1. Handling sync errors
The data-modification methods
(set(),
remove(), etc)
can take an onComplete callback parameter:
A callback function that will be called when synchronization to the Firebase servers
has completed. The callback will be passed an Error object on failure; else null.
var onComplete = function(error) {
if (error) {
console.log('Synchronization failed');
} else {
console.log('Synchronization succeeded');
}
};
fredRef.remove(onComplete);
In the example above, what kind of errors should the fredRef.remove() callback expect to receive?
Temporary errors?
Client is offline (network connection lost) ?
Firebase server is temporarily overloaded or down for maintenance, but will be available again soon?
Permanent errors?
Permission denied (due to security rules) ?
Database location does not exist?
Is there a way to distinguish between temporary and permanent errors?
How should we handle / recover from these errors?
For temporary errors, do we need to call fredRef.remove() again after a short period of time, to retry the operation?
2. Global sync status
I realise that each call to set() and remove() will receive an individual sync success/failure
result in the onComplete callback.  But I'm looking for a way to determine the
global sync status of the whole Firebase client.
I'd like to use a beforeunload event listener
to warn the user when they attempt to leave the page before all modified data has been synced to the server,
and I'm looking for some function like firebase.isAllModifiedDataSynced().  Something like this:
window.addEventListener('beforeunload', function (event) {
if (!firebase.isAllModifiedDataSynced()) {
event.returnValue = 'Some changes have not yet been saved. If you ' +
'leave this page, your changes will be lost.';
}
});
Here's an example of the same functionality in Google Drive:
I'm aware of the special /.info/connected location:
it is useful for a client to know when it is online or offline.
Firebase clients provide a special location at /.info/connected which is updated every time the client's connection state changes.
Here is an example:
var connectedRef = new Firebase("https://<YOUR-FIREBASE-APP>.firebaseio.com/.info/connected");
connectedRef.on("value", function(snap) {
if (snap.val() === true) {
alert("connected");
} else {
alert("not connected");
}
});
The special /.info/connected location can be connected to a beforeunload event listener like this:
var connectedRef = new Firebase('https://myapp.firebaseio.com/.info/connected');
var isConnected = true;
connectedRef.on('value', function (snap) {
isConnected = snap.val();
});
window.addEventListener('beforeunload', function (event) {
if (!isConnected) {
event.returnValue = 'Some changes have not yet been saved. If you ' +
'leave this page, your changes will be lost.';
}
});
My question is:
If isConnected is true, does this also mean that all modified data has been synced to the server?
i.e. Does "connected" also mean "synced"?
If not, how can the app determine the global sync status of the whole Firebase client?
Is there a special /.info/synchronized location?
Does the app need to manually keep track of the sync success/failure result of every onComplete callback?

In the example above, what kind of errors should the fredRef.remove() callback expect to receive?
Client is offline (network connection lost) ?
No, this will not cause an error to be passed to the completion listener. It will simply cause the completion listener to not be called (yet).
Firebase server is temporarily overloaded or down for maintenance, but will be available again soon?
No. This is essentially the same as being without a network connection.
Permission denied (due to security rules) ?
Yes, this is will indeed cause an error to be passed to the completion handler.
Database location does not exist?
No, this will not cause an error to be caused to the completion listener.
If isConnected is true, does this also mean that all modified data has been synced to the server? i.e. Does "connected" also mean "synced"?
No it does not. .info/connected will fire with true when a connection is made to the database.
If not, how can the app determine the global sync status of the whole Firebase client?
There is currently no way to determine whether your local data is up to date with the server.
Is there a special /.info/synchronized location?
No, such a location doesn't exist.
Does the app need to manually keep track of the sync success/failure result of every onComplete callback?
That depends on the use-case. But if you want to simply know when all your writes are executed, push a dummy value and wait for that to complete. Since Firebase executes the writes in order, you can be certain at that stage that you've gotten the other events.

Understanding anonymous functions in Express js

I am new to express and am trying to wrap my head around callbacks in RESTful actions. In my PUT request below, I'm confused about the following line that I have bolded below. Why is response.pageInfo.book being set to the second parameter in the anonymous function (result)? that seems kind of arbitrary.
Also, what is the best way to inspect some of these parameters (req, res, result, etc)? When I console.log it, doesn't show up in my terminal or in my browser console.
exports.BookEdit = function(request, response) {
var id = request.params.id;
Model.BookModel.findOne({
_id: id
}, function(error, result) {
if (error) {
console.log("error");
response.redirect('/books?error=true&message=There was an error finding a book with this id');
} else {
response.pageInfo.title = "Edit Book";
**response.pageInfo.book = result;**
response.render('books/BookEdit', response.pageInfo)
}
})
}

The findOne function takes a query ({_id : id}) and a callback as arguments. The callback gets called after findOne has finished querying the database. This callback pattern is very common in nodejs. Typically the callback will have 2 arguments
the first one error is only set if there was an error.
the second one usually contains the value being returned. In this case you are finding one book in the database.
The line you have bolded is where the book object is assigned to a variable which will be sent back to be rendered in the browser. It is basically some javascript object.
Your second request, to debug this stuff, here is what you can do:
In you code type the word debugger;
e.g.
var id = request.params.id;
debugger;
Next, instead of running your program like this:
node myprogram.js
... run with debug flag, i.e.
node debug myprogram.js
It will pause at the beginning and you can continue by pressing c then Enter
Next it will stop at that debugger line above. Type repl and then Enter and you'll be able to inspect objects and variables by typing their names.
This works very well and requires no installation. However, you can also take a more visual approach and install a debugger such as node-inspector which does the same thing but in a web browser. If you use a good IDE (e.g. webstorm) you can also debug node.js pretty easily.

In the above, the document that is the result of the findOne() query is being added to the pageInfo key of the response and is then being rendered in a template. The first parameter is a potential error that must be checked and the remainder contain data. It's the standard node idiom that an asynchronous call returns to a callback where you do your work.
The writer of the code has also decided to decorate the response object with an extra attribute. This is often done when a request passes through a number of middleware functions and you might want to build up the response incrementally (for example having a middleware function that adds information about the current user to the pageInfo key).
Look and see what else is on response.pageInfo. Information was probably put there by previous middleware (especially since the function above expects the pageInfo key to exist). Just do a console.log(response.pageInfo) and look on your server log or standard out.