For a scheduling app I'm building in Flutter I'm trying to write data to my cloud Firestore database with cloud functions and cron jobs to make my app more self-sustaining. I'm stuck at the first step, which is trying to get my cloud function to write data to my cloud Firestore database.
Below I've included links to pictures on github of how my current data is structured in Firestore. What I want is to add named documents to the collection 'days' with a subcollection of 'hours', in which there are more named documents with the fields 'hour' and 'reserved'.
Picture 1: https://github.com/winckles/rooster/blob/master/Schermafbeelding%202019-11-07%20om%2014.27.55.png?raw=true
Picture 2: https://github.com/winckles/rooster/blob/master/Schermafbeelding%202019-11-07%20om%2014.28.26.png?raw=true
Below I have also included my try on getting data in Firestore.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
// admin.firestore().collection('days').add({original: original}). then(writeResult => {
//
// });
exports.updateData = functions.https.onCall((data, context) => {
const days = admin.firestore().collection('days');
return days.doc(data['2019-11-08/hours/001']).set({
hour: '08:00-09:00',
reserved: '',
});
});
Ideally I would want the cloud function to add 14 documents (eg. 2019-11-06 to 2019-11-19) to the collection 'days' at the same time. Those documents would then each have a subcollection 'hours' with the data from the second screenshot (so the documents 001, 002, 003 etc. with the fields hour and reserved).
I read the documentation on cloud functions and mostly found triggers when data is written to Firestore, but this is not what I want. I also tried the quickstart function samples but with no success. It shouldn't be too hard but I can't seem to figure it out. I'm using Javascript to write the cloud function in. Any help/tips would be greatly appreciated!
You didn't specify exactly what wasn't working for you. However, the first thing I notice is that you're not calling initilizeApp properly from the cloud functions context. For cloud functions, you don't need any parameters, as the default credentials should work for you (unless you have done something very unusual already).
Here is a cloud function that will model the behavior you want. It does not do the full behavior, as I found that writing the date handling code would likely distract from the main part of the problem you are asking about, which is the firestore and functions code itself.
Likewise, this uses an https function (as it is a bit easier for me to test :), to use a callable function (or any other function type, e.g. a scheduled function if you're using a cron job) you would need to adjust it slightly (e.g., for a callable function you would need to change the declaration back to onCall and changing the final .then() call to return the value you want to return).
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();
exports.doIt = functions.https.onRequest((request, response) => {
const days = db.collection('days');
const writePromises = [];
['2019-11-08', '2019-11-09'].forEach((day) => {
const documentReference = days.doc(day);
writePromises.push(documentReference.set({reserved: ''}));
const hoursReference = documentReference.collection('hours');
const dataMap = { '001': '08:00-09:00',
'002': '09:00-10:00',
'003': '10:00-11:00',
'004': '11:00-12:00' };
Object.keys(dataMap).forEach((hour) => {
writePromises.push(hoursReference.doc(hour).set({
hour: dataMap[hour],
reserved: ''
}))
});
});
return Promise.all(writePromises)
.then(() => { response.send('ok'); })
.catch((err) => { console.log(err); });
});
Note that when you write this entire structure, you will be billed for a write to each document. There isn't really a way to avoid that though, as writes are billed per document, not per request.
Likewise, you may want to consider doing this as a batched write -- the above example is just showing a basic approach to writing. A batched write would put all of the documents into the database atomically. However, a batched write is limited to 500 updates, which you would hit at around 21 days of data (21 days * 24 hours). This would look very similar to the above (below is just the content of the function itself:
const days = db.collection('days');
const batch = db.batch();
['2019-11-08', '2019-11-09'].forEach((day) => {
const documentReference = days.doc(day);
batch.set(documentReference, {reserved: ''});
const hoursReference = documentReference.collection('hours');
const dataMap = { '001': '08:00-09:00',
'002': '09:00-10:00',
'003': '10:00-11:00',
'004': '11:00-12:00' };
Object.keys(dataMap).forEach((hour) => {
batch.set(hoursReference.doc(hour), {
hour: dataMap[hour],
reserved: ''
});
});
});
return batch.commit()
.then(() => { response.send('ok'); })
.catch((err) => { console.log(err); });
I do wonder a small bit about why you need to do this in a cloud function, rather than via a Firestore call directly from your app. Regardless, the above should allow you to get started.
if you wanna do a cron in firebase maybe its better if you use Google Cloud Scheduler, but be careful this approach have a different kind of facturation
exports.scheduledFunction = functions.pubsub.schedule('every 5 minutes').onRun((context) => {
console.log('This will be run every 5 minutes!');
return null;
});
You can learn more about this in:
https://firebase.google.com/docs/functions/schedule-functions
Related
I have a firebase cloud function, which triggers when a value is updated. Say, If an order is delivered and it's status is marked as delivered in firestore, i need the function to execute after 15 minutes whicn sends notification to the user.
exports.notificationOnDelivered = functions.firestore
.document('orders/{orderId}')
.onUpdate(async (change, context) => {
const after = change.after.data();
if (after.status === 'Delivered') {
setTimeout(async () => {
const notification = new OneSignal.Notification();
notification.app_id = ONESIGNAL_APP_ID;
notification.include_player_ids = [after.uid.fcm_token];
notification.contents = {
en: "Rate your Order"
};
notification.headings = {
en: "Go to -> My orders -> Rate now to rate your order"
}
await client.createNotification(notification);
}, 900000)
}
})
Currently, i'm using settimeout. But after some SO answers, i came to know that using settimeout is not appropriate. What's the right way of doing this?
I need the function to execute after 15 minutes which sends
notification to the user... What's the right way of doing this?
A correct approach to do that is described in this article: How to schedule a Cloud Function to run in the future with Cloud Tasks (to build a Firestore document TTL).
This approach is based on Cloud Tasks and on two Cloud Functions:
One Cloud Function to trigger when the order document is updated (i.e. the Cloud Function you already wrote but that needs to be adapted) and which schedules a task;
Another Cloud Function, of type HTTPS, for Cloud Tasks to invoke when it’s time for it to send the notification (i.e. 15 minutes later). It's in this CF that you'll send the notifcation.
I am writing an app that features an inventory in which users can reserve products. I want to ensure that 2 users cannot simultaneously reserve a product at the same time, for this, I intend on using transactions. When using transactions from the Firebase SDK, everything works as intended, but I am getting unexpected behavior when using transactions from a callable cloud function. To simulate the use case where 2 users happen to reserve the same product, I use setTimeout in my cloud function to halt the function for 3 seconds. I am launching this function from 2 different clients with different user contexts.
export const reserveProduct = functions.https.onCall(async (data,context) => {
function testTimeout(){
return new Promise((resolve,reject) => {
setTimeout(()=> {
return resolve(true)
},3000)
})
}
if(!context.auth){
return {
error: `You must be logged in to reserve products`
}
}else{
const productRef = admin.firestore().collection('products').doc(data.productID)
const userRef = admin.firestore().collection('users').doc(context.auth.uid)
return admin.firestore().runTransaction((transaction) => {
return transaction.get(productRef).then(async(doc) => {
if(doc.get('status') == 'reserved'){
throw "Document already reserved!"
}else{
console.log("Product not reserved, reserving now!")
}
await testTimeout()
transaction.update(productRef, {status: 'reserved'});
transaction.update(userRef, {reserved: admin.firestore.FieldValue.arrayUnion(data.productID)})
})
}).then(() => {
console.log("Transaction Successfully committed !")
}).catch((error) => {
throw "Transaction failed, product already reserved"
})
}
After running this function call from 2 different clients simultaneously, The function call from my first client returns successfully as expected, but only after roughly 35s (which is way too long for the simplicity of the transaction). However, the second function call times out without returning any value. I have not seen any documentation explicitly stating the use of transactions in callable cloud functions, nor should it be affected when used within the emulator.
I am expecting to simply get a return value for whichever function call is able to modify the data first, and catch the error from the function which has retried and validated the reserved state.
Any help would be appreciated, thanks!
One major difference between the two places is in the way the SDKs used handle transactions:
The client-side SDKs use an optimistic compare-and-set approach for transactions, meaning that they pass the values you read in the transaction with the data you're writing. The server then only writes the new data if the documents you read haven't been updated.
The server-side SDKs (used in your Cloud Function) use a more traditional pessimistic approach for transactions, and place a lock on each document that you read in the transaction.
You can read more about database contention in the SDKs in the documentation.
While I'm not exactly certain how this is affecting your code, I suspect it is relevant to the difference in behavior you're seeing between the client-side and server-side implementations.
I am trying to build an app using FireStore and React JS (Web)
My Firestore database basically has:
A collection of ChatRooms ChatRooms
Every chat-room has many messages which is a subcollection, for example:
this.db.collection("ChatRooms").doc(phone-number-here).collection("messages")
Also, every chat-room has some client info like first-name, last-name etc, and one that's very important:
lastVisited which is a timestamp (or firestamp whatever)
I figured I would put a React Hook that updates every second the lastVisited field, which means to try to record as accurately as possible on Firestore the last time I left a chat-room.
Based on that, I want to retrieve all the messages for every customer (chat-room) that came in after the last visit,
=> lastVisited field. :)
And show a notification.
I have tried from .onSnapshot listener on the messages subcollection, and a combination of Firestore Transactions but I haven't been lucky. My app is buggy and it keeps showing two, then one, then nothing, back to two, etc, and I am suffering much.
Here's my code!
Please I appreciate ANY help!!!
unread_messages = currentUser => {
const chatRoomsQuery = this.db.collection("ChatRooms");
// const messagesQuery = this.db.collection("ChatRooms");
return chatRoomsQuery.get().then(snapshot => {
return snapshot.forEach(chatRoom => {
const mess = chatRoomsQuery
.doc(chatRoom.id)
.collection("messages")
.where("from", "==", chatRoom.id)
.orderBy("firestamp", "desc")
.limit(5);
// the limit of the messages could change to 10 on production
return mess.onSnapshot(snapshot => {
console.log("snapshot SIZE: ", snapshot.size);
return snapshot.forEach(message => {
// console.log(message.data());
const chatRef = this.db
.collection("ChatRooms")
.doc(message.data().from);
// run transaction
return this.db
.runTransaction(transaction => {
return transaction.get(chatRef).then(doc => {
// console.log("currentUser: ", currentUser);
// console.log("doc: ", doc.data());
if (!doc.exists) return;
if (
currentUser !== null &&
message.data().from === currentUser.phone
) {
// the update it
transaction.update(chatRef, {
unread_messages: []
});
}
// else
else if (
new Date(message.data().timestamp).getTime() >
new Date(doc.data().lastVisited).getTime()
) {
console.log("THIS IS/ARE THE ONES:", message.data());
// newMessages.push(message.data().customer_response);
// the update it
transaction.update(chatRef, {
unread_messages: Array.from(
new Set([
...doc.data().unread_messages,
message.data().customer_response
])
)
});
}
});
})
.then(function() {
console.log("Transaction successfully committed!");
})
.catch(function(error) {
console.log("Transaction failed: ", error);
});
});
});
});
});
};
Searching about it, it seems that the best option for you to achieve that comparison, would be to convert your timestamps in milliseconds, using the method toMillis(). This way, you should be able to compare the results better and easier - more information on the method can be found in the official documentation here - of the timestamps of last message and last access.
I believe this would be your best option as it's mentioned in this Community post here, that this would be the only solution for comparing timestamps on Firestore - there is a method called isEqual(), but it doesn't make sense for your use case.
I would recommend you to give it a try using this to compare the timestamps for your application. Besides that, there is another question from the Community - accessible here: How to compare firebase timestamps? - where the user has a similar use cases and purpose as yours, that I believe might help you with some ideas and thoughts as well.
Let me know if the information helped you!
I'm trying to start writing tests for my application. I'm using Jest & Supertest to run all of my tests. When I try and run my test suite, I'm getting an error regarding a foreign key constraint.
The error:
error: truncate "users" restart identity - cannot truncate a table referenced in a foreign key constrainterror: cannot truncate a table referenced in a foreign key constraint
This is my server.spec.js file:
const request = require('supertest');
const server = require('./server.js');
const db = require('../data/db-config.js');
describe('server.js', () => {
describe('POST /register', () => {
it('should return 201 created', async () => {
const user =
{
name: "test",
username: "test",
email: "test77#test.com",
password: "password"
}
const res = await request(server).post('/api/auth/register').send(user);
expect(res.status).toBe(201);
})
beforeEach(async () => {
await db("graphs").truncate();
await db("users").truncate();
});
})
})
And here is my knex migration file:
exports.up = function(knex) {
return (
knex.schema
.createTable('users', tbl => {
tbl.increments();
tbl.string('username', 255).notNullable();
tbl.string('password', 255).notNullable();
tbl.string('name', 255).notNullable();
tbl.string('email', 255).unique().notNullable();
})
.createTable('graphs', tbl => {
tbl.increments();
tbl.string('graph_name', 255).notNullable();
tbl.specificType('dataset', 'integer[]').notNullable();
tbl
.integer('user_id')
.unsigned()
.notNullable()
.references('id')
.inTable('users')
.onDelete('CASCADE')
.onUpdate('CASCADE');
})
)
};
exports.down = function(knex) {
return (
knex.schema
.dropTableIfExists('graphs')
.dropTableIfExists('users')
)
};
I came across this answer in my research: How to test tables linked with foreign keys?
I'm new to both Postgres as well as testing. It makes sense that I would need to drop the tables in the reverse order like I have in my migration. But when I try to truncate them in the beforeEach section of my test, it doesn't seem to matter what order the tables are listed in.
I'm not sure where exactly to go from here. Any help would be greatly appreciated.
I think the trick here will be to resort to a bit of knex.raw:
await db.raw('TRUNCATE graphs, users RESTART IDENTITY CASCADE');
CASCADE because you don't want foreign key constraints getting in the way, and RESTART IDENTITY because the default Postgres behaviour is not to reset sequences. See TRUNCATE.
While we're on a related subject, allow me to introduce something that might make your life a lot easier: template databases! Templates are databases that Postgres can use to very rapidly recreate a database from a known state (by copying it). This can be faster than even truncating tables, and it allows us to skip all the annoying foreign key stuff when testing.
For example, you could use raw to do the following:
DROP DATABASE testdb;
CREATE DATABASE testdb TEMPLATE testdb_template;
This is a surprisingly inexpensive operation, and is great for testing because you can begin with a known state (not necessarily an empty one) each time you do a test run. I guess the caveats are that your knexfile.js will need to specify a connection with a user sufficiently credentialled to create and delete databases (so maybe an 'admin' connection that knows about localhost only) and that the template must be created and maintained. See Template Databases for more.
I am trying to query my firestore database using cloud functions.
I want to trigger an email notification every time a new reading in my database is under the value of 10.
Here is the relevant database structure for reference: database structure.
The "readings" field is an array and each "reading" is a map which holds the fields "date" and "value".
Currently I am at the point where I can send an email notification every time a new user is created however I want this to work for the database. I am unsure how to query for the "readings" array and then for each individual reading.
Here is my code so far which sends an email when a new user is created
exports.sendNotification = functions.auth.user().onCreate((user) => {
const mailOptions = {
from: '"Spammy Corp." <noreply#firebase.com>',
to:"fakeEmail#btopenworld.com",
text: "TEST"
};
return mailTransport.sendMail(mailOptions)
.then(() => console.log("It worked"))
.catch((error) =>
console.error('There was an error while sending the email:', error));
});
See: https://firebase.google.com/docs/firestore/extend-with-functions
For example, to fire on all new readings added to that first child:
exports.sendEmail = functions.firestore
.document('sensor/UGt.../readings')
.onCreate((snap, context) => {
const newValue = snap.data();
const value = newValue.value;
if (value < 10) {
// send email
}
});
In further comments you mentioned listening for new readings in all sensor elements, not just your first one. This is unfortunately not possible in an efficient / simple way (source). Instead you will have to listen to all onUpdate events on /sensor/, check if the update is adding a reading, then check the value & send your email.
It may be easier to call the cloud function directly from wherever adds the reading, depending on how many times the /sensor/ path is going to be updated for other reasons (since every time this happens, it's a waste of resources).