I am trying to migrate from Firebase Real time Database to Cloud Firestore in order to use "complex" queries among collections but I found some "issues" that I don't know how to solve.
My server uses Express JS and Firebase Admin SDK. I am trying to create a CRUD router for my admin dashboard but I am getting the following error:
"9 FAILED_PRECONDITION: The query requires an index. You can create it here: ...
When I access that URL (...) I get some error I don't have enough permissions... Well, anyway, after some research I understood that "complex" queries requires indexes and they have to be more than one.
This is my current router code so you can see what I want to achieve:
// GET - Get users
router.get('/', async (req: Request, res: Response) => {
...
let ref: any = db.collection('users').orderBy('createdAt');
Object.keys(req.query).forEach(key => {
ref = ref.where(key, '==', req.query[key]);
});
...
});
As you can see when I iterate req.query key I am adding a conditional statement to the query. My idea is to progamatically add custom filters from the client.
According to what I understood from documentation, I should create a complex index type for every document property.
Therefore, what should I do in order to achieve the mentioned idea? Thanks in advance.
Firestore compound queries have some limitations.
One is that to use more than one method, for example "where" and "orderBy", you need to create an index for this query.
So in your case, as your queries are dynamic, for each query you will have to create an index.
My suggestion is that you only use the "where" in your queries and in the other filters use javascript to filter. Or migrate to another database like MongoDB.
This is the link to the Firestore documentation explaining the compound queries: https://firebase.google.com/docs/firestore/query-data/queries?hl=en-us
Related
I have just started working with mongoose/mongodb and node.js to build the backend of a react native app I am working on. I'm getting fairly comfortable with querying documents in the database from node but how do I set those "filters" in a GET request (using fetch) from the front end?
If a user applies a filter to a list to only return products of a certain price, how do I get that filter from the client side over to the server side so node can make the proper query on the database?
Same question for if I only want to return specific fields as opposed to returning every field in a mongodb document. How can I let the backend know what fields to return from the client side?
Hopefully that all makes sense, like I said I'm very new to both mongo and node so I apologize if my explanations are not the best.
Thank you to anyone who can help!
What you're looking for is a query language - a syntax for conveying the parameters of a query to a server. There are several options.
Some sites decide to only support simple queries. The simplest query language, in this case, is the GET query string. For instance, if you're building an on-line store, your customers will typically need to search for products by name and by category, and so you may end up with two query string parameters that you support in URLs, like this:
/products?category=clothes&search=t-shirt
Then, your application must process each of these parameters independently - your back-end must know how to turn this into a MongoDB query. For example:
products.find({
category: { $eq: req.query.category },
$text: { $search: req.query.search }
// similarly, you could accept params such as "pricemin" and "pricemax"
// and construct a { $gt, $lt } query
});
For more complex systems or applications that require more flexibility, you can allow the client to construct a complete query on their own and pass that to the back-end for evaluation. Multiple solutions exist that let you achieve this - GraphQL is a popular choice lately due to its good tooling support and ease of use. Another well-known solution is OData, which is used in some high-profile APIs, such as Microsoft Graph API.
If using an advanced query language, your back-end will always need to translate between the format that's sent to the server and the format that the database speaks. The simplest, non-generic example is the code snippet above - though batteries-included solutions will normally come with their own translators, so you may not need to write any code yourself if a package exists that does that for you.
I want to remove one value from an array in Firestore. I am using firebase functions and am updating this field after the deletion of a User. I have tried updating the array, I have tried arrayRemove, and a variety of other lines. Because I have to deploy every time I want to run a test it is getting time consuming and data is difficult to build and delete every time.
So I was hoping someone could help with this solution I found here:
I want to loop through a list of IDs (Users in the DB) and pull out their matches, and remove a specific ID from their list of matches.
When I run this, it doesn't know what firebase.firestore is. So I am not sure how to register firestore with this type of FieldValue.delete(). What is this section looking to target? "admin.FieldValue.delete()"?
admin.firestore()
.collection("Users")
.doc(matches[i])
.update({
[userId]: firebase.firestore.FieldValue.delete(),
});
I have also tried this:
admin
.firestore()
.collection("Users")
.doc(matches[i])
.update({
matches: admin.firebase.firestore.FieldValue.arrayRemove(userId),
});
but again it doesn't know what firebase.firestore is. What is the correct method to removing one value from an array?
Also, I am concerned about making a callout in a for loop but I am not seeing a way around it. Does anyone have any suggestions? These don't need to be async as all of the transactions are not reliant on one another.
error message i get in firebase functions:
ReferenceError: firebase is not defined
If you imported the Admin SDK like this:
const admin = require("firebase-admin");
You will want to use admin.firestore.FieldValue.delete(). Same pattern for arrayRemove().
I'm trying to delete specific document in the collection based on timestamp of that document. Posts that are out of date I set them to, should be deleted from the cloud on the script call.
The problem is that I couldn't manage to find a way to iterate over all the documents in the collection, so that I can access the fields and compare Date.now() to post['expireDate].
I'm not using the realtime database, but the firestore cloud for my project. I've found a way to do it in the db, but no on the cloud, and have tried different ways to do it.
exports.removeOldPosts = functions.https.onRequest((req, res) => {
const timeNow = Date.now();
let postsRef = admin.firestore().collection('accesories/').listDocuments();
postsRef.forEach(post => {
if (post['expiredDate'] < timeNow) {
post.delete();
}
})
return res.status(200).end();
});
You're using the listDocuments() API, which returns (asynchronously) a list of document references. But your code :
does not deal with the fact that the call is asynchronous. I.e. you don't have a then() callback, as in the example of the documentation.
assumes that the data of the document is retrieved, while listDocuments in reality only returns document references.
To fix this, I recommend using the get() API that is shown in the Firebase documentation on getting all documents from a collection. You'll still have to deal with the asynchronous nature yourself though, even with that API, so I recommend studying up on the asynchronous nature of Cloud Functions and modern web APIs. One placeto get started could be Doug's excellent blog post Why are the Firebase API asynchronous? and video series Learn JavaScript Promises.
As a separate note: instead of reading all documents from the database, and then determining in your code which ones to delete, I'd recommend using a query to only retrieve the documents that you want to delete. This leads to reading fewer documents, which saves you some cost in both document reads and bandwidth.
I need a little assistance with my app. I'm not asking for a handout, just some guidance as to where to begin. The basic idea is for logged in users to be able to favorite videos, and to have them persist when logged in.
My app uses the YouTube API (Playlist Items) to display videos from my public playlists within my app. I'm using firebase auth to register and login users, but I have yet to implement the RTD. This is where I need some assistance in structuring my data & organizing my app.
I don't want favorite lists created for every user by default, nor do I want to store false values and have to loop through them. I'd only like to set a favorites list if the user requests to do so, and the values are true. I'm open to suggestions regarding structuring my data, but I was thinking something simple like this:
"favorites": {
"John Doe": {
"video1ID": true,
"video2ID": true,
}
}
Videos are contained within cards using a .each function from within the API response. Included in these cards are "favorite" toggle switches that I'd like a user to be able to toggle and add a favorite video to their list.
YouTube provides Video ID's from within their JSON response. I was thinking that assigning a boolean to that video ID would get the job done, but I have no idea where to begin.
Something like:
function writeFavoritesList (name, videoID, toggleValue) {
firebase.database().ref('favorites/' + userId).set({
name: displayName,
videoID: videoID,
toggleValue: true
});
}
I'm very much a newb to anything outside of WordPress, so I hope I'm on the right track. Any help appreciated. Thanks! :)
Looks great. If this were another database, you could consider storing the video IDs in an array, but this being the firebase RealTime Database, you're much better off with objects, which you've already got.
You could modify your structure slightly to take advantage of RTDs push() key generation if you ever intend on sorting your favourite videos. To do so, instead of making the key the videoID and the value the boolean status, you could generate a key using firebase's push() key generation and make the value the videoID. "The unique key generated by push() are ordered by the current time, so the resulting list of items will be chronologically sorted. The keys are also designed to be unguessable (they contain 72 random bits of entropy)."
"favorites": {
"uid1": {
"uniqueKey1": videoID1,
"uniqueKey2": videoID2,
}
}
To generate a push() key, use: const key = firebase.database().ref().push().key.
More info: https://firebase.google.com/docs/reference/js/firebase.database.Reference#push
Saw your Guru post. I think the best way for you to learn is to delve deep into the documentation and figure this out for yourself. If you're truly committed to learning this stuff you'd be doing yourself a disservice to have someone else write the code for you.
I'd start with the GCP(Google Cloud Platform) cloud firestore docs and read through the Concepts section in its entirety:
https://cloud.google.com/firestore/docs/concepts
The firebase site mirrors parts of the GCP documenation, but also covers client implementations:
https://firebase.google.com/docs/firestore/
To get the most out of these docs use the nav sidebar on the left to drill down into all the various Cloud Firestore topics. They go into how to structure your database and provide sample code for you to analyse and play with.
You'll see the terms Documents and Collections thrown around a lot. A Document is somewhat equivalent to a JSON Object. A Collection is a list of documents; similar to an array of JSON objects. But here's where things get interesting; Documents can reference Collections (aka Subcollections):
So I would structure your database as follows:
Create a Users collection
Whenever a new user signs into your app, create a user document and add it to the Users collection.
The first time a user selects a favorite video create a Favorites collection and add it to the user document; then add favorite documents to the Favorites collection for this user
There is a Javascript/Web client (you've seem to already have it loaded from what I've seen in the repo link you provided on Guru). Here's the reference documentation for it:
https://firebase.google.com/docs/reference/js/firebase.firestore
The classes, methods and properties defined in those reference docs are what you'll be calling from within your jquery code blocks.
Good luck and stick with it.
I'm trying to query my firebase realtime database for the list of User objects. I have a favourites field and in it I'm storing a list of id's of favourited users. How would I go about writing a http endpoint with cloud functions so that it returns a json list of User objects corresponding to these ids in database?
Thanks a lot.
You can configure your development environment for FCF by following this. Once you've initialized your project, write a function inside functions/index.js like this one:
exports.getFavouriteUsers = functions.https.onRequest((req, res) => {
var favouriteUsers = admin.database.ref('users/favourites').val();
res.status(200).send(favouriteUsers);
});
Finally deploy and you'll be able to make requests to the endpoint shown in the console.
Look at this documentation for HTTP triggers