I have the following batch, where I delete three documents in a collection,
the data-set-c collection however has one nested collection for massages where each message is a doc each.
the problem is that the (data-set-c) collection never gets deleted, I don't know if it is due to the nesting taking place? will deleting this way affect also sub-collection? or is it the rules that are blocking it since my rules are specific to the deepest level endpoint, or should use the cloud function instead since this doc will be massive later on, and each day is massages collection but the main issue here is how to delete this one level nested structure.
could you please take a look and see what I am doing wrong.
// Batch
const batch = writeBatch(db);
// data-set-a/{Id}/sub-set-a/{subSetId} ---> the direct document
const colA = collection(db, data-set-a,_authContext.currentUser.uid, sub-set-a);
// data-set-b/{Id}/sub-set-b/{subSetId} ---> the direct document
const colB = collection(db, data-set-b, _authContext.currentUser.uid, sub-set-b);
// data-set-c/{Id}/sub-set-c/{subSetId}/masseges/{mgsId} /*nested collection*/
const colC = collection(db, 'data-set-c', _authContext.currentUser.uid, sub-set-c);
const docA = doc(colA, subSetId);
const docB = doc(colB, subSetId);
const docC = doc(colC, subSetId);
const docsArr = [docA, docB, docC];
docsArr.forEach((col: any) => {
batch.delete(col)
});
await batch.commit();
// sub-set-a + sub-set-b SECURITY RULES
match /data-set-a/Id}sub-set-a/{subId} {
allow delete: if request.auth != null
&& request.auth.token.email_verified
&& request.auth.uid == Id
}
// sub-set-c SECURITY RULES
match /data-set-c/{Id}/sub-set-c/{subSetId}/masseges/{mgsId} {
allow delete: if request.auth != null
&& request.auth.token.email_verified
&& request.auth.uid == Id
}
the problem is that the (data-set-c) collection never gets deleted.
That's the expected behavior. If you delete a document, doesn't mean that you delete all sub-collections that exist within that document. Besides that, that document ID will continue to exist and the Firebase Console will display it in italic.
will deleting this way affect also sub-collection?
No, it won't.
or is it the rules that are blocking it since my rules are specific to the deepest level endpoint.
If it was about the security rules, then you would have received an error indicating that.
or should use cloud function instead since this doc will be massive later on
Yes, you can indeed use a Cloud Function to delete all documents that exist in all sub-collection of the document that was deleted.
How to delete this one-level nested structure.
You can achieve this by iterating each sub-collection, and by deleting each document. But don't do it on the client!
Related
I am working with a project where we create a bunch of entries in firestore based on results from an API endpoint we do not control, using a firestore cloud function. The API endpoint returns ids which we use for the document ids, but it does not include any timestamp information. Since we want to include a createdDate in our documents, we are using admin.firestore.Timestamp.now() to set the timestamp of the document.
On subsequent runs of the function, some of the documents will already exist so if we use batch.commit with create, it will fail since some of the documents exist. However, if we use batch.commit with update, we will either not be able to include a timestamp, or the current timestamp will be overwritten. As a final requirement, we do update these documents from a web application and set some properties like a state, so we can't limit the permissions on the documents to disallow update completely.
What would be the best way to achieve this?
I am currently using .create and have removed the batch, but I feel like this is less performant, and I occasionally do get the error Error: 4 DEADLINE_EXCEEDED on the firestore function.
First prize would be a batch that can create or update the documents, but does not edit the createdDate field. I'm also hoping to avoid reading the documents first to save a read, but I'd be happy to add it in if it's the best solution.
Thanks!
Current code is something like this:
const createDocPromise = docRef
.create(newDoc)
.then(() => {
// success, do nothing
})
.catch(err => {
if (
err.details &&
err.details.includes('Document already exists')
) {
// doc already exists, ignore error
} else {
console.error(`Error creating doc`, err);
}
});
This might not be possible with batched writes as set() will overwrite the existing document, update() will update the timestamp and create() will throw an error as you've mentioned. One workaround would be to use create() for each document with Promise.allSettled() that won't run catch() if any of the promise fails.
const results = [] // results from the API
const promises = results.map((r) => db.doc(`col/${r.id}`).create(r));
const newDocs = await Promise.allSettled(promises)
// either "fulfilled" or "rejected"
newDocs.forEach((result) => console.log(result.status))
If any documents exists already, create() will throw an error and status for that should be rejected. This way you won't have to read the document at first place.
Alternatively, you could store all the IDs in a single document or RTDB and filter out duplicates (this should only cost 1 read per invocation) and then add the data.
Since you prefer to keep the batch and you want to avoid reading the documents, a possible solution would be to store the timestamps in a field of type Array. So, you don't overwrite the createdDate field but save all the values corresponding to the different writes.
This way, when you read one of the documents you sort this array and take the oldest value: it is the very first timestamp that was saved and corresponds to the document creation.
This way you don't need any extra writes or extra reads.
So I have a followers collection and a users collection. Creating a doc in the followers collection with a certain ID requires a doc to exist with the same ID in the users collection.
So, in the security rules, I check for the existence of that document.
match /followers/{followers} {
function loggedInUserMatching() {
return (request.auth != null) && (request.auth.uid == followers);
}
function userExistsAndLoggedIn() {
return loggedInUserMatching() && exists(/databases/$(database)/documents/users/$(request.auth.uid));
}
allow create, delete: if userExistsAndLoggedIn();
}
In my tests, I try to create a doc in followers without the corresponding doc in the users collection. This should fail.
const database = testEnv.authenticatedContext('user1').firestore();
let testFollowersDoc = database.collection("followers").doc("user1");
expect(assertFails(testFollowersDoc.set({followers: []}))).resolves.toBeDefined();
Then, I create the doc in the users collection and then try to create the doc in followers again. This should succeed, but it always fails.
const testUserDoc = database.collection("users").doc("user1");
testUserDoc.set({about: "Wow", following: []});
expect(assertSucceeds(testFollowersDoc.set({followers: []}))).resolves.toBeUndefined();
The create rule for the users collection is correct and the document is actually created. I can access its data and verify it exists. But in my security rules, the exists() function always returns false, so the permission is denied and the document is not created.
What could be the cause of this? Am I using the exists() function incorrectly? Or am I not creating the document properly in my test?
I've been trying to solve this for a long time so any help would be appreciated!
These lines are asynchronous and should have await.
expect(assertFails(await testFollowersDoc.set({followers: []}))).resolves.toBeDefined();
expect(assertSucceeds(await testFollowersDoc.set({followers: []}))).resolves.toBeUndefined();
Hoi, I would like to check, using React javascript, if a collection in the Firestore already exists, no matter if it's empty or not. I tried:
if (collection(db, ref)) // is always true somehow
Any ideas? Thanks!
You would need to try to fetch from the collection and see if anything is returned:
const snap = await query(collection(db, ref), limit(1));
if (snap.empty) {
// no docs in collection
}
There is no function available in the SDK that can help you can check if a particular collection exists. A collection will start to exist only if it contains at least one document. If a collection doesn't contain any documents, then that collection doesn't exist at all. So that being said, it makes sense to check whether a collection contains or not documents. In code, it should look as simple as:
const snapshot = await query(collection(db, yourRef), limit(1));
if (snapshot.empty) {
//The collection doesn't exist.
}
One thing to mention is that I have used a call to limit(1) because if the collection contains documents, then we limit the results so we can pay only one document read. However, if the collection doesn't exist, there is still one document read that has to be paid. So if the above query yields no resul## Heading ##t, according to the official documentation regarding Firestore pricing, it said that:
Minimum charge for queries
There is a minimum charge of one document read for each query that you perform, even if the query returns no results.
You have to fetch the collection out of the database and check if it has more than 0 documents. Even, if the collection doesn't exist, it will return 0.
const db = firebase.firestore();
db.collection("YOUR COLLECTION NAME").get().then((res) =>{
if(res.size==0){
//Collection does not exist
}else{
//Collection does exist
}
Im getting my documents based on a list of ids.
db.collection("fruits").where(db.FieldPath.documentId(), "in", fruitIds).get()
How should I write my security rules to allow the above call and to deny the below call
db.collection("fruits").get()
It's not possible exactly as you require. What you can do is set your rules like this:
match /fruits/{id} {
allow get: true;
allow list: false;
}
This allows clients to get a document if they know the ID, but make it impossible to query documents in bulk.
You will have to then code your client app request each document individually with a DocumentReference get() (instead of a Query with a where clause). The performance hit for this is negligible (no, there is not any noticeable performance gain for using an "in" query the way that you show here - and you are limited to 10 documents per batch anyway).
As #Doug covered in their answer, this is not currently supported in the way you expect.
However, by looking at the reference, you can at least limit list (query) operations by placing conditions on the orderBy and limit used by any queries to make it more difficult rather than outright blocking it.
Consider this answer educational, just fetch the items one-by-one instead and disable list/query access in your security rules. It is included here for those who simply want to obfuscate rather than outright block such queries.
This would mean changing your query to:
db.collection("fruits")
.where(db.FieldPath.documentId(), "in", fruitIds)
.orderBy(db.FieldPath.documentId()) // probably implicitly added by the where() above, but put here for good measure
.limit(10) // this limit applies to `in` operations anyway, but for this to work needs to be added
.get()
service cloud.firestore {
match /databases/{database}/documents {
// Matches any document in the cities collection as well as any document
// in a subcollection.
match /fruits/{fruit} {
allow read: if <condition>
// Limit documents per request to 10 and only if they provide an orderBy clause
allow list: if <condition>
&& request.query.limit <= 10
&& request.query.orderBy = "__name asc" // __name is FieldPath.documentId()
allow write: if <condition>;
}
}
}
Using those restrictions, this should no longer work:
db.collection("fruits").get()
But you could still scrape everything in smaller chunks using:
const fruits = [];
const baseQuery = db.collection("fruits")
.orderBy(db.FieldPath.documentId())
.limit(10);
while (true) {
const snapshot = await (fruits.length > 0
? baseQuery.startAt(fruits[fruits.length-1]).get()
: baseQuery.get())
Array.prototype.push.apply(fruits, snapshot.docs);
if (snapshot.empty) {
break;
}
}
// here, fruits now contains all snapshots in the collection
https://medium.com/#haimrait/how-to-add-a-redis-cache-layer-to-mongoose-in-node-js-a9729181ad69
In this guide. So I mostly do queries like
{
id: <guild id>
}
so whenever new document is created.
const book = new Book({
title,
content,
author
});
try {
await book.save();
clearKey(Book.collection.collectionName);
res.send(book);
} catch (err) {
res.send(400, err);
}
will it remove stuff from caches if i use {id: } or will it delete only data on cache that is like empty object or like Model#find()?
I also have another problem which is not related to that but could ask.
Imagine I do this
const result = Model.findOne()
Cache.set(<anything>, JSON.stringify(result));
const cached = Cache.get(<anything>)
const result = new Model(cached);
result.message++;
await result.save().catch(console.error)
it throws the MongoError: E11000 duplicate key error collection:
How to fix that?
clearKey(Book.collection.collectionName) In short, it will clear all the cache for the collection.
TLDR
In your case this.hashKey = JSON.stringify(options.key || this.mongooseCollection.name); is collectionName
https://redis.io/commands/hget
Returns the value associated with field in the hash stored at key.
clearKey(hashKey) {
client.del(JSON.stringify(hashKey));
}
https://redis.io/commands/del
Removes the specified keys. A key is ignored if it does not exist.
So when you call clearKey(Book.collection.collectionName); it calls client.del which will delete all the records for that particular collection. as the complete hash is deleted.
To delete specific fields not the full hash :-
https://redis.io/commands/HDEL
Removes the specified fields from the hash stored at key. Specified fields that do not exist within this hash are ignored. If key does not exist, it is treated as an empty hash and this command returns 0.
You can use my library, which has event-based logic to clear cache if some related changes appear. It also has an option to pass a custom callback to check, if your record was marked as deleted, but not physically removed from database.
https://www.npmjs.com/package/speedgoose