I am working with a PostgreSQL database using Prisma. I have a bulk update command which I want to fail if any of the records have changed since my last read.
My schema:
model OrderItem {
id String #id #default(uuid()) #db.Uuid
quantity Int
lastUpdated DateTime #updatedAt #map("last_updated")
##map("order_item")
}
I have written a query which works, but I built the query manually rather than using Prisma's safe query builder tools.
My query:
type OrderItemType = {
id: string;
quantity: number;
lastUpdated: Date;
}
type OrderItemUpdateDataType = {
quantity: number;
}
const updateByIds = async (
orderItemIdLastUpdatedTuples: ([OrderItemType['id'], OrderItemType['lastUpdated']])[],
orderItemUpdateData: OrderItemUpdateDataType,
) => {
// Optimistic concurrency - try updating based on last known "last updated" state. If mismatch, fail.
await prisma.$transaction(async (prisma) => {
// TODO: Prefer prisma.$queryRaw. Prisma.join() works on id[], but not on [id, lastUpdated][]
const idLastUpdatedPairs = orderItemIdLastUpdatedTuples
.map(([id, lastUpdated]) => `(uuid('${id}'), '${lastUpdated.toISOString()}')`)
.join(', ');
const query = `SELECT * FROM order_item WHERE (id, last_updated) in ( ${idLastUpdatedPairs} )`;
const items = await prisma.$queryRawUnsafe<OrderItem[]>(query);
// If query doesn't match expected update count then another query has outraced and updated since last read.
const itemIds = orderItemIdLastUpdatedTuples.map(([id]) => id);
if (items.length !== orderItemIdLastUpdatedTuples.length) {
throw new ConcurrentUpdateError(`Order Items ${itemIds.join(', ')} were stale. Failed to update.`);
}
await prisma.orderItem.updateMany({
where: { id: { in: itemIds } },
data: orderItemUpdateData,
});
});
};
This function wants to update a set of items. It accepts a list of tuples - id/lastUpdated pairs. It starts an explicit transaction, then performs an unsafe SELECT query to confirm the items to affect haven't been updated, then updates. This is following the guidance of Prisma's docs here - https://www.prisma.io/docs/concepts/components/prisma-client/transactions#interactive-transactions-in-preview
I was hoping to achieve the same results using prisma.$queryRaw rather than prisma.$queryRawUnsafe or even using implicit transactions rather than an explicit transaction wrapper. I wasn't able to find a syntax for expressing "where in tuple" using either of these approaches, though.
I am able to express what I want using implicit transactions when updating a single record. An example here would look like:
const { count } = await prisma.orderItem.updateMany({
where: { id, lastUpdated },
data: orderItemUpdateData,
});
and when using an explicit, safe query I stumbled on joining the array of tuples properly.
From the Prisma documentation, https://www.prisma.io/docs/concepts/components/prisma-client/raw-database-access#tagged-template-helpers, there exists a Prisma.join command (which happens implicitly when using their tagged template helper syntax) but I wasn't able to generate a valid output when feeding it an array of tuples.
Did I miss anything? Does Prisma support joining a tuple using their safe query template syntax?
I'm currently trying Firestore, and I'm stuck at something very simple: "updating an array (aka a subdocument)".
My DB structure is super simple. For example:
proprietary: "John Doe",
sharedWith:
[
{who: "first#test.com", when:timestamp},
{who: "another#test.com", when:timestamp},
],
I'm trying (without success) to push new records into shareWith array of objects.
I've tried:
// With SET
firebase.firestore()
.collection('proprietary')
.doc(docID)
.set(
{ sharedWith: [{ who: "third#test.com", when: new Date() }] },
{ merge: true }
)
// With UPDATE
firebase.firestore()
.collection('proprietary')
.doc(docID)
.update({ sharedWith: [{ who: "third#test.com", when: new Date() }] })
None works. These queries overwrite my array.
The answer might be simple, but I could'nt find it...
Firestore now has two functions that allow you to update an array without re-writing the entire thing.
Link: https://firebase.google.com/docs/firestore/manage-data/add-data, specifically https://firebase.google.com/docs/firestore/manage-data/add-data#update_elements_in_an_array
Update elements in an array
If your document contains an array field, you can use arrayUnion() and
arrayRemove() to add and remove elements. arrayUnion() adds elements
to an array but only elements not already present. arrayRemove()
removes all instances of each given element.
Edit 08/13/2018: There is now support for native array operations in Cloud Firestore. See Doug's answer below.
There is currently no way to update a single array element (or add/remove a single element) in Cloud Firestore.
This code here:
firebase.firestore()
.collection('proprietary')
.doc(docID)
.set(
{ sharedWith: [{ who: "third#test.com", when: new Date() }] },
{ merge: true }
)
This says to set the document at proprietary/docID such that sharedWith = [{ who: "third#test.com", when: new Date() } but to not affect any existing document properties. It's very similar to the update() call you provided however the set() call with create the document if it does not exist while the update() call will fail.
So you have two options to achieve what you want.
Option 1 - Set the whole array
Call set() with the entire contents of the array, which will require reading the current data from the DB first. If you're concerned about concurrent updates you can do all of this in a transaction.
Option 2 - Use a subcollection
You could make sharedWith a subcollection of the main document. Then
adding a single item would look like this:
firebase.firestore()
.collection('proprietary')
.doc(docID)
.collection('sharedWith')
.add({ who: "third#test.com", when: new Date() })
Of course this comes with new limitations. You would not be able to query
documents based on who they are shared with, nor would you be able to
get the doc and all of the sharedWith data in a single operation.
Here is the latest example from the Firestore documentation:
firebase.firestore.FieldValue.ArrayUnion
var washingtonRef = db.collection("cities").doc("DC");
// Atomically add a new region to the "regions" array field.
washingtonRef.update({
regions: firebase.firestore.FieldValue.arrayUnion("greater_virginia")
});
// Atomically remove a region from the "regions" array field.
washingtonRef.update({
regions: firebase.firestore.FieldValue.arrayRemove("east_coast")
});
You can use a transaction (https://firebase.google.com/docs/firestore/manage-data/transactions) to get the array, push onto it and then update the document:
const booking = { some: "data" };
const userRef = this.db.collection("users").doc(userId);
this.db.runTransaction(transaction => {
// This code may get re-run multiple times if there are conflicts.
return transaction.get(userRef).then(doc => {
if (!doc.data().bookings) {
transaction.set({
bookings: [booking]
});
} else {
const bookings = doc.data().bookings;
bookings.push(booking);
transaction.update(userRef, { bookings: bookings });
}
});
}).then(function () {
console.log("Transaction successfully committed!");
}).catch(function (error) {
console.log("Transaction failed: ", error);
});
Sorry Late to party but Firestore solved it way back in aug 2018 so If you still looking for that here it is all issues solved with regards to arrays.
https://firebase.googleblog.com/2018/08/better-arrays-in-cloud-firestore.htmlOfficial blog post
array-contains, arrayRemove, arrayUnion for checking, removing and updating arrays. Hope it helps.
To build on Sam Stern's answer, there is also a 3rd option which made things easier for me and that is using what Google call a Map, which is essentially a dictionary.
I think a dictionary is far better for the use case you're describing. I usually use arrays for stuff that isn't really updated too much, so they are more or less static. But for stuff that gets written a lot, specifically values that need to be updated for fields that are linked to something else in the database, dictionaries prove to be much easier to maintain and work with.
So for your specific case, the DB structure would look like this:
proprietary: "John Doe"
sharedWith:{
whoEmail1: {when: timestamp},
whoEmail2: {when: timestamp}
}
This will allow you to do the following:
var whoEmail = 'first#test.com';
var sharedObject = {};
sharedObject['sharedWith.' + whoEmail + '.when'] = new Date();
sharedObject['merge'] = true;
firebase.firestore()
.collection('proprietary')
.doc(docID)
.update(sharedObject);
The reason for defining the object as a variable is that using 'sharedWith.' + whoEmail + '.when' directly in the set method will result in an error, at least when using it in a Node.js cloud function.
#Edit (add explanation :) )
say you have an array you want to update your existing firestore document field with. You can use set(yourData, {merge: true} ) passing setOptions(second param in set function) with {merge: true} is must in order to merge the changes instead of overwriting. here is what the official documentation says about it
An options object that configures the behavior of set() calls in DocumentReference, WriteBatch, and Transaction. These calls can be configured to perform granular merges instead of overwriting the target documents in their entirety by providing a SetOptions with merge: true.
you can use this
const yourNewArray = [{who: "first#test.com", when:timestamp}
{who: "another#test.com", when:timestamp}]
collectionRef.doc(docId).set(
{
proprietary: "jhon",
sharedWith: firebase.firestore.FieldValue.arrayUnion(...yourNewArray),
},
{ merge: true },
);
hope this helps :)
addToCart(docId: string, prodId: string): Promise<void> {
return this.baseAngularFirestore.collection('carts').doc(docId).update({
products:
firestore.FieldValue.arrayUnion({
productId: prodId,
qty: 1
}),
});
}
i know this is really old, but to help people newbies with the issue
firebase V9 provides a solution using the arrayUnion and arrayRemove
await updateDoc(documentRef, {
proprietary: arrayUnion( { sharedWith: [{ who: "third#test.com", when: new Date() }] }
});
check this out for more explanation
Other than the answers mentioned above. This will do it.
Using Angular 5 and AngularFire2. or use firebase.firestore() instead of this.afs
// say you have have the following object and
// database structure as you mentioned in your post
data = { who: "third#test.com", when: new Date() };
...othercode
addSharedWith(data) {
const postDocRef = this.afs.collection('posts').doc('docID');
postDocRef.subscribe( post => {
// Grab the existing sharedWith Array
// If post.sharedWith doesn`t exsit initiated with empty array
const foo = { 'sharedWith' : post.sharedWith || []};
// Grab the existing sharedWith Array
foo['sharedWith'].push(data);
// pass updated to fireStore
postsDocRef.update(foo);
// using .set() will overwrite everything
// .update will only update existing values,
// so we initiated sharedWith with empty array
});
}
We can use arrayUnion({}) method to achive this.
Try this:
collectionRef.doc(ID).update({
sharedWith: admin.firestore.FieldValue.arrayUnion({
who: "first#test.com",
when: new Date()
})
});
Documentation can find here: https://firebase.google.com/docs/firestore/manage-data/add-data#update_elements_in_an_array
Consider John Doe a document rather than a collection
Give it a collection of things and thingsSharedWithOthers
Then you can map and query John Doe's shared things in that parallel thingsSharedWithOthers collection.
proprietary: "John Doe"(a document)
things(collection of John's things documents)
thingsSharedWithOthers(collection of John's things being shared with others):
[thingId]:
{who: "first#test.com", when:timestamp}
{who: "another#test.com", when:timestamp}
then set thingsSharedWithOthers
firebase.firestore()
.collection('thingsSharedWithOthers')
.set(
{ [thingId]:{ who: "third#test.com", when: new Date() } },
{ merge: true }
)
If You want to Update an array in a firebase document.
You can do this.
var documentRef = db.collection("Your collection name").doc("Your doc name")
documentRef.update({
yourArrayName: firebase.firestore.FieldValue.arrayUnion("The Value you want to enter")});
Although firebase.firestore.FieldValue.arrayUnion() provides the solution for array update in firestore, at the same time it is required to use {merge:true}. If you do not use {merge:true} it will delete all other fields in the document while updating with the new value. Here is the working code for updating array without loosing data in the reference document with .set() method:
const docRef = firebase.firestore().collection("your_collection_name").doc("your_doc_id");
docRef.set({yourArrayField: firebase.firestore.FieldValue.arrayUnion("value_to_add")}, {merge:true});
If anybody is looking for Java firestore sdk solution to add items in array field:
List<String> list = java.util.Arrays.asList("A", "B");
Object[] fieldsToUpdate = list.toArray();
DocumentReference docRef = getCollection().document("docId");
docRef.update(fieldName, FieldValue.arrayUnion(fieldsToUpdate));
To delete items from array user: FieldValue.arrayRemove()
If the document contains a nested object in the form of an array, .dot notation can be used to reference and update nested fields.
Node.js example:
const users = {
name: 'Tom',
surname: 'Smith',
favorites: {
sport: 'tennis',
color: 'red',
subject: 'math'
}
};
const update = await db.collection('users').doc('Tom').update({
'favorites.sport': 'snowboard'
});
or Android sdk example:
db.collection("users").document("Tom")
.update(
'favorites.sport': 'snowboard'
);
There is a simple hack in firestore:
use path with "." as property name:
propertyname.arraysubname.${id}:
db.collection("collection")
.doc("docId")
.update({arrayOfObj: fieldValue.arrayUnion({...item})})
// my db structure now
rcv : {
visible: 'all',
ids: [
[0] : userId,
[1] : user2Id ]
}
this is how i query to get the data it works.
//service.ts
getAlbumByUserId(userId) {
return this.afs.collection('albums', ref => ref.where('rcv.visible', '==', 'all').where('rcv.ids', 'array-contains', userId)).valueChanges();
}
//component.ts
this.service.getAlbumByUserId(this.userId);
but i want to set the structure like this but i don't know how to query nested objects in firebase
// database structure
rcv : {
visible: 'all',
ids: {
userId: {
id: userId
}
user2Id: {
id: user2Id
}
}
}
You're looking for the array-contains operator, which can check if a field that is an array contains a certain value.
You're already using the correct array-contains operator, but not with the correct syntax. The array-contains operator checks whether any element of your array is exactly the same as the value you pass in. So you need to pass in the complete value that exists in the array:
ref.where('rcv.visible', '==', 'all').where('rcv.ids', 'array-contains', { id: userId })
As you add more data to the array, it may become unfeasible to reproduce the entire array element for the query. In that case, the common approach is to add an additional field where you keep just the IDs.
So you'd end up with one field (say rcv.users) where you keep all details about the receiving users, and one field (say rcv.ids) where you just keep their IDs, and that you use for querying.
Is there a way to sort by a given array?
something like this:
const somes = await SomeModel.find({}).sort({'_id': {'$in': [ObjectId('sdasdsd), ObjectId('sdasdsd), ObjectId('sdasdsd)]}}).exec()
What i looking for is a way to get a solution, to get all document of the collection and sort by if the document's _id match with one of the given array.
An example:
we have albums collection and songs collection. In albums collection we store the ids of the songs that belongs to the albums.
I want to get the songs, but if the song is in the album take them front of the array.
I solved this as follow, but its looks a bit hacky:
const songs = await SongMode.find({}).skipe(limit * page).limit(limit).exec();
const album = await AlbumModel.findById(id).exec();
if(album) {
songArr = album.songs.slice(limit * page);
for(let song of album.songs) {
songs.unshift(song);
songs.pop();
}
}
This cannot be accomplished using an ordinary .find().sort(). Instead, you will need to use the MongoDB aggregation pipeline (.aggregate()). Specifically, you will need to do the following:
Perform a $projection such that if the _id is $in the array, your new sort_field is given the value 1, otherwise it's given a value of 0.
Perform a $sort such that you're doing a descending sort on the new sort_field.
If you're using MongoDB version 3.4 or greater, then this is easy because of the $addFields operator:
const your_array_of_ids = [
ObjectId('objectid1'),
ObjectId('objectid2'),
ObjectId('objectid3')
];
SomeModel.aggregate([
{ '$addFields': {
'sort_field': { '$cond': {
'if': { '$in': [ '$_id', your_array_of_ids ] },
'then': 1,
'else': 0
}}
}},
{ '$sort': {
'sort_field': -1
}}
]);
If you're using an older version of MongoDB, then the solution is similar, but instead of $addFields you will be using $project. Additionally, you will need to explicitly include all of the other fields you want included, otherwise they will be excluded from the results.
Im using Firebase Firestore and want to update an array field under a userprofile with the latest chat thread's id.. Im guessing that I have to pull the entire array (if it exists) from the chat node under that user, then I need to append the new id (if it doesnt exist) and update the array.. It works when theres only 1 value in the array then it fails after that with the following error:
Transaction failed: { Error: Cannot convert an array value in an array value.
at /user_code/node_modules/firebase-admin/node_modules/grpc/src/node/src/client.js:554:15 code: 3, metadata: Metadata { _internal_repr: {} } }
and here is my firebase cloud function, can anyone tell me where im going wrong ?
exports.updateMessages = functions.firestore.document('messages/{messageId}/conversation/{msgkey}').onCreate( (event) => {
/// console.log('function started');
const messagePayload = event.data.data();
const userA = messagePayload.userA;
const userB = messagePayload.userB;
// console.log("userA " + userA);
// console.log("userB " + userB);
// console.log("messagePayload " + JSON.stringify(messagePayload, null, 2) );
const sfDocRef = admin.firestore().doc(`users/${userB}`);
return admin.firestore().runTransaction( (transaction) => {
return transaction.get(sfDocRef).then( (sfDoc) => {
const array = [];
array.push(...[event.params.messageId, sfDoc.get('chats') ]);
transaction.update(sfDocRef, { chats: array } );
});
}).then( () => {
console.log("Transaction successfully committed!");
}).catch( (error) => {
console.log("Transaction failed: ", error);
});
});
You're nesting arrays in your code here:
const array = [];
array.push(...[event.params.messageId, sfDoc.get('chats') ]);
This leads to an array with two values, the first one being the new messageId and the second value contains an array all of your previous values, e.g.
[ "new message id", ["previous id", "older id"] ]
This type of nested array is something that Firestore (apparently) doesn't allow to be stored.
The solution is simple:
const array = [event.params.messageId, ...sfDoc.get('chats')];
The fact that you have to first load the array to then add a single element to it is one of reasons Firebasers recommend not storing data in arrays. Your current data looks like it'd be better off as a set, as shown in the Firestore documenation:
{
"new message id": true,
"previous id": true,
"older id": true
}
That way adding a chat ID is as simple as:
sfDocRef.update({ "chats."+event.params.messageId, true })
I have looked further into the matter, and I would follow the advice that Frank gave you in his post; allocate the data in collections rather than with arrays as they have greater versatility for Firebase 1. Researching under the examples listed in the Firebase website looking for anything related to a chat, I’ve found the data structure and code for messages that are used by Firechat as they might be of use for you.
In the source code, they use a collection for the their message-id -userId pair with the following topology 2 :
The exact way how the saving is executed at the repository is 3 :
It executes an append of the message into the Room-id collection. Instead of this structure, you could use an userID - messageID pair as it might fit you better.