I'm trying to execute the following query with Firestore REST API, and I find the documentation to be very limited on more complex queries. Basically I want to use WHERE IN query.
const allowedIds = ['xxxx', 'zzzz', 'yyyy'];
let myQuery = firebase.firestore().collection('myCollection').where(firebase.firestore.FieldPath.documentId(), 'in', allowedIds);
This one works perfectly, but now I want to convert it around. Here is what I've tried so far:
const allowedIds = ['xxxx', 'zzzz', 'yyyy'];
'structuredQuery': {
'from': [{
'collectionId': 'myCollection',
}],
'where': {
'fieldFilter': {
'field': {
'fieldPath': firebase.firestore.FieldPath.documentId(),
},
'op': 'IN',
'arrayValue': {
'values': [
{
'stringValue': 'xxxx',
},
{
'stringValue': 'zzzz',
},
],
},
},
},
},
It seems that the problem is in using firebase.firestore.FieldPath.documentId() to reference a document ID, but is there another way to achieve that?
The error message I currently get:
"Invalid value at 'structured_query.where.field_filter.field' (field_path), Starting an object on a scalar field
Invalid JSON payload received. Unknown name "arrayValue" at 'structured_query.where.field_filter': Cannot find field."
My data structure: I have vehicles and users collections. Users have a field called vehicles which is an array of strings(vehicles collection document id-s). So I would like to query vehicles collection WHERE IN the user.vehicles
Try replacing firebase.firestore.FieldPath.documentId() by "__name__".
Related
I'm working on building a database from a set of CSV files and am running into issues pushing elements to a deeply nested array. I've seen examples for 2D arrays and the use of positional operators but I can't quite figure out how to use them for my situation.
What I am trying to do read the CSV file which has columns for the answer_id and the url string for the photo. I want to push the url string to the photos array for the corresponding answer_id. When I try to use the code below, I get a long error message which starts with:
MongoBulkWriteError: Cannot create field 'answers' in element
{
results: [
{
question_id: "1",
_id: ObjectId('6332e0b015c1d1f4eccebf4e'),
answers: [
{
answer_id: "5",
//....
}
],
}
]
}
I formatted the above message to make things easier to read. It may be worth noting the first row of my CSV file has '5' in the answer_id column which makes me think things are failing at the first try to update my document.
Here is an example of my code:
const ExampleModel = new Schema({
example_id: String,
results: [
{
question_id: String,
answers: [
{
answer_id: String,
photos: [
{ url: String },
]
}
]
}
]
});
// Example Operation
// row.answer_id comes from CSV file
updateOne: {
'filter': {
'results.answers.answer_id': row.answer_id,
},
'update': {
'$push': {
'results.answers.$.photos' : { 'url': 'test'}
}
}
}
I guess my question is can I update an array this deeply nested using Mongoose?
I have got a data structure:
{
field: 1,
field: 3,
field: [
{ _id: xxx , subfield: 1 },
{ _id: xxx , subfield: 1 },
]
}
I need to update a certain element in the array.
So far I can only do that by pulling out old object and pushing in a new one, but it changes the file order.
My implementation:
const product = await ProductModel.findOne({ _id: productID });
const price = product.prices.find( (price: any) => price._id == id );
if(!price) {
throw {
type: 'ProductPriceError',
code: 404,
message: `Coundn't find price with provided ID: ${id}`,
success: false,
}
}
product.prices.pull({ _id: id })
product.prices.push(Object.assign(price, payload))
await product.save()
and I wonder if there is any atomic way to implement that. Because this approach doesn't seem to be secured.
Yes, you can update a particular object in the array if you can find it.
Have a look at the positional '$' operator here.
Your current implementation using mongoose will then be somewhat like this:
await ProductModel.updateOne(
{ _id: productID, 'prices._id': id },//Finding Product with the particular price
{ $set: { 'prices.$.subField': subFieldValue } },
);
Notice the '$' symbol in prices.$.subField. MongoDB is smart enough to only update the element at the index which was found by the query.
I'm trying to count the associated entries using the separate attribute in my includes to improve performance (without it the request it's taking 5s). But I'm receiving the following error:
"message": "missing FROM-clause entry for table "likedPosts""
Sorry for bad english, it's not my first. I hope you understand and can help me.
My code:
#Query((returns) => [Post], {
nullable: true
})
async getAllFeedPostsByUserId(#Arg('user_id') user_id: number): Promise < Post[] > {
const result = await Post.findAll({
attributes: {
include: [
[Sequelize.fn("COUNT", Sequelize.col("likedPosts.feed_post")), "likesAmount"]
]
},
include: [{
model: LikedPosts,
as: 'likedPosts',
attributes: [],
separate: true,
}, ]
});
return result;
}
I think group is must to count entries.
Post.findAll({
attributes: {
include: [[Sequelize.fn('COUNT', Sequelize.col('likedPosts.feed_post')), 'likesAmount']]
},
include: [{
model: LikedPosts,
attributes: []
}],
group: ['likedPosts.feed_post'] // groupBy is necessary else it will generate only 1 record with all rows count
})
I can see seperate
separate desc
To elaborate: by default, to retrieve the related model instance, Sequelize will use a SQL JOIN. By enabling separate, Sequelize will perform a separate query for each of the associated models, and join the resulting documents in code (instead of letting the database perform the join).
I'm using normalizr util to process API response based on non-ids model. As I know, typically normalizr works with ids model, but maybe there is a some way to generate ids "on the go"?
My API response example:
```
// input data:
const inputData = {
doctors: [
{
name: Jon,
post: chief
},
{
name: Marta,
post: nurse
},
//....
}
// expected output data:
const outputData = {
entities: {
nameCards : {
uniqueID_0: { id: uniqueID_0, name: Jon, post: uniqueID_3 },
uniqueID_1: { id: uniqueID_1, name: Marta, post: uniqueID_4 }
},
positions: {
uniqueID_3: { id: uniqueID_3, post: chief },
uniqueID_4: { id: uniqueID_4, post: nurse }
}
},
result: uniqueID_0
}
```
P.S.
I heard from someone about generating IDs "by the hood" in normalizr for such cases as my, but I did found such solution.
As mentioned in this issue:
Normalizr is never going to be able to generate unique IDs for you. We
don't do any memoization or anything internally, as that would be
unnecessary for most people.
Your working solution is okay, but will fail if you receive one of
these entities again later from another API endpoint.
My recommendation would be to find something that's constant and
unique on your entities and use that as something to generate unique
IDs from.
And then, as mentioned in the docs, you need to set idAttribute to replace 'id' with another key:
const data = { id_str: '123', url: 'https://twitter.com', user: { id_str: '456', name: 'Jimmy' } };
const user = new schema.Entity('users', {}, { idAttribute: 'id_str' });
const tweet = new schema.Entity('tweets', { user: user }, {
idAttribute: 'id_str',
// Apply everything from entityB over entityA, except for "favorites"
mergeStrategy: (entityA, entityB) => ({
...entityA,
...entityB,
favorites: entityA.favorites
}),
// Remove the URL field from the entity
processStrategy: (entity) => omit(entity, 'url')
});
const normalizedData = normalize(data, tweet);
EDIT
You can always provide unique id's using external lib or by hand:
inputData.doctors = inputData.doctors.map((doc, idx) => ({
...doc,
id: `doctor_${idx}`
}))
Have a processStrategy which is basically a function and in that function assign your id's there, ie. value.id = uuid(). Visit the link below to see an example https://github.com/paularmstrong/normalizr/issues/256
I have the following schemas for the document Folder:
var permissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
var folderSchema = new Schema({
name: { type: string },
permissions: [ permissionSchema ]
});
So, for each Page I can have many permissions. In my CMS there's a panel where I list all the folders and their permissions. The admin can edit a single permission and save it.
I could easily save the whole Folder document with its permissions array, where only one permission was modified. But I don't want to save all the document (the real schema has much more fields) so I did this:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }, function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
but the problem is that perm is always undefined! I tried to "statically" fetch the permission in this way:
var perm = data.permissions[0];
and it works great, so the problem is that Underscore library is not able to query the permissions array. So I guess that there's a better (and workgin) way to get the subdocument of a fetched document.
Any idea?
P.S.: I solved checking each item in the data.permission array using a "for" loop and checking data.permissions[i]._id == permission._id but I'd like a smarter solution, I know there's one!
So as you note, the default in mongoose is that when you "embed" data in an array like this you get an _id value for each array entry as part of it's own sub-document properties. You can actually use this value in order to determine the index of the item which you intend to update. The MongoDB way of doing this is the positional $ operator variable, which holds the "matched" position in the array:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$": permission
}
},
function(err,doc) {
}
);
That .findOneAndUpdate() method will return the modified document or otherwise you can just use .update() as a method if you don't need the document returned. The main parts are "matching" the element of the array to update and "identifying" that match with the positional $ as mentioned earlier.
Then of course you are using the $set operator so that only the elements you specify are actually sent "over the wire" to the server. You can take this further with "dot notation" and just specify the elements you actually want to update. As in:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$.role": permission.role
}
},
function(err,doc) {
}
);
So this is the flexibility that MongoDB provides, where you can be very "targeted" in how you actually update a document.
What this does do however is "bypass" any logic you might have built into your "mongoose" schema, such as "validation" or other "pre-save hooks". That is because the "optimal" way is a MongoDB "feature" and how it is designed. Mongoose itself tries to be a "convenience" wrapper over this logic. But if you are prepared to take some control yourself, then the updates can be made in the most optimal way.
So where possible to do so, keep your data "embedded" and don't use referenced models. It allows the atomic update of both "parent" and "child" items in simple updates where you don't need to worry about concurrency. Probably is one of the reasons you should have selected MongoDB in the first place.
In order to validate subdocuments when updating in Mongoose, you have to 'load' it as a Schema object, and then Mongoose will automatically trigger validation and hooks.
const userSchema = new mongoose.Schema({
// ...
addresses: [addressSchema],
});
If you have an array of subdocuments, you can fetch the desired one with the id() method provided by Mongoose. Then you can update its fields individually, or if you want to update multiple fields at once then use the set() method.
User.findById(userId)
.then((user) => {
const address = user.addresses.id(addressId); // returns a matching subdocument
address.set(req.body); // updates the address while keeping its schema
// address.zipCode = req.body.zipCode; // individual fields can be set directly
return user.save(); // saves document with subdocuments and triggers validation
})
.then((user) => {
res.send({ user });
})
.catch(e => res.status(400).send(e));
Note that you don't really need the userId to find the User document, you can get it by searching for the one that has an address subdocument that matches addressId as follows:
User.findOne({
'addresses._id': addressId,
})
// .then() ... the same as the example above
Remember that in MongoDB the subdocument is saved only when the parent document is saved.
Read more on the topic on the official documentation.
If you don't want separate collection, just embed the permissionSchema into the folderSchema.
var folderSchema = new Schema({
name: { type: string },
permissions: [ {
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
} ]
});
If you need separate collections, this is the best approach:
You could have a Permission model:
var mongoose = require('mongoose');
var PermissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
module.exports = mongoose.model('Permission', PermissionSchema);
And a Folder model with a reference to the permission document.
You can reference another schema like this:
var mongoose = require('mongoose');
var FolderSchema = new Schema({
name: { type: string },
permissions: [ { type: mongoose.Schema.Types.ObjectId, ref: 'Permission' } ]
});
module.exports = mongoose.model('Folder', FolderSchema);
And then call Folder.findOne().populate('permissions') to ask mongoose to populate the field permissions.
Now, the following:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }).populate('permissions').exec(function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
The perm field will not be undefined (if the permission._id is actually in the permissions array), since it's been populated by Mongoose.
just try
let doc = await Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{ "permissions.$": permission},
);