Develop Reference

Using lodash isEqual() to compare single object property with matching properties from large array with multiple objects

I've looked at lodash documentation and played around with comparing simple objects. I've also found a number of explanations online for comparing entire objects and other types of comparisons, but I want to compare one property value in a single object with the values of all properties of a certain name in a large array with multiple objects.
Is lodash smart enough to do this as is, and, if so, what would be the proper syntax to handle this? Or do I need some sort of loop to work through the larger object and recursively compare its properties of a certain name with the small object property?
The javascript comparison I'm looking for would be something like this, but I don't know how to indicate that I want to compare all itemURL properties in the large array:
// guard clause to end the larger function if test is true, any match found
if (_.isEqual(feedItem.link, rssDataFileArr.itemURL)) {
return;
}
Small object example:
const feedItem = {
link: 'https://news.google.com/rss/search?q=nodejs',
otherProperty: 'whatever'
}
Large array of objects example:
const rssDataFileArr = [
{
"itemURL": "https://news.google.com/rss/search?q=rss-parser",
"irrelevantProperty": "hello"
},
{
"itemURL": "https://news.google.com/rss/search?q=nodejs",
"irrelevantProperty": "world"
},
{
"itemURL": "https://news.google.com/rss/search?q=javascript",
"irrelevantProperty": "hello"
}
]
Any and all help appreciated.

As per suggestion in comment, I went with a built-in javascript method instead of lodash. I used some() because I only needed a true/false boolean result, not a find() value.
const feedItem = {
link: 'https://news.google.com/rss/search?q=nodejs',
otherProperty: 'whatever',
};
const rssDataFileArr = [
{
itemURL: 'https://news.google.com/rss/search?q=rss-parser',
irrelevantProperty: 'hello',
},
{
itemURL: 'https://news.google.com/rss/search?q=nodejs',
irrelevantProperty: 'world',
},
{
itemURL: 'https://news.google.com/rss/search?q=javascript',
irrelevantProperty: 'hello',
},
{
itemURL: 'https://news.google.com/rss/search?q=nodejs',
irrelevantProperty: 'world',
},
];
const linkMatch = rssDataFileArr.some(
({ itemURL }) => itemURL === feedItem.link
);
// guard clause to end the larger function if test is true, any match found
if (linkMatch) {
console.log('linkMatch is true');
return;
}

How can I list all the records to see if they are duplicated?

I have a problem that I cannot resolve. I have a table in MongoDB, and this is structure:
const shopEconomy = new mongoose.Schema({
guildID: { type: String },
name: { type: String },
value: { type: Number },
description: { type: String },
rolereq: { type: String },
roleadd: { type: String },
roleremove: { type: String },
buyinfo: { type: String }
});
I need to list all names from the table (shopData.name) and then check if the typed name exists in the database. I tried to do something like the one below, but it doesn't work.
const shopData = await shopEconomy.find({ guildID: message.guild.id });
let categories = [];
let data = new Object();
for(const i in shopData){
data += `${shopData[i].name}\n`
categories.push(data)
}
Could someone take a look at this and help me out?

The title of the question does not quite match the description of the question. Given the description, let's assume the typed name is assigned to var typedName.
Let's also assume that you have bound your shopEconomy schema to a model that will actually interact with a mongodb collection called shopData. Then this will iterate all the docs in the shopData:
var found = false;
cursor = db.shopData.find(); // get EVERYTHING
cursor.forEach(function(doc) {
print(doc['name']);
if(doc['name'] == typedName) {
found = true;
}
});
if(found) {
print(typedName,"was found");
}
It is likely that the OP wants to find duplicate name in the collection, for which this pipeline will work:
db.shopData.aggregate([
{$group: {_id: '$name', N:{$sum:1}} },
{$match: {'N':{$gt:1}}}
]);

Part of the issue here comes from the use of a for...in loop which treats shopData as an object and loops over all properties of it. Instead try using a for...of loop which treats shopData as an array and loops over all objects in it.
...
for(const i of shopData) {
data += `${i.name}\n`
...
}
See also this question on for...in vs for...of and this question on JavaScript loops.

Is it possible to have mongoose populate a Mixed field only if the field contains an ObjectId?

Setup
Let's say we have schemas for Foo and Bar. Foo has a field bar. This field can either contain an ObjectId that reference a document Bar, or it could contain other arbitrary objects that are not ObjectIds.
const fooSchema = new mongoose.Schema({
bar: {
type: mongoose.Schema.Types.Mixed,
ref: 'Bar'
}
});
const Foo = <any>mongoose.model<any>('Foo', fooSchema);
const barSchema = new mongoose.Schema({
name: String
});
const Bar = <any>mongoose.model<any>('Bar', barSchema);
Problem
Now suppose we have a bunch of Foo documents.
I would like to be able to use mongoose's populate on the bar field to automatically replace references to a Bar document with the actual Bar document itself. I would also like to leave all other objects that are not references to a Bar document unchanged.
Normally, I would use something like this to get all the Foo documents and then populate the bar field:
Foo.find().populate('bar')
However, this method will throw an exception when it encounters objects in the bar field that are not ObjectIds, as opposed to leaving them untouched.
UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): CastError: Cast to ObjectId failed for value "Some arbitrary object" at path "_id" for model "Bar"
Attempt at finding a solution
I have examined using the match option on populate, by requiring that a field on Bar exists:
Foo.find().populate({
path: 'bar',
match: {
name: {
$exists: true
}
}
}
Unfortunately, the error is the same.
Question
So my question is then, is there any way to get mongoose to only populate a field if the field contains an ObjectId, and leave it alone otherwise?

As far as I know you cannot use populate that way. Select property works after trying to get values for population and there's no way to filter it before that.
You would have to do it manually. You could do it manually.
let foos = await Foo.find({});
foos = foos.map(function (f) {
return new Promise(function (resolve) {
if (condition()) {
Foo.populate(f, {path: 'bar'}).then(function(populatedF){
resolve(f);
});
} else {
resolve(f);
}
});
});
await Promise.all(foos).then(function (fs) {
res.status(200).json(fs);
});
Elegantly would be to wrap it in post hook or static method on your Model.
Another option would be to send 2 queries:
const foosPopulated = Foo.find({ alma: { $type: 2 } }).populate('bar'); // type of string
const foosNotPopulated = Foo.find({ alma: { $type: 3 } }); // type of object
const foos = foosPopulated.concat(foosNotPopulated);
This is of course suboptimal because of 2 queries (and all population queries) but maybe this will not be a problem for you. Readability is much better. Of course you could then change find queries to match your case specifically.

How to make Mongoose not insert empty array or object fields into a document

Let's say we have a Mongoose schema in our Node.js project:
let coolSchema = new mongoose.Schema({
field_1 : Number,
field_2 : String,
field_3 : [ String ],
});
And let's we have an according object:
var data = {
field_1 : 123,
field_2 : 'blah',
field_3 : ['aa', 'bb'],
};
Now to save this data into MongoDB we can use this code:
let Model = require('mongoose').model('CoolModel', coolSchema);
(new Model(data)).save();
Ok, while it's all cool.
But if data does not contain field_3 (array field, and the same will be for an object field) Mongoose will anyway add this field into the being created document with empty value.
Can we somehow tell Mongoose not to create this field if it's not contained in the data object?

you can do it easily skip the array field and array of object field.. This will let you skip saving empty array in new documents.but you have to use pre hook for this .
var brandSchema = new Schema({
name : {type:String},
email:String,
check:[]
})
brandSchema.pre('save', function (next) {
if (this.isNew && 0 === this.check.length) {
this.check = undefined;
}
next();
})
when new document is inserted in your schema you have to use this middlware.this works fine so try this.
this is the response when we want to insert any document
"data": {
"__v": 0,
"name": "testing",
"email": "testing#gmail.com",
"_id": "5915b018e292833edda8837f"
}
so i have send only email and name but check(array) field is skipped(Not send any value).

The accepted answer is good. But if you wouldn't want to use pre-hook, then you can add default: undefined to the array fields. For example:
var schema = new Schema({
myArr: { type: [String], default: undefined }
});
Refer to this comment for more explanation.

Not particularly an answer to the question itself but some thought on the matter.
It's not clear exactly what you're trying to achieve here. You defined a schema that is supposed to contain a list of string. Mongoose correctly does so that the data saved in your schema is consistent with the definition of the schema.
In this case, the list is more of a structural part of the schema. If you have different behaviour, you'd have to handle special case in your code in case the list isn't present. Now, you can safely assume that you schema is always returning a list so fetching some data will always allow you to do:
coolData.field_3.forEach(function(x) {
do_cool_things(x)
})
What you're asking is to make the schema allow inconsistent data being returned from mongodb... In other words, you'd have to do this in order to prevent accessing attributes on undefined:
if (coolData.field_3) {
coolData.field_3.forEach(function(x) {
do_cool_things(x)
})
}
Also, I you were trying to optimize the size of you objects/database, you could fill a bug report so mongoose doesn't define empty values while saving the objects and autofill them with defaults when the field is missing from mongodb. (I could be wrong but did you actually check if the data in mongodb was containing empty values or you were just looking at data coming from mongoose?)

It's because you're not marking the fields as required in your schema definition.
Do this:
let coolSchema = new mongoose.Schema({
field_1 : { type: Number, required: true },
field_2 : { type: String, required: true },
field_3 : { type: [ String ], required: true },
});

Mongoose find/update subdocument

I have the following schemas for the document Folder:
var permissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
var folderSchema = new Schema({
name: { type: string },
permissions: [ permissionSchema ]
});
So, for each Page I can have many permissions. In my CMS there's a panel where I list all the folders and their permissions. The admin can edit a single permission and save it.
I could easily save the whole Folder document with its permissions array, where only one permission was modified. But I don't want to save all the document (the real schema has much more fields) so I did this:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }, function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
but the problem is that perm is always undefined! I tried to "statically" fetch the permission in this way:
var perm = data.permissions[0];
and it works great, so the problem is that Underscore library is not able to query the permissions array. So I guess that there's a better (and workgin) way to get the subdocument of a fetched document.
Any idea?
P.S.: I solved checking each item in the data.permission array using a "for" loop and checking data.permissions[i]._id == permission._id but I'd like a smarter solution, I know there's one!

So as you note, the default in mongoose is that when you "embed" data in an array like this you get an _id value for each array entry as part of it's own sub-document properties. You can actually use this value in order to determine the index of the item which you intend to update. The MongoDB way of doing this is the positional $ operator variable, which holds the "matched" position in the array:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$": permission
}
},
function(err,doc) {
}
);
That .findOneAndUpdate() method will return the modified document or otherwise you can just use .update() as a method if you don't need the document returned. The main parts are "matching" the element of the array to update and "identifying" that match with the positional $ as mentioned earlier.
Then of course you are using the $set operator so that only the elements you specify are actually sent "over the wire" to the server. You can take this further with "dot notation" and just specify the elements you actually want to update. As in:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$.role": permission.role
}
},
function(err,doc) {
}
);
So this is the flexibility that MongoDB provides, where you can be very "targeted" in how you actually update a document.
What this does do however is "bypass" any logic you might have built into your "mongoose" schema, such as "validation" or other "pre-save hooks". That is because the "optimal" way is a MongoDB "feature" and how it is designed. Mongoose itself tries to be a "convenience" wrapper over this logic. But if you are prepared to take some control yourself, then the updates can be made in the most optimal way.
So where possible to do so, keep your data "embedded" and don't use referenced models. It allows the atomic update of both "parent" and "child" items in simple updates where you don't need to worry about concurrency. Probably is one of the reasons you should have selected MongoDB in the first place.

In order to validate subdocuments when updating in Mongoose, you have to 'load' it as a Schema object, and then Mongoose will automatically trigger validation and hooks.
const userSchema = new mongoose.Schema({
// ...
addresses: [addressSchema],
});
If you have an array of subdocuments, you can fetch the desired one with the id() method provided by Mongoose. Then you can update its fields individually, or if you want to update multiple fields at once then use the set() method.
User.findById(userId)
.then((user) => {
const address = user.addresses.id(addressId); // returns a matching subdocument
address.set(req.body); // updates the address while keeping its schema
// address.zipCode = req.body.zipCode; // individual fields can be set directly
return user.save(); // saves document with subdocuments and triggers validation
})
.then((user) => {
res.send({ user });
})
.catch(e => res.status(400).send(e));
Note that you don't really need the userId to find the User document, you can get it by searching for the one that has an address subdocument that matches addressId as follows:
User.findOne({
'addresses._id': addressId,
})
// .then() ... the same as the example above
Remember that in MongoDB the subdocument is saved only when the parent document is saved.
Read more on the topic on the official documentation.

If you don't want separate collection, just embed the permissionSchema into the folderSchema.
var folderSchema = new Schema({
name: { type: string },
permissions: [ {
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
} ]
});
If you need separate collections, this is the best approach:
You could have a Permission model:
var mongoose = require('mongoose');
var PermissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
module.exports = mongoose.model('Permission', PermissionSchema);
And a Folder model with a reference to the permission document.
You can reference another schema like this:
var mongoose = require('mongoose');
var FolderSchema = new Schema({
name: { type: string },
permissions: [ { type: mongoose.Schema.Types.ObjectId, ref: 'Permission' } ]
});
module.exports = mongoose.model('Folder', FolderSchema);
And then call Folder.findOne().populate('permissions') to ask mongoose to populate the field permissions.
Now, the following:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }).populate('permissions').exec(function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
The perm field will not be undefined (if the permission._id is actually in the permissions array), since it's been populated by Mongoose.

just try
let doc = await Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{ "permissions.$": permission},
);