Develop Reference

Unable to update Mongoose Nested Array

I've spent a day going through Mongoose documentation and posts on here and while I'm almost certain I have the code as prescribed in all those solved posts I simply cannot get my code to work. :(
The schema is relatively complex but by no means as complex as I've seen:
const hrUserSchema = new mongoose.Schema({
username: String,
displayName: String,
school: String,
leaveAuthoriser: String,
holidayLeft: Number,
holidayAllowed: Number,
discretionaryDays: Number,
emailAddress: String,
termTimeOnly: Boolean,
noOfAdditionalContractedDays: Number,
noOfAdditionalContractedDaysLeft: Number,
teachingStaff: Boolean,
employStartDate: String,
notes: String,
loginAttempts: Number,
lineManages: [{
staffName: String
}],
bookedHoliday: [{
dateEntered: String,
startDate: String,
endDate: String,
daysTaken: Number,
status: String,
approvedBy: String,
leaveType: String,
declineReason: String
}],
additionalContractedDays: [{
acdDateEntered: String,
acdStartDate: String,
acdEndDate: String,
acdDaysTaken: Number,
acdStatus: String,
acdApprovedBy: String,
acdDeclineReason: String
}],
perfMan: [{
year: Number,
selfReview: {
sref1: String,
sref2: String,
sref3: String,
sref4: String,
sref5: String,
status: String,
dateStarted: String,
dateCompleted: String,
signedOff: Boolean
},
stage1: {
objectives : [
{
objective: String,
objectiveLinkToSchoolTeam: String,
objectiveProgress: Boolean
}
],
personalDevelopment: String,
resourcesTraining: String,
appraiserSignOff: Boolean,
appraiseeSignOff: Boolean
},
stage2: {
feedback: String,
appraiserSignOff: Boolean,
appraiseeSignOff: Boolean
},
stage3: {
feedback: String,
appraiserSignOff: Boolean,
appraiseeSignOff: Boolean
}
}]
});
Basically I want to update perfMan.stage1.objectives.objectiveProgress, example of what data in that might look like is:
"perfMan" : [
{
"_id" : ObjectId("60cb502631dcea3eaaae6853"),
"selfReview" : {
"sref1" : "I have no strength",
"sref2" : "No developments",
"sref3" : "None I'm brill",
"sref4" : "The department has no aims",
"signedOff" : true
},
"stage1" : {
"objectives" : [
{
"_id" : ObjectId("60cb502631dcea3eaaae6854"),
"objective" : "Objective is pants",
"objectiveLinkToSchoolTeam" : "I hate objectives!",
"objectiveProgress" : false
},
{
"_id" : ObjectId("60cb502631dcea3eaaae6855"),
"objective" : "My second Objectoves",
"objectiveLinkToSchoolTeam" : "My Second reasons",
"objectiveProgress" : false
}
],
"personalDevelopment" : "My personal Development",
"resourcesTraining" : "My Resources"
},
"stage2" : {
"feedback" : "Keep working HARD",
"appraiserSignOff" : true
},
"stage3" : {
"feedback" : "Yoy've done really well",
"appraiserSignOff" : true
},
"year" : NumberInt(2021)
}
]
I've read about and tried to implement this by using arrayFilters which to me seems to exactly what I want, I've checked my Mongoose version (hosted) and it's 4.4.6 so I'm easily running above 3.6 which I think is what's needed.
My current code looks like this, I've confirmed the find is getting the right data:
HRUser.findOneAndUpdate(
{"username": username},
{"$set": {"perfMan.$[perfMan].stage1.objectives.$[objectives].objectiveProgress" : true}},
{"arrayFilters" : [{ "perfMan._id": ObjectId("" + perfmanID + "") },{ "objectives._id": ObjectId("" + objectiveID + "") }]}
),
function (err, response) {
console.log(err)
console.log(response);
if (!err) {
res.send("Successfully updated staff member details.");
} else {
res.send(err);
}
};
If somebody could spot my obviously glaring error I would be for ever grateful!
Thanks

After not looking at a screen all weekend and reading through the documentation for the billionth time I decided to change tack! I can't see any limitations on the number of arrayFilters, or for that matter objectID use within them, and I'm 99% certain both object IDs are correct but, after changing the code to the below, basically using a mix of a positional operator $ based on the search of the same first objectID AND an arrayFilter element it's now working! Code below:
HRUser.findOneAndUpdate({
perfMan: {
$elemMatch: {
_id: ObjectId("" + perfManID + "")
}
}
},
{"$set": {"perfMan.$.stage1.objectives.$[objectives].objectiveProgress" : false}},
{"arrayFilters" : [{ "objectives._id": ObjectId("" + objectiveID + "") }]},
function (err, response) {
console.log(err)
console.log(response);
});

mongoose duplicated key error with empty string

I have a schema like this:
const MemberSchema = mongoose.Schema({
name: {
type: String,
},
email: {
type: String,
unique: true
},
phone: {
type: String,
unique: true
}
})
and of course email and phone needs to be unique but the thing is either email or phone is optional so they can be empty string.
and when you set them as unique index I get this error
E11000 duplicate key error collection: model.members index: email_1 dup key: { : "" }'
Is there any way to add exception to unique?
Edit
I changed my code like below but still doesn't work.
email: {
type: String,
trim: true, index: true, unique: true, sparse: true
},
\
'E11000 duplicate key error collection: email-collector.members index: email_1 dup key:{ : null }'
when I run getIndexes, I get
> db.members.getIndexes()
[
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "email-collector.members"
},
{
"v" : 2,
"unique" : true,
"key" : {
"email" : 1
},
"name" : "email_1",
"ns" : "email-collector.members",
"sparse" : true,
"background" : true
},
{
"v" : 2,
"unique" : true,
"key" : {
"email" : 1,
"phone" : 1
},
"name" : "email_1_phone_1",
"ns" : "email-collector.members",
"sparse" : true
}
]

MongoError: E11000 duplicate key error collection with passport

I have the following userSchema
var mongoose = require("mongoose"),
passportLocalMongoose = require("passport-local-mongoose");
var userSchema = new mongoose.Schema({
email: String,
password: String,
userName: String,
fname: String,
lname: String,
userType: Number,
subscribedThreads: [
{
type: mongoose.Schema.Types.ObjectId,
ref: "Threads"
}
]
});
// add passport methods
userSchema.plugin(passportLocalMongoose);
// export modules to be used by the file requiring it
module.exports = mongoose.model("Users",userSchema);
The first entry into the collection occurs as it should but the next ones give
{ [MongoError: E11000 duplicate key error collection: KManV3.users
index: username_1 dup key: { : null }]
name: 'MongoError',
message: 'E11000 duplicate key error collection: KManV3.users index:
username_1 dup key: { : null }',
driver: true,
code: 11000,
index: 0,
errmsg: 'E11000 duplicate key error collection: KManV3.users index:
username_1 dup key: { : null }',
getOperation: [Function],
toJSON: [Function],
toString: [Function]
}
Also, dbname.users.getIndexes() gives:
> db.users.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "KManV3.users"
},
{
"v" : 1,
"key" : {
"password" : 1
},
"name" : "password_1",
"ns" : "KManV3.users",
"background" : true
},
{
"v" : 1,
"unique" : true,
"key" : {
"username" : 1
},
"name" : "username_1",
"ns" : "KManV3.users",
"background" : true
}
]
Apparently every property of schema has been set as unique and I can't add data into collections even if the data is totally different. I'm not sure if it's due to integration of passport.

Looking at the options for passport-local-mongoose:
usernameField: specifies the field name that holds the username. Defaults to 'username'.
usernameUnique : specifies if the username field should be enforced to be unique by a mongodb index or not. Defaults to true.
Which explains why your collection has a unique index on the (non-existent-in-your-schema) username field.
If you don't actually set this field in documents that you add to the database, MongoDB will use null, and once the first document has been inserted, a subsequent document (also with the field value null for username) will throw an E11000 error.
So first, remove the index on username (and also password, I assume you once marked that field as unique in your schema), and set the proper field name for passport-local-mongoose to use:
userSchema.plugin(passportLocalMongoose, { usernameField : 'userName' });
(or email, if you want that field to be used as unique user identifier)

Error reason - The index is not present in your collection, in which you are trying to insert.
Solution - drop that collection and run your program again.

MongoDB index search works in console, but not from browser

All of this was working yesterday so I'm really not sure whats wrong. I'v searched on SO and been through the docs and steps a number of times but I havnt got it to work. Any help much appreciated as always!
If I type db.movies.find({ '$text': { '$search': 'elephant' } }) into the shell, it returns a list of titles with 'elephant' in the title. But if I run the same search from my browser it crashes the node server and I get MongoError: text index required for $text query.
The error comes from the error response from the find():
Movie.find(
{ '$text': { '$search': 'elephants' } },
(err, movies) => {
if (err) throw err; // MongoError: text index required for $text query
if (!movies) {
res.json({
data: null,
success: false,
message: 'No movies data'
});
} else {
res.json({
success: true,
data: movies
});
}
})
The index on my DB looks like this:
{
"v" : 2,
"key" : {
"_fts" : "text",
"_ftsx" : 1
},
"name" : "title_text",
"ns" : "db.movies",
"weights" : {
"title" : 1
},
"default_language" : "english",
"language_override" : "language",
"textIndexVersion" : 3
}
I also have mongoose schemas defined and I'm using an auto increment plugin to increment a custom id field on each movie save. (Saving movies works, so I am connecting to the DB ok)
mongoose.model('Job', new Schema({
id: Number,
title: String,
desc: String,
genre: String
})
.index({ title: 'text' })
.plugin(autoIncrement.mongoosePlugin));

As pointed out in the comments by #Chris Satchell
Adding the index to the mongoose schema solves this.
instead of:
mongoose.model('Job', new Schema({
id: Number,
title: String,
desc: String,
genre: String
})
do this:
mongoose.model('Job', new Schema({
id: Number,
title: { type: String, text: true },
desc: String,
genre: String
})
even though I still had .index({ title: 'text' }) on the Schema, you still need to define the { type: String, text: true } on the property.

compound index not working?

I am trying to create a compound index using mongoose:
var ProjectSchema = new mongoose.Schema({
name: {type: String, required: true},
user: {type: mongoose.Schema.ObjectId, ref: 'User', required: true}
});
ProjectSchema.index({user: 1, name: 1}, {unique: true});
after that I dropped the old database in mongo
db.dropDatabase()
but I still can insert multiple documents with the same name and user id. why?
the index that it created shows in mongo as
> db.projects.getIndexes();
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "mydatabase.projects",
"name" : "_id_"
}
]

This is the pure mongo console function and it works,
Click Here for more detail. This is not descibe in mongoose's API.
I think it might be work.
db.collection.ensureIndex( { a: 1 }, { unique: true, dropDups: true } )

Actually your index does not appear to have been created. You are showing just the default primary key. Your output from .getIndexes() should be more like:
> db.projects.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "project.projects",
"name" : "_id_"
},
{
"v" : 1,
"key" : {
"user" : 1,
"name" : 1
},
"unique" : true,
"ns" : "project.projects",
"name" : "user_1_name_1",
"background" : true,
"safe" : null
}
]
There might be something up in your code, but this works for me:
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/project');
var db = mongoose.connection;
var Schema = mongoose.Schema;
var UserSchema = new Schema({
name: { type: String, required: true },
info: String
});
var ProjectSchema = new Schema({
name: { type: String, required: true},
user: { type: Schema.ObjectId, ref: 'User', required: 'true'}
});
ProjectSchema.index({ user: 1, name: 1}, { unique: true });
var User = mongoose.model( "User", UserSchema );
var Project = mongoose.model( "Project", ProjectSchema );
var user = new User({ name: "me" });
user.save(function(){
var project = new Project({ name: "Project1", user: user._id });
project.save(function(err, project, numAffected){
if (err) { // Should be fine
console.log(err);
}
console.log("First project created");
User.findOne({}).exec(function(err, user) {
console.log(user._id);
var project = new Project({ name: "Project1", user: user._id });
project.save(function(err, project, numAffected){
if (err) {
console.log(err); // Expect a problem here
}
console.log({ project: project, num: numAffected });
});
});
});
});

I had the exact same problem and this Github issue explained what was happening.
Firstly, compound indexes are only created after ensureIndex() is called. The problem for me is that I was using an import script that would drop my database and re-create it. ensureIndex() is not called until the server is restarted, so the compound index was not re-created after this.
The solution for me was to not drop my database in my import script, but instead iterate through my models and remove all the documents. That way the data was destroyed, but the indexes remained and hence the compound index would work.

I just had this problem, the compound index was not created at startup and I was checking the mongo logs, I could see that it was starting the build of the index but nothing was created, no errors etc...
Then I tried to manually create the index - in mongo console - and here I got an error (duplicate error in my case), so I removed the duplicates and I was able to create the index. I don't know why this was not popping up on my mongo logs.
Mongo v4