I've spent a day going through Mongoose documentation and posts on here and while I'm almost certain I have the code as prescribed in all those solved posts I simply cannot get my code to work. :(
The schema is relatively complex but by no means as complex as I've seen:
const hrUserSchema = new mongoose.Schema({
username: String,
displayName: String,
school: String,
leaveAuthoriser: String,
holidayLeft: Number,
holidayAllowed: Number,
discretionaryDays: Number,
emailAddress: String,
termTimeOnly: Boolean,
noOfAdditionalContractedDays: Number,
noOfAdditionalContractedDaysLeft: Number,
teachingStaff: Boolean,
employStartDate: String,
notes: String,
loginAttempts: Number,
lineManages: [{
staffName: String
}],
bookedHoliday: [{
dateEntered: String,
startDate: String,
endDate: String,
daysTaken: Number,
status: String,
approvedBy: String,
leaveType: String,
declineReason: String
}],
additionalContractedDays: [{
acdDateEntered: String,
acdStartDate: String,
acdEndDate: String,
acdDaysTaken: Number,
acdStatus: String,
acdApprovedBy: String,
acdDeclineReason: String
}],
perfMan: [{
year: Number,
selfReview: {
sref1: String,
sref2: String,
sref3: String,
sref4: String,
sref5: String,
status: String,
dateStarted: String,
dateCompleted: String,
signedOff: Boolean
},
stage1: {
objectives : [
{
objective: String,
objectiveLinkToSchoolTeam: String,
objectiveProgress: Boolean
}
],
personalDevelopment: String,
resourcesTraining: String,
appraiserSignOff: Boolean,
appraiseeSignOff: Boolean
},
stage2: {
feedback: String,
appraiserSignOff: Boolean,
appraiseeSignOff: Boolean
},
stage3: {
feedback: String,
appraiserSignOff: Boolean,
appraiseeSignOff: Boolean
}
}]
});
Basically I want to update perfMan.stage1.objectives.objectiveProgress, example of what data in that might look like is:
"perfMan" : [
{
"_id" : ObjectId("60cb502631dcea3eaaae6853"),
"selfReview" : {
"sref1" : "I have no strength",
"sref2" : "No developments",
"sref3" : "None I'm brill",
"sref4" : "The department has no aims",
"signedOff" : true
},
"stage1" : {
"objectives" : [
{
"_id" : ObjectId("60cb502631dcea3eaaae6854"),
"objective" : "Objective is pants",
"objectiveLinkToSchoolTeam" : "I hate objectives!",
"objectiveProgress" : false
},
{
"_id" : ObjectId("60cb502631dcea3eaaae6855"),
"objective" : "My second Objectoves",
"objectiveLinkToSchoolTeam" : "My Second reasons",
"objectiveProgress" : false
}
],
"personalDevelopment" : "My personal Development",
"resourcesTraining" : "My Resources"
},
"stage2" : {
"feedback" : "Keep working HARD",
"appraiserSignOff" : true
},
"stage3" : {
"feedback" : "Yoy've done really well",
"appraiserSignOff" : true
},
"year" : NumberInt(2021)
}
]
I've read about and tried to implement this by using arrayFilters which to me seems to exactly what I want, I've checked my Mongoose version (hosted) and it's 4.4.6 so I'm easily running above 3.6 which I think is what's needed.
My current code looks like this, I've confirmed the find is getting the right data:
HRUser.findOneAndUpdate(
{"username": username},
{"$set": {"perfMan.$[perfMan].stage1.objectives.$[objectives].objectiveProgress" : true}},
{"arrayFilters" : [{ "perfMan._id": ObjectId("" + perfmanID + "") },{ "objectives._id": ObjectId("" + objectiveID + "") }]}
),
function (err, response) {
console.log(err)
console.log(response);
if (!err) {
res.send("Successfully updated staff member details.");
} else {
res.send(err);
}
};
If somebody could spot my obviously glaring error I would be for ever grateful!
Thanks
After not looking at a screen all weekend and reading through the documentation for the billionth time I decided to change tack! I can't see any limitations on the number of arrayFilters, or for that matter objectID use within them, and I'm 99% certain both object IDs are correct but, after changing the code to the below, basically using a mix of a positional operator $ based on the search of the same first objectID AND an arrayFilter element it's now working! Code below:
HRUser.findOneAndUpdate({
perfMan: {
$elemMatch: {
_id: ObjectId("" + perfManID + "")
}
}
},
{"$set": {"perfMan.$.stage1.objectives.$[objectives].objectiveProgress" : false}},
{"arrayFilters" : [{ "objectives._id": ObjectId("" + objectiveID + "") }]},
function (err, response) {
console.log(err)
console.log(response);
});
Related
I am trying to add the fancyItem to the below model. Whenever I try to add the next fancyITem I get below mongo db error:
MongoError: E11000 duplicate key error collection: natours.fancyitems index: books.name_1 dup key: { : null }\n
To avoid this I am trying to add the default values to the objects so that I don't get this duplicate error.
Kindly suggest better way to handle this issue!
const mongoose = require("mongoose");
const fancyItemSchema = new mongoose.Schema({
firstName: {
type: String,
required: [true, "Please enter fancyItem's first name"]
},
lastName: {
type: String
},
genre: {
type: String,
enum: [
"guidingLights",
"luminaries",
"mavericScientists",
"menOfLetters",
"theGrandPhilosophers",
"architectsOfTheFuture"
],
required: true
},
notableWork: {
type: String,
required: [true, "Please enter notable work"]
},
quotes: [
{
quote: {
type: String,
default: "There is no quote"
}
}
],
books: [
{
bookName: {
type: String,
sparse: true,
default: "There is no quote"
},
bookURL: {
type: String,
sparse: true,
default: "There is no quote"
}
}
],
videos: [
{
videoName: {
type: String,
maxlength: [
50,
"A video description must have less or equal to 50 characters"
],
sparse: true,
default: "There is no quote"
},
videoURL: {
type: String,
sparse: true,
default: "There is no quote"
}
}
],
courses: [
{
courseName: {
type: String,
sparse: true,
default: "There is no quote"
},
courseURL: {
type: String,
sparse: true,
default: "There is no quote"
},
platform: {
type: String,
sparse: true,
default: "There is no quote"
}
}
]
});
const FancyItem = mongoose.model("FancyItem", fancyItemSchema);
module.exports = FancyItem;
Error you have provided says that there's already a record with null as the name. In other words, you already have a book without a name.
The relevant documentation for this:
If a document does not have a value for the indexed field in a unique
index, the index will store a null value for this document. Because of
the unique constraint, MongoDB will only permit one document that
lacks the indexed field. If there is more than one document without a
value for the indexed field or is missing the indexed field, the index
build will fail with a duplicate key error.
You can combine the unique constraint with the sparse index to filter
these null values from the unique index and avoid the error.
Unique Indexes
Sparse indexes only contain entries for documents that have the
indexed field, even if the index field contains a null value.
Sparse Indexes
All of this was working yesterday so I'm really not sure whats wrong. I'v searched on SO and been through the docs and steps a number of times but I havnt got it to work. Any help much appreciated as always!
If I type db.movies.find({ '$text': { '$search': 'elephant' } }) into the shell, it returns a list of titles with 'elephant' in the title. But if I run the same search from my browser it crashes the node server and I get MongoError: text index required for $text query.
The error comes from the error response from the find():
Movie.find(
{ '$text': { '$search': 'elephants' } },
(err, movies) => {
if (err) throw err; // MongoError: text index required for $text query
if (!movies) {
res.json({
data: null,
success: false,
message: 'No movies data'
});
} else {
res.json({
success: true,
data: movies
});
}
})
The index on my DB looks like this:
{
"v" : 2,
"key" : {
"_fts" : "text",
"_ftsx" : 1
},
"name" : "title_text",
"ns" : "db.movies",
"weights" : {
"title" : 1
},
"default_language" : "english",
"language_override" : "language",
"textIndexVersion" : 3
}
I also have mongoose schemas defined and I'm using an auto increment plugin to increment a custom id field on each movie save. (Saving movies works, so I am connecting to the DB ok)
mongoose.model('Job', new Schema({
id: Number,
title: String,
desc: String,
genre: String
})
.index({ title: 'text' })
.plugin(autoIncrement.mongoosePlugin));
As pointed out in the comments by #Chris Satchell
Adding the index to the mongoose schema solves this.
instead of:
mongoose.model('Job', new Schema({
id: Number,
title: String,
desc: String,
genre: String
})
do this:
mongoose.model('Job', new Schema({
id: Number,
title: { type: String, text: true },
desc: String,
genre: String
})
even though I still had .index({ title: 'text' }) on the Schema, you still need to define the { type: String, text: true } on the property.
I have this error i cannot seem to understand what I'm doing wrong.
In my database i have an object called Question, questions have a reference to Subject and to User. When I'm trying to post to Question i get a strange error.
E11000 duplicate key error index: codenoname.questions.$subject.name_1 dup key: { : null }
My Question schema:
var questionSchema = mongoose.Schema({
title: { type : String , required : true},
text: { type : String , required : true},
subject: {type:String, ref: 'Subject', required: true},
createdBy: {type: String, ref:'User', required: true},
difficulty: { type : String , required : true},
alternatives: [{alternative: {type:String, required:true}, isCorrect: {type:Boolean, required:true}}]
});
and my Subject
var subjectSchema = mongoose.Schema({
name: { type : String , required : true, unique:true}
});
Save method:
var question = new Question(
{title: title,
text: text,
subject: ObjectId(subject),
difficulty: difficulty,
createdBy: id,
alternatives:alternatives
});
question.save( function(err, newQuestion) {
if(err) {
res.status(400).json({err:err});
} else {
res.status(200).json({status:"Question added"});
}
});
What i have tried
Delete all Questions, then I can post, but just one...
Remove the reference and just keep it as a string. No difference.
Restarted the server a few times.
Try removing the unique: true from subject. I think questionSchema inherits the unique property and once you try to save two different questions with the same subject, you'll get duplicate key.
Follow these steps:
Remove unique: true from your model
Find the index name by typing db.questions.getIndexes() in your terminal.
Remove it by typing db.questions.dropIndex(name) where name is the "name"-property from step 2
Example from my database where i'll remove the unique-property from usernames:
> db.accounts.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "passport_local_mongoose_express4.accounts"
},
{
"v" : 1,
"unique" : true,
"key" : {
"email" : 1
},
"name" : "email_1",
"ns" : "passport_local_mongoose_express4.accounts",
"background" : true
},
{
"v" : 1,
"unique" : true,
"key" : {
"username" : 1
},
"name" : "username_1",
"ns" : "passport_local_mongoose_express4.accounts",
"background" : true
}
]
> db.accounts.dropIndex('username_1')
I wish to retrieve several information from my User model that looks like this:
var userSchema = new mongoose.Schema({
email: { type: String, unique: true, lowercase: true },
password: String,
created_at: Date,
updated_at: Date,
genre : { type: String, enum: ['Teacher', 'Student', 'Guest'] },
role : { type: String, enum: ['user', 'admin'], default: 'user' },
active : { type: Boolean, default: false },
profile: {
name : { type: String, default: '' },
headline : { type: String, default: '' },
description : { type: String, default: '' },
gender : { type: String, default: '' },
ethnicity : { type: String, default: '' },
age : { type: String, default: '' }
},
contacts : {
email : { type: String, default: '' },
phone : { type: String, default: '' },
website : { type: String, default: '' }
},
location : {
formattedAddress : { type: String, default: '' },
country : { type: String, default: '' },
countryCode : { type: String, default: '' },
state : { type: String, default: '' },
city : { type: String, default: '' },
postcode : { type: String, default: '' },
lat : { type: String, default: '' },
lng : { type: String, default: '' }
}
});
In Homepage I have a filter for location where you can browse Users from Country or City.
All the fields contains also the number of users in there:
United Kingdom
All Cities (300)
London (150)
Liverpool (80)
Manchester (70)
France
All Cities (50)
Paris (30)
Lille (20)
Nederland
All Cities (10)
Amsterdam (10)
Etc...
This in the Homepage, then I have also the Students and Teachers pages where I wish to have information only about how many teachers there are in those Countries and Cities...
What I'm trying to do is to create a query to MongoDB to retrieve all these information with a single query.
At the moment the query looks like this:
User.aggregate([
{
$group: {
_id: { city: '$location.city', country: '$location.country', genre: '$genre' },
count: { $sum: 1 }
}
},
{
$group: {
_id: '$_id.country',
count: { $sum: '$count' },
cities: {
$push: {
city: '$_id.city',
count: '$count'
}
},
genres: {
$push: {
genre: '$_id.genre',
count: '$count'
}
}
}
}
], function(err, results) {
if (err) return next();
res.json({
res: results
});
});
The problem is that I don't know how to get all the information I need.
I don't know how to get the length of the total users in every Country.
I have the users length for each Country.
I have the users length for each city.
I don't know how to get the same but for specific genre.
Is it possible to have all these information with a single query in Mongo?
Otherwise:
Creating few promises with 2, 3 different requests to Mongo like this:
getSomething
.then(getSomethingElse)
.then(getSomethingElseAgain)
.done
I'm sure it would be easier storing every time specified data but: is it good for performance when there are more than 5000 / 10000 users in the DB?
Sorry but I'm still in the process of learning and I think these things are crucial to understand MongoDB performance / optimisation.
Thanks
What you want is a "faceted search" result where you hold the statistics about the matched terms in the current result set. Subsequently, while there are products that "appear" to do all the work in a single response, you have to consider that most generic storage engines are going to need multiple operations.
With MongoDB you can use two queries to get the results themselves and another to get the facet information. This would give similar results to the faceted results available from dedicated search engine products like Solr or ElasticSearch.
But in order to do this effectively, you want to include this in your document in a way it can be used effectively. A very effective form for what you want is using an array of tokenized data:
{
"otherData": "something",
"facets": [
"country:UK",
"city:London-UK",
"genre:Student"
]
}
So "factets" is a single field in your document and not in multiple locations. This makes it very easy to index and query. Then you can effectively aggregate across your results and get the totals for each facet:
User.aggregate(
[
{ "$unwind": "$facets" },
{ "$group": {
"_id": "$facets",
"count": { "$sum": 1 }
}}
],
function(err,results) {
}
);
Or more ideally with some criteria in $match:
User.aggregate(
[
{ "$match": { "facets": { "$in": ["genre:student"] } } },
{ "$unwind": "$facets" },
{ "$group": {
"_id": "$facets",
"count": { "$sum": 1 }
}}
],
function(err,results) {
}
);
Ultimately giving a response like:
{ "_id": "country:FR", "count": 50 },
{ "_id": "country:UK", "count": 300 },
{ "_id": "city:London-UK", "count": 150 },
{ "_id": "genre:Student": "count": 500 }
Such a structure is easy to traverse and inspect for things like the discrete "country" and the "city" that belongs to a "country" as that data is just separated consistently by a hyphen "-".
Trying to mash up documents within arrays is a bad idea. There is a BSON size limit of 16MB to be respected also, from which mashing together results ( especially if you are trying to keep document content ) is most certainly going to end up being exceeded in the response.
For something as simple as then getting the "overall count" of results from such a query, then just sum up the elements of a particular facet type. Or just issue your same query arguments to a .count() operation:
User.count({ "facets": { "$in": ["genre:Student"] } },function(err,count) {
});
As said here, particularly when implementing "paging" of results, then the roles of getting "Result Count", "Facet Counts" and the actual "Page of Results" are all delegated to "separate" queries to the server.
There is nothing wrong with submitting each of those queries to the server in parallel and then combining a structure to feed to your template or application looking much like the faceted search result from one of the search engine products that offers this kind of response.
Concluding
So put something in your document to mark the facets in a single place. An array of tokenized strings works well for this purpose. It also works well with query forms such as $in and $all for either "or" or "and" conditions on facet selection combinations.
Don't try and mash results or nest additions just to match some perceived hierarchical structure, but rather traverse the results received and use simple patterns in the tokens. It's very simple to
Run paged queries for the content as separate queries to either facets or overall counts. Trying to push all content in arrays and then limit out just to get counts does not make sense. The same would apply to a RDBMS solution to do the same thing, where paging result counts and the current page are separate query operations.
There is more information written on the MongoDB Blog about Faceted Search with MongoDB that also explains some other options. There are also articles on integration with external search solutions using mongoconnector or other approaches.
I have the following model:
var Customer = mongoose.model('Customer', {
firstname : String,
lastname : String,
phone : String,
street : String,
city : String,
state : String,
zip : String,
fixed : Boolean,
readings: [
{
reading: Number,
date: String
}],
billing: {
charges: [Number],
payments: [Number]
}
});
Im trying to add individual charges and payments to billing.
I am using the following code but doesn't seem to be updating....
Customer.update({_id: req.params.cust_id},
{ billing: { $push:{charges: 100}, $push:{payments:50}}},
{upsert:true},
function(err, customer){
if(err){
console.log(err);
}
else{
Customer.findOne({
_id : req.params.cust_id
}, function(err, customer) {
if (err)
res.send(err);
res.json(customer);
});
}
}
)
IF anyone has any suggestions they would be greatly appreciated. Ive found a lot of examples of $push but all seem to be on the top level and not a subdoc.
Use dot notation for the fields, which are an argument to $push
Customer.update({_id: req.params.cust_id},
{ $push: { "billing.charges": 100, "billing.payments": 50} },
{upsert:true},