Trying to create MongoDB indexes. Using the Mongoose ODM and in my schema definition below I have the username field set to a unique index. The collection and document all get created properly, it's just the indexes that aren't working. All the documentation says that the ensureIndex command should be run at startup to create any indexes, but none are being made. I'm using MongoLab for hosting if that matters. I have also repeatedly dropped the collection. What is wrong.
var schemaUser = new mongoose.Schema({
username: {type: String, index: { unique: true }, required: true},
hash: String,
created: {type: Date, default: Date.now}
}, { collection:'Users' });
var User = mongoose.model('Users', schemaUser);
var newUser = new Users({username:'wintzer'})
newUser.save(function(err) {
if (err) console.log(err);
});
Hook the 'index' event on the model to see if any errors are occurring when asynchronously creating the index:
User.on('index', function(err) {
if (err) {
console.error('User index error: %s', err);
} else {
console.info('User indexing complete');
}
});
Also, enable Mongoose's debug logging by calling:
mongoose.set('debug', true);
The debug logging will show you the ensureIndex call it's making for you to create the index.
In my case I had to explicitly specify autoIndex: true:
const User = new mongoose.Schema({
name: String,
email: String,
password: String
},
{autoIndex: true}
)
Mongoose declares "if the index already exists on the db, it will not be replaced" (credit).
For example if you had previous defined the index {unique: true} but you want to change it to {unique: true, sparse: true} then unfortunately Mongoose simply won't do it, because an index already exists for that field in the DB.
In such situations, you can drop your existing index and then mongoose will create a new index from fresh:
$ mongo
> use MyDB
> db.myCollection.dropIndexes();
> exit
$ restart node app
Beware that this is a heavy operation so be cautious on production systems!
In a different situation, my indexes were not being created, so I used the error reporting technique recommended by JohnnyHK. When I did that I got the following response:
E11000 duplicate key error collection
This was because my new index was adding the constraint unique: true but there were existing documents in the collection which were not unique, so Mongo could not create the index.
In this situation, I either need to fix or remove the documents with duplicate fields, before trying again to create the index.
When I hooked the index event on the model that wasn't working, I started getting an error on the console that indicated "The field 'retryWrites' is not valid for an index specification." The only place in my app that referenced 'retryWrites' was at the end of my connection string. I removed this, restarted the app, and the index rebuild was successful. I put retryWrites back in place, restarted the app, and the errors were gone. My Users collection (which had been giving me problems) was empty so when I used Postman to make a new record, I saw (with Mongo Compass Community) the new record created and the indexes now appear. I don't know what retryWrites does - and today was the first day I used it - but it seemed to be at the root of my issues.
Oh, and why did I use it? It was tacked onto a connection string I pulled from Mongo's Atlas Cloud site. It looked important. Hmm.
It might be solve your problem
var schema = mongoose.Schema({
speed: Number,
watchDate: Number,
meterReading: Number,
status: Number,
openTrack: Boolean,
});
schema.index({ openTrack: 1 });
As you can see in mongoose documentations https://mongoosejs.com/docs/guide.html#indexes, we need define schema.index to create our indexes. Take a look in code below to test:
Note, after update your schema restart the server to check.
const schemaUser = new mongoose.Schema(
{
username: {
type: String,
required: true,
index: true,
unique: true,
dropDups: true,
},
hash: String,
created: {
type: Date,
default: Date.now,
},
},
{
autoCreate: true, // auto create collection
autoIndex: true, // auto create indexes
}
)
// define indexes to be create
schemaUser.index({ username: 1 })
const User = mongoose.model('Users', schemaUser)
const newUser = new Users({ username: 'wintzer' })
newUser.save(function (err) {
if (err) console.log(err)
})
Connections that set "readPreference" to "secondary" or "secondaryPreferred" may not opt-in to the following connection options: autoCreate, autoIndex.
Check your readPreference option in the mongoose connection
I had this problem when writing a data import command-line utility. After execution end, some indexes were created, some were not.
The solution was to call await model.ensureIndexes() before terminating the script. It worked regardless to autoIndex option value.
Сheck that the mongoose connect options do not specify :
autoIndex: false
Related
So I have a situation where I need to delete elements in an array of reference / ObjectIds, but the delete condition will be based on a field in the reference.
For example, I have schemas like the following:
const UserSchema = new mongoose.Schema({
firstName: String,
lastName: String,
homeFeeds:[{type: Schema.Types.ObjectId, requried: true, ref: "Activity"}];
}); // User , is the referenece name
const ActivitySchema = new mongoose.Schema({
requester: {type: Schema.Types.ObjectId, requried: true, ref: "User"},
message: String,
recipient: {type: Schema.Types.ObjectId, requried: true, ref: "User"},
}) // Activity, is the reference name
Now I need to delete some of the homeFeeds for a user, and the ones that should be deleted need to be by certain requester. That'll require the homeFeeds (array of 'Activity's) field to be populated first, and then update it with the $pull operator, with a condition that the Activity requester matches a certain user.
I do not want to read the data first and do the filtering in Nodejs/backend code, since the array can be very long.
Ideally I need something like:
await User.find({_id: ID})
.populate("homeFeeds", "requester")
.updateMany({
$pull: {
homeFeeds.requester: ID
}
});
But it does not work, Id really appreciate if anyone can help me out with this?
Thanks
MongoDB doesn't support $lookup in update as of version v6.0.1.
MongoServerError: $lookup is not allowed to be used within an update.
Though, this doesn't have to do with Mongoose's populate as populate doesn't depend on $lookup and fires additional queries to get the results. Have a look at here. Therefore, even if, you could achieve what you intend, that is avoiding fetching a large array on nodejs/backend, using mongoose will do the same thing for you behind the scenes which defeats your purpose.
However you should raise an issue at Mongoose's official github page and expect a response.
I am working on a mongoose schema similar to this:
const actionSchema = {
actions: {
type: [{
actionName: {
type: String,
required: true
},
count: {
type: Number,
default: 0,
required: true
},
users: [{
type: Schema.Types.ObjectId,
ref: 'User'
}]
}]
}};
It is a nested schema of a post schema.
Here, actions are dynamically generated and number of people does that action are maintained by count and their identity is maintained by users array.
As you see, actions is an array of objects which further contain users array.
I want to check if a provided user id is present in any of the action object and then remove it from array and also reduce the count.
Being totally new to mongoose and mongodb, one simple way I see is to find the post using Post.findById() which has to be updated, run js loops, update the post and call .save(). But it can be very costly when users array has thousands of user ids.
I tried .update() but can't understand how to use it in this case.
How about adding a method to the Post Model (like postSchema.methods.removeUserAction)? This gives access to document from this and allows to update the document and thus call .save(). Does it loads the full document to the client node application?
So please suggest the right way.
Thank you.
You should simplify your model, for example
// Model - Actions Model
const actionSchema = {
actionName: {
type: String,
required: true
},
user: {
type: Schema.Types.ObjectId,
ref: 'User'
}
};
And you can easily get the total actions via Model.count(), get specific action count with Model.count({ actionName: 'action name'}), and removing entries with Model.delete(condition). Unless there's a reason why you have it modeled this way.
I'm currently using mongodb with mongoose.
When I connect to the database via the terminal and run the command:
db.locphoto.find({})
It successfully returns the list of items that I'm looking for.
Alternatively, on my application I do the following and unfortunately it constantly returns []. I was hoping someone could show me the way to see which mongodb query is generated so that I'm able to check if it is correctly generating db.locphoto.find({}).
My controller code is as follows:
var LocPhoto = require('../models/locPhoto');
module.exports.getGalleryPictures = function(req, res) {
LocPhoto.find({}, function(err, results) {
res.json(results);
});
}
And my model code is as follows:
var mongoose = require('mongoose');
var locPhotoSchema = mongoose.Schema({
challengeId: String,
image: String,
main: Number,
});
module.exports = mongoose.model('LocPhoto', locPhotoSchema);
I'd really appreciate if someone knows how to see the command that is being generated so that I can check this better in the future, since I've had this issue a few times already, it's usually to do with capital letters etc.
You're not properly creating the schema , you have to use new keyword
var locPhotoSchema = new mongoose.Schema({
challengeId: String,
image: String,
main: Number,
});
I'm trying to enforce uniqueness by field on a mongoose model, in an existing database.
const UserSchema = new mongoose.Schema({
email: {
type: String,
required: true,
unique: true,
}
})
Then when I create a user with an email that is already assigned, it passes, and I have 2 users with the same email.
I have read all related SO answers > 2015 and all about dropDups, which is deprecated.
I think I fixed this issue by manually running
db.users.createIndex({email:1}, {unique:true})
However this obviously becomes cumbersome both in development and in production, especially considering the mongoose docs state that the attribute takes care of it:
unique: boolean, whether to define a unique index on this property.
Can someone provide a clear solution to enforcing uniqueness by field on a Mongoose model, considering an existing database and a fresh collection? Thanks
Two user has been created before unique index has been created.So you can insert two users with same email.You can try code below, it make sure all index has been created:
UserModel.on('index', function(error) {
const user1 = new UserModel({
email: '3489'
});
const user2 = new UserModel({
email: '3489'
});
user1.save(console.log);
user2 .save(console.log);
});
Not sure when this issue cropped up but I am not able to fetch items from mongo consistently. I have 4000+ items in the db. Here's the schema.
var Order = new Schema({
code: {
type: String,
unique: true
},
...
});
Now run some queries:
Order.find().exec(function(err, orders) {
console.log(orders.length); // always 101
})
Order.find().limit(100000).exec(function(err, orders) {
console.log(orders.length); // varies, sometimes 1150, 1790, 2046 - never more
})
Now if I remove the 'unique: true' from schema it will always return the total amount:
Order.find().exec(function(err, orders) {
console.log(orders.length); // always 4213 (correct total)
})
Any idea as to why this behavior occurs? afaik the codes are all unique (orders from a merchant). This is tested on 3.8.6, 3.8.8
Ok issue was indeed unique index being not there/corrupted. I a guilty of adding the unique index later on in the game and probably had some dups already which prevented Mongo from creating indexes.
I removed the duplicates and then in the mongo shell did this:
db.orders({name: 1}, {unique: true, dropDubs: true});
I would think the above would remove dups but it would just die because of the dups. I am sure there is a shell way to do this but I just did it with some js code then ran the above to recreate the index which can be verified with:
db.orders.getIndexes()