Develop Reference

MongoDB: How can populate reference, and delete element in array after based on the ID of the reference

So I have a situation where I need to delete elements in an array of reference / ObjectIds, but the delete condition will be based on a field in the reference.
For example, I have schemas like the following:
const UserSchema = new mongoose.Schema({
firstName: String,
lastName: String,
homeFeeds:[{type: Schema.Types.ObjectId, requried: true, ref: "Activity"}];
}); // User , is the referenece name
const ActivitySchema = new mongoose.Schema({
requester: {type: Schema.Types.ObjectId, requried: true, ref: "User"},
message: String,
recipient: {type: Schema.Types.ObjectId, requried: true, ref: "User"},
}) // Activity, is the reference name
Now I need to delete some of the homeFeeds for a user, and the ones that should be deleted need to be by certain requester. That'll require the homeFeeds (array of 'Activity's) field to be populated first, and then update it with the $pull operator, with a condition that the Activity requester matches a certain user.
I do not want to read the data first and do the filtering in Nodejs/backend code, since the array can be very long.
Ideally I need something like:
await User.find({_id: ID})
.populate("homeFeeds", "requester")
.updateMany({
$pull: {
homeFeeds.requester: ID
}
});
But it does not work, Id really appreciate if anyone can help me out with this?
Thanks

MongoDB doesn't support $lookup in update as of version v6.0.1.
MongoServerError: $lookup is not allowed to be used within an update.
Though, this doesn't have to do with Mongoose's populate as populate doesn't depend on $lookup and fires additional queries to get the results. Have a look at here. Therefore, even if, you could achieve what you intend, that is avoiding fetching a large array on nodejs/backend, using mongoose will do the same thing for you behind the scenes which defeats your purpose.
However you should raise an issue at Mongoose's official github page and expect a response.

How to best reference another schema in a Mongoose schema property?

I am defining a mongoose User schema.
It will contain a reference to an Address schema:
const AddressSchema = mongoose.Schema({
street: String,
city: String,
zip: String,
});
const UserSchema = mongoose.Schema({
...
});
I see from the docs I can do both:
const UserSchema = mongoose.Schema({
name: String,
address: {
type: AddressSchema,
},
...
});
or
const Address = mongoose.model("Address", AddressSchema);
const UserSchema = mongoose.Schema({
name: String,
address: {
{
type: mongoose.Schema.Types.ObjectId,
ref: "Address"
}
},
...
});
I suppose the only difference is that when querying a User document in the first case I always get address populated, while in the second one I will have to populate() it...
I'm asking just to be sure this is the difference, and if there are subtler ones I should take care of... From the docs I couldn't tell...

It really depends on the type of queries you'll need.
If you only need the inner ones in connection to the outer, like the user's private account actions that are supposed to be shown only to him - the subdocument wat is a better way since it doesn't need to populate.
If you will need to show a list of all actions together it'll be a pain to loop over each user to get their actions, then sum it up, and then you'll have that array, so the better way here is what in SQL called "normalization", you'll end up with one collection of all users so you will be able to display it, but each action will have a reference to the user so you can filter it by the specific user

Users schema with poking other users schema in node.js

I am making an application in which a user can poke other users. Here is the code for the schema designs I have considered. The first is using only a users schema:
const userSchema = new Schema({
name: { type : String},
pokes: [{ type : Schema.Types.ObjectId, ref: 'Users' ,default:null}],
});
Another way is using a pokes schema. Here I'm storing the object ids of the pokes schema in the users schema.
const pokesSchema = new Schema({
from_user_id: { type : Schema.Types.ObjectId, ref: 'Users' ,default:null},
to_user_id: { type : Schema.Types.ObjectId, ref: 'Users' ,default:null},
});
const userSchema = new Schema({
name: { type : String},
pokes: [{ type : Schema.Types.ObjectId, ref: 'Pokes' ,default:null}],
});
In the third way I totally remove the relation between the two schemas:
const pokesSchema = new Schema({
from_user_id: { type : Schema.Types.ObjectId, ref: 'Users' ,default:null},
to_user_id: { type : Schema.Types.ObjectId, ref: 'Users' ,default:null},
});
const userSchema = new Schema({
name: { type : String},
});
In the second and third ways I can query for pokes easily.
I want to know which of the three is the best design and why. Also if userA pokes userB then it can be the case that userB can also poke back to userA. I'm learning node.js currently and am confused about the design in mongoDb.

Alright, so here's the best I can do. You briefly answered my question in my comment above but I'd like to point out it's important to think about what you are doing (or expect to be doing) more and how much more. That aside though, let's take a look at each schema.
const userSchema = new Schema({
name: { type : String},
pokes: [{ type : Schema.Types.ObjectId, ref: 'Users' ,default:null}],
});
When we look at this first one it seems inadequate for your needs. It's a collection of users who have a name and an array of pokes they have made. If we need to know who a user has poked then that's a really easy and fast query - it's right there under their name, search by name or _id and we're done! But what happens when you want to look up who has poked this user? You will need to query every single user and then search through every single pokes array for the original user. If we have m-many users and each has n-many pokes, that's doing m* n tests. Yikes. If m and n get big that's going to be a lot (think of the difference of 100 * 100 vs 10,000 * 10,000 or even more!). Even if you personally are not coding that search in your node then mongo is doing that search. So unless you're sure that looking up who has poked a user is going to be something that is pretty rare this is probably not a good option. Moving on:
const pokesSchema = new Schema({
from_user_id: { type : Schema.Types.ObjectId, ref: 'Users' ,default:null},
to_user_id: { type : Schema.Types.ObjectId, ref: 'Users' ,default:null},
});
const userSchema = new Schema({
name: { type : String},
pokes: [{ type : Schema.Types.ObjectId, ref: 'Pokes' ,default:null}],
});
Now we have a pokes schema, nice! If we wanted to do the search we discussed above we can instead query pokes directly based on to_user_id, and then if we need a name of all the users who initiated the pokes we can just query the users based on its _id. Not bad! We also still have the fast way to get the reverse, aka search for pokes a user has initiated, because there is still pokes in our user schema. What happens when a poke occurs, though? We have to update both schemas. So not only will we do a (relatively easy) insert into pokes, we will have to also update our pokes array of the user who did the poking. This might not be so bad, but what happens if one update fails? Now our data is inconsistent - users and pokes don't match. We're also doubling our updates every poke. This might not be a big deal, and if we're getting a user's pokes much more than we're poking then it might be an ok trade-off, but it becomes a little riskier because we've introduced somewhere we can be inconsistent. Alright, last one:
const pokesSchema = new Schema({
from_user_id: { type : Schema.Types.ObjectId, ref: 'Users' ,default:null},
to_user_id: { type : Schema.Types.ObjectId, ref: 'Users' ,default:null},
});
const userSchema = new Schema({
name: { type : String},
});
First, note that these schemas are still related - the pokes schema reference users. It's just not doubly-related like the last one. Anyway, now we've removed the pokes array from the user schema. Ok! We don't run the risk of having inconsistent data anymore, noice! We're also not doing two updates for every poke, toit! The trade-off is now when we want to get the list of users a user has poked we have to do a similar query to the one we did above when we wanted to get a list of users a user has been poked by. Which isn't so bad, but is certainly not as fast as having the pokes array already sitting there and waiting.
In my opinion, unless you're searching for who users have poked (and not been poked by) significantly more often than doing anything else this third scenario is best. The schemas make logical sense, you're not updating twice. It's what I would use. But as I said, it's very important to consider your particular need and design.
Hope that helps!

Setting up an array of collections in MongoDB using Node.js

I am creating a game where, as part of a collection named Game, I am trying to label one of the elements of it. The element in question is supposed to be an array of userNames from another collection. I can't seem to figure out how to access that. Here is what I have in the games collection:
var mongoose = require('mongoose');
var schema = mongoose.Schema;
var ObjectId = schema.ObjectId;
module.exports.Game = mongoose.model('Game', new schema({
id: ObjectId,
gameRoomName: { type: String, required: '{PATH} is required.' },
players: { }
}));
The users collection:
var mongoose = require('mongoose');
var schema = mongoose.Schema;
module.exports.users = mongoose.model('Users', new schema({
userName: {type: String, required: '{PATH} is required.'}
}));
Basically, the usernames for a game will be saved in the Users schema. Then, I'd like to access that and insert it into the Game schema in the players space. I'm imagining it to be something like {type: collection.users}, however, that doesn't seem to be doing the trick.

You can store players as the array of references to Users model
module.exports.Game = mongoose.model('Game', new schema({
.
.
players: [{type: Schema.Types.ObjectId, ref: 'Users'}],
)}
Access later by:
Game.find()
// filter 'players' field
.select('players')
// populate players with only 'username' field
.populate('players', 'username')
.exec(function(err, username) {
// anything with players
});
Long story. You will be good to go after finishing the article

There's a few ways to solve the situation here...but ultimately I think it depends on what data you have readily available at the time you want to add the users into the game object and also how you want to retrieve the data when you need it.
If you have all the usernames cached, whether as objects or just the username itself, it would be more efficient to just add them into the game object.
Example:
var usernamesExample = ["Mike", "Ike", "Clara", "Joe"];
Game.findById(gameIdExample, function(error, foundGame){
// handle errors/checks/etc.
foundGame.players = usernamesExample;
foundGame.save();
})
I personally think this approach is best performance wise. Then again it might not work for your situation, in which case I would need further clarification into how you obtain the username data for the game.

Mongoose Not Creating Indexes

Trying to create MongoDB indexes. Using the Mongoose ODM and in my schema definition below I have the username field set to a unique index. The collection and document all get created properly, it's just the indexes that aren't working. All the documentation says that the ensureIndex command should be run at startup to create any indexes, but none are being made. I'm using MongoLab for hosting if that matters. I have also repeatedly dropped the collection. What is wrong.
var schemaUser = new mongoose.Schema({
username: {type: String, index: { unique: true }, required: true},
hash: String,
created: {type: Date, default: Date.now}
}, { collection:'Users' });
var User = mongoose.model('Users', schemaUser);
var newUser = new Users({username:'wintzer'})
newUser.save(function(err) {
if (err) console.log(err);
});

Hook the 'index' event on the model to see if any errors are occurring when asynchronously creating the index:
User.on('index', function(err) {
if (err) {
console.error('User index error: %s', err);
} else {
console.info('User indexing complete');
}
});
Also, enable Mongoose's debug logging by calling:
mongoose.set('debug', true);
The debug logging will show you the ensureIndex call it's making for you to create the index.

In my case I had to explicitly specify autoIndex: true:
const User = new mongoose.Schema({
name: String,
email: String,
password: String
},
{autoIndex: true}
)

Mongoose declares "if the index already exists on the db, it will not be replaced" (credit).
For example if you had previous defined the index {unique: true} but you want to change it to {unique: true, sparse: true} then unfortunately Mongoose simply won't do it, because an index already exists for that field in the DB.
In such situations, you can drop your existing index and then mongoose will create a new index from fresh:
$ mongo
> use MyDB
> db.myCollection.dropIndexes();
> exit
$ restart node app
Beware that this is a heavy operation so be cautious on production systems!
In a different situation, my indexes were not being created, so I used the error reporting technique recommended by JohnnyHK. When I did that I got the following response:
E11000 duplicate key error collection
This was because my new index was adding the constraint unique: true but there were existing documents in the collection which were not unique, so Mongo could not create the index.
In this situation, I either need to fix or remove the documents with duplicate fields, before trying again to create the index.

When I hooked the index event on the model that wasn't working, I started getting an error on the console that indicated "The field 'retryWrites' is not valid for an index specification." The only place in my app that referenced 'retryWrites' was at the end of my connection string. I removed this, restarted the app, and the index rebuild was successful. I put retryWrites back in place, restarted the app, and the errors were gone. My Users collection (which had been giving me problems) was empty so when I used Postman to make a new record, I saw (with Mongo Compass Community) the new record created and the indexes now appear. I don't know what retryWrites does - and today was the first day I used it - but it seemed to be at the root of my issues.
Oh, and why did I use it? It was tacked onto a connection string I pulled from Mongo's Atlas Cloud site. It looked important. Hmm.

It might be solve your problem
var schema = mongoose.Schema({
speed: Number,
watchDate: Number,
meterReading: Number,
status: Number,
openTrack: Boolean,
});
schema.index({ openTrack: 1 });

As you can see in mongoose documentations https://mongoosejs.com/docs/guide.html#indexes, we need define schema.index to create our indexes. Take a look in code below to test:
Note, after update your schema restart the server to check.
const schemaUser = new mongoose.Schema(
{
username: {
type: String,
required: true,
index: true,
unique: true,
dropDups: true,
},
hash: String,
created: {
type: Date,
default: Date.now,
},
},
{
autoCreate: true, // auto create collection
autoIndex: true, // auto create indexes
}
)
// define indexes to be create
schemaUser.index({ username: 1 })
const User = mongoose.model('Users', schemaUser)
const newUser = new Users({ username: 'wintzer' })
newUser.save(function (err) {
if (err) console.log(err)
})

Connections that set "readPreference" to "secondary" or "secondaryPreferred" may not opt-in to the following connection options: autoCreate, autoIndex.
Check your readPreference option in the mongoose connection

I had this problem when writing a data import command-line utility. After execution end, some indexes were created, some were not.
The solution was to call await model.ensureIndexes() before terminating the script. It worked regardless to autoIndex option value.

Сheck that the mongoose connect options do not specify :
autoIndex: false