Develop Reference

Mongoose populate returns an empty array | multiple levels of embedded documents

I am trying to populate my ChatRoom model with the User reference. However, it returns a ChatRoom object with only _ids where I expected usernames, as if I never applied populate on it.
Here is an extract of my ChatRoom model :
const ChatRoom = mongoose.model("ChatRoom", {
sender: {
type: mongoose.Schema.Types.ObjectId,
ref: "User",
},
roomname: { type: String, default: "new room" },
messages: [
{
messages: {
type: mongoose.Schema.Types.ObjectId,
ref: "Message",
},
meta: [
{
user: {
type: mongoose.Schema.Types.ObjectId,
ref: "User",
},
delivered: Boolean,
read: Boolean,
},
],
},
],
participants: [
{
user: { type: mongoose.Schema.Types.ObjectId, ref: "User" },
},
],
isPrivate: { type: Boolean, default: "false" },
});
My User model :
const User = mongoose.model("User", {
username: { required: true, unique: true, type: String },
avatar: Object,
token: String,
hash: String,
salt: String,
chatroom: {
type: mongoose.Schema.Types.ObjectId,
ref: "ChatRoom",
},
});
As this seems to be a recurrent issue, I tested several StackOverflow answers for my populate code :
Using populate('participants.user') and 'model: user' or just populate('participants.user'), same solution here:
const chatroom = await ChatRoom.findById(req.params.id)
.populate([
{
path: "participants.user",
model: "User",
},
])
.exec((err, user) => {
if (err) {
console.log("error", err);
} else {
console.log("Populated User " + user);
}
});
The console.log returns :
Populated User { _id: new ObjectId("62262b342e28298eb438d9eb"),
sender: new ObjectId("6225d86c9340237fe2a3f067"), roomname:
'Hosmeade', participants: [ { _id: new
ObjectId("6225d86c9340237fe2a3f067") } ], isPrivate: false,
messages: [], __v: 0 }
As if no populate method was ever applied. On the client side, I get an empty string.
Checking my documents aren't empty, this link mentions that Mongoose get problems with detecting referring model across multiple files but the solution doesn't work for me :
_id:6225d86c9340237fe2a3f067 username:"Berlioz" token:"rTCLAiU7jq3Smi3B"
hash:"wvJdqq25jYSaJjfiHAV4YRn/Yub+s1KHXzGrkDpaPus="
salt:"hZdiqIQQXGM1ryYK" avatar:Object
__v:0
If I remove the .exec(...) part, I get the info on the client side, but participants is still filled with only id :
chatroom response : Object { _id: "62262bb14e66d86fb8a041e8",
sender: "6225d86c9340237fe2a3f067", roomname: "Very secret room",
participants: (1) […], isPrivate: false, messages: [], __v: 0 }
I also tried with select: 'username' and get the same result as above :
const chatroom = await ChatRoom.findById(req.params.id).populate({
path: "participants.user",
select: "username",
});
Populating it "as an array"
Changing type of participants.user in my ChatRoom model into an Object (nothing changes)
If needed hereafter are my repos:
Client side and Backend
I run out of ideas on how to debbug my code. Any help would be great !

How to only get UserID from message.guild.members.fetch() in discord.js v12?

I'm trying to get a random winner for prizes and I would like to take the whole object variable and just sorta isolate the whole thing so i just get a list of the UserID's
Heres an example of what I get when I do console.log(message.guild.members.fetch()):
Promise {
Collection [Map] {
'123456789012345678' => GuildMember {
guild: [Guild],
user: [User],
joinedTimestamp: 1234567890123,
lastMessageID: '012345678901234567',
lastMessageChannelID: '012345678901234567',
deleted: false,
_roles: [Array]
},
'234567890123456789' => GuildMember {
guild: [Guild],
user: [User],
joinedTimestamp: 1234567890123,
lastMessageID: null,
lastMessageChannelID: null,
premiumSinceTimestamp: null,
deleted: false,
_roles: [Array]
},
'345678901234567890' => GuildMember {
guild: [Guild],
user: [ClientUser],
joinedTimestamp: 1234567890123,
lastMessageChannelID: null,
premiumSinceTimestamp: null,
deleted: false,
_roles: [Array]
}
}
}
I would like to only get the 123456789012345678 part of the '123456789012345678' => GuildMember {line

Use the keys method (inherited from Map):
message.guild.messages.fetch().then(members => {
const userIDs = [...members.keys()]
// Do something with the IDs here
})
With async/await:
const userIDs = [...(await message.guild.messages.fetch()).keys()]
// Do something with the IDs here

Node JS [Object Map] while writing object to file

I read many posts on stackoverflow and other tutorial sites for writing object/JSON to a file, but none of the solutions worked for me.
1) Code:
let messages = await fetchMessages()
console.log(messages) // Prints an object
fs.writeFileSync('./msgdata.json', messages , 'utf-8');
2) Also tried,
fs.writeFileSync('./msgdata.json', JSON.stringify(messages) , 'utf-8');
3) Also tried,
fs.writeFile(), but get same output as above.
msgdata.json: (For 1)
[object Map]
msgdata.json: (For 2 & 3)
{}
Can someone please point out what could be causing this?
## Output of console.log(messages) ##
OUTPUT contains 10 more such objects having different id's:
channel:
TextChannel {
type: 'text',
id: '424825532274253312',
name: 'request',
position: 69,
parentID: '397363224286473987',
permissionOverwrites: [Object],
topic: null,
nsfw: false,
lastMessageID: '427143105410760704',
guild: [Object],
messages: [Object],
_typing: Map {} },
id: '427142596817846272',
type: 'DEFAULT',
content: '**Select your emoji:** __***Group 9:***__',
author:
ClientUser {
id: '407083773537350272',
username: 'Bot',
discriminator: '9256',
avatar: '9c374e719ba2ab4e69fd577005b635bf',
bot: true,
lastMessageID: null,
lastMessage: null,
verified: true,
email: null,
localPresence: {},
_typing: Map {},
friends: Collection {},
blocked: Collection {},
notes: Collection {},
premium: null,
mfaEnabled: false,
mobile: null,
settings: [Object],
guildSettings: Collection {} },
member:
GuildMember {
guild: [Object],
user: [Object],
_roles: [Array],
serverDeaf: false,
serverMute: false,
selfMute: undefined,
selfDeaf: undefined,
voiceSessionID: undefined,
voiceChannelID: undefined,
speaking: false,
nickname: null,
joinedTimestamp: 1517127343434,
lastMessageID: null,
lastMessage: null },
pinned: false,
tts: false,
nonce: undefined,
system: false,
embeds: [],
attachments: Collection {},
createdTimestamp: 1521909131007,
editedTimestamp: null,
reactions:
Collection {
'taillow:417281639777959940' => [Object],
'shroomish:417281639899463680' => [Object],
'sableye:417281640197521419' => [Object],
'ralts:417281642735075329' => [Object],
'sentret:417281644001624076' => [Object],
'shuppet:417281644291162132' => [Object],
'torchic:417281647210397706' => [Object],
'snubbull:417281647692480522' => [Object],
'sunkern:417281647763783681' => [Object],
'slowpoke:417281648653107200' => [Object],
'teddiursa:417281649537974273' => [Object],
'sneasel:417281649613471747' => [Object],
'snorunt:417281649819123712' => [Object],
'surskit:417281650163056640' => [Object],
'qwilfish:417281654629859348' => [Object],
'shelgon:417281654730522624' => [Object] },
mentions:
MessageMentions {
everyone: false,
users: Collection {},
roles: Collection {},
_content: '**Select your emoji:** __***Group
9:***__',
_client: [Object],
_guild: [Object],
_members: null,
_channels: null },
webhookID: null,
hit: null,
_edits: [] },

This is because the object returned is a Map object (MDN). When you JSON.strigify a Map object, it always returns {}. There are two ways to get readable JSON that stores your map. The first is to loop over all the entries and create a JSON (taking in consideration the keys and values). That should be simple to achieve.
Another way is to create an array from the Map and then stringify it.
fs.writeFileSync('./msgdata.json', JSON.stringify([...myObject]) , 'utf-8');

You can define Map.prototype.toJSON which works, but it isn't standard and is discouraged. It is automatically called by JSON.stringify if it finds it.
Map.prototype.toJSON = function() {return [...this];};
console.log(JSON.stringify(messages));
A better way is to define your own replacer function (and reviver function too if you need to reconstruct map from JSON). Here I also wrap the map representing array in a special object to tell maps and ordinary arrays apart in JSON:
function replacer (key, value) {
if (value instanceof Map) {
return {
_type: "map",
map: [...value],
}
} else return value;
}
function reviver (key, value) {
if (value._type == "map") return new Map(value.map);
else return value;
}
let str = JSON.stringify(messages, replacer);
let msg = JSON.parse(str, reviver);
The error you are getting about circular structure means that some objects/arrays there contain references to itself. When you store messages you don't need to store all the information related to the channels etc and instead only store ids. So that means extending replacer for TextChannels, ClientUsers, GuildMembers etc:
if (value instanceof TextChannel) {
return {
_type: "TextChannel",
id: value.id,
};
}

var util = require('util');
fs.writeFileSync('./data.json', util.inspect(obj) , 'utf-8');
Ref. Write objects into file with Node.js

Loop through Snowflake array

I'm looking for a way to get data from this Collection.
The data looks like:
'0000000' => GuildMember {
guild:
Guild {
members: [Object],
id: '000000',
name: 'Zombie',
_rawVoiceStates: [Object] },
user:
User {
id: '0000000',
username: 'Orc',
_roles: [ '0000' ],
nickname: 'Orc',
joinedTimestamp: 00000,
lastMessageID: null },
'0000000' => GuildMember {
guild:
Guild {
members: [Object],
id: '000000',
name: 'Zombie',
_rawVoiceStates: [Object] },
user:
User {
id: '0000001',
username: 'Orc1',
_roles: [ '0000' ],
nickname: 'Orc',
joinedTimestamp: 00000,
lastMessageID: null },
_array: null,
_keyArray: null }
My current loop is:
var user;
for(var u in test.members){
user = test.members[u];
console.log("["+u+"] "+user.username);
}
It currently kicks back a TypeError: Cannot read property 'user' of null
I originally thought this the data was an array, but it's not according to the Discord.js docs, but I'm still not sure how to pull the username data from the collection.
Any help would be helpful.

Now i looked into the discord.js API and i think what u got todo is something like this (assuming test is your guild object):
test.members.forEach(function(guildMember, guildMemberId) {
console.log(guildMemberId, guildMember.user.username);
})
If that doesn't work try along the lines of:
var membersArray = test.members.array();
for(var guildMemberId in membersArray) {
console.log(guildMemberId, membersArray[guildMemberId].user.username);
}

TypeError: Cannot read property 'user' of null
means your user variable is null which means test.members[u] is null.
Try logging the test.members first and see if its filled.
user.user.username
is probably wrong. As it looks it should be just user.username

How do you iterate over Mongoose model properties within a middleware function?

I'm attempting to iterate over fields in a mongoose model within a middleware function. The current context this is the model object itself. So I have a function in which the context is a Javascript object like this:
{
lastName: 'Wilkens',
firstName: 'Elepart',
name: 'Elepart Wilkens',
username: 'eK',
bio: '<script>alert(\'this is a hack!!\')',
_id: 53b17dd0e8c5af50c1d73bc6,
language: 'en',
country: 'USA',
online: true
}
I want to iterate over this object (which is represented in the current function with this). Every time I attempt to iterate with loops, it prints out values that look like internal Javascript metadata. Is it possible to iterate over this within a function if this represents an object?
Here's the actual middleware function:
userSchema.pre('save', function (next) {
console.log(this); // This prints precisely the same object I have copied above
var fields = Object.keys(this);
for(var i = 0; i < fields.length; i++) {
console.log(this[fields[i]]);
}
for(var key in this) {
if(this.hasOwnProperty(key)) {
console.log(this[key]);
}
}
});
And the output is:
{
strictMode: true,
selected: undefined,
shardval: undefined,
saveError: undefined,
validationError: undefined,
adhocPaths: undefined,
removing: undefined,
inserting: undefined,
version: undefined,
getters: {},
_id: undefined,
populate: undefined,
populated: undefined,
wasPopulated: false,
scope: undefined,
activePaths:
{ paths:
{ username: 'modify',
firstName: 'modify',
name: 'modify',
online: 'default',
country: 'default',
language: 'default',
_id: 'default',
bio: 'modify',
lastName: 'modify' },
states: { default: [Object], init: {}, modify: [Object], require: {} },
stateNames: [ 'require', 'modify', 'init', 'default' ],
map: [Function] },
ownerDocument: undefined,
fullPath: undefined }
true
undefined
0
{ online: true,
country: 'USA',
language: 'en',
_id: 53b1825a00ed9af7c12eedf9,
bio: '<script>alert(\'this is a hack!!\')',
username: 'yK',
name: 'Yershay Wilkens',
firstName: 'Yershay',
lastName: 'Wilkens' }
{ save:
[ { [Function] isAsync: false },
{ [Function: checkForExistingErrors] isAsync: false },
{ [Function: validation] isAsync: false },
{ [Function] isAsync: false },
{ [Function] isAsync: false } ],
validate: [ { [Function] isAsync: false } ] }
{ save: [], validate: [] }
{ [Function] numAsyncPres: 0 }
{ [Function] numAsyncPres: 0 }
{}

You're using integer indexes instead of the string references from the fields array. It should be:
var fields = Object.keys(this);
for(var i = 0; i < fields.length; i++) {
console.log(this[fields[i]]);
}
(e.g., you were doing this[1], this[2], instead of this[fields[1]])

#JohnnyHK's comment worked for me:
const user = new User();
const schemaKeys = Object.keys(user.toObject());

I found a slightly different way to accomplish what I wanted which was to iterate over the model properties of a Mongoose model within a middleware function. This uses async.js, but you could refactor it to use a generic JS loop or any other control flow library. The key is to get an array of the document's fields, then you can iterate over those and get/set the values using the current context with this. As far as I know, this will not coerce non-string values into strings. I've tested it with strings, numbers, booleans and objectIds and they are successfully saved as their original data types.
yourSchema.pre('save', function (next) {
var self = this;
// Get the document's fields
var fields = Object.keys(this._doc);
// Do whatever you want for each field
async.each(fields, function(field, cb) {
self[field] = validator.escape(self[field]);
cb();
}, function(err){
next();
});
});