i am trying to build a chat application.. i am not sure how to structure my mongodb schema..
currently i am using this schema..
collection name: conversations
{
conversation: {
id: 1,
name: test
},
messages: [
{
_id: mongoid("123"),
userID: 1,
message: "test message"
date: date..,
status: 1,
pins: [2, 3] //message pin by user ids
mentions: [4, 5],
replyMsgID: mongoid(456)
},
...
],
users: [
{
userID: 1,
blocked: 1,
lastActiveTime: dateTime,
},
...
]
}
...
i am thinking another schema like this...
{
conversation: {
id: 1,
name: test
},
messages: [
{
_id: mongoid("999")
msgID: mongoID("123")
},
{
_id: mongoid("888")
msgID: mongoID("456")
},
...
],
users: [
{
userID: 1,
blocked: 1,
lastActiveTime: dateTime,
},
...
]
}
messages collection..
messages: [
{
_id: mongoid("123"),
userID: 1,
message: "test message"
date: date..,
status: 1,
pins: [2, 3] //message pin by user ids
mentions: [4, 5],
replyMsgID: mongoid(456),
conversationID: 1
},
...
]
i am new to mongodb and i am confused which schema should be use for a scalable for chat application..
i thinks the problems i will face with schema 1 are..
at some point the document size might get pass 16mb limit of mongo...
when fetch or update anything like if i need only 1 message or just a single user the mongo will return the whole document with thousands of messages
the problem with 2nd schema ...
i will need to join two collections with aggregate framework and might need to use $unwind, $group a lot... which i think will be very performance heavy...
to create a new message i will have to insert the message in the messages collection first then insert the the newly added message to the conversations collections property messages field, same goes for delete.. so lots of query.. :|
it would be really helpful if someone could help me out to select a schema.. or create a better schema..
thanks in advance :)
Related
I have created a successful mysql query that joins 2 tables that have foreign keys to each other. I got almost everything to work in Sequelize other than getting the user attached to the responses. My query does that successfully but I cannot understand how to get it.
This is my mysql query that works specifically gets the "users" that has the same id as the response "user_id"
SELECT *
FROM meetup_db.posts p
INNER JOIN meetup_db.responses r
INNER JOIN meetup_db.users u
WHERE p.id = r.post_id AND
r.user_id = u.id
This is my sequelize query that almost works but does not give me back the user in each response where user_id = id
const dbPostData = await Posts.findOne({
where: { id: req.params.id },
include: [
{ model: Users },
{
model: Responses,
include: {
model: Users,
where: { id: Responses.user_id }, //this is the line in question i need the user that has the same id as the Response.user_id
}
}
]
});
The sequelize correctlys outputs everything but the user, heres a sample output as you can see the response has Users listed inside but the users just show as [Object]
{
id: 1,
title: 'BBQ At My House!!',
description: 'Hey gang! Im having a BBQ at my house! I hope all can attend!!',
upvotes: 44,
location: '2113 Main St. Austin, TX.',
date_occuring: 2021-08-04T18:00:00.000Z,
created_at: 2021-08-06T04:42:01.000Z,
edited: false,
user_id: 1,
createdAt: 2021-08-06T04:42:01.000Z,
updatedAt: 2021-08-06T04:42:01.000Z,
User: {
id: 1,
username: 'Jennifer Wylan',
email: 'jwylan#gmail.com',
password: 'ewfchijwnsj',
image_url: '/assets/fake-pfp/fakeperson1.png'
},
Responses: [
{
id: 4,
response: 'Im in there like swimwear!',
user_id: 4,
post_id: 1,
createdAt: 2021-08-06T04:42:01.000Z,
updatedAt: 2021-08-06T04:42:01.000Z,
User: [Object] //these lines right here need to look like the above User: {} Object
},
{
id: 5,
response: 'weeeeeeeeeeeeeeeeeeeeeeeeeeeeee',
user_id: 1,
post_id: 1,
createdAt: 2021-08-06T04:42:01.000Z,
updatedAt: 2021-08-06T04:42:01.000Z,
User: [Object] //these lines right here need to look like the above User: {} Object
}
],
}
I figured it out. So it worked already.
In my terminal the output of responses was showing the User key as [Object]
... Turns out when its like 3 tables deep the terminal just shows [Object] but it was actually there already, you also dont need the where: because it already looks for the foreign key
I have a Schema named user on mongoose, and that schema has lastExams property as below:
lastExams: [{
lecture: {
type: String,
required: true
},
exams: [{
examID: {type: [mongoose.Schema.Types.ObjectId], ref: Exam, required: true},
resultID: {type: [mongoose.Schema.Types.ObjectId], ref: Result, required: true},
date: {type: Date, required: true},
result: {}
}]}]
With this, I want to keep the last 10 exams user have taken for each lecture they have. So after each exam, I want to check if the corresponding 'lastExams.lecture' subdocument already exists, if so, push the result that lastExams.$.exams array. Else, upsert that subdocument with first element of the exams array.
for example, thats an user document without any exams on it;
user: {
name: { firstName: '***', lastName: '***' },
email: '****#****.***',
photo: 'https://****.jpg',
role: 0,
status: true,
program: {
_id: 6017b829c878b5bf117dfb92,
dID: '***',
eID: '****',
pID: '****',
programName: '****',
__v: 0
},
lectures: [
{
some data
}
],
currentExams: [
some data
],
lastExams: []
}}
If user sends an exam data for math-1 lecture, since there is no exam with that lecture name, I need to upsert that document to get user document to become as below;
user: {
name: {
firstName: '***',
lastName: '***'
},
email: '****#****.***',
photo: 'https://****.jpg',
role: 0,
status: true,
program: {
_id: 6017 b829c878b5bf117dfb92,
dID: '***',
eID: '****',
pID: '****',
programName: '****',
__v: 0
},
lectures: [{
some data
}],
currentExams: [
some data
],
lastExams: [{
lecture: 'math-1',
exams: [
examID: 601 ba71e62c3d45a4f10f080,
resultID: '602c09b2148214693694b16c',
date: 2021 - 02 - 16 T18: 06: 42.559 Z,
result: {
corrects: 11,
wrongs: 9,
empties: 0,
net: 8.75,
score: 43.75,
qLite: [
'some question objects'
]
}
]
}]
}
}
I can do that like this;
User.findOneAndUpdate({email: result.user}, {$addToSet: {'lastExams': {
lecture: result.lecture,
exams: [{
examID: doc.examID, // btw, idk why, these id's saving to database as arrays
resultID: doc.id,
date: doc.createdAt,
result: doc.results
}]
}}})
But since this adds new subdoc with same lecture value each time. I am having to check if there is a subdoc with that lecture value first manually. if not so, run the above code, else, to push just exam data to that lectures subdoc, I am using below code;
User.findOneAndUpdate({email: result.user, 'lastExams.lecture': result.lecture }, {$addToSet: {'lastExams.$.exams': {
examID: doc.examID,
resultID: doc.id,
date: doc.createdAt,
result: doc.results
}}})
So, I am having to make User.find() query first to see if that lecture is already there, and pop an item if it's exams.lengt is 10. then deciding to what kind of User.findOneAndUpdate() to use.
Do you think there is any way to make this proccess in a single query? Without going to database 2-3 times for each exam save?
I know it's too long, but i couldn't put it straight with my poor english. Sorry.
Two methods:
Like you already did, multiple queries.
In the first query, you have to check if the subdocument exists, if it does, update it in the second query, else create subdocument with first item in the second query.
Using $out in an aggregation pipeline
If multiple round trips is only the issue, checkout $out aggregation pipeline method, which allows to write aggregation output to a collection. In there, you can first match you document, check if the subdocument exists, using $filter followed by $cond. Once you have the data ready, use $out to write it back to the collection.
NB: Aggregation pipeline is expensive operation than findOneAndUpdate IMO, so make sure you test the average latency for both 2 query method and single aggregation method and decide which is faster in your case.
PS: Sorry for not providing an example, I simply don't know it very well to create a working example for you. You can refer to the mongoDB docs for details.
https://docs.mongodb.com/manual/reference/operator/aggregation/out/
Also, there is a Jira ticket discussion going on for this specific use case in MongoDB. Hoping some simple solution will be implemented in MongoDB in the upcoming versions
I have a NodeJS based application using Mongoose. I am wanting to create a response where there values are an average of ratings provided by people who have responded to a questionnaire, for each question asked.
My Schema looks as follows:
const questionnaireResultSchema = new Schema({
user: { type: ObjectId, ref: 'User' },
questionnaire: { type: ObjectId, ref: 'Questionnaire' },
rating: [{
id: Number,
question: String,
value: Number
}]
},{
timestamps: true
}).index({user: 1, questionnaire: 1}, {unique: true});
I have looked at the Mongoose aggregator operator, but I am not sure how I would apply it to my case. Pseudo code would look as follows:
Find all questionnaire results for questionnaire of id xyz
Provide a result where the ratings for each questionnaire result has be averaged
For example the response would look as follows:
[{
id: 1,
question: 'How strongly do you feel about candidate A?',
value: 12 // averaged value
},{
id: 2,
question: 'How strongly do you think we should change the sky color to green?',
value: 31 // averaged value
},{
id: 3,
question: 'How strongly do you think your answers count?',
value: 20 // averaged value
}]
I have tried:
QuestionnaireResultSchema.aggregate([{
$match: {
questionnaire: questionnaire
}},
{$project: {
scores: { $avg: '$scores'}
}}
]);
This just provides the JSON:
[{
"_id": "57bbd4b495407f6145b3ba9f",
"scores": null
}]
A sample document in a collection would like:
{
user: ObjectId("57bca30536e376c653f439bb")
questionnaire: ObjectId("37bca0feedb0bc470353ab")
scores: [{
id: 1,
question: 'How strongly do you feel about candidate A?',
value: 3
},{
id: 2,
question: 'How strongly do you think we should change the sky color to green?',
value: 4 // averaged value
},{
id: 3,
question: 'How strongly do you think your answers count?',
value: 5 // averaged value
}]
}
While I could calculate the averages myself, if Mongoose provides the functionality, I would rather leverage that.
Any help would be appreciated.
This should work for your aggregation pipeline:
[
{
$match: {
questionnaire: ObjectId("37bca0feedb0bc470353ab")
}
},
{
$unwind: "$rating"
},
{
$group:{
_id:{
"rating_id": "$rating.id",
"question": "$rating.question",
},
avg_rating: {$avg:"rating.value"}
}
},
{
$project:{
"id": "$_id.rating_id",
"question": "$_id.question",
"avg_rating": "$avg_rating"
}
}
]
Although your sample doc has "scores" instead of "rating" in which case you'd use:
[
{
$match: {
questionnaire: ObjectId("237bca0feedb0bc470353aba")
}
},
{
$unwind: "$scores"
},
{
$group:{
_id:{
"rating_id": "$scores.id",
"question": "$scores.question",
},
avg_rating: {$avg:"scores.value"}
}
},
{
$project:{
"id": "$_id.rating_id",
"question": "$_id.question",
"avg_rating": "$avg_rating"
}
}
]
Also, some of the ObjectId's that you are using are not valid. I'm assuming those are just stubbed.
I am not sure how I am going to solve this problem:
I want to search in a mongoDB collection and return only the nested objects that fits the search query (using text search on all of the fields).
All documents in the collection have this format:
{
arr: [
{
_id: 1,
name: 'Random',
description: 'Hello world'
},
{
_id: 2,
name: 'World',
description: 'This is a random description'
},
{
_id: 3,
name: 'Random',
description: 'Hi'
}
]
}
In this case, if my search query is 'world', then this should be the result:
[
{
_id: 1,
name: 'Random',
description: 'Hello world'
},
{
_id: 2,
name: 'World',
description: 'This is a random description'
},
//... objects from other documents in the collection that fits the query
]
If this is not possible in mongoDB, are there any JavaScript libraries that can achieve this? Would greatly appreciate the help!
With the aggregation framework it could look like so
db.getCollection('yourCollection').aggregate([
{
$unwind: '$arr'
},
{
$match: {
$or: [
{ 'arr.name': /world/i },
{ 'arr.description': /world/i }
]
}
},
{
$project: {
_id: '$arr._id',
name: '$arr.name',
description: '$arr.description'
}
}
])
which will result in the following output for your example data:
{
"_id" : 1,
"name" : "Random",
"description" : "Hello world"
}
{
"_id" : 2,
"name" : "World",
"description" : "This is a random description"
}
If you have the need for a single array with the resulting documents as shown in your question, you can simply chain a toArray() call at the end of the pipeline - keep in mind though that this may cause increased memory consumption in case of large result sets as pointed out by SSDMS in the comments.
I have 3 collections,
Users,
Clan, which has many Clanmembers
Clanmembers, which has many Users
So when I do
Clan.find().populate('clanmembers');
It works fine and returns the result but then I get something like:
{ clanmembers:
[ { role: 'Entry',
owner: 2,
member: 2,
id: 1,
createdAt: null,
updatedAt: null } ],
owner:
{ username: 'tester',
email: 'tester',
about: 'This is\n\na test hello\n\n\n\n\n\n\n\n\nhiiii',
profileurl: '',
avatar: 'asd.jpg',
rank: '1',
id: 2,
createdAt: '2015-10-25T10:15:20.000Z',
updatedAt: '2015-10-25T21:31:47.000Z' },
clanname: 'testset',
clantag: 'testin',
id: 2,
createdAt: '2015-10-26T01:07:02.000Z',
updatedAt: '2015-10-26T01:07:02.000Z' }
What I'd like is also to be able to populate the clanmembers array's owners so that I can get usernames and what not of the clanmembers, how can I populate this nested array? Or is it not possible and I'll have to loop/find each one by their ID. Any information would be great thank you.
This can't be done as of now, although it has been in the talks for quite some time.
As you mentioned, you'll have to do the population manually. But you can avoid making separate calls for each subdocument.
What I usually do is collect the required ids in an array and run a single query to the database, then maintain a map of ids to documents which makes it pretty convenient to use in multiple places.