I am trying to cache friends from social media in the user's doc and am having issues storing/getting them back.
First, I blindly strip the their friends cache and would just fill it with my fresh fetch of the data, but through that I found out that the embedded Documents must be unique among all the users (I have a couple of friends that are the same in my test accounts and I get:
{"name":"MongoError","err":"E11000 duplicate key error index: test-dev.users.$friends.outsideID_1 dup key: { : \"1205314313242\" }","code":11001,"n":0,"connectionId":274,"ok":1}
so from this I know that because that embedded document exists in my other already registered and updated account it can't create that document (outsideID is index because I plan on searching using it later))
So then I started trying to OutsideFriend.find({'outsideID':{$in:friendsIDs}}(where friendsIDs is an array of the ids I got from the SM query), etc.. and then in the callback I go through and try added the new friends and just add the existing docs to the user in question so that it's not trying to duplicate it in the system. However, for whatever reason OutsideFrind.find() never returns any docs... which makes me think that the embedded docs aren't centralized... but then why would the first attempt fail?
How am I supposed to do this? (schema below, please let me know if you need any other info!)
current schema:
//External friend model
var OutsideFriend = new Schema({
outsideID:{type:String, index:true, unique:true}
,username:String
,location:String
,avatarURL:String
});
//User model
var User = new Schema({
avatarURL:String
,mySMID:String
//Social Media info
,smID:{type:String, index:true, unique:true}
,tok:String
,tokSec:String
,friends:[OutsideFriend]
};
There's a lot going on in your question, but if you want to query OutsideFriend independent of User, then you shouldn't be embedding them in the User model. Instead, just store ObjectId references to the friends and then populate that in User when you need the full details. Don't prematurely optimize with embedded copies unless you find you need to as that introduces consistency challenges.
So your User model would change to:
var User = new Schema({
avatarURL:String
//Social Media info
,smID:{type:String, index:true, unique:true}
,tok:String
,tokSec:String
,friends:[{type: ObjectId, ref: 'OutsideFriend'}]
};
Related
While using .populate() in mongoose how to fetch and add, ObjectIds with additional properties to array(returned by .populate()) whose document in the collection no longer exists?
My question is regarding MongoDB mongoose .populate()
I have 2 collections:
users:
likedSongs: [objectId()]
playlists:
songs: [ObjectId()}
creator:
songs: [ObjectId()]
songs:
_id: ObjectId() ( Default )
There are other fields but irrelevant to this question
a user uA is a creator of song sA and some other users [oU] have added the sA to their likedSongs and playlists.
while adding a new song and while removing a song, I also add and remove songId from uA.creator,
but it is not possible for me to remove songId of sA from every single user [oU]'s likedSongs and playlist.
'cause it will increase the load on the server.
So what I want to do is while populating the ObjectIds,
I want to add an "empty" property to every songId which was not found,
and then ask the user in the frontend if he wants to remove the song that "no longer exists",
If so then make an API call from there to remove it
But the problem is, after .populate() mongoose only returns an array of songs whose ObjectIds were matched,
I don't know how to fetch the ObjectIds whose song no longer exists,
and add them with the array which I got from .populate()
I know, I'm not very good at explaining problems, but if you can understand please give a solution.
Is there any method or function in mongoose which I can use?
That is not possible. During .populate(), MongoDB will actually go and make another database call to fetch additional information from the other collection. Since you already deleted some documents, MongoDB can not find them during population.
What you can do it refactoring your schema model, so instead of saving only ObjectIds of each song, you would in addition store some other properties for display purposes (song name, song duration...). Then, even when song is removed, you can still display something to the user.
However, if original song document is changed, you would still need to updated these additional information for all users.
I am making a "Like" function (facebook or instagram kind of), but not sure what is the right way to do it.
I could think of 2 ways.... (User cannot like same article twice)
A. "User" data has an array of "Article" IDs...
// simplified user schema MongoDB
const UserSchema ={
id:ObjectID,
username:String,
likes:[{type:ObjectID,ref:"Article"}]
}
// simplified article schema
const AriticleSchema = {
id:ObjectID,
title:String,
content:String,
likes:Number,
}
B. "Article" data has an array of "User" IDs...
// simplified user schema MongoDB
const UserSchema ={
id:ObjectID,
username:String,
}
// simplified article schema
const AriticleSchema = {
id:ObjectID,
title:String,
content:String,
likes:[{type:ObjectID,ref:"User"}],
}
I tried both ways and they all worked fine when I only have few users and few articles.
but What if I have thousands of "User"s and thousands of "Article"s? I am worrying that everytime I request "User" data or "Article" data(let's say several at a time), I also have to bring arrays of thousands? I think there must be better way to do this...
Do you know how people or companies do this? I want to know the concept of how "Like" function works.
Thank you.
** Adding some details **
I want "User" can login for reading articles, and press "like button" to like it. Article "like" will go up by 1 every time unique user likes it (no duplicate). Somebody who already liked the " article" they can "unlike" it or "user" will see they already liked the "article", which means we gotta know that "user" like this "article" or not. Other people dosen't need to know.
I'll extrapolate on a solution by going through the requirements you elaborated step by step.
User can login
This will require some system for authorization. You could perhaps use another Mongo table dedicated to this sort of thing - at the minimum it should link some authorization token to a user id.
const AuthSchema = {
user_id: ObjectID,
auth_token: string,
}
The way you get this auth token is through various means - really depends on how you auth your users, e.g. phone auth or username/password. All that's a bit beyond the scope of this answer.
Article "like" will go up by 1 every time unique user likes it (no duplicate). Somebody who already liked the " article" can "unlike" it or "user" will see they already liked the "article"
A Redis Set would model this quite well. Every time a user likes an article, add their user_id into a Redis Set for that article. The key for such set could look like this:
article:${article_id}:likes
Get the number of people who liked the article by taking the size of its set (full of unique user_ids). Whenever someone unlikes an article, remove their user_id from the set.
Further reading:
Redis Sets and other Redis datatypes
Redis sadd command.
I have an array property called courses on my User table in Parse. Any idea why I might getting Cannot modify user XTC9aiDZlL. code=206, message=Cannot modify user XTC9aiDZlL. when I do the following:
user.remove('courses', deletedCourse);
user.save()
where deleteCourse is the course PFObject to delete
Are you signed in as the user you're trying to modify? That can cause problems like this, as Parse usually just lets a user modify themself & the objects they've created.
If you're signed in as the same user you're trying to edit that's another story. This is a glitch that seems to be popping up in the Parse server recently. It's not the best solution but for now you'll need to modify the ACL when you create the user, like this:
let user = PFUser()
let acl = PFACL()
acl.getPublicWriteAccess = true
acl.getPublicReadAccess = true
user.acl = acl
I am building a kind of social network for students and teachers with MEAN stack.
Students has their own account page different from the one of the teachers.
I have one single registration page for both teachers and students.
When registering, the user has to choose the "type" of the account with a select html tag (student or teacher),
I want to have one single model representating two different type of user (teacher and student) that contains common fields like email and password etc, and some specific fields depending on the select tag ; because I am using email-verification-node npm and it only permits to use a single model in the PersistentUserModel function.
I suggest use this approach.
You should have a separate Schemas for Account, Teacher and Student so the different info between the teachers and students should not be mixed in one place.
Account
var Account = new Schema({
email:String,
password:String,
_teacher:{type:Schema.Types.ObjectId, ref:'Teacher'},
_student:{type:Schema.Types.ObjectId, ref:'Student'}
})
Under account, you should reference the Teacher model if teacher account otherwise reference Students model.
To check if Account is Teacher or Student you could just check _teacher, if it has a value then it's a Teacher account else it's a student. But to make the condition more unique you, check both _teacher and _student.
This approach will save you a lot of refactoring in the future if you will decided to allow the teacher to be a student as well (which is not impossible to happen), he/she can just use the same account and register as a student. Just like what google is doing, on account/email multiple types of app to use.
Teacher
var Teacher = new Schema({
name:{type:Schema.Types.ObjectId, ref:'Name'}
// Other teachers info
})
Student
var Student = new Schema({
name:{type:Schema.Types.ObjectId, ref:'Name'}
// Other students info
})
Name
On this part maybe you are wondering why you need a separate model for name. Well that is because in this approach you can just use one route or endpoint or query to search users across your app. When you search for a name, all students and teachers with matching result will be queried without looking into 2 different collections (Teacher collection and Student collection).
On good use case for this is, for sure you will have an admin dashboard where you can manage all students and teachers. In that dashboard you can just one search field for both teacher and student.
var Name = new Schema({
firstName:String,
middleName:String,
lastName:String
})
Good reads
Referencing a document
Embedded documents
Schema Types
Schema Options
Other tips
You could also separate the Address like I did with the name here. Reason? Same purpose as with the Name, you might want to add search by location feature or something like that.
I hope this helps.
Set up a Boolean field inside your data model that has the key teacher (or student if you prefer) and set that when the user signs up.
Edit:
You can have two different schemas, one for each user type. Declaring them would look something like this.
const usersSchema = new Schema({/* base schema */});
const teachersSchema = new Schema({ /* teachers schema */});
const studentsSchema = new Schema({ /* students schema */});
And
const User = mongoose.model('User', usersSchema, 'users');
User.Teacher = mongoose.model('Teacher', teachersSchema, 'users');
User.Student = mongoose.model('Student', studentsSchema, 'users');
Have a look at the docs here
Edit 2:
I found a better way of doing this is using discriminators...thanks!
const options = {discriminatorKey: 'kind'};
const userSchema = new mongoose.Schema({/* user schema */}, options);
const User = mongoose.model('User', userSchema);
// Schema that inherits from User
const teacherSchema = User.discriminator('Teacher',
new mongoose.Schema({/* Schema specific to teacher */}, options));
const studentSchema = User.discriminator('Student',
new mongoose.Schema({/* Schema specific to student */}, options));
const teacher = new Teacher({/* you can add everything here! */});
const student = new Student({/* you can add everything here! */});
Look up by calling either Teacher or Student
Now you have one model with two Schema! More info in the docs here.
Edit with more info:
You would create two types of data structure, a teacher and student which would both be held in the User collection. When you are calling the database you call using Teacher or Student.
Any data that is going to be common to both is put in the User schema, while anything that is specific you put in the relevant schema.
When you receive a call to the api you direct to the relevant look up. You could use a boolean or a string in the request params, and then separate out the logic with an if statement or a switch statement. I would use a string and a switch.
In your client set two constants TEACHER = 'teacher', STUDENT = 'student', and call with the relevant constant to the api in the body of your request. That way when it hits the api the request will be parsed to the correct lookup and send back the relevant data.
PROBLEM
I'm running an application with AngularJS, Node JS, Express and MongoDB. I'm acessing MongoDB trought mongoose. My problem is that I have a lists of swords, each forged by someone. But, to access someone's profile, I need to use the ID of someone. The ID is unique. But I can't display the link as something link '352384b685326vyad6'. So, when someone create's a sword, I will store his or her name within the sword info.
To display a list of swords, for example, I could do something like this:
<div ng-repeat='sword in swords'>
<p> Sword name: {{sword.name}} </p>
<p> Author: <a href='#/user/{{sword.createdByID}}'> {{sword.createdByName}} </a> </p>
</div>
But, if the User changes his name, the sword will not update his creator name accordingly. What should I do? I have thought of some solutions, but I don't know which and if any could solve this in a good manner.
When someone changes own name, I could make a POST request with the new username and the ID, updating all sword that has createdByID equals to user ID. But I see this too strange.
SwordModel.find({ createdByID: req.body.id}, [...]);
When loading the swords in the controller via GET request, make another GET request for each sword and update sword.username based on the sword.createdById.
UserModel.findById(req.body.id), [...]);
Forget UX and use ugly links.
I want to know how can I maintain the username of each sword updated without affecting too much my DB Thanks for any advice.
MODELS - Just for reference.
sword.js
var mongoose = require('mongoose');
var SwordModel = mongoose.model('SwordModel',
{
name: String, //Sword's name
createdById: String, //ID of the user who created.
createdByName: String //Name of the user
});
module.exports = mongoose.model('SwordModel', SwordModel);
user.js
var mongoose = require('mongoose');
var UserModel = mongoose.model('UserModel',
{
name: String, //Name of the user.
ID: String //ID of the user.
});
module.exports = mongoose.model('UserModel', UserModel);
You can do it by referencing the User inside Swords if you make use of Mongoose schema.
After you make a reference to another schema you can use populate method to get the desired results.
Example (May be not exactly, but something similar to following) :
Sword Schema:
var swordSchema = new Schema({
name: String,
createdBy: {type: mongoose.Schema.Types.ObjectId, ref: 'User'}
});
Make model using the schema :
var swordModel = mongoose.model('Sword', swordSchema);
Find what you are looking for using populate.
See full documentation for populate here - http://mongoosejs.com/docs/populate.html
EDIT: note that I am recommending to keep user's name only in User model and just reference it.
I think we have less of a technical problem and more of a conceptual problem here.
Constraints
I take it for granted that...
users may change their usernames
usernames are unique
you only need to provide a link to a user, displayed by name
Furthermore, I will use plain JSON and MongoDB and trust that you can translate this to Mongoose.
The solution
Albeit users can change their usernames, this is not going to happen often. The more common use case is that you need to link to a username. So we first need to find out how to efficiently deal with that use case.
Since you only need the name of the smith for a given sword, there is nothing wrong with a sword model like
{
_id: new ObjectId(),
name: "Libertas",
smith: "Foobar"
}
to efficiently find "Foobar" in the users collection, we simply add an index here (if not already done):
db.users.createIndex({username:1}, {unique:true})
and your service can query efficiently by using
db.users.find({name: "Foobar"})
No need to save the _id of the user within the sword document, but still you can query for it efficiently.
Dealing with a change in the username is a rather rarely executed use case, so optimizing here does not make sense. However, if a user changes his or her username, your service can easily achieve that through
db.swords.update(
{ smith:"Foobar" },
{ $set:{ smith: "CoolNewUsername" },
{ multi: true, writeConcern: { w:1, j:true }
)
The last line of the above needs a little explanation. The multi: true option tells MongoDB to change all documents matching {smith: "Foobar"}, not only the first one found, that's easy to understand. But why to set the write concern to journaled? The first reason for it is that regardless of the write concern configured for the connection (which may even be unacknowledged), we need those changes to be durable. However, we usually do not need to have the changes to be propagated to more replica set members, so the chosen write concern gives you the best performance while still you can be sure that the changes were synced to a disk. If you need higher durability, of course you can set the write concern to {w:2} or {w:"majority"}.
Advantages
For the most common use case of this relationship (displaying a link to the user who forged a given sword), all information needed to do this is included in the sword's document, preventing possibly unnecessary queries.
Still, the smith of a given sword can be queried efficiently if a user clicks said link.
Changing a users name is possible and can be achieved pretty efficiently and durable
Disadvantages
The main disadvantage here is that you actually have to modify all affected swords when a user changes his or her username, whereas with a Mongoose reference that would be unnecessary. However, since this is a rare use case and using populate would result in the whole user document to be loaded where only the users name is needed, I see this disadvantage as negligible. Basically you are trading to cut down the queries needed for a common use case by half against the need for a manual update which occurs rather rarely.
I fail to see any other disadvantage.