How to remove duplicates in mongoDB with mongoose(NodeJS) - javascript

I have a collection in MongoDB where there are around (~200k records). My sample record would look like,
{
name:String,
slug:String,
metaDes:String,
content:String,
parentID:String,
thumbnail:String,
display:Boolean,
}
I am having a lot of duplicate records in the collection having same slug
I want to remove duplicate records based on slug
Is there any fast way to remove all duplicates with mongoose(Nodejs)?
Thanks!

Remove duplicate records in the collection having the same slug
db.table.aggregate([
{
"$group": {
_id: {slug: "$slug"},
slugs: { $addToSet: "$_id" } ,
count: { $sum : 1 }
}
},
{
"$match": {
count: { "$gt": 1 }
}
}
]).forEach(function(doc) {
doc.slugs.shift();
db.table.remove({
_id: {$in: doc.slugs}
});
})
Refarnce link

Related

return what was not found in array in MongoDB

say my database collection has
* user collection*
[
{id:'1'}
{id:'2'}
]
I have an array of object
[
{id:'1'}
{id:'2'}
{id:'3'}
]
I want the object that was not found in the collection.
I want
[
{id:'3'}
]
I'm currently have this
const records = await dbo
.collection('user collection')
.find({
'id': { $in: newArr },
})
.toArray();
I'm a bit stumped on what to do! ... hope someone can help Thanks!
Option 1:
Looks like this is what you need via the not in operation ( $nin ) when you need to check the not exisitng id in collection documents from provided array:
db.collection.aggregate([
{
$match: {
id: {
"$nin": [
1,
2
]
}
}
},
{
$group: {
_id: null,
"idnotIntheArray": {
$push: "$id"
}
}
}
])
Explained:
$match for any documents with id not in provided array.
$group all id's in an array
plaground1
Option 2:
And this is the option where you output only the array elements not existing in the collection:
db.collection.aggregate([
{
$group: {
_id: null,
ids: {
$push: "$id"
}
}
},
{
$project: {
missingFromCollection: {
"$setDifference": [
[
1,
5,
4
],
"$ids"
]
}
}
}
])
Explained:
Push all id elements from collection to array ids ( note this solution will not allow more then 16MB total size of id's )
Use $setDifference to identify the difference between the two arrays.
playground2
You can use this aggregation:
db.entity.aggregate([
{
$match : {
"myObjList.id" : 1
}
},
{
$unwind : "$myObjList"
},
{
$match : {
"myObjList.id" : 1
}
}
])
and my aggregation result:
{
"_id" : ObjectId("6225a0f78d435fd2845f1dd1"),
"myObjList" : {
"id" : 1
}
}

Get some elements from an array mongoDB

In MongoDB shell version v4.4.6
the following code works perfectly.
db['pri-msgs'].findOne({tag:'aaa&%qqq'},{msgs:{$slice:-2}})
But in nodeJs mongoDB the following code doesn't work.
db.collection('pri-msgs').findOne({
tag: 'aaa&%qqq'
}, {
msgs: {
slice: -2
}
})
My document-->
{"_id":{"$oid":"60c4730fadf6891850db90f9"},"tag":"aaa&%qqq","msgs":[{"msg":"abc","sender":0,"mID":"ctYAR5FDa","time":1},{"msg":"bcd","sender":0,"mID":"gCjgPf85z","time":2},{"msg":"def","sender":0,"mID":"lAhc4yLr6","time":3},{"msg":"efg","sender":0,"mID":"XcBLC2rGf","time":4,"edited":true},{"msg":"fgh","sender":0,"mID":"9RWVcEOlD","time":5},{"msg":"hij","sender":0,"mID":"TJXVTuWrR","time":6},{"msg":"jkl","sender":0,"mID":"HxUuzwrYN","time":7},{"msg":"klm","sender":0,"mID":"jXEOhARC2","time":8},{"msg":"mno","sender":0,"mID":"B8sVt4kCy","time":9}]}
Actually what I'm trying to do is Get last 2 itmes from msgs Array where time is greater than 'n'. Here 'n' is a number.
You can use aggregation-pipeline to get the results you are looking for. The steps are the following.
Match the documents you want by tag.
Unwind the msgs array.
Sort descending by msgs.time.
Limit first 2 elements.
Match the time you are looking for using a range query.
Group the documents back by _id.
Your query should look something like this:
db['pri-msgs'].aggregate([
{ $match: { tag: 'aaa&%qqq' } },
{ $unwind: '$msgs' },
{
$sort: {
'msgs.time': -1 //DESC
}
},
{ $limit: 2 },
{
$match: {
'msgs.time': {
$gt: 2 //n
}
}
},
{
$group: {
_id: '$_id',
tag: { $first: '$tag' },
msgs: {
$push: { msg: '$msgs.msg', sender: '$msgs.sender', mID: '$msgs.mID', time: '$msgs.time' }
}
}
}
]);

Mongoose aggregate Get count and append new value to the query result

Given that I have this two COLLECTIONS:.
1st.col
users:
{
_id :34,
name :"mama mia"
}
2nd.col
posts:
{
_id :67
body :" hello mongoose"
likes:[ 0: ObjectId("34") ]
}
I wanna get every posts with likes count.
And let suppose if I have a auth user id ready and a want to map through the likes and if the user._id(auth id) is found in the post likes i wanna append a new attribute to the collection result not in the db , i just want to modifies the result i'm gonna get.
To make more since of my question , this is the result expected :
[
{
_id : 67
body : " hello mongoose"
likesCount : 1
liked :true
likes :{
"34":{
name :"mama mia"
}
}
}
]
$lookup to join users collection
$map to iterate loop of likes array and return key-value format result
$arrayToObject to convert key-value array of object to object
$size to get total elements in likes array
$in to check current auth user id in likes array or not
let auth_user_id = ObjectId("34");
db.posts.aggregate([
{
$lookup: {
from: "users",
localField: "likes",
foreignField: "_id",
as: "likes"
}
},
{
$project: {
likes: {
$arrayToObject: {
$map: {
input: "$likes",
in: {
k: { $toString: "$$this._id" },
v: "$$this.name"
}
}
}
},
likesCount: { $size: "$likes" },
liked: { $in: [auth_user_id, "$likes"] },
body: 1
}
}
])
Playground

Use mongoDB $lookup to find documents in another collection not present inside an array

I'm using the aggregate framework to query a collection and create an array of active players (up until the last $lookup) after which I'm trying to use $lookup and $pipeline to select all the players from another collection (users) that are not present inside the activeUsers array.
Is there any way of doing this with my current setup?
Game.aggregate[{
$match: {
date: {
$gte: ISODate('2021-04-10T00:00:00.355Z')
},
gameStatus: 'played'
}
}, {
$unwind: {
path: '$players',
preserveNullAndEmptyArrays: false
}
}, {
$group: {
_id: '$players'
}
}, {
$group: {
_id: null,
activeUsers: {
$push: '$_id'
}
}
}, {
$project: {
activeUsers: true,
_id: false
}
}, {
$lookup: {
from: 'users',
'let': {
active: '$activeUsers'
},
pipeline: [{
$match: {
deactivated: false,
// The rest of the query works fine but here I would like to
// select only the elements that *aren't* inside
// the array (instead of the ones that *are* inside)
// but if I use '$nin' here mongoDB throws
// an 'unrecognized' error
$expr: {
$in: [
'$_id',
'$$active'
]
}
}
},
{
$project: {
_id: 1
}
}
],
as: 'users'
}
}]
Thanks
For negative condition use $not before $in operator,
{ $expr: { $not: { $in: ['$_id', '$$active'] } } }

How can I update all Mongo documents within an array of _ids?

I have an array of User _id's in foundStory.authors like so:
"authors": [
{
"$oid": "5814ef8cafc25327a572eee5"
},
{
"$oid": "5814ef80afc25327a572eee4"
}
],
I would like to run through that array of authors, and increment their scores by two. Currently, I am trying to do this with the following:
User.update({ _id: { $in: foundStory.authors } },{ $inc: { score : 2 } })
But, this is only incrementing the author at the last index of my array. From what I have read, I expected this to work. Any ideas?
Figured it out... adding {multi: true} seemed to solve the problem.
User.update({ _id: { $in: foundStory.authors } },{ $inc: { score : 2 } }, { multi: true })

Categories

Resources