Related
I am using mongoDB as backend server, I have nested array(one level array) like this
{
"_id" : ObjectId("60b1fc6d3c43f74e0c1dba92"),
"seriesId" : "60acebf73acb5b3a98d14331",
"name" : "Season 1",
"logoURL" : "uploads/season/1622277216401.png",
"yearOfPublish" : "2021-05-29",
"description" : "Season 1",
"createdBy" : ObjectId("609cbf49ba46cc3924859ab5"),
"createdOn" : "2021-05-29T08:33:49.480Z",
"episode" : [
{
"seasonId" : "60b1fc6d3c43f74e0c1dba92",
"name" : "Episode 1",
"id" : 0,
"logoURL" : "uploads/episode/1622278616899.png",
"dateOfTelecast" : null,
"description" : "sadfgh",
"duration" : "30",
"videoType" : "customURL",
"embedCode" : "",
"url" : "https://youtu.be/kbpsXMUr7ss",
"liveboxChannel" : "",
"createdOn" : "2021-05-29T08:56:59.230Z",
"createdBy" : ObjectId("609cbf49ba46cc3924859ab5"),
"_id" : "ZaVrpOLO5"
},
{
"seasonId" : "60b1fc6d3c43f74e0c1dba92",
"name" : "Episode 2",
"id" : 0,
"logoURL" : "uploads/episode/1622279206607.png",
"dateOfTelecast" : null,
"description" : "adfd",
"duration" : "30",
"videoType" : "customURL",
"embedCode" : "",
"url" : "https://youtu.be/kbpsXMUr7ss",
"liveboxChannel" : "",
"createdOn" : "2021-05-29T09:06:48.637Z",
"createdBy" : ObjectId("609cbf49ba46cc3924859ab5"),
"_id" : "9GKqXhxcH"
},}
I have more no of seasons. from the season collection,i have episode array under the name of Episode.
Now My frontend page required that episode array alone.
response = {episode: all the episode data} and this episode data is based on skip and limit value
I have tried something in mongodb,
db.getCollection('season_copy').aggregate([
{$project: {
episodes: {
$cond:{ if: { $isArray: "$episode" }, then: { input:"$episode" }, else: 0 }
},
},
},
])
Can anyone suggest me some idea?
Check this out:
Without aggregate:
db.getCollection('season_copy')
.find({ _id: ObjectID(id)})
.project({ episode: 1 }).toArray();
With aggregate:
MongoDB playground
db.getCollection('season_copy')
.aggregate([
{
$match: {
_id: ObjectId("60b1fc6d3c43f74e0c1dba92")
}
},
{
$project: {
episode: 1
}
}
])
I'm using nodejs with official mongodb's package. I got many documents in mongodb containing "type" and "timestamp" field. I want to sort it by prioritizing "type" (only specific content) and then "timestamp".
As example I have following documents:
{ type: "book", timestamp: 1580825471 }
{ type: "house", timestamp: 1580825502 }
{ type: "water", timestamp: 1580825515 }
{ type: "book", timestamp: 1580825478 }
{ type: "smartphone", timestamp: 1580825522 }
{ type: "book", timestamp: 1580825424 }
My goal is to have sorted by that way to priority the type "book" first (and then sort it by timestamp)
{ type: "book", timestamp: 1580825478 }
{ type: "book", timestamp: 1580825471 }
{ type: "book", timestamp: 1580825424 }
{ type: "smartphone", timestamp: 1580825522 }
{ type: "water", timestamp: 1580825515 }
{ type: "house", timestamp: 1580825502 }
I was trying to use the db.collection.aggregate with following $sort value:
$sort: {
type: "book",
timestamp: -1
}
But that didn't worked out because the $sort field's value can only have the value of "1", "-1" or "{ $meta: "textScore" }".
Does anybody have an idea how to solve that issue?
Thanks in advance
EDIT:
This solution by using
$sort: {
type: 1,
timestamp: -1
}
is not a solution since then all types are also sorted which I don't want it. I just want to have "book" as first result then after that, types can be randomized (but timestamp is still being sorted.). Reason for that is that I want to list history entries (that's why I'm using timestamp to sort it), but I want to show type "book" at first. Even if the document are older than other documents.
So yeah, for other types expect "book", I want it to be sorted by timestamp.
You can add an extra field in a project stage that creates a sort priority, then use that to sort on.
For example:
db.data.aggregate([
{ $addFields : { sortPriority: { $eq: [ "$type", "book" ] } } },
{ $sort: { sortPriority: -1, timestamp: -1} }
])
This will output the following:
{ "_id" : ObjectId("5e39892e0f18de54afe4d874"), "type" : "book", "timestamp" : 1580825478, "sortPriority" : true }
{ "_id" : ObjectId("5e39892e0f18de54afe4d871"), "type" : "book", "timestamp" : 1580825471, "sortPriority" : true }
{ "_id" : ObjectId("5e39892e0f18de54afe4d876"), "type" : "book", "timestamp" : 1580825424, "sortPriority" : true }
{ "_id" : ObjectId("5e39892e0f18de54afe4d875"), "type" : "smartphone", "timestamp" : 1580825522, "sortPriority" : false }
{ "_id" : ObjectId("5e39892e0f18de54afe4d873"), "type" : "water", "timestamp" : 1580825515, "sortPriority" : false }
{ "_id" : ObjectId("5e39892e0f18de54afe4d872"), "type" : "house", "timestamp" : 1580825502, "sortPriority" : false }
If you want to ommit the extra field add $unset stage:
db.data.aggregate([
{ $addFields : { sortPriority: { $eq: [ "$type", "book" ] } } },
{ $sort: { sortPriority: -1, timestamp: -1} },
{ $unset: "sortPriority" }
])
This will then output:
{ "_id" : ObjectId("5e39892e0f18de54afe4d874"), "type" : "book", "timestamp" : 1580825478 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d871"), "type" : "book", "timestamp" : 1580825471 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d876"), "type" : "book", "timestamp" : 1580825424 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d875"), "type" : "smartphone", "timestamp" : 1580825522 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d873"), "type" : "water", "timestamp" : 1580825515 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d872"), "type" : "house", "timestamp" : 1580825502 }
You can create a sorting key by your own:
db.col.aggregate([
{
$addFields: {
sortBy: {
$cond: {
if: { $eq: ["$type", "book"] }, then: 0, else: 1
}
}
}
},
{ $sort: { sortBy: 1, timestamp: 1 } },
{ $unset: "sortBy" }
])
Output:
{ "_id" : ObjectId("5e398952227b6d209de231bb"), "type" : "book", "timestamp" : 1580825424, "sortPriority" : true }
{ "_id" : ObjectId("5e398952227b6d209de231b6"), "type" : "book", "timestamp" : 1580825471, "sortPriority" : true }
{ "_id" : ObjectId("5e398952227b6d209de231b9"), "type" : "book", "timestamp" : 1580825478, "sortPriority" : true }
{ "_id" : ObjectId("5e398952227b6d209de231b7"), "type" : "house", "timestamp" : 1580825502, "sortPriority" : false }
{ "_id" : ObjectId("5e398952227b6d209de231b8"), "type" : "water", "timestamp" : 1580825515, "sortPriority" : false }
{ "_id" : ObjectId("5e398952227b6d209de231ba"), "type" : "smartphone", "timestamp" : 1580825522, "sortPriority" : false }
My mongodb data is like this,i want to filter the memoryLine.
{
"_id" : ObjectId("5e36950f65fae21293937594"),
"userId" : "5e33ee0b4a3895a6d246f3ee",
"notes" : [
{
"noteId" : ObjectId("5e36953665fae212939375a0"),
"time" : ISODate("2020-02-02T17:24:06.460Z"),
"memoryLine" : [
{
"_id" : ObjectId("5e36953665fae212939375ab"),
"memoryTime" : ISODate("2020-02-03T17:54:06.460Z")
},
{
"_id" : ObjectId("5e36953665fae212939375aa"),
"memoryTime" : ISODate("2020-02-03T05:24:06.460Z")
}
]
}
]}
i want to get the item which memoryTime is great than now as expected like this,
"userId" : "5e33ee0b4a3895a6d246f3ee",
"notes" : [
{
"noteId" : ObjectId("5e36953665fae212939375a0"),
"time" : ISODate("2020-02-02T17:24:06.460Z"),
"memoryLine" : [
{
"_id" : ObjectId("5e36953665fae212939375ab"),
"memoryTime" : ISODate("2020-02-03T17:54:06.460Z")
},
{
"_id" : ObjectId("5e36953665fae212939375aa"),
"memoryTime" : ISODate("2020-02-03T05:24:06.460Z")
}
]
}]
so is use code as below.i use a $filter in memoryLine to filter to get the right item.
aggregate([{
$match: {
"$and": [
{ userId: "5e33ee0b4a3895a6d246f3ee"},
]
}
}, {
$project: {
userId: 1,
notes: {
noteId: 1,
time: 1,
memoryLine: {
$filter: {
input: "$memoryLine",
as: "mLine",
cond: { $gt: ["$$mLine.memoryTime", new Date(new Date().getTime() + 8 * 1000 * 3600)] }
}
}
}
}
}]).then(doc => {
res.json({
code: 200,
message: 'success',
result: doc
})
});
but i got this,memoryLine is null,why?I try to change $gt to $lt, but also got null.
"userId" : "5e33ee0b4a3895a6d246f3ee",
"notes" : [
{
"noteId" : ObjectId("5e36953665fae212939375a0"),
"time" : ISODate("2020-02-02T17:24:06.460Z"),
"memoryLine" : null <<<------------- here is not right
}]
You can use $addFields to replace existing field, $map for outer collection and $filter for inner:
db.collection.aggregate([
{
$addFields: {
notes: {
$map: {
input: "$notes",
in: {
$mergeObjects: [
"$$this",
{
memoryLine: {
$filter: {
input: "$$this.memoryLine",
as: "ml",
cond: {
$gt: [ "$$ml.memoryTime", new Date() ]
}
}
}
}
]
}
}
}
}
}
])
$mergeObjects is used to avoid repeating fields from source memoryLine object.
Mongo Playground
I have the following Product Schema: (partial, using mongoose)
attributes: [
{
set: {
ref: 'AttributeSet',
type: Schema.Types.ObjectId
},
items: [
{
attribute: {
ref: 'Attributes',
type: Schema.Types.ObjectId
},
values: [
{
ref: 'AttributeValues',
type: Schema.Types.ObjectId
}
]
}
],
_id: 0
}
],
example document 1: (partial)
"attributes" : [
{
"set" : ObjectId("5ccc079c846de44116182890"),
"items" : [
{
"values" : [
ObjectId("5ccc0900846de441161828a1")
],
"_id" : ObjectId("5dee638d72fa520f53d0d1c4"),
"attribute" : ObjectId("5ccc0900846de441161828a0")
},
{
"values" : [
ObjectId("5ccc0a51846de441161828cc")
],
"_id" : ObjectId("5dee638d72fa520f53d0d1c3"),
"attribute" : ObjectId("5ccc0a51846de441161828cb")
},
{
"values" : [
ObjectId("5ccc0c7d846de44116182906")
],
"_id" : ObjectId("5dee638d72fa520f53d0d1c2"),
"attribute" : ObjectId("5ccc0c7d846de44116182904")
},
{
"values" : [
ObjectId("5ccc0d64846de44116182911")
],
"_id" : ObjectId("5dee638d72fa520f53d0d1c1"),
"attribute" : ObjectId("5ccc0d64846de4411618290f")
},
{
"values" : [
ObjectId("5ccc079f846de44116182892")
],
"_id" : ObjectId("5def6acf66910405e07e1e9f"),
"attribute" : ObjectId("5ccc079f846de44116182891")
}
]
}
]
example document 2: (partial)
"attributes" : [
{
"set" : ObjectId("5ccc079c846de44116182890"),
"items" : [
{
"values" : [
ObjectId("5ccc079f846de44116182892")
],
"_id" : ObjectId("5dee635c72fa520f53d0d1c0"),
"attribute" : ObjectId("5ccc079f846de44116182891")
},
{
"values" : [
ObjectId("5ccc0900846de441161828a2")
],
"_id" : ObjectId("5dee635c72fa520f53d0d1bf"),
"attribute" : ObjectId("5ccc0900846de441161828a0")
},
{
"values" : [
ObjectId("5ccc0ea4846de44116182941")
],
"_id" : ObjectId("5dee635c72fa520f53d0d1be"),
"attribute" : ObjectId("5ccc0ea4846de44116182940")
},
{
"values" : [
ObjectId("5ccc08ba846de4411618289c")
],
"_id" : ObjectId("5def56c537e877042d5abeb5"),
"attribute" : ObjectId("5ccc08ba846de4411618289a")
},
{
"values" : [
ObjectId("5ccc09ca846de441161828aa"),
ObjectId("5ccc09ca846de441161828a9")
],
"_id" : ObjectId("5def56c537e877042d5abeb4"),
"attribute" : ObjectId("5ccc09ca846de441161828a7")
}
]
}
],
I want to aggregate and find all products that have attributes and then group the attributes in the output.
Pipeline:
db.getCollection("products").aggregate(
[
{ $unwind: "$attributes" },
{
$group: {
_id: "$attributes",
attributes: { $first: "$attributes.items" }
}
},
{ $unwind: "$attributes" },
{
$lookup: {
from: "attributes",
let: { attribute: "$attributes.attribute" },
pipeline: [
{
$match: {
$expr: {
$eq: ["$_id", "$$attribute"]
}
}
},
{ $project: { display_name: 1, _id: 1 } }
],
as: "attrs"
}
},
{
$lookup: {
from: "attributevalues",
let: { attribute: "$attributes.values" },
pipeline: [
{
$match: {
$expr: {
$in: ["$_id", "$$attribute"]
}
}
}
],
as: "values"
}
},
{ $project: { attrs: 1, values: 1, _id: 0 } },
{
$group: { _id: "$attrs", items: { $push: "$values" }, total: { $sum: 1 } }
}
],
{
allowDiskUse: true
}
);
Pipeline output:
[{
"_id" : [
{
"_id" : ObjectId("5ccc079f846de44116182891"),
"display_name" : "Caliber (cal.)"
}
],
"items" : [
[
{
"_id" : ObjectId("5ccc079f846de44116182892"),
"sort_order" : NumberInt(0),
"label" : "12",
"attribute_id" : ObjectId("5ccc079f846de44116182891")
}
],
[
{
"_id" : ObjectId("5ccc079f846de44116182892"),
"sort_order" : NumberInt(0),
"label" : "12",
"attribute_id" : ObjectId("5ccc079f846de44116182891")
}
]
],
"total" : 2.0
},
{
"_id" : [
{
"_id" : ObjectId("5ccc0900846de441161828a0"),
"display_name" : "Mechanism"
}
],
"items" : [
[
{
"_id" : ObjectId("5ccc0900846de441161828a2"),
"sort_order" : NumberInt(1),
"label" : "Inaction",
"attribute_id" : ObjectId("5ccc0900846de441161828a0")
}
],
[
{
"_id" : ObjectId("5ccc0900846de441161828a1"),
"sort_order" : NumberInt(0),
"label" : "Gas",
"attribute_id" : ObjectId("5ccc0900846de441161828a0")
}
]
],
"total" : 2.0
}]
Problem is that e.x. in 1st element in the array, I have a duplicate inside the items array.
This is the desired output:
[{
"_id" : [
{
"_id" : ObjectId("5ccc079f846de44116182891"),
"display_name" : "Caliber (cal.)"
}
],
"items" : [
[
{
"_id" : ObjectId("5ccc079f846de44116182892"),
"sort_order" : NumberInt(0),
"label" : "12",
"attribute_id" : ObjectId("5ccc079f846de44116182891"),
"total": 'current _id total, in this case it should be 2'
}
],
...other items goes below, grouped as above
]
}]
Add the solution to group the values by attribute and values and count the occurrences followed by look up and pushing all the values for the attribute with their count.
db.products.aggregate(
[
{"$unwind":"$attributes"},
{"$unwind":"$attributes.items"},
{"$replaceRoot":{"newRoot":"$attributes.items"}},
{"$unwind":"$values"},
{"$group":{
"_id":{"attribute":"$attribute","values":"$values"},
"total":{"$sum":1}
}},
{"$lookup":{
"from":"attributes",
"let":{"attribute":"$_id.attribute"},
"pipeline":[
{"$match":{"$expr":{"$eq":["$_id","$$attribute"]}}},
{"$project":{"display_name":1,"_id":1}}],
"as":"attrs"
}},
{"$lookup":{
"from":"attributevalues",
"localField":"_id.values",
"foreignField":"_id",
"as":"values"
}},
{"$unwind":"$values"},
{"$addFields":{"values.total":"$total"}},
{"$group":{
"_id":{"$arrayElemAt":["$attrs", 0]},
"values":{"$push":"$values"}
}}
])
Use the below aggregation query. Use $addToSet to keep unique values.
db.products.aggregate(
[
{"$unwind":"$attributes"},
{"$unwind":"$attributes.items"},
{"$replaceRoot":{"newRoot":"$attributes.items"}},
{"$unwind":"$values"},
{"$group":{
"_id":"$attribute",
"values":{"$addToSet":"$values"},
"total":{"$sum":1}
}},
{"$lookup":{
"from":"attributes",
"let":{"attribute":"$_id"},
"pipeline":[
{"$match":{"$expr":{"$eq":["$_id","$$attribute"]}}},
{"$project":{"display_name":1,"_id":1}}],
"as":"attrs"
}},
{"$addFields":{"attrs":{"$arrayElemAt":["$attrs", 0]}},
{"$lookup":{
"from":"attributevalues",
"localField":"values",
"foreignField":"_id",
"as":"values"
}}
])
Old answer
You could use below aggregation query. I tried to clean up your current query and change to group only by values field.
Something like
db.products.aggregate(
[
{"$unwind":"$attributes"},
{"$unwind":"$attributes.items"},
{"$replaceRoot":{"newRoot":"$attributes.items"}},
{"$unwind":"$values"},
{"$group":{
"_id":"$values",
"items":{"$first":"$$ROOT"},
"total":{"$sum":1}
}},
{"$lookup":{
"from":"attributes",
"let":{"attribute":"$items.attribute"},
"pipeline":[
{"$match":{"$expr":{"$eq":["$_id","$$attribute"]}}},
{"$project":{"display_name":1,"_id":1}}],
"as":"attrs"
}},
{"$lookup":{
"from":"attributevalues",
"localField":"items.values",
"foreignField":"_id",
"as":"values"
}},
{"$unwind":"$values"},
{"$group":{
"_id":{"$arrayElemAt":["$attrs", 0]},
"values":{"$push":"$values"},
"total":{"$first":"$total"}
}},
{"$addFields":{"_id":0, "attribute":"$_id"}}
])
I have two data structures "database" and "components"
const DatabaseSchema = mongoose.Schema({
components: [{ type: Schema.Types.ObjectId, ref: 'Components', required: false }],
});
const ComponentsSchema = mongoose.Schema({
name: { type: String, required: true, trim: true, unique: true, lowercase: true },
updatedAt: Date,
});
I want to filter all items in the database by component names
search rule I'm using
Database.find({
components: { $elemMatch: { name: /antr/i } }
}).populate({
path: 'components',
select: 'name -_id'
}).select(['descript','components']).exec( (err,data) => {
console.log(err);
res.json(data);
});
however always return an empty element
Please try this :
As I've already suggested you can use this :
Database.find({})
.populate({ path: 'components', match: { name: /antr/i }, select: 'name -_id' })
.exec((err, data) => { console.log(err); res.json(data); });
Since you're seeing empty array's is because of the filter query in match which doesn't find appropriate documents in components collection w.r.t. ObjectIds in components array of database document, this is normal. May be you can filter those out in code, as you aren't looking in that way, You can use mongoDB's $lookup from aggregation framework which is equivalent to .populate() from mongoose.
Database.aggregate(
[{
$lookup: {
from: "components",
"let": { "ids": "$components" },
pipeline: [
{ $match: { $expr: { $in: ['$_id', '$$ids'] } } }],
as: "dbComponentsArray"
}
}, { $unwind: '$dbComponentsArray' }, { $match: { 'dbComponentsArray.name': /antr/i } },
{ $group: { _id: '$_id', dbComponentsArray: { $push: '$dbComponentsArray' }, data: { $first: '$$ROOT' } } }, { $addFields: { 'data.dbComponentsArray': '$dbComponentsArray' } },
{ $replaceRoot: { 'newRoot': '$data' } }])
Sample Data in collections :
components :
/* 1 */
{
"_id" : ObjectId("5d481cd098ba991c0857959f"),
"name" : "antracito",
"updatedAt" : ISODate("2019-08-05T12:10:56.777Z"),
"__v" : 0
}
/* 2 */
{
"_id" : ObjectId("5d481cd098ba991c0857958f"),
"name" : "anacito",
"updatedAt" : ISODate("2019-08-05T12:10:56.777Z"),
"__v" : 0
}
/* 3 */
{
"_id" : ObjectId("5d481cd098ba991c0857951f"),
"name" : "antracito",
"updatedAt" : ISODate("2019-08-05T12:10:56.777Z"),
"__v" : 0
}
/* 4 */
{
"_id" : ObjectId("5d481cd098ba991c0857952f"),
"name" : "anacito",
"updatedAt" : ISODate("2019-08-05T12:10:56.777Z"),
"__v" : 0
}
database :
/* 1 */
{
"_id" : ObjectId("5d4979d52a17d10a6c8de81b"),
"components" : [
ObjectId("5d481cd098ba991c0857951f"),
ObjectId("5d481cd098ba991c0857952f"),
ObjectId("5d481cd098ba991c0857953f"),
ObjectId("5d481cd098ba991c0857959f")
]
}
Output :
/* 1 */
{
"_id" : ObjectId("5d4979d52a17d10a6c8de81b"),
"components" : [
ObjectId("5d481cd098ba991c0857951f"),
ObjectId("5d481cd098ba991c0857952f"),
ObjectId("5d481cd098ba991c0857953f"),
ObjectId("5d481cd098ba991c0857959f")
],
"dbComponentsArray" : [
{
"_id" : ObjectId("5d481cd098ba991c0857959f"),
"name" : "antracito",
"updatedAt" : ISODate("2019-08-05T12:10:56.777Z"),
"__v" : 0
},
{
"_id" : ObjectId("5d481cd098ba991c0857951f"),
"name" : "antracito",
"updatedAt" : ISODate("2019-08-05T12:10:56.777Z"),
"__v" : 0
}
]
}