Develop Reference

Referencing and Inserting ObjectId's from one collection to another as an array

I have two collections. One to store all the user Details and another to store movies. I have a user_id field which has the objectId of the user who uploads it in the movie collection.
Now I need to store all the movie ObjectId's as a array in the corresponding user collection. Like one to many relationship.
say,
I have some movie documents :
[{
'id' : '1',
'title' : 'Pk',
'user_id' : '25'
},
{
'id' : '2',
'title' : 'Master',
'user_id' : '25'
}]
In user collection, I want to store all the Movie_id's as a array to the corresponding user.
{
'user_id' : '25',
'user_name' : 'john',
'Movie_ids' : ['1','2']
}
How can I achieve this using mongodb and express.js?

Query
$lookup does what you want (see mongodb documentation is very good)
$map is used to keep only the ids in the array
Test code here
db.users.aggregate([
{
"$lookup": {
"from": "movies",
"localField": "user_id",
"foreignField": "user_id",
"as": "movies"
}
},
{
"$set": {
"movies": {
"$map": {
"input": "$movies",
"in": "$$this.id"
}
}
}
}
])

Ok, I'm not sure if this is entirely what you are looking for but you can use javascript function .filter on the movie object to get all the movies with user_id=25 and then map the id's of those movies to a new array like this:
let movies = [
{
"id": "1",
"name": "pk",
"user_id": "25"
},{
"id": "2",
"name": "Master",
"user_id": "25"
}]
let user = {
"id": "25",
"name": "john"
}
let sortedMovies = movies.filter(movie => movie.user_id === user.id).map(movie => movie.id);
user.movieIDs = sortedMovies;
A link to test the code: https://playcode.io/816962/

mongoose find from nested array of objects

hey I am quite new to mongoose and can't get my head around search.
models
User->resumes[]->employments[]
UserSchema
{
resumes: [ResumeSchema],
...
}
ResumeSchema
{
employments: [EmploymentSchema],
...
}
EmploymentSchema
{
jobTitle: {
type: String,
required: [true, "Job title is required."]
},
...
}
Background
User has to enter job title and needs suggestions from the existing data of the already present resumes and their employment's job title
I have tried the following code.
let q = req.query.q; // Software
User.find({ "resumes.employments.jobTitle": new RegExp(req.query.q, 'ig') }, {
"resumes.employments.$": 1
}, (err, docs) => {
res.json(docs);
})
Output
[
{
_id: '...',
resumes:[
{
employments: [
{
jobTitle: 'Software Developer',
...
},
...
]
},
...
]
},
...
]
Expected OutPut
["Software Developer", "Software Engineer", "Software Manager"]
Problem
1:) The Data returned is too much as I only need jobTitle
2:) All employments are being returned whereas the query matched one of them
3:) Is there any better way to do it ? via index or via $search ? I did not find much of information in mongoose documentation to create search index (and I also don't really know how to create a compound index to make it work)
I know there might be a lot of answers but none of them helped or I was not able to make them work ... I am really new to mongodb I have been working with relational databases via SQL or through ORM so my mongodb concepts and knowledge is limited.
So please let me know if there is a better solution to do it. or something to make the current one working.

You can use one of the aggregation query below to get this result:
[
{
"jobTitle": [
"Software Engineer",
"Software Manager",
"Software Developer"
]
}
]
Query is:
First using $unwind twice to deconstructs the arrays and get the values.
Then $match to filter by values you want using $regex.
Then $group to get all values together (using _id: null and $addToSet to no add duplicates).
And finally $project to shown only the field you want.
User.aggregate({
"$unwind": "$resumes"
},
{
"$unwind": "$resumes.employments"
},
{
"$match": {
"resumes.employments.jobTitle": {
"$regex": "software",
"$options": "i"
}
}
},
{
"$group": {
"_id": null,
"jobTitle": {
"$addToSet": "$resumes.employments.jobTitle"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here
Also another option is using $filter into $project stage:
Is similar as before but using $filter instead of $unwind twice.
User.aggregate({
"$unwind": "$resumes"
},
{
"$project": {
"jobs": {
"$filter": {
"input": "$resumes.employments",
"as": "e",
"cond": {
"$regexMatch": {
"input": "$$e.jobTitle",
"regex": "Software",
"options": "i"
}
}
}
}
}
},
{
"$unwind": "$jobs"
},
{
"$group": {
"_id": null,
"jobTitle": {
"$addToSet": "$jobs.jobTitle"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here

How to properly iterate and parse nested objects in javascript with or without lodash

I'm currently working on a service that returns the following payload:
{
"account1": {
"app1": {
"status": "Online",
"comments": "blah blah",
"events": [
{
"date": "some date",
"generated_by": "some user"
}
]
}
},
"account2": {
"app1": {
"status": "Offline",
"comments": "blah blah bleh",
"events": [
{
"date": "some date",
"generated_by": "some user"
}
]
},
"app2": {
"status": "Online",
"comments": "blah blah",
"events": [
{
"date": "some date",
"generated_by": "some user"
}
]
}
}
}
I'm trying to render a table with the following fields:
-------------------------------
Application | Account | Status
-------------------------------
app1 | account1 | Online
app1 | account2 | Offline
app2 | account2 | Online
Normally this would be easy to do if my payload would be something like a list of objects but I'm kinda stuck here.
I tried to normalize this payload by extracting each of the fields and creating a new payload by doing something like the following:
const extractAccountNumber = Object.values(payload).map(account => ({account: account}))
which would return:
[
{
"account": "account1"
},
{
"account": "account2"
}
]
I wanted to move on to app name the same way and once I get all my fields I would merge the payload. This has proven to be super cumbersome and I'm sure there is a better, more efficient way to achieve this which I'm probably missing. Any feedback would help me a ton to understand how this can be achieved using javascript with or without lodash.

Iterating by first level and then by second level:
table = [];
for (accountName in apiResponse) {
account = apiResponse[accountName];
for (appName in account) {
app = account[appName];
table.push({
application: appName,
account: accountName,
status: app.status,
});
}
}
Then table is something like this:
[
{
"application": "app1",
"account": "account1",
"status": "Online"
},
{
"application": "app1",
"account": "account2",
"status": "Offline"
},
{
"application": "app2",
"account": "account2",
"status": "Online"
}
]

Can try something like
Object.entries(o).map(([accountName, value]) => ({
account: accountName,
apps: Object.entries(value)
.map(([appName, value]) => ({name: appName, ...value }))
}))
Not sure about structure. Where to put app1 from account2 in that table?

MongoDB: Fetching documents from different collections in one query

I have two different collections for two different type of products. Now, I want to fetch all documents from both collections for a particular user.
I know I can do that with 2 queries for each collection, merging them on the server side and sending the result to the user. Downside of this is that I have to fetch all documents for a user from both collections, which is not good for pagination. That is why I want to do it in one query, so I can leave a pagination logic to MongoDB as well.
Here is the example of collections and expected result:
Products_type_1
[
{
"name": "product_1",
"user": "user_1",
...
},
{
"name": "product_2",
"user": "user_2",
...
}
]
Products_type_2
[
{
"name": "product_3",
"user": "user_1",
...
},
{
"name": "product_4",
"user": "user_2",
...
}
]
The expected result:
[
{
"type": "Products_type_1",
"name": "product_1",
"user": "user_1",
...
},
{
"type": "Products_type_2",
"name": "product_3",
"user": "user_1",
...
}
]

You can use aggregation framework with $unionWith stage:
db.Products_type_1.aggregate([
{
"$match": {
"user": "user_1"
}
},
{
$unionWith: {
coll: "Products_type_2",
pipeline: [
{
"$match": {
"user": "user_1"
}
}
]
}
}
])
Playground: https://mongoplayground.net/p/v0dKCwiKsZU
If you want to use pagination you will need to add sort stage to ensure consistent order of the documents in the result.

Firstly I would query the logic of having a different collection for the different 'product_type_x'. If you had a single collection with an added field...
{ "productType" : 1,
...
},
That way that issue has just been resolved, everything to do with Procts is now accessible in a single collection. Aggregation of your data now becomes simple (by comparison)

Sequelize include inside another include

The problem:
I can't retrieve an include inside include with Sequelize#5.21.2.
The first include its OK. But when i try to grab the other include, i only receive null.
The code:
let planCodes = ['monthly', 'annual'];
const plans = await Plan.findAll({
where: {
code: {
[Op.or]: planCodes
}
},
include: [{
model: PlanItem,
as: 'plan_items',
include: [{
model: Product,
as: 'product',
}]
}]
});
Model associations:
Plan
Plan.hasMany(PlanItem, { foreignKey: "plan_id" });
PlanItem
PlanItem.hasOne(Product, { foreignKey: "plan_item_id" });
The response:
[
{
"id": 175231,
"name": "mensal",
"interval": "months",
"interval_count": 1,
"code": "monthly",
"status": "active",
"metadata": {},
"created_at": "XXXX-XX-XX",
"updated_at": "XXXX-XX-XX",
"plan_items": [
{
"id": 190651,
"plan_id": 175231,
"product_id": 655939,
"created_at": "XXXX-XX-XX",
"updated_at": "XXXX-XX-XX",
"product": null
}
]
}
]
I already checked all details with the IDs at database and everything is OK.

I will admit that Sequelize likes to take all the spontaneity out of relationships, but you should use your alias in both the model definition AND the query. And they have to be the same. I'm sure there is some leeway, as you are, in fact, returning plan_items values. But whether "best practice" or "mandate", give this a go:
Plan.hasMany(PlanItem, { as: "plan_items", foreignKey: "plan_id" });
PlanItem.hasOne(Product, { as: "product", foreignKey: "plan_item_id" });
// You may also try defining the Product/PlanItem relationship the other way
Product.belongsTo(PlanItem, { as: "plan_item", foreignKey: "plan_item_id" });