Since nested population isn't available I need to pass my custom attributes manually. In my specific case this means: a customer has many projects, a project has many contributors.
Customer.find().populate('projects').exec(function(err, customer) {
Response looks like
[
{
"projects": [
{ "name": "First project" }
],
"customer": "John Doe"
},
{
"projects": [
{ "name": "Another project" },
{ "name": "And another one" }
],
"customer": "Susan Doe"
}
]
I'm iterating through the projects and want to attach a contributors attribute. I've tried
customer.forEach(function(customer, index) {
customer.projects.forEach(function(project, index) {
ProjectContributor.find({
project: project.id
}).exec(function(err, contributor) {
project.contributors = contributors;
});
But project.contributors is still undefined. Why? And how to attach these custom attributes?
There are many errors in you code.
Customer.find().populate('projects').exec(function(err, customers) {
customers.forEach(function(customer, index) {
customer.projects.forEach(function(project, index) {
ProjectContributor.findOne({project: project.id}) // use findOne since you only want one project at a time
.populate('contributors')
.exec(function(err, projectContributor) {
project.contributors = projectContributor.contributors; // contributors is in projectContributor
});
});
});
});
Related
I have two different collections for two different type of products. Now, I want to fetch all documents from both collections for a particular user.
I know I can do that with 2 queries for each collection, merging them on the server side and sending the result to the user. Downside of this is that I have to fetch all documents for a user from both collections, which is not good for pagination. That is why I want to do it in one query, so I can leave a pagination logic to MongoDB as well.
Here is the example of collections and expected result:
Products_type_1
[
{
"name": "product_1",
"user": "user_1",
...
},
{
"name": "product_2",
"user": "user_2",
...
}
]
Products_type_2
[
{
"name": "product_3",
"user": "user_1",
...
},
{
"name": "product_4",
"user": "user_2",
...
}
]
The expected result:
[
{
"type": "Products_type_1",
"name": "product_1",
"user": "user_1",
...
},
{
"type": "Products_type_2",
"name": "product_3",
"user": "user_1",
...
}
]
You can use aggregation framework with $unionWith stage:
db.Products_type_1.aggregate([
{
"$match": {
"user": "user_1"
}
},
{
$unionWith: {
coll: "Products_type_2",
pipeline: [
{
"$match": {
"user": "user_1"
}
}
]
}
}
])
Playground: https://mongoplayground.net/p/v0dKCwiKsZU
If you want to use pagination you will need to add sort stage to ensure consistent order of the documents in the result.
Firstly I would query the logic of having a different collection for the different 'product_type_x'. If you had a single collection with an added field...
{ "productType" : 1,
...
},
That way that issue has just been resolved, everything to do with Procts is now accessible in a single collection. Aggregation of your data now becomes simple (by comparison)
Goal
I want to develop a middleware in TypeScript that filters the response of a REST API and returns only defined properties.
It should work generically, i.e. independent of specific entities. Neither their properties nor the exact depth (e.g. with any number of relations) should be necessarily known.
Example
An author has any number of articles with any number of comments.
[
{
"name": "John Doe",
"email": "john#doe.com",
"articles": [
{
"title": "Lalilu 1",
"text:": "la li lu",
"comments": [
{
"author": "Bendthatdict Cumberstone",
"text": "Great article!"
},
{
"author": "Bendthatdict Cumberstone",
"text": "Great article!"
}
]
},
{
"title": "Lalilu 1",
"text:": "la li lu",
"comments": [
{
"author": "Bendthatdict Cumberstone",
"text": "Great article!"
},
{
"author": "Bendthatdict Cumberstone",
"text": "Great article!"
}
]
}
]
},
{
"name": "Jane Doe",
"email": "jane#doe.com",
"articles": [
{
"title": "Lalilu 1",
"text:": "la li lu",
"comments": [
{
"author": "Bendthatdict Cumberstone",
"text": "Great article!"
},
{
"author": "Bendthatdict Cumberstone",
"text": "Great article!"
}
]
},
{
"title": "Lalilu 1",
"text:": "la li lu",
"comments": [
{
"author": "Bendthatdict Cumberstone",
"text": "Great article!"
},
{
"author": "Bendthatdict Cumberstone",
"text": "Great article!"
}
]
}
]
}
]
Now I want to specify that it should return everything except the "text" of each article and the "author" of each comment.
Syntax could look like this with glob notation:
select("*,!articles.text,!articles.comments.author")
Approach
For objects and nested objects it is quite simple, e.g. with pick() and omit() of "lodash", but I fail when arrays step into the game.
I did some research and came across packages such as json-mask, node-glob or glob-object but none of them exactly met my needs and I was not able to combine them for success.
Question
What is the most efficient way to generically filter an arbitrarily nested JSON with any number of further objects / arrays?
Also, how could the TypeScripts type system be used to advantage?
I would be very grateful for general coding approaches or even tips for a package that can already do this!
In short I would break this up into functions. You could create helpers that do more or less what you want with a string/filter as you show however I'd work it in reverse. Get a nice way to iterate so any post processing can be done, then build your helpers as you wish against that. Here's what I mean:
Example
export interface IComment {
author: string;
text: string;
}
export interface IArticle {
title: string;
text: string;
comments: IComment[];
}
export interface IComposer {
name: string,
email: string,
articles: IArticle[];
}
// Remove items from list for brevity sake...
const authorList = [
{
"name": "John Doe",
"email": "john#doe.com",
"articles": [
{
"title": "Lalilu 1",
"text": "la li lu",
"comments": [
{
"author": "Bendthatdict Cumberstone",
"text": "Great article!"
}
]
}
]
}
] as IComposer[];
/**
* Accepts JSON string or array of type.
*
* #param arr a JSON string containing array of type or array of type.
*/
export function selectFrom<T extends Record<string, any>>(arr: string | T[]) {
// If you want to use this route I would suggest
// also a function to validate that the JSON is
// shaped correctly.
if (typeof arr === 'string')
arr = JSON.parse(arr);
const collection = arr as T[];
const api = {
filters: [],
register,
run
};
/**
* Register a search op.
* #param fn function returning whether or not to filter the result.
*/
function register(fn: (obj: T) => Partial<T>) {
if (typeof fn === 'function')
api.filters.push(fn);
return api;
}
/**
* Run registered ops and filter results.
*/
function run() {
return collection.reduce((results, obj) => {
let result = obj;
// Don't use reducer here as you can't break
// and would unnecessarily loop through filters
// that have no need to run, use for of instead.
for (const filter of api.filters) {
// if we set the result to null
// don't continue to run filters.
if (!result) break;
// Pipe in the previous result, we start with
// original object but it's shape could change
// so we keep iterating with the previous result.
const filtered = filter(result);
// update the result.
if (filtered)
result = filtered;
}
if (result)
results.push(result);
return results;
// If changing the object you're going to
// end up with partials of the original
// shape or interface.
}, [] as Partial<T>[]);
}
return api;
}
Usage
By making this function based at the core you have a lot more flexibility. From there you could make a simple helper that maps your Glob or SQL like string to the pre-defined filter functions. Let me know if you have further questions.
const filtered =
selectFrom(authorList)
.register((composer) => {
composer.articles = composer.articles.map(article => {
const { text, ...filteredArticle } = article;
filteredArticle.comments = filteredArticle.comments.map(comment => {
const { author, ...filteredComment } = comment;
return filteredComment as typeof comment;
});
// Note setting to type of IArticle here so typescript
// doesn't complain, this is because you are removing props
// above so the shape changes so you may want to consider
// setting the props you plan to strip as optional or make
// everything a partial etc. I'll leave that to you to decide.
return filteredArticle as typeof article;
});
return composer;
})
.run();
What's Next
From here to get where you want it's about string parsing. Keep in mind Lodash does support gets down into nested values in an array. You can see this here in the docs.
Given that you could leverage Lodash using both _.get _.omit... etc along with a little parsing using dot notation.
Done this very thing with permissions. As such I feel strongly you need to start with a simple api to process then from there make your map from either Glob like or SQL string to those helpers.
I am using mongoose in nodejs(express) in backend. My array structure has THREE levels. At third level, some files are present. But I need to add entries at any level as per user demand.
[
{
"name": "A folder at",
"route": "level1_a"
},
{
"name":"Another folder at Level1",
"route": "level1_b",
"children":
[
{
"name": "A folder at Level2",
"route": "level1_b/level2_a",
"children":
[
{
"name": "A folder at Level3",
"route": "level1_b/level2_a/level3_a",
"children":
[
{
"name": "A file at last level",
"route": "level1_b/level2_a/level3_a/file1"
},
{
"name": "Add a new File",
"route":"level1_b/level2_a/level3_a/new_file"
}
]
},
{
"name": "Add Folder at Level3",
"route":"level1_b/level2_a/new_folder"
}
]
},
{
"name": "Add Folder at level2",
"route":"level1_b/new_folder"
}
]
},
{
"name": "Add Folder at Level1",
"route":"new_folder"
}
]
Now I have to add an entry at a specified position. Suppose at level2, I need to add a folder. For adding, two parameters are sent from angular to the backend. These will be 'name' and a 'route'. So my entry would be having {name: 'Products', route: 'level1_a/products'} and similarily should be placed at correct position i.e. inside the children of level1_a.
My backend has a schema which would be like:
const navSchema = mongoose.Schema({
name:{type:String,required:true},
route:{type:String},
children:{
type: {
name:{type:String,required:true},
route:{type:String},
}}
});
module.exports = mongoose.model('NatItems',navSchema);
And the API would be like:
router.post('/navlist',(req,res,next)=>{
const name= req.body.folder;
const route= req.body.url;
console.log(folder,url);//it will be required parameters like name: 'Products', route:'level1_a/products'
//let pathArray = route.split('/'); //if you want you can split the urls in the array
//Help me write the code here
res.status(201).json({
message:"Post added successfully!"
})
})
Please help me in adding entries in db. I know navlist.save() adds an entry directly but I am not able to add entries in a nested manner.
PS: I can't change the array structure because this array is easily read by angular and a complete navigation menu is made!! I am working for first time in nodejs and mongoose, so I am having difficulty in writing code with mongoose function.
For the scenario you've provided ({name: 'Products', route: 'level1_a/products'}) the update statement is pretty straightforward and looks like this:
Model.update(
{ route: "level1_a" },
{ $push: { children: {name: 'Products', route: 'level1_a/products'} } })
Things are getting a little bit more complicated when there are more than two segments in the incoming route, e.g.
{ "name": "Add a new File", "route":"level1_b/level2_a/level3_a/new_file2" };
In such case you need to take advantage of the positional filtered operator and build arrayFilters and your query becomes this:
Model.update(
{ "route": "level1_b"},
{
"$push": {
"children.$[child0].children.$[child1].children": {
"name": "Add a new File",
"route": "level1_b/level2_a/level3_a/new_file2"
}
}
},
{
"arrayFilters": [
{
"child0.route": "level1_b/level2_a"
},
{
"child1.route": "level1_b/level2_a/level3_a"
}
]
})
So you need a function which loops through the route and builds corresponding update statement along with options:
let obj = { "name": "Add a new File", "route":"level1_b/level2_a/level3_a/new_file2" };
let segments = obj.route.split('/');;
let query = { route: segments[0] };
let update, options = {};
if(segments.length === 2){
update = { $push: { children: obj } }
} else {
let updatePath = "children";
options.arrayFilters = [];
for(let i = 0; i < segments.length -2; i++){
updatePath += `.$[child${i}].children`;
options.arrayFilters.push({ [`child${i}.route`]: segments.slice(0, i + 2).join('/') });
}
update = { $push: { [updatePath]: obj } };
}
console.log('query', query);
console.log('update', update);
console.log('options', options);
So you can run:
Model.update(query, update, options);
According to the smartsheet API Docs, I should be able to use "level" parameter in my options to get a complex object for Multi-Contact columns.
Unfortunately all I'm getting in return is value and displayValue.
Am I doing something wrong here?
var options = {
id: SHEET_ID, //Id of sheet
queryParameters = {
include: ["objectValue"],
level: 1
}
}
ss.sheets.getSheet(options)
.then(function (results) {
console.log(results.rows[args[0]].cells[6])
})
The above code returns:
{ columnId: 8746190272522116, displayValue: 'John Smith, Danny Doe' }
I've verified (using Postman) that Smartsheet API does indeed support the scenario you've described. i.e., if I submit this Get Sheet request:
https://api.smartsheet.com/2.0/sheets/5831916227192708?include=objectValue&level=1
...then the response does include the complex object for a multi-contact cell in my sheet:
{
"id": 5831916227192708,
...
"rows": [
{
"id": 5942480978372484,
...
"cells": [
{
"columnId": 3992195570132868,
"objectValue": {
"objectType": "MULTI_CONTACT",
"values": [
{
"objectType": "CONTACT",
"email": "john_doe#test.com",
"name": "John Doe"
},
{
"objectType": "CONTACT",
"email": "jane_doe#test.com",
"name": "Jane Doe"
}
]
},
"displayValue": "John Doe, Jane Doe"
},
...
]
},
...
]
}
However, it looks like the Smartsheet JavaScript SDK doesn't yet support this scenario.
It's not unusual for SDK updates to lag a bit behind the release of new API features. You might consider logging an issue in the JavaScript SDK repo to request that support for this scenario be added -- or better yet, submit a PR to that repo that adds support for this scenario. In the meantime, you'll need to implement this functionality yourself within your integration (i.e., since you can't rely on the out-of-the-box SDK functionality to provide it at this time).
You just need to remove the array notations from your options definition:
var options = {
id: SHEET_ID, //Id of sheet
queryParameters = {
include: "objectValue",
level: 1
}
}
ss.sheets.getSheet(options)
.then(function (results) {
console.log(results.rows[args[0]].cells[6])
})
I'm wondering how I can compare arrays of (nested) objects in Mongoose.
Considering the data below, I would like to get results when the name properties match. Could anyone help me with this?
Organisation.find( {
$or: [
{ "category_list": { $in: cat_list } },
{ "place_topics.data": { $in: place_tops } }
]
}
)
Let's say that this is the data stored in my MongoDB:
"category_list": [
{
"id": "197750126917541",
"name": "Pool & Billiard Hall"
},
{
"id": "197871390225897",
"name": "Cafe"
},
{
"id": "218693881483234",
"name": "Pub"
}
],
"place_topics": {
"data": [
{
"name": "Pool & Billiard Hall",
"id": "197750126917541"
},
{
"name": "Pub",
"id": "218693881483234"
}
]
}
And let's say that these are the arrays I want to compare against (almost the same data):
let cat_list = [
{
"id": "197750126917541",
"name": "Pool & Billiard Hall"
},
{
"id": "197871390225897",
"name": "Cafe"
},
{
"id": "218693881483234",
"name": "Pub"
}
]
let place_tops = [
{
"name": "Pool & Billiard Hall",
"id": "197750126917541"
},
{
"name": "Pub",
"id": "218693881483234"
}
]
When there are "multiple conditions" required for each array element is when you actually use $elemMatch, and in fact "need to" otherwise you don't match the correct element.
So to apply multiple conditions, you would rather make an array of conditions for $or instead of shortcuts with $in:
Organizations.find({
"$or": [].concat(
cat_list.map( c => ({ "category_list": { "$elemMatch": c } }) ),
place_tops.map( p => ({ "place_topics": { "$elemMatch": p } }) )
)
})
However, if you take a step back and think logically about it, you actually named one of the properties "id". This would generally imply in all good practice that the value is in fact ""unique".
Therefore, all you really should need to do is simply extract those values and stick with the original query form:
Organizations.find({
"$or": [
{ "category_list.id": { "$in": cat_list.map(c => c.id) } },
{ "place_topics.id": { "$in": place_tops.map(p => p.id) } }
]
})
So simply mapping both the values and the property to "match" onto the "id" value instead. This is a simple "dot notation" form that generally suffices when you have one condition per array element to test/match.
That is generally the most logical approach given the data, and you should apply which one of these actually suits the data conditions you need. For "multiple" use $elemMatch. But if you don't need multiple because there is a singular match, then simply do the singular match