Develop Reference

MongoDB/JS - Advanced Faceted search - How to get only relevant categories/values

First of all thank you for opening my question. Before I write the actual question I write some details related to my project to fully understand what I want to achieve.
I have Product collection. In this collection each document has different keys and values (the datas are dynamic). On the example below the first two document has the same keys but the values are different, and the third document is missing the key:'Color':
{
_id:1
MainCategory:Vehicle
SubCategory:Car
Model:BMW
Color:Black
}
{
_id:2
MainCategory:Vehicle
SubCategory:Car
Model:Audi
Color:Red
}
{
_id:3
MainCategory:Vehicle
SubCategory:Car
Model:BMW
}
I have a Category collection this is based on MainCategory+SubCategory and here I store the keys and values, this will be used for a better filtering speed/experience.
_id:..
MainCategory:Vehicle
SubCategory:Car
Fields:{
Model: {BMW:[_id1,_id3], Audi:[_id2]},
Color: {Black:[_id1],Red:[_id2]}
}
I want to make a filtering option based on Category collection, where all of the 'Fields' keys will be the filtering categories, and the values what these keys have will be the filtering options. The example above is the initial filtering state:
MainCategory-Vehicle
SubCategory-Car
Model - BMW, Audi
Color - Black, Red
And now here comes the question part: If a user selects the Model=BMW I want to retrieve only the relevant categories and values from my Category collection. In this case relevant means the following steps:
1.From Fields->Model->BMW I want to get the array of products which has the category value BMW -> i will get [_id1,_id3] . This will be the data which I will show on my UI.
The Advanced Part:
2.When I get the IDs I want to check if other objects from 'Fields' are containing any of the products from the above list:in this case the Fields->Color->Black contains the _id1
3.If an object is containing one of my products I want to retrieve the key as filtering category -> Color and the value which contained it -> Red
After this I will want to get the output something like this:
Products:[_id1,_id3]
Model:[BMW]
Color:[Black]
By this my filtering will show only the relevant categories and values and will look like this:
MainCategory:Car
SubCategory:Vehicle
Model: BMW
Color:Black
Example 2
Another example and this part makes hard to achieve the given output is: If a user selects the Color:Red value I want to retrieve the _id2 product and get from the Model only the 'Audi' value previously.
So in this case the filtering will look like:
Product:[_id2]
Model:Audi
Color:Red
Example 3
A third problem can be when a user selects more than one filtering category-value. For example if BMW has an addiotnal _id4, and _id4 is present in Color Red too. If a user selects Model:BMW and Color:Red for filtering option, we should join at the final output the BMW id array with the Red id array to get back only the BMW and Red options.
This case the filtering:
Product [_id1,_id3,_id4]
Model:BMW
Color:Red
How can I achieve to find the array in my object and check inside that object if any of the array values are present in other arrays? Is it possible to do it with only one query or it would be optimal to make two queries? Thanks in advance.
Update:
Just an additional question: Is this kind of data structuring is good for Category collection or maybe I should think in totally other structure? Maybe can you advice something else for handling easier and faster the data?
Update2
//Product creation part -> the productbod contains the above mentioned key-value pairs
export const createProduct = async (productbod) => {
try {
if (!ObjectID.isValid(productbod.userId)) throw 'Invalid MongoDB ID.'
const db=getDB()
const product= db.collection('products')
const finalProduct = await product.insertOne(productbod)
return finalProduct
} catch (err) {
throw error
}
}

At the very basic, you can create the 2 categories as follows
Category
- _id
- category
Products
_ _id (the product _id )
- category (use the _id from Category)
- subcategory (use string, or if you want, create a subcategory collection)
- name
- attributes (color, model)
When you want to find all the categories
db.category.find()
When you need to find all products in a category, use the Category _id
db.products.find({ cateogry: _id})
When you need to find all products in a category, AND sub category
db.products.find({ category: _id, subcategory: 'salon' }) //or subcategory_id
When you need to find a single product
db.products.findOne({ _id }) //with the product_id
When you want to find a product by name
db.products.find({ name: { $regex : 'some search term', $option: 'i' } }) // where search term can be part of the product name
When you want to find all categories that contains BMW
db.products.aggregate([
{
$match: { "attributes.model": "BMW"
},
{
$group: { _id: "$category" }
}
])
Once you have started creating the database, then you move on to creating the views for your db with aggregation, as well as javascript. (that you can create a separate stackoverflow question)
The above data structure and example queries should be sufficient to handle your "advanced filtering"

How to assign users to a group and sort them using firestore reference field type

I have a collection of employees that has data sent to it. Right now there is 4 employees but eventually there will be many more.
I want to add a grouping feature so that the user can sort the employees by their group. I am trying to find the best way to assign these employees groups and I found the reference field type in cloud firestore and thought I could use it to solve my problem. But I am stuck and not sure the most efficeient way to use it to link employees to a group.
This is my database. Right now I have the employees doc (ex. 2569) and inside that is a sub-collection with 2 documents in itself.
So end goal is to assign employees groups and then be able to sort and display them separately. Right now I have the group name assigned in articles/group -> groupName: "example".
(display them hopefully with ".Where( "groupName" "==" "example" ) somehow in code without hard-coding the group name. The group name will be created by the user so it could be anything)
Is what I am doing a good start? I know this question is a little odd but I am stuck and could really use some pointers on where to head next.

A collection group query would allow you to query all articles regardless of which employee contained them:
db.collectionGroup('articles')
.where('groupName', '==', 'X')
.get()
This would match documents in any collection (i.e. employees) where the last part of the collection path is articles. If you would like to find the employees who belong to a certain groupName, you may want to find the parent by retrieving the collection this DocumentReference belongs to.
Once you have the parent of the CollectionReference, you will get a reference to the containing DocumentReference of your subcollection.

Find Documents whose Subdocument Fields partially match given Criteria with Mongoose

I have using nodeJS and mongoose/mongoDB to build an API. Consider the following model for a generic Model (Gmodel):
{
name: "teleni",
type: "person",
data: mixed
}
The data field in this document is a mixed type so it can be anything, but I intend for it to be a subdocument/object in most cases (depending on the value of the type field). If I wanted to find all the documents that were of type person I would execute the following code:
Gmodel.find({type: "person"}, callbackFunction)
Now consider another instance of the Model:
{
name: "teleni",
type: "person",
data: {
data_type: "event",
date: "Jan 30 2019"
}
}
My issue is that I want to use mongoose to find all documents where the data field is an object/document where the data_type field is "event". I have tried the following code:
Gmodel.find({data: {data_type: "event"}}, callbackFunction)
However nothing is returned when I execute this code because find is looking for Documents where the data field is exactly {data_type: "event"}. So how do I use find to retrieve Documents whose subdocuments (of mixed type) partially match a criteria, similar to how find works on top level fields.

You can use the dot syntax data.data_type to specify a field in subdocument. This way it would match on the basis on this field only, not the entire subdocument as in your original code.
Gmodel.find({data.data_type: "event"}, callbackFunction)

rally lookback is lowest portfolio Item an indexed field?

The lookback api docs say PortfolioItem field is an index. Is the lowest portfolio item type also an index?
E.g: the Portfolio Item types in my workspace are Product, Milestone and Feature. will Feature be an index in Lookback API in addition to PortfolioItem?
The reason I ask is because only top-level UserStories have the PortfolioItem field, but both top-level and child UserStories have the Feature field. I want to query all User Stories under a particular Feature, which means I can't use PortfolioItem field, because it will not include child User Stories, only top-level User Stories.
Example of what i want to do if Feature is indexed:
Ext.create('Rally.data.lookback.SnapshotStore', {
listeners: {
load: function(store, data, success) {
//do stuff
}
},
autoLoad:true,
limit:Infinity,
fetch: ['ScheduleState', 'PlanEstimate', 'Feature', 'ObjectID'],
compress:true,
find: {
_TypeHierarchy: 'HierarchicalRequirement',
Children: null,
Release: //a release OID
},
hydrate: ['ScheduleState']
});

There may be some confusion with the use of the word 'index'. Some fields are "indexed" for fast lookup..."Feature" isn't one of them, though it is a valid field and you can search for it. More correctly, the field that is the lowest-level Portfolio Item type is kept in the snapshots.* Given what you're asking for, adding "Feature": {oid} to the find should give you what you want.
* The distinction is due to the fact that the label "Feature" can be changed to something else, so what is "Feature" in one workspace might be "Thing" in another.

The _ItemHierarchy field includes all levels of PortfolioItems through all levels of Stories to Defects, Tasks and (I'm pretty sure) TestCases. So, if you want "all User Stories under a particular Feature", simply specify find: {_ItemHierarchy: 1234567} where 1234567 is the ObjectID of the Feature. You could combine this with the _TypeHierarchy and Release clauses. If you combine it with the Children and _TypeHiearchy clauses as you proposes, that will give you only leaf stories as opposed to all the levels. This is ideal if you are doing aggregations on fields like sum of PlanEstimate (points) or TaskActual, etc.
Note, I don't think this has anything to do with being indexed, so I may be misunderstanding your question. Please accept my apology if that's the case.

Mongoose - recursive query (merge from multiple results)

I have the following generic schema to represent different types of information.
var Record = new Schema (
{
type: {type: String}, // any string (foo, bar, foobar)
value: {type: String}, // any string value
o_id: {type:String}
}
);
Some of the records based on this schema have:
type="car"
value="ferrari" or
value="ford"
Some records have type "topspeed" with value "210" but they always share o_id (e.g. related "ferrari has this topspeed"). So if "ferrari has top speed 300", then both records have same o_id.
How can I make query to find "ferrari with topspeed 300" when I don't know o_id?
The only solution I found out is to select cars "ferrari" first and then with knowledge of all o_id for all "ferrari" use it to find topspeed.
In pseudocode:
Record.find({type:"car", value:"ferrari"}, function(err, docs)
{
var condition = [];// create array of all found o_id;
Record.find({type:"topspeed", value:"300"}...
}
I know that some merging or joining might not be possible, but what about some chaining these conditions to avoid recursion?
EDIT:
Better example:
Lets imagine I have a HTML document that contains DIV elements with certain id (o_id).
Now each div element can contain different type of microdata items (Car, Animal...).
Each microdata item has different properties ("topspeed", "numberOfLegs"...) based on the type (Car has a topspeed, animal numberOfLegs)
Each property has some value (310 kph, 4 legs)
Now I'm saving these microdata items to the database but in a general way, agnostic of the type and values they contain since the user can define custom schemas from Car, to Animal, to pretty much anything). For that I defined the Record schema: type consists of "itemtype_propertyname" and value is value of the property.
I would eventually like to query "Give me o_id(s) of all DIV elements that contain item Ferrari and item Dog" at the same time.
The reason for this general approach is to allow anyone the ability to define custom schema and corresponding parser that stores the values.
But I will have only one search engine to find all different schemas and value combinations that will treat all possible schemas as a single definition.

I think it'd be far better to combine all records that share an o_id into a single record. E.g.:
{
_id: ObjectId(...),
car: "ferarri",
topspeed: 300
}
Then you won't have this problem, and your schema will be more efficient both in speed and storage size. This is how MongoDB is intended to be used -- heterogenous data can be stored in a single collection, because MongoDB is schemaless. If you continue with your current design, then no, there's no way to avoid multiple round-trips to the database.