I am quite new to using databases and have only used MongoDB so far. I was planning on making a app for a shop to create their menu(s).
There are several interfaces I would define beforehand.
interface FoodItem {
id: number
name: string,
price: number
}
interface Category {
id: number,
name: string,
food: FoodItem[]
}
interface Menu {
id: number
name:string,
categories: Category[]
}
From the above schemas, I was wondering if this is ok for a Postgres database, where all the data resides in the Menus table, and fetching a Menu will yield all nested Categories and FoodItems.
I am confused because from other examples of such data models, the array of Categories will simply be replaced by an array of IDs that points to their respective Categories and the same for FoodItems as well. In that case, to fetch a Menu, the database will also have to fetch, say 3 other Categories and in those categories fetch another say 10 other FoodItems via their IDs. Wouldn't that be a performance issue since the database has to query each nested item in the array, as opposed to doing it all in one single query fetch from the root Menu?
For my app that I'm designing, each Category and FoodItem is unique to their parents and will not be shared with other Categories/Menus. In that case there wouldn't be duplicate entries of FoodItem when using the first data model. Of course I would not mind using the second data model since it is much cleaner and easy to reference without deeply nested objects.
So for the second data model, would there a big performance decrease when my dataset starts to grow larger and larger?
Related
First of all thank you for opening my question. Before I write the actual question I write some details related to my project to fully understand what I want to achieve.
I have Product collection. In this collection each document has different keys and values (the datas are dynamic). On the example below the first two document has the same keys but the values are different, and the third document is missing the key:'Color':
{
_id:1
MainCategory:Vehicle
SubCategory:Car
Model:BMW
Color:Black
}
{
_id:2
MainCategory:Vehicle
SubCategory:Car
Model:Audi
Color:Red
}
{
_id:3
MainCategory:Vehicle
SubCategory:Car
Model:BMW
}
I have a Category collection this is based on MainCategory+SubCategory and here I store the keys and values, this will be used for a better filtering speed/experience.
_id:..
MainCategory:Vehicle
SubCategory:Car
Fields:{
Model: {BMW:[_id1,_id3], Audi:[_id2]},
Color: {Black:[_id1],Red:[_id2]}
}
I want to make a filtering option based on Category collection, where all of the 'Fields' keys will be the filtering categories, and the values what these keys have will be the filtering options. The example above is the initial filtering state:
MainCategory-Vehicle
SubCategory-Car
Model - BMW, Audi
Color - Black, Red
And now here comes the question part: If a user selects the Model=BMW I want to retrieve only the relevant categories and values from my Category collection. In this case relevant means the following steps:
1.From Fields->Model->BMW I want to get the array of products which has the category value BMW -> i will get [_id1,_id3] . This will be the data which I will show on my UI.
The Advanced Part:
2.When I get the IDs I want to check if other objects from 'Fields' are containing any of the products from the above list:in this case the Fields->Color->Black contains the _id1
3.If an object is containing one of my products I want to retrieve the key as filtering category -> Color and the value which contained it -> Red
After this I will want to get the output something like this:
Products:[_id1,_id3]
Model:[BMW]
Color:[Black]
By this my filtering will show only the relevant categories and values and will look like this:
MainCategory:Car
SubCategory:Vehicle
Model: BMW
Color:Black
Example 2
Another example and this part makes hard to achieve the given output is: If a user selects the Color:Red value I want to retrieve the _id2 product and get from the Model only the 'Audi' value previously.
So in this case the filtering will look like:
Product:[_id2]
Model:Audi
Color:Red
Example 3
A third problem can be when a user selects more than one filtering category-value. For example if BMW has an addiotnal _id4, and _id4 is present in Color Red too. If a user selects Model:BMW and Color:Red for filtering option, we should join at the final output the BMW id array with the Red id array to get back only the BMW and Red options.
This case the filtering:
Product [_id1,_id3,_id4]
Model:BMW
Color:Red
How can I achieve to find the array in my object and check inside that object if any of the array values are present in other arrays? Is it possible to do it with only one query or it would be optimal to make two queries? Thanks in advance.
Update:
Just an additional question: Is this kind of data structuring is good for Category collection or maybe I should think in totally other structure? Maybe can you advice something else for handling easier and faster the data?
Update2
//Product creation part -> the productbod contains the above mentioned key-value pairs
export const createProduct = async (productbod) => {
try {
if (!ObjectID.isValid(productbod.userId)) throw 'Invalid MongoDB ID.'
const db=getDB()
const product= db.collection('products')
const finalProduct = await product.insertOne(productbod)
return finalProduct
} catch (err) {
throw error
}
}
At the very basic, you can create the 2 categories as follows
Category
- _id
- category
Products
_ _id (the product _id )
- category (use the _id from Category)
- subcategory (use string, or if you want, create a subcategory collection)
- name
- attributes (color, model)
When you want to find all the categories
db.category.find()
When you need to find all products in a category, use the Category _id
db.products.find({ cateogry: _id})
When you need to find all products in a category, AND sub category
db.products.find({ category: _id, subcategory: 'salon' }) //or subcategory_id
When you need to find a single product
db.products.findOne({ _id }) //with the product_id
When you want to find a product by name
db.products.find({ name: { $regex : 'some search term', $option: 'i' } }) // where search term can be part of the product name
When you want to find all categories that contains BMW
db.products.aggregate([
{
$match: { "attributes.model": "BMW"
},
{
$group: { _id: "$category" }
}
])
Once you have started creating the database, then you move on to creating the views for your db with aggregation, as well as javascript. (that you can create a separate stackoverflow question)
The above data structure and example queries should be sufficient to handle your "advanced filtering"
I am working on my first Ext JS project and have my source data downloaded from a web server held in a store. Now I want to have various grids all show some variation of that source data. For example, grid 1 would be the source data grouped by 1 particular column. Grid 2 would be grouped by a particular column, and where column X = 'variable'. So on and so on.
I am coming from a c# background working in WPF. So normally I would use observable collections, and run LINQ on the observable collection to form a new observable collection, then bind that new collection to my grid.
My question is, how can I run LINQ type queries on stores in EXT JS?
You can run LINQ type queries on stores in ExtJS: You can easily get a Collection from a store using the query or queryBy method.
But you can't work like this for your grids, because grids can't run solely on collections, they need a store each. You wouldn't want to use LINQ type queries for these stores.
If you are using ExtJS6, you can look into ChainedStore. You can bind each grid to a ChainedStore which has your data store as the source. Each ChainedStore can be filtered, sorted and grouped independently from the source and from other stores, by using the sort(), group() and addFilter() functions on the respective store.
If you are using an earlier version, you have to use multiple stores and copy all data from the main store manually. Please be aware that store.add() takes an array of records, not a collection. Instead of store.add(collection), use store.add(collection.getRange()). The getRange() function will return an array that contains all items in the collection.
To answer your question from the comment:
what if I need to do something as simple as create a new store, from store1, group everything by column1, and sum column2?
In ExtJS6, if you want to show the grouping and the sum in a grid, that would be something along:
var store2 = Ext.create('Ext.data.ChainedStore',{
source:store1, // use the data from the other store.
grouper:{
property:'someTestField' // grouping is done inside this store.
}
});
Ext.create('Ext.grid.Panel',{
store:store2,
features:[{
ftype:'groupingsummary' // a summary row for each group is provided by this
}],
columns:[{
xtype:'numbercolumn'
dataIndex: 'someTestField'
summaryType: 'sum' // this column should have the sum in the summary row.
}]
})
Untested and without warranty. If you want to do it without the grid, just calculate the sum, that would have to be done manually like this:
var sums = {};
store1.getGroups().each(function(group,i) {
sums[i] = 0;
group.each(function(record) {
sums[i] += record.get(propertyName);
});
});
I am trying to find which approach is more scalable.
I have a user who has requested a seat in a carpool trip, and the user needs to be able to see all trips that apply to them. My models look like this:
var UserSchema = new mongoose.Schema({
id: String,
name: String,
trips: [String] // An array of strings, which holds the id of trips
});
var TripSchema = new mongoose.Schema({
id: String,
description: String,
passengers: [String] // An array of strings, which holds the id of users
});
So when the user goes to see all trips that apply to them, my backend will search through all the trips in the Mongo database.
I am deciding between 2 approaches:
Search through all trips and return the trips where the user's id is in the passengers array
Search through all trips and return the trips with an id matching an id in the user's trips array.
I believe approach #2 is better because it does not have to search deeper in the Trip model. I am just seeking confirmation and wondering if there is anything else I should consider.
If you don't do big data, I would simply say that it does not matter - both are good enough, but if you really have millions of queries on millions of users and trips...
for option 1 you only have one query but you would have to make sure, that you have your field passengers indexed, so you would need to maintain another index for this to be efficient. Another index impacts your write performance.
for option 2 you always have to do two queries.
First query for the user object in the user collection, then do an in style query to load the trip items that match any of those tripIds from user.trips. You will query on on the _id field which is always indexed. Of course, when you always load your user anyway there is only one query which really counts.
You would also have to consider whether write or read performance matters more. Your model is pretty inefficient for write because for every new trip you need to update two collections (the trip and the user). So currently you double your writes and usually writes are more expensive than reads.
And finally: to have easy and maintainable code is mostly more imporant than a bit of performance --> just use the mongoose populate feature, and all is done automatically for you. Don't store the references as Strings but as type ObjectId and use the ref keywoard in your model.
I have found a couple of solutions for inheritance using Mongoose here and here. These seem to work fine when the documents are stored in a normal collection. But, I am having trouble figuring out how to be able to store an array of the subclassed documents in a sub document on a model.
Let's say we have an object 'drawer' and it contains a collection of 'clothing' objects, where clothing objects can actually be one of several types of clothing 'sock', 'shirt', 'shorts'.
Sock, Shirt, and Shorts are all subclasses of Clothing.
So, I want to have my model Drawer look something like this...
var drawerSchema = mongoose.Schema({
// some drawer properties here
// ...
contents: [clothingSchema]
});
I have tried this approach, but when saved, only the properties of the actual clothingSchema are saved to the DB. For example, if my clothing schema had a common size property, it would be saved, but a property on my shirt object called 'buttonDown' would not be saved.
Has anyone else had a need to do this kind of modeling and if so found a solution?
Thanks!
If you can have inherited Schemas work on independent collections, you can use some Foreign Keys.
var drawerSchema = new Schema({
contents: [{type: Schema.Types.ObjectId, ref: 'cloth'}],
});
var clothSchema = new Schema({
size: Number
});
// Then define extended Schemas (Shirt, Sock, Short...)
This approach also lets you have a cloth definition only once in your DB so prevents unnecessary repeat of same data.
I have the following generic schema to represent different types of information.
var Record = new Schema (
{
type: {type: String}, // any string (foo, bar, foobar)
value: {type: String}, // any string value
o_id: {type:String}
}
);
Some of the records based on this schema have:
type="car"
value="ferrari" or
value="ford"
Some records have type "topspeed" with value "210" but they always share o_id (e.g. related "ferrari has this topspeed"). So if "ferrari has top speed 300", then both records have same o_id.
How can I make query to find "ferrari with topspeed 300" when I don't know o_id?
The only solution I found out is to select cars "ferrari" first and then with knowledge of all o_id for all "ferrari" use it to find topspeed.
In pseudocode:
Record.find({type:"car", value:"ferrari"}, function(err, docs)
{
var condition = [];// create array of all found o_id;
Record.find({type:"topspeed", value:"300"}...
}
I know that some merging or joining might not be possible, but what about some chaining these conditions to avoid recursion?
EDIT:
Better example:
Lets imagine I have a HTML document that contains DIV elements with certain id (o_id).
Now each div element can contain different type of microdata items (Car, Animal...).
Each microdata item has different properties ("topspeed", "numberOfLegs"...) based on the type (Car has a topspeed, animal numberOfLegs)
Each property has some value (310 kph, 4 legs)
Now I'm saving these microdata items to the database but in a general way, agnostic of the type and values they contain since the user can define custom schemas from Car, to Animal, to pretty much anything). For that I defined the Record schema: type consists of "itemtype_propertyname" and value is value of the property.
I would eventually like to query "Give me o_id(s) of all DIV elements that contain item Ferrari and item Dog" at the same time.
The reason for this general approach is to allow anyone the ability to define custom schema and corresponding parser that stores the values.
But I will have only one search engine to find all different schemas and value combinations that will treat all possible schemas as a single definition.
I think it'd be far better to combine all records that share an o_id into a single record. E.g.:
{
_id: ObjectId(...),
car: "ferarri",
topspeed: 300
}
Then you won't have this problem, and your schema will be more efficient both in speed and storage size. This is how MongoDB is intended to be used -- heterogenous data can be stored in a single collection, because MongoDB is schemaless. If you continue with your current design, then no, there's no way to avoid multiple round-trips to the database.