Develop Reference

Strapi v4: no relational fields when populating

I'm trying to populate a specific relation, using the relation name (categories) in combination with the populate parameter but it doesn't populate the categories.
When I look at my schema, I see that the relational field is present in the attributes object. But I still only get the non-relational fields in my response.
I tried every combination mentioned on the Strapi documentation but none of them worked.
The find permission is also enabled for the content-types that are being populated which in this case is categories.
/api/products?populate=*
/api/products?populate[0]=categories
/api/products?populate[categories]=*
My Product schema
{
"kind": "collectionType",
"collectionName": "products",
"info": {
"singularName": "product",
"pluralName": "products",
"displayName": "Product",
"description": ""
},
"options": {
"draftAndPublish": true
},
"pluginOptions": {},
"attributes": {
"title": {
"type": "string"
},
"images": {
"type": "media",
"multiple": true,
"required": false,
"allowedTypes": [
"images"
]
},
"categories": {
"type": "relation",
"relation": "oneToMany",
"target": "api::category.category"
}
}
}
System
Strapi version: 4.1.8
NPM version: 8.3.2
Node.js version: 16.13.2
Database: MySQL

You need to enable find permission on the product as well. You have to set the find permission on all the relationships (or sub-tables).
Use the "role" option.

Edit: Other choice is to use populate deep from strapi marketplace, just install, configure the default depth and then on every request where you want deep just add populate: deep
you will have to populate inside your controller, as in Strapi documentation
Query Engine API: Populating
strapi.db.query('api::article.article').findMany({
populate: true,
});
Entity Service API: Populating
const entries = await strapi.entityService.findMany('api::article.article', {
populate: '*',
});
REST API: Population & Field Selection
const qs = require('qs');
const query = qs.stringify({
fields: ['title', 'body'],
}, {
encodeValuesOnly: true,
});
If you want to populate everything just use populate: '*'
If you want to populate a relation or more use populate: [relationOne, relationTwo]

As an Example If you want to fetch data from db based on some conditions and populate one relationship
const transactions = await strapi
.query("api::transaction.transaction")
.findMany({
where: { user: userId },
populate: { offer: true },
});
here the result
If you want to populate all relations
const transactions = await strapi
.query("api::transaction.transaction")
.findMany({
where: { user: userId },
populate: "*",
});

mongoose find from nested array of objects

hey I am quite new to mongoose and can't get my head around search.
models
User->resumes[]->employments[]
UserSchema
{
resumes: [ResumeSchema],
...
}
ResumeSchema
{
employments: [EmploymentSchema],
...
}
EmploymentSchema
{
jobTitle: {
type: String,
required: [true, "Job title is required."]
},
...
}
Background
User has to enter job title and needs suggestions from the existing data of the already present resumes and their employment's job title
I have tried the following code.
let q = req.query.q; // Software
User.find({ "resumes.employments.jobTitle": new RegExp(req.query.q, 'ig') }, {
"resumes.employments.$": 1
}, (err, docs) => {
res.json(docs);
})
Output
[
{
_id: '...',
resumes:[
{
employments: [
{
jobTitle: 'Software Developer',
...
},
...
]
},
...
]
},
...
]
Expected OutPut
["Software Developer", "Software Engineer", "Software Manager"]
Problem
1:) The Data returned is too much as I only need jobTitle
2:) All employments are being returned whereas the query matched one of them
3:) Is there any better way to do it ? via index or via $search ? I did not find much of information in mongoose documentation to create search index (and I also don't really know how to create a compound index to make it work)
I know there might be a lot of answers but none of them helped or I was not able to make them work ... I am really new to mongodb I have been working with relational databases via SQL or through ORM so my mongodb concepts and knowledge is limited.
So please let me know if there is a better solution to do it. or something to make the current one working.

You can use one of the aggregation query below to get this result:
[
{
"jobTitle": [
"Software Engineer",
"Software Manager",
"Software Developer"
]
}
]
Query is:
First using $unwind twice to deconstructs the arrays and get the values.
Then $match to filter by values you want using $regex.
Then $group to get all values together (using _id: null and $addToSet to no add duplicates).
And finally $project to shown only the field you want.
User.aggregate({
"$unwind": "$resumes"
},
{
"$unwind": "$resumes.employments"
},
{
"$match": {
"resumes.employments.jobTitle": {
"$regex": "software",
"$options": "i"
}
}
},
{
"$group": {
"_id": null,
"jobTitle": {
"$addToSet": "$resumes.employments.jobTitle"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here
Also another option is using $filter into $project stage:
Is similar as before but using $filter instead of $unwind twice.
User.aggregate({
"$unwind": "$resumes"
},
{
"$project": {
"jobs": {
"$filter": {
"input": "$resumes.employments",
"as": "e",
"cond": {
"$regexMatch": {
"input": "$$e.jobTitle",
"regex": "Software",
"options": "i"
}
}
}
}
}
},
{
"$unwind": "$jobs"
},
{
"$group": {
"_id": null,
"jobTitle": {
"$addToSet": "$jobs.jobTitle"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here

Counting relationships in many to many table between joined tables in Sequelize.js

I am building a project using sequelize.js that includes a Tags table and a Stories table. They have a many to many relationship, which I created in sequelize with a through table of StoryTag. This all works perfectly so far, but I want to get a list of most popluar tags, as in how many stories they are associated with in the StoryTag table, and order them by the number of stories that use this tag.
This is the MySQL syntax of what I am trying to do. This works perfectly in MySQL Workbench:
SELECT tagName, COUNT(StoryTag.TagId)
FROM Tags
LEFT JOIN StoryTag on Tags.id = StoryTag.TagId
GROUP BY Tags.tagName ORDER BY COUNT(StoryTag.TagId) DESC;
This is what works in sequelize.js. It's a raw query, which is not ideal, but since this doesn't handle any sensitive information, it's not a huge worry, just very inelegant.
//DIRECT QUERY METHOD (TEST)
app.get("/api/directags", function (req, res) {
db.sequelize.query("select tags.id, tags.TagName, COUNT(stories.id) as num_stories
from tags left join storytag on storytag.TagId = tags.id
left join stories on storytag.StoryId = stories.id
group by tags.id order by num_stories desc;", {
type: db.Sequelize.QueryTypes.SELECT
}).then(function(result) {
res.send(result);
});
});
This outputs
[
{
"id": 3,
"TagName": "fiction",
"num_stories": 3
},
{
"id": 5,
"TagName": "Nursery Rhyme",
"num_stories": 2
},
...
{
"id": 4,
"TagName": "nonfiction",
"num_stories": 0
}
]
As it should. What doesn't quite work is:
//Sequelize count tags
//Known issues: will not order by the count
//Includes a random 'storytag' many-to-many table row for some reason
app.get("/api/sequelizetags", function (req, res) {
db.Tag.findAll({
attributes: ["id","TagName"],
include: [{
model: db.Story,
attributes: [[db.sequelize.fn("COUNT", "stories.id"), "Count_Of_Stories"]],
duplicating: false
}],
group: ["id"]
}).then(function (dbExamples) {
res.send(dbExamples);
});
});
Which outputs:
[
{
"id": 1,
"TagName": "horror",
"Stories": [
{
"Count_Of_Stories": 1,
"StoryTag": {
"createdAt": "2018-11-29T21:09:46.000Z",
"updatedAt": "2018-11-29T21:09:46.000Z",
"StoryId": 1,
"TagId": 1
}
}
]
},
{
"id": 2,
"TagName": "comedy",
"Stories": []
},
{
"id": 3,
"TagName": "fiction",
"Stories": [
{
"Count_Of_Stories": 3,
"StoryTag": {
"createdAt": "2018-11-29T21:10:04.000Z",
"updatedAt": "2018-11-29T21:10:04.000Z",
"StoryId": 1,
"TagId": 3
}
}
]
},
{
"id": 4,
"TagName": "nonfiction",
"Stories": []
},
...
{
"id": 8,
"TagName": "Drama",
"Stories": [
{
"Count_Of_Stories": 1,
"StoryTag": {
"createdAt": "2018-11-30T01:13:56.000Z",
"updatedAt": "2018-11-30T01:13:56.000Z",
"StoryId": 3,
"TagId": 8
}
}
]
},
{
"id": 9,
"TagName": "Tragedy",
"Stories": []
}
]
This is not in order, and the count of stories is buried. This seems like the sort of thing that would be a common and frequent request from a database, but I am at a loss of how to do this correctly with sequelize.js.
Resources that have failed me:
Sequelize where on many-to-many join
Sequelize Many to Many Query Issue
How to query many-to-many relationship data in Sequelize
Select from many-to-many relationship sequelize
The official documentation for sequelize: http://docs.sequelizejs.com/manual/tutorial/
Some less official and more readable documentation for sequelize: https://sequelize.readthedocs.io/en/v3/docs/querying/

Here's what finally worked, in case anyone else has this question. We also added a where to the include Story, but that's optional.
This resource is easier to understand than the official sequelize docs: https://sequelize-guides.netlify.com/querying/
I also learned that being familiar with promises is really helpful when working with sequelize.
db.Tag.findAll({
group: ["Tag.id"],
includeIgnoreAttributes:false,
include: [{
model: db.Story,
where: {
isPublic: true
}
}],
attributes: [
"id",
"TagName",
[db.sequelize.fn("COUNT", db.sequelize.col("stories.id")), "num_stories"],
],
order: [[db.sequelize.fn("COUNT", db.sequelize.col("stories.id")), "DESC"]]
}).then(function(result){
return result;
});

Please, use the same name if you mean the same thing (num_stories - Count_Of_Stories, etc.).
For ordering use order option.
Include count in top level attributes for get it on top level of instance.
I can't find include[].duplicating option in doc.
Your case:
db.Tag.findAll({
attributes: [
"id",
"TagName",
[db.sequelize.fn("COUNT", "stories.id"), "Count_Of_Stories"]
],
include: [{
model: db.Story,
attributes: [],
duplicating: false
}],
group: ["id"],
order: [
[db.sequelize.literal("`Count_Of_Stories`"), "DESC"]
]
});

Use through: {attributes: []} in options

Create ID with data of other fields

I'm new to mongodb, and I'm using mongoose to validate and order the data (I'm open to change it to MySQL if this doesn't work).
The app will be an e-shop, to buy merchandising related to movies, games, ext.
My schema is as follows:
var productSchema = {
id: {
type: String,
required: true
},
name: {
type: String,
required: true
},
img: {
type: String,
required: true
},
price: {
type: Number,
required: true
},
stock: {
type: Number,
required: true
},
category: {
object: {
type: String,
required: true
},
group: {
type: String,
required: true
},
name: {
type: String,
required: true
}
}
};
This is what I would like to do:
If I have the following data in category:
category.object = "ring"
category.group = "movies"
category.name= "lord of the rings"
I want the id to be made of the first letters of every field in category and a number (the number of the last item added plus 1). In this case, It would be RMLOTR1.
What I'm doing right now
I'm adding a lot of data at the same time, so every time I do it, I made a function that iterates through all the items added and does what I want but...
My question is
Is there a built-in way to do this with mongodb or mongoose, adding the data and creating the id at the same time? I know I can do a virtual, but I want the data to be stored.
Extras
If it's not posible to do this with mongodb, is there a way to do this with MySQL?
Is doing this kind of thing considered a correct/wrong approach?

You are basically looking for a "pre" middleware hook on the "save" event fired by creating new documents in the collection. This will inspect the current document content and extract the "strings" from values in order to create your "prefix" value for _id.
There is also another part, where the "prefix" needs the addition of the numeric counter when there is already a value present for that particular "prefix" to make it distinct. There is a common technique in MongoDB used to "Generate an auto-incrementing sequence field", which basically involves keeping a "counters" collection and incrementing the value each time you access it.
As a complete and self contained demonstration, you combine the techniques as follows:
var async = require('async'),
mongoose = require('mongoose'),
Schema = mongoose.Schema;
mongoose.connect('mongodb://localhost/warehouse');
var counterSchema = new Schema({
"type": { "type": String, "required": true },
"prefix": { "type": String, "required": true },
"counter": Number
});
counterSchema.index({ "type": 1, "prefix": 1 },{ "unique": true });
counterSchema.virtual('nextId').get(function() {
return this.prefix + this.counter;
});
var productSchema = new Schema({
"_id": "String",
"category": {
"object": { "type": String, "required": true },
"group": { "type": String, "required": true },
"name": { "type": String, "required": true }
}
},{ "_id": false });
productSchema.pre('save', function(next) {
var self = this;
if ( !self.hasOwnProperty("_id") ) {
var prefix = self.category.object.substr(0,1).toUpperCase()
+ self.category.group.substr(0,1).toUpperCase()
+ self.category.name.split(" ").map(function(word) {
return word.substr(0,1).toUpperCase();
}).join("");
mongoose.model('Counter').findOneAndUpdate(
{ "type": "product", "prefix": prefix },
{ "$inc": { "counter": 1 } },
{ "new": true, "upsert": true },
function(err,counter) {
self._id = counter.nextId;
next(err);
}
);
} else {
next(); // Just skip when _id is already there
}
});
var Product = mongoose.model('Product',productSchema),
Counter = mongoose.model('Counter', counterSchema);
async.series(
[
// Clean data
function(callback) {
async.each([Product,Counter],function(model,callback) {
model.remove({},callback);
},callback);
},
function(callback) {
async.each(
[
{
"category": {
"object": "ring",
"group": "movies",
"name": "lord of the rings"
}
},
{
"category": {
"object": "ring",
"group": "movies",
"name": "four weddings and a funeral"
}
},
{
"category": {
"object": "ring",
"group": "movies",
"name": "lord of the rings"
}
}
],
function(data,callback) {
Product.create(data,callback)
},
callback
)
},
function(callback) {
Product.find().exec(function(err,products) {
console.log(products);
callback(err);
});
},
function(callback) {
Counter.find().exec(function(err,counters) {
console.log(counters);
callback(err);
});
}
],
function(err) {
if (err) throw err;
mongoose.disconnect();
}
)
This gives you output like:
[ { category: { name: 'lord of the rings', group: 'movies', object: 'ring' },
__v: 0,
_id: 'RMLOTR1' },
{ category:
{ name: 'four weddings and a funeral',
group: 'movies',
object: 'ring' },
__v: 0,
_id: 'RMFWAAF1' },
{ category: { name: 'lord of the rings', group: 'movies', object: 'ring' },
__v: 0,
_id: 'RMLOTR2' } ]
[ { __v: 0,
counter: 2,
type: 'product',
prefix: 'RMLOTR',
_id: 57104cdaa774fcc73c1df0e8 },
{ __v: 0,
counter: 1,
type: 'product',
prefix: 'RMFWAAF',
_id: 57104cdaa774fcc73c1df0e9 } ]
To first understand the Counter schema and model, you are basically defining something where you are going to look up a "unique" key and also attach a numeric field to "increment" on match. For convenience this just has a two fields making up the unique combination and a compound index defined. This could just also be a compound _id if so wanted.
The other convenience is the virtual method of nextId, which just does a concatenation of the "prefix" and "counter" values. It's also best practice here to include something like "type" here since your Counter model can be used to service "counters" for use in more than one collection source. So here we are using "product" whenever accessing in the context of the Product model to differentiate it from other models where you might also keep a similar sequence counter. Just a design point that is worthwhile following.
For the actual Product model itself, we want to attach "pre save" middleware hook in order to fill the _id content. So after determining the character portion of the "prefix", the operation then goes off and looks for that "prefix" with the "product" type data in combination in the Counter model collection.
The function of .findOneAndUpdate() is to look for a document matching the criteria in the "counters" collection and then where a document is found already it will "increment" the current counter value by use of the $inc update operator. If the document was not found, then the "upsert" option means that a new document will be created, and at any rate the same "increment" will happen in the new document as well.
The "new" option here means that we want the "modified" document to be returned ( either new or changed ) rather than what the document looked like before the $inc was applied. The result is that "counter" value will always increase on every access.
Once that is complete and a document for Counter is either incremented or created for it's matching keys, then you now have something you can use to assign to the _id in the Product model. As mentioned earlier you can use the virtual here for convenience to get the prefix with the appended counter value.
So as long as your documents are always created by either the .create() method from the model or by using new Product() and then the .save() method, then the methods attached to your "model" in your code are always executed.
Note here that since you want this in _id, then as a primary key this is "immutable" and cannot change. So even if the content in the fields referenced was later altered, the value in _id cannot be changed, and therefore why the code here makes no attempt when an _id value is already set.

Change MongoDB find results dynamically

I'm using mongoose with node.js.
Let's say I have 'Posts' DB where each document in it is a post.
Each post has a 'ReadBy' array which holds names of users that had read this post.
When I'm searching for documents in this DB, I want to "change" the 'ReadBy' value to show by Boolean value if the user that is searching for it is in this array or not.
For example, let's say these are 2 documents that are in this DB:
{ "PostName": "Post Number 1", "ReadBy": ["Tom", "John", "Adam"] }
{ "PostName": "Post Number 2", "ReadBy": ["John", "Adam"] }
If I'm user 'Tom', I want to get the results like this:
[
{
"PostName": "Post Number 1",
"ReadBy": true,
},
{
"PostName": "Post Number 2",
"ReadBy": false,
}
]
Now, I know that I can get the documents and go over each one of them with forEach function, and then use forEach again on the "ReadBy" array and change this field.
I'm asking if there is more efficient way to do it in the mongoDB query itself, or some other way in the code.
If there is another way with mongoose - even better.

Using mongoDb $setIntersection in aggregation you get the result like this :
db.collectionName.aggregate({
"$project": {
"ReadBy": {
"$cond": {
"if": {
"$eq": [{
"$setIntersection": ["$ReadBy", ["Tom"]]
},
["Tom"]
]
},
"then": true,
"else": false
}
},
"PostName": 1
}
})
So above working first like this
{ $setIntersection: [ [ "Tom", "John", "Adam"], [ "Tom"] ] }, return [ "Tom"]
{ $setIntersection: [ [ "John", "Adam"], [ "Tom"] ] }, return [ ]
and $eq to check whether setIntersection results matched with ["Tom"] if yes then return true else false

You can try something similar to
var unwind = {"$unwind": "$ReadBy"}
var eq = {$eq: ["$ReadBy", "Bob"]}
var project = {$project: {PostName: 1, seen: eq}}
db.posts.aggregate([unwind, project])
Just notice that you solution is highly inefficient. Both for storing the data ( growing array) and for searching.