I'm trying to count the associated entries using the separate attribute in my includes to improve performance (without it the request it's taking 5s). But I'm receiving the following error:
"message": "missing FROM-clause entry for table "likedPosts""
Sorry for bad english, it's not my first. I hope you understand and can help me.
My code:
#Query((returns) => [Post], {
nullable: true
})
async getAllFeedPostsByUserId(#Arg('user_id') user_id: number): Promise < Post[] > {
const result = await Post.findAll({
attributes: {
include: [
[Sequelize.fn("COUNT", Sequelize.col("likedPosts.feed_post")), "likesAmount"]
]
},
include: [{
model: LikedPosts,
as: 'likedPosts',
attributes: [],
separate: true,
}, ]
});
return result;
}
I think group is must to count entries.
Post.findAll({
attributes: {
include: [[Sequelize.fn('COUNT', Sequelize.col('likedPosts.feed_post')), 'likesAmount']]
},
include: [{
model: LikedPosts,
attributes: []
}],
group: ['likedPosts.feed_post'] // groupBy is necessary else it will generate only 1 record with all rows count
})
I can see seperate
separate desc
To elaborate: by default, to retrieve the related model instance, Sequelize will use a SQL JOIN. By enabling separate, Sequelize will perform a separate query for each of the associated models, and join the resulting documents in code (instead of letting the database perform the join).
Related
I have two Entities, Organization and Applications. One organization can have many Applications.
const organization = mongoose.Schema(
{
name: {
type: String,
required: [true, 'Organization name is required.']
}, // min 1 max 255
// administrators get it from user table with org and role.
applications: [
// use populate in query
{
type: mongoose.Types.ObjectId,
ref: 'Application'
}
]
I am trying to query Organization with two applications and its returning a blank array
const organizations = await Organization.find({
'applications': {
$all: [
'636bdf70bcd2d24005061023',
'6373ba91f53f95ca187809d6'
]
}
}).populate('applications');
I tried running the same expression in MongoDB compass and it works. What am I doing wrong here ?
Based on your schema model, the applications field is an array of ObjectIDs, not an array of objects where each object would have _id property.
So instead of applications._id, it should be applications:
const organizations = await Organization.find({
applications: {
$all: ['636bdf70bcd2d24005061023', '6373ba91f53f95ca187809d6']
}
})
.populate('applications');
I have a table with an array column called tokens
I can query it via npm sequelize with no issues, Sometimes this column may have upto 20k elements in the array which i dont always need. I need just 10 elements from it
In SQL this would be
select tokens[:10] from schema.table
How I do this using sequelize ?
This is what I'm doing now
const whereClause = {
where: { active: true },
attributes: {
exclude: ['tokens'],
include: ['tokens[:10]'],
},
};
table.findAll(whereClause);
This gives the following error
original: error: column "tokens[:10]" does not exist
It is looking for a column named "tokens[:10]" instead of taking a subset.
What am I doing wrong ?
You can use literals when you select attributes in sequelize. You can try something like this,
const whereClause = {
where: { active: true },
attributes: [
[Sequelize.literal(`tokens[:${sub-array-length}]`), 'tokens']
]
};
table.findAll(whereClause);
Note the use of colons to denote the sub-array slicing index
Trying to use sequelize findAndCountAll method, to get items.
I've to use distinct, and offset with limit due to my task.
The problem is, that in associated model i've column with array type, and i need to order parent model by that array length.
My query looks like :
const { rows, count } = await this.repo.findAndCountAll({
where: { someField: someValue },
col: 'someCol',
distinct: true,
include: [
{
model: someNestedModel,
as: 'someNestedModelAssociation',
include: [{ model: someInnerNestedModel, as: 'someInnerNestedAssociation' }]
}
],
// eslint-disable-next-line #typescript-eslint/ban-ts-comment
//#ts-ignore
order: this.getOrderByOptions(sortOrder, orderBy),
limit,
offset
});
getOrderByOptions(sortOrder, orderBy) {
switch (orderBy) {
case Sort_By_Array_Length:
return Sequelize.literal('json_array_length(someModelName.someColumnWithArrayName) ASC');
default:
return [[orderBy, sortOrder]];
}
}
The problem is, that my order by query is used both in subQuery and mainQuery.
And using it into subQuery leads to error, cz there is no such field.
If i use subQuery:false flag, it works, but then i got messed with returning results, due to problems with subQuery:false and offset&limits.
So the question is, is there a way, to exclude orderBy field from subQuery?
P.S. Models have many to many association with through table.
I have two models
Units and Filters which are related with M:N association through unit_filter table
this.belongsToMany(Unit, {
through: "unit_filter",
as: "units",
});
this.belongsToMany(Filter, {
through: 'unit_filter',
as: 'filters',
});
The goal is to fetch units which have more than 1 filter associated with and condition.
let result = await Unit.findAll({
include: [
{
model: Filter,
where: {
id: {
[Op.and] : [2,252,4,80]
}
},
as: 'filters',
},
],
});
The above is only working if there is only one id in the array which does not make any sense.
Seqeulize documenation states
Post.findAll({
where: {
[Op.and]: [{ a: 5 }, { b: 6 }], // (a = 5) AND (b = 6)
}
})
So the code I have should work theoritically. I also tried with
where: {
[Op.and] : [{id:2},{id:252},{id:4},{id:80}]
}
which results in getting all the items from the db. It does not even care about the where condition in this case.
Would be of great help if any one points me in right direction.
You need to use Sequelize.literal with a subquery in where option in order to filter units that have more than 1 filter because simply indicating several ids of filters you will get units that have one of indicated filters (from 1 to N).
let result = await Unit.findAll({
where: Sequelize.where(
Sequelize.literal('(SELECT COUNT(*) FROM unit_filter as uf WHERE uf.unit_id=unit.id AND uf.filter_id in ($filter_ids))'),
Op.gt,
'1'),
bind: {
filter_ids: [2,252,4,80]
}
});
I have models called "Activities" that I am querying for (using Mongoose). Their schema looks like this:
var activitySchema = new mongoose.Schema({
actor: {
type: mongoose.Schema.ObjectId,
ref: 'User',
required: true
},
recipient: {
type: mongoose.Schema.ObjectId,
ref: 'User'
},
timestamp: {
type: Date,
default: Date.now
},
activity: {
type: String,
required: true
},
event: {
type: mongoose.Schema.ObjectId,
ref: 'Event'
},
comment: {
type: mongoose.Schema.ObjectId,
ref: 'Comment'
}
});
When I query for them, I am populating the actor, recipient, event, and comment fields (all the references). After that, I also deep-populate the event field to get event.creator. Here is my code for the query:
var activityPopulateObj = [
{ path: 'event' },
{ path: 'event.creator' },
{ path: 'comment' },
{ path: 'actor' },
{ path: 'recipient' },
{ path: 'event.creator' }
],
eventPopulateObj = {
path: 'event.creator',
model: User
};
Activity.find({ $or: [{recipient: user._id}, {actor: {$in: user.subscriptions}}, {event: {$in: user.attending}}], actor: { $ne: user._id} })
.sort({ _id: -1 })
.populate(activityPopulateObj)
.exec(function(err, retrievedActivities) {
if(err || !retrievedActivities) {
deferred.reject(new Error("No events found."));
}
else {
User.populate(retrievedActivities, eventPopulateObj, function(err, data){
if(err) {
deferred.reject(err.message);
}
else {
deferred.resolve(retrievedActivities);
}
});
}
});
This is already a relatively complex query, but I need to do even more. If it hits the part of the $or statement that says {actor: {$in: user.subscriptions}}, I also need to make sure that the event's privacy field is equal to the string public. I tried using $elemMatch, but since the event has to be populated first, I couldn't query any of its fields. I need to achieve this same goal in multiple other queries, as well.
Is there any way for me to achieve this further filtering like I have described?
The answer is to change your schema.
You've fallen into the trap that many devs have before you when coming into document database development from a history of using relational databases: MongoDB is not a relational database and should not be treated like one.
You need to stop thinking about foreign keys and perfectly normalized data and instead, keep each document as self-contained as possible, thinking about how to best embed relevant associated data within your documents.
This doesn't mean you can't maintain associations as well. It might mean a structure like this, where you embed only necessary details, and query for the full record when needed:
var activitySchema = new mongoose.Schema({
event: {
_id: { type: ObjectId, ref: "Event" },
name: String,
private: String
},
// ... other fields
});
Rethinking your embed strategy will greatly simplify your queries and keep the query count to a minimum. populate will blow your count up quickly, and as your dataset grows this will very likely become a problem.
You can try below aggregation. Look at this answer: https://stackoverflow.com/a/49329687/12729769
And then, you can use fields from $addFields in your query. Like
{score: {$gte: 5}}
but since the event has to be populated first, I couldn't query any of its fields.
No can do. Mongodb cannot do joins. When you make a query, you can work with exactly one collection at a time. And FYI all those mongoose populates are additional, distinct database queries to load those records.
I don't have time to dive into the details of your schema and application, but most likely you will need to denormalize your data and store a copy of whatever event fields you need to join on in the primary collection.