Trying to use sequelize findAndCountAll method, to get items.
I've to use distinct, and offset with limit due to my task.
The problem is, that in associated model i've column with array type, and i need to order parent model by that array length.
My query looks like :
const { rows, count } = await this.repo.findAndCountAll({
where: { someField: someValue },
col: 'someCol',
distinct: true,
include: [
{
model: someNestedModel,
as: 'someNestedModelAssociation',
include: [{ model: someInnerNestedModel, as: 'someInnerNestedAssociation' }]
}
],
// eslint-disable-next-line #typescript-eslint/ban-ts-comment
//#ts-ignore
order: this.getOrderByOptions(sortOrder, orderBy),
limit,
offset
});
getOrderByOptions(sortOrder, orderBy) {
switch (orderBy) {
case Sort_By_Array_Length:
return Sequelize.literal('json_array_length(someModelName.someColumnWithArrayName) ASC');
default:
return [[orderBy, sortOrder]];
}
}
The problem is, that my order by query is used both in subQuery and mainQuery.
And using it into subQuery leads to error, cz there is no such field.
If i use subQuery:false flag, it works, but then i got messed with returning results, due to problems with subQuery:false and offset&limits.
So the question is, is there a way, to exclude orderBy field from subQuery?
P.S. Models have many to many association with through table.
Related
I have a table with an array column called tokens
I can query it via npm sequelize with no issues, Sometimes this column may have upto 20k elements in the array which i dont always need. I need just 10 elements from it
In SQL this would be
select tokens[:10] from schema.table
How I do this using sequelize ?
This is what I'm doing now
const whereClause = {
where: { active: true },
attributes: {
exclude: ['tokens'],
include: ['tokens[:10]'],
},
};
table.findAll(whereClause);
This gives the following error
original: error: column "tokens[:10]" does not exist
It is looking for a column named "tokens[:10]" instead of taking a subset.
What am I doing wrong ?
You can use literals when you select attributes in sequelize. You can try something like this,
const whereClause = {
where: { active: true },
attributes: [
[Sequelize.literal(`tokens[:${sub-array-length}]`), 'tokens']
]
};
table.findAll(whereClause);
Note the use of colons to denote the sub-array slicing index
I'm trying to count the associated entries using the separate attribute in my includes to improve performance (without it the request it's taking 5s). But I'm receiving the following error:
"message": "missing FROM-clause entry for table "likedPosts""
Sorry for bad english, it's not my first. I hope you understand and can help me.
My code:
#Query((returns) => [Post], {
nullable: true
})
async getAllFeedPostsByUserId(#Arg('user_id') user_id: number): Promise < Post[] > {
const result = await Post.findAll({
attributes: {
include: [
[Sequelize.fn("COUNT", Sequelize.col("likedPosts.feed_post")), "likesAmount"]
]
},
include: [{
model: LikedPosts,
as: 'likedPosts',
attributes: [],
separate: true,
}, ]
});
return result;
}
I think group is must to count entries.
Post.findAll({
attributes: {
include: [[Sequelize.fn('COUNT', Sequelize.col('likedPosts.feed_post')), 'likesAmount']]
},
include: [{
model: LikedPosts,
attributes: []
}],
group: ['likedPosts.feed_post'] // groupBy is necessary else it will generate only 1 record with all rows count
})
I can see seperate
separate desc
To elaborate: by default, to retrieve the related model instance, Sequelize will use a SQL JOIN. By enabling separate, Sequelize will perform a separate query for each of the associated models, and join the resulting documents in code (instead of letting the database perform the join).
I've got a problem that I've been stuck on, to no avail - seemingly similar in nature to Where condition for joined table in Sequelize ORM, except that I'd like to query on a previous join. Perhaps code will explain my problem. Happy to provide any extra info.
Models:
A.hasMany(B);
B.belongsTo(A);
B.hasMany(C);
C.belongsTo(B);
This is what I'd like to be able to achieve with Sequelize:
SELECT *
FROM `A`AS `A`
LEFT OUTER JOIN `B` AS `B` ON `A`.`id` = `B`.`a_id`
LEFT OUTER JOIN `C` AS `B->C` ON `B`.`id` = `B->C`.`b_id`
AND (`B`.`b_columnName` = `B->C`.`c_columnName`);
This is how I imagine this working: (instead it will create a raw query (2 raw queries, for A-B/C) with AND ( `C`.`columnName` = '$B.columnName$')) on the join (second arg is a string). Have tried sequelize.col, sequelize.where(sequelize.col..., etc..)
A.findOne({
where: { id: myId },
include: [{
model: B,
include: [{
model: C,
where: { $C.c_columnName$: $B.b_columnName$ }
}]
}]
});
Use the Op.col query operator to find columns that match other columns in your query. If you are only joining a single table you can pass an object instead of an array to make it more concise.
const Op = Sequelize.Op;
const result = await A.findOne({
include: {
model: B,
include: {
model: C,
where: {
c_columnName: {
[Op.col]: 'B.b_columnName',
},
}
},
},
});
If a collection have a list of dogs, and there is duplicate entries on some races. How do i remove all, but a single specific/non specific one, from just one query?
I guess it would be possible to get all from a Model.find(), loop through every index except the first one and call Model.remove(), but I would rather have the database handle the logic through the query. How would this be possible?
pseudocode example of what i want:
Model.remove({race:"pitbull"}).where(notFirstOne);
To remove all but one, you need a way to get all the filtered documents, group them by the identifier, create a list of ids for the group and remove a single id from
this list. Armed with this info, you can then run another operation to remove the documents with those ids. Essentially you will be running two queries.
The first query is an aggregate operation that aims to get the list of ids with the potentially nuking documents:
(async () => {
// Get the duplicate entries minus 1
const [doc, ...rest] = await Module.aggregate([
{ '$match': { 'race': 'pitbull'} },
{ '$group': {
'_id': '$race',
'ids': { '$push': '$_id' },
'id': { '$first': '$_id' }
} },
{ '$project': { 'idsToRemove': { '$setDifference': [ ['$id'], '$ids' ] } } }
]);
const { idsToRemove } = doc;
// Remove the duplicate documents
Module.remove({ '_id': { '$in': idsToRemove } })
})();
if purpose is to keep only one, in case of concurrent writes, may as well just write
Module.findOne({race:'pitbull'}).select('_id')
//bla
Module.remove({race:'pitbull', _id:{$ne:idReturned}})
If it is to keep the very first one, mongodb does not guarantee results will be sorted by increasing _id (natural order refers to disk)
see Does default find() implicitly sort by _id?
so instead
Module.find({race:'pitbull'}).sort({_id:1}).limit(1)
I hit an API which follows 50 members' data in a game once a day, and use mongoose to convert the JSON into individual documents in a collection. Between days there is data which is consistent, for example each member's tag (an id for the member in game), but there is data which is different (different scores etc.). Each document has a createdAt property.
I would like to find the most recent document for each member, and thus have an array with each member's tag.
I an currently using the following query to find all documents where tags match, however they are returning all documents, not just one. How do I sort/limit the documents to the most recent one, whilst keep it as one query (or is there a more "mongodb way")?
memberTags = [1,2,3,4,5];
ClanMember.find({
'tag': {
$in: memberTags
}
}).lean().exec(function(err, members) {
res.json(members);
});
Thanks
You can query via the aggregation framework. Your query would involve a pipeline that has stages that process the input documents to give you the desired result. In your case, the pipeline would have a $match phase which acts as a query for the initial filter. $match uses standard MongoDB queries thus you can still query using $in.
The next step would be to sort those filtered documents by the createdAt field. This is done using the $sort operator.
The preceding pipeline stage involves aggregating the ordered documents to return the top document for each group. The $group operator together with the $first accumulator are the operators which make this possible.
Putting this altogether you can run the following aggregate operation to get your desired result:
memberTags = [1,2,3,4,5];
ClanMember.aggregate([
{ "$match": { "tag": { "$in": memberTags } } },
{ "$sort": { "tag": 1, "createdAt: -1 " } },
{
"$group": {
"_id": "$tag",
"createdAt": { "$first": "$createdAt" } /*,
include other necessary fields as appropriate
using the $first operator e.g.
"otherField1": { "$first": "$otherField1" },
"otherField2": { "$first": "$otherField2" },
...
*/
}
}
]).exec(function(err, members) {
res.json(members);
});
Or tweak your current query using find() so that you can sort on two fields, i.e. the tag (ascending) and createdAt (descending) attributes. You can then select the top 5 documents using limit, something like the following:
memberTags = [1,2,3,4,5];
ClanMember.find(
{ 'tag': { $in: memberTags } }, // query
{}, // projection
{ // options
sort: { 'createdAt': -1, 'tag': 1 },
limit: memberTags.length,
skip: 0
}
).lean().exec(function(err, members) {
res.json(members);
});
or
memberTags = [1,2,3,4,5];
ClanMember.find({
'tag': {
$in: memberTags
}
}).sort('-createdAt tag')
.limit(memberTags.length)
.lean()
.exec(function(err, members) {
res.json(members);
});
Ok, so, first, let's use findOne() so you get only one document out of the request
Then to sort by the newest document, you can use .sort({elementYouWantToSort: -1}) (-1 meaning you want to sort from newest to oldest, and 1 from the oldest to the newest)
I would recommend to use this function on the _id, which already includes creation date of the document
Which gives us the following request :
ClanMember.findOne({
'tag': {
$in: memberTags
}
}).sort({_id: -1}).lean().exec(function(err, members) {
res.json(members);
});