Very slow Sequelize findAndCountAll query with nested "include" - javascript

I'm very new to using sequelize. I have a query with nested includes as follows:
Site.findAndCountAll({
where,
order: orderCondition,
...(orderAfter ? {} : limits),
include: [
{
model: Version,
as: 'versions',
limit: 1,
separate: true,
order: [['version_id', 'DESC']],
attributes: ['status', 'config'],
include: [
{
model: User,
as: 'updatedByUser',
attributes: ['email'],
paranoid: false,
},
],
},
{
model: Version,
as: 'prod_version',
separate: false,
required: false,
attributes: ['status', 'config'],
where: {
is_prod: 1,
status: 'SUCCESS',
},
include: [
{
model: User,
as: 'updatedByUser',
attributes: ['email'],
paranoid: false,
},
],
}
],
distinct: true,
});
And I have these associations between Site and Version:
Site.hasMany(models.Version, {
foreignKey: 'site_id',
as: 'versions',
});
Site.hasOne(models.Version, {
foreignKey: 'site_id',
as: 'prod_version',
});
This query can take several seconds to load a page when navigating through the pagination. It takes longer the further along I go through the pages. It's fine for the first ~10-15 pages but then starts getting increasingly slower and by page 30-40 it sometimes takes more than 5 seconds to get a response.
I know there's an inherent issue with the way OFFSET works in SQL but I feel it shouldn't be this slow unless I'm hundreds of pages in. How else could I potentially optimize this query? A big part of the slowdown also seems to be the second item in include, prod_version. With that commented out I get a huge improvement of the response time. Is there a more efficient way to achieve the same result?

Related

Sequelize - best way to query latest entry AND entry that meets a specific requirement

I currently have the following query in Sequelize:
const sites = await Site.findAndCountAll({
include: [
{
model: Version,
as: 'versions',
limit: 1,
order: [['version_id', 'DESC']],
attributes: ['status', 'config', 'user_id', 'updated_at', 'version_id', 'updated_by', 'is_abtest_parent', 'is_prod'],
include: [
{
model: User,
as: 'updatedByUser',
attributes: ['email'],
paranoid: false,
},
],
},
],
distinct: true,
});
This gets sites from the sites table and the version with the latest version_id for a specific site. The Site and Version models have the following associations:
Version.belongsTo(models.Site, {
foreignKey: 'site_id',
});
Site.hasMany(models.Version, {
foreignKey: 'site_id',
as: 'versions',
});
I now need to also get the version that is in production and has its is_prod attribute set to 1, while also keeping the version with the latest version_id. What would be the best way to query these two versions?

How to write formula in ORDER using fields from subquery

I need to implement sorting on computed data from a subquery. There is a request like this:
const res = await db.order.findAndCountAll({
limit: 10,
offset: (page - 1) * 10,
distinct: true,
order: [[db.sequelize.literal('product.taxes * product.count'), 'ASC']],
where,
include: [
{ model: db.location },
{ model: db.address },
{
model: db.order_product,
as: 'products',
required: true,
include: [
{
model: db.product,
include: [{ model: db.location }],
},
],
},
],
});
This code is not working. I get the error:
missing FROM-clause entry for table "product"
In line:order: [[db.sequelize.literal('product.taxes * product.count'), 'ASC']], i want to use to calculate the fields that are in the db.product model which i am accessing in a sub-sub-query. Moreover, this sub-sub-query will return an array, and I need to execute (db.product.amount * db.product.price) for each element of the array and then add up all the resulting products. How to describe this formula in "order"?
Maybe I chose the wrong path. What is the best way to sort by calculated data from a subquery?

How do i count a data that includes another data in sequelize?

So I was trying to get a record about car dealers who successfully sold a specific car, sorted from how much the dealer successfully sold that car so the dealer that successfully sold the most of that specific car will appear first. The problem is that in the end I also need to group the car id, which makes the counting inaccurate. Can anyone please help me solve this problem where I can just get the record without needing to group the car id as well?
Here is my code:
const data = await models.Car.findAll({
paranoid: false,
attributes: [[Sequelize.fn('COUNT', Sequelize.col('userId')), 'carsCount'], 'userId'],
where: { brandId, groupModelId, status: 2 },
include: [
{
model: models.User,
as: 'user',
required: true,
duplicating: false,
attributes: ['name', 'phone', 'email'],
include: [
{
model: models.UserAddress,
as: 'userAddress'
}
]
}
],
group: ['userId', 'user.id'],
offset,
limit
})
Thank you in advance and sorry if my English is not perfect

Order and Limit does not work simultaneously with associations in sequelize

Here, I'm trying to order an associated model and limit it to 10 records. I'm using sequelize version 5.19.1 and sequelize-cli version 5.5.1. I've tried it to order without limiting the records and it worked absolutely fine, while working with the limit it does not order the way it should work. It just limits the records to 10 which is expected while does not order those records simultaneously.
Have tried using subquery: false and separate: true options but it didn't work in this case.
Associations
surgery
surgery.hasMany(models.surgeryDoctors);
surgery.belongsTo(models.species, {
foreignKey: 'speciesId',
targetKey: 'id',
});
surgery.hasMany(models.surgerySpecialities);
surgery doctor
surgeryDoctors.belongsTo(models.doctors, {
foreignKey: 'doctorId',
targetKey: 'id',
});
surgerySpecialities.belongsTo(models.doctorSpecialities, {
foreignKey: 'specialityId',
targetKey: 'id',
});
Query
// here db comprises the sequelize it self.
const { rows, count } = await db.surgery.findAndCountAll({
include: [
{
model: db.surgeryDoctors,
include: [
{
model: db.doctors,
},
],
},
{
model: db.species,
},
{
model: db.surgerySpecialities,
include: [
{
model: db.doctorSpecialities,
},
],
},
],
order: [['species','name','ASC']],
limit: 10,
});
Is there any way to fix it or any better alternative in order to achieve this? (except raw query)

Sequelize how to return column of joined table in the results

I'm using the sequelize module for my node.js mvc project and the query i'd like to execute is the following
SELECT answer_text, isNew, name FROM answer JOIN topic ON answer.topic_id = topic.id
answer_text and isNew are columns of the answer table while name is a column that only exists in the topic table.
How can i have the topic table name column appear in the results next to isNew column so that i can access it easily? Does sequelize provide such a feature or it's my responsibility to format the result?
I've tried to add various things in attributes like 'topic.name' but none worked.
The way i've set up the file structure is based on their documentation Sequelize usage with express
var models = require('../models')
var answers = await models.Answer.findAll({
include: [{
model: models.Topic
}],
attributes: [
'answer_text',
'isNew'
]
})
console.log(answers)
The output of the following is
{ answer_text: 'maybe it is robots',
isNew: true,
Topic:
Topic {
dataValues:
{ id: 830,
mid: 'm.0bjmp5',
name: 'Robot Arena',
description:
'Robot Arena is a computer game made by Infogrames. It features robotic combat similar to that of Battlebots Robotica and Robot Wars. There are a number of different chassis and on top of that there are numerous attachments. Weapons accessories tires and other forms of mobility batteries and air tanks are among the customization choices. A sequel called Robot Arena 2 Design and Destroy was made which allows for total customization of your
robot.',
type: 'cvg.computer_videogame' },
_previousDataValues:
{ id: 830,
mid: 'm.0bjmp5',
name: 'Robot Arena',
description:
'Robot Arena is a computer game made by Infogrames. It features robotic combat similar to that of Battlebots Robotica and Robot Wars. There are a number of different chassis and on top of that there are numerous attachments. Weapons accessories tires and other forms of mobility batteries and air tanks are among the customization choices. A sequel called Robot Arena 2 Design and Destroy was made which allows for total customization of your
robot.',
type: 'cvg.computer_videogame' },
_changed: {},
_modelOptions:
{ timestamps: false,
validate: {},
freezeTableName: false,
underscored: false,
paranoid: false,
rejectOnEmpty: false,
whereCollection: null,
schema: null,
schemaDelimiter: '',
defaultScope: {},
scopes: {},
indexes: [],
name: [Object],
omitNull: false,
sequelize: [Sequelize],
hooks: {} },
_options:
{ isNewRecord: false,
_schema: null,
_schemaDelimiter: '',
include: undefined,
includeNames: undefined,
includeMap: undefined,
includeValidated: true,
raw: true,
attributes: undefined },
isNewRecord: false } }
Please try the following sequelize statement -
var answers = await models.Answer.findAll({
include: [{
model: models.Topic,
attributes: ['name']
}],
attributes: [
'answer_text',
'isNew'
],
raw: true
})
I hope it helps!
Working answer:
Sequelize must be required in order to use [sequelize.col('Topic.name'), 'name'] inside attributes so that we can fetch name column of Topic table and rename 'Topics.name' to name. (Tried models.col but it is not a function)
raw: true is required if you want to get only the columns inside answers[0]
attributes:[] is required inside include because if you don't put it the result will include all the columns from the joined table (Topic).
const models = require('../models')
const sequelize = require('sequelize');
var answers = await models.Answer.findAll({
include: [{
model: models.Topic,
attributes: []
}],
attributes: [
'answer_text',
'isNew',
[sequelize.col('Topic.name'), 'name']
],
raw: true
})
console.log(answers[0])
output:
{ answer_text: 'robot arena',
isNew: 'true',
name: 'Robot Arena' }

Categories

Resources