Develop Reference

How to write formula in ORDER using fields from subquery - javascript

I need to implement sorting on computed data from a subquery. There is a request like this:
const res = await db.order.findAndCountAll({
limit: 10,
offset: (page - 1) * 10,
distinct: true,
order: [[db.sequelize.literal('product.taxes * product.count'), 'ASC']],
where,
include: [
{ model: db.location },
{ model: db.address },
{
model: db.order_product,
as: 'products',
required: true,
include: [
{
model: db.product,
include: [{ model: db.location }],
},
],
},
],
});
This code is not working. I get the error:
missing FROM-clause entry for table "product"
In line:order: [[db.sequelize.literal('product.taxes * product.count'), 'ASC']], i want to use to calculate the fields that are in the db.product model which i am accessing in a sub-sub-query. Moreover, this sub-sub-query will return an array, and I need to execute (db.product.amount * db.product.price) for each element of the array and then add up all the resulting products. How to describe this formula in "order"?
Maybe I chose the wrong path. What is the best way to sort by calculated data from a subquery?

Related

I'm very new to using sequelize. I have a query with nested includes as follows:
Site.findAndCountAll({
where,
order: orderCondition,
...(orderAfter ? {} : limits),
include: [
{
model: Version,
as: 'versions',
limit: 1,
separate: true,
order: [['version_id', 'DESC']],
attributes: ['status', 'config'],
include: [
{
model: User,
as: 'updatedByUser',
attributes: ['email'],
paranoid: false,
},
],
},
{
model: Version,
as: 'prod_version',
separate: false,
required: false,
attributes: ['status', 'config'],
where: {
is_prod: 1,
status: 'SUCCESS',
},
include: [
{
model: User,
as: 'updatedByUser',
attributes: ['email'],
paranoid: false,
},
],
}
],
distinct: true,
});
And I have these associations between Site and Version:
Site.hasMany(models.Version, {
foreignKey: 'site_id',
as: 'versions',
});
Site.hasOne(models.Version, {
foreignKey: 'site_id',
as: 'prod_version',
});
This query can take several seconds to load a page when navigating through the pagination. It takes longer the further along I go through the pages. It's fine for the first ~10-15 pages but then starts getting increasingly slower and by page 30-40 it sometimes takes more than 5 seconds to get a response.
I know there's an inherent issue with the way OFFSET works in SQL but I feel it shouldn't be this slow unless I'm hundreds of pages in. How else could I potentially optimize this query? A big part of the slowdown also seems to be the second item in include, prod_version. With that commented out I get a huge improvement of the response time. Is there a more efficient way to achieve the same result?

I currently have the following query in Sequelize:
const sites = await Site.findAndCountAll({
include: [
{
model: Version,
as: 'versions',
limit: 1,
order: [['version_id', 'DESC']],
attributes: ['status', 'config', 'user_id', 'updated_at', 'version_id', 'updated_by', 'is_abtest_parent', 'is_prod'],
include: [
{
model: User,
as: 'updatedByUser',
attributes: ['email'],
paranoid: false,
},
],
},
],
distinct: true,
});
This gets sites from the sites table and the version with the latest version_id for a specific site. The Site and Version models have the following associations:
Version.belongsTo(models.Site, {
foreignKey: 'site_id',
});
Site.hasMany(models.Version, {
foreignKey: 'site_id',
as: 'versions',
});
I now need to also get the version that is in production and has its is_prod attribute set to 1, while also keeping the version with the latest version_id. What would be the best way to query these two versions?

I've come across a problem or bug where if I order by an associated column, and limit the results, they appear to be limited first and then ordered.
This doesn't happen when ordering by columns of the model I've queried, just the associated model's column. The order is also perfect if I don't limit the results at all.
Practical example:
Job A (Company 2), Job B (Company 1), Job C (Company 4), Job D (Company 3)
Ordering by Company name with no limit property gives the results: B, A, D, C, as expected
Adding a limit property of 3 seems to return the limited results: A, B, C and then orders them, resulting in B, A, C, when it should be B, A, D
Association:
Company.hasMany(Job, { foreignKey: { name: 'companyId', allowNull: false} });
Job.belongsTo(Company, { foreignKey: { name: 'companyId', allowNull: false} });
Models:
const Company = sequelize.define('company', {
id: {
type: Sequelize.INTEGER,
allowNull: false,
primaryKey: true,
autoIncrement: true
},
name: {
type: Sequelize.STRING,
allowNull: false,
}
});
const Job = sequelize.define('job', {
id: {
type: Sequelize.INTEGER,
autoIncrement: true,
allowNull: false,
primaryKey: true
},
title: {
type: Sequelize.STRING,
allowNull: false
},
// Lots more boring fields
});
Query & options object:
switch(req.query.orderField) {
case 'company': {
order = [
'company', 'name', orderDirection
];
break;
}
default:
order = [ orderField, orderDirection ];
}
const options = {
attributes: [
'id',
'title',
[Sequelize.fn('date_format', Sequelize.col('job.createdAt'), '%d/%m/%y'), 'jobDate'],
],
order: [ order ],
distinct: true,
include: [
{
model: Company,
attributes: ['name'],
},
]
};
// Commenting out this line works fine
if(req.query.limit) options.limit = parseInt(limit, 10);
if(req.query.index) options.offset = parseInt(index);
const results = await Job.findAndCountAll(options);
I'm pretty stumped tbh, would really appreciate some help!
**Update:
So in order to make the options model more concise, I left out one other included model: Applicant. So Job has an include with both a Company model and an Applicant model (with its own nested model).
It actually looks like this:
include: [
{
model: Company,
attributes: ['name'],
},
{
model: Applicant,
attributes: [
'id',
'cvUrl',
'personId',
[Sequelize.fn('date_format', Sequelize.col('applicants.createdAt'), '%d/%m/%y'), 'createdAt'],
],
include: [
{
model: Person,
attributes: [ 'firstName', 'lastName', 'phone', 'email' ]
}
]
}
]
I didn't think it would make any difference, but I checked out the query, and it does have a subquery: (TLDR for your convenience followed by real query)
TLDR pseudo query:
SELECT job.* ... FROM(SELECT job.id ... FROM jobs AS job LIMIT 0 8) AS job ... JOINS HERE ORDER BY company.name ASC
Actual query:
Executing (default): SELECT `job`.*, `company`.`id` AS `company.id`, `company`.`name` AS `company.name`, `applicants`.`id` AS `applicants.id`, `applicants`.`cvUrl` AS `applicants.cvUrl`, `applicants`.`personId` AS `applicants.personId`, date_format(`applicants`.`createdAt`, '%d/%m/%y') AS `applicants.createdAt`, `applicants->application`.`id` AS `applicants.application.id`, `applicants->application`.`createdAt` AS `applicants.application.createdAt`, `applicants->application`.`updatedAt` AS `applicants.application.updatedAt`, `applicants->application`.`applicantId` AS `applicants.application.applicantId`, `applicants->application`.`jobId` AS `applicants.application.jobId`, `applicants->person`.`id` AS `applicants.person.id`, `applicants->person`.`firstName` AS `applicants.person.firstName`, `applicants->person`.`lastName` AS `applicants.person.lastName`, `applicants->person`.`phone` AS `applicants.person.phone`, `applicants->person`.`email` AS `applicants.person.email` FROM (SELECT `job`.`id`, `job`.`title`, `job`.`wage`, `job`.`location`, `job`.`description`, `job`.`jobType`, `job`.`position`, `job`.`pqe`, `job`.`featured`, `job`.`createdAt`, date_format(`job`.`createdAt`, '%d/%m/%y') AS `jobDate`, `job`.`companyId` FROM `jobs` AS `job` LIMIT 0, 8) AS `job` LEFT OUTER JOIN `companies` AS `company` ON `job`.`companyId` = `company`.`id` LEFT OUTER JOIN ( `applications` AS `applicants->application` INNER JOIN `applicants` AS `applicants` ON `applicants`.`id` = `applicants->application`.`applicantId`) ON `job`.`id` = `applicants->application`.`jobId` LEFT OUTER JOIN `people` AS `applicants->person` ON `applicants`.`personId` = `applicants->person`.`id` ORDER BY `company`.`name` ASC;
So it is limiting the results first.
But here's the fix: Removing the included Applicant model:
(This query isn't so bad)
Executing (default): SELECT `job`.`id`, `job`.`title`, `job`.`wage`, `job`.`location`, `job`.`description`, `job`.`jobType`, `job`.`position`, `job`.`pqe`, `job`.`featured`, `job`.`createdAt`, date_format(`job`.`createdAt`, '%d/%m/%y') AS `jobDate`, `job`.`companyId`, `company`.`id` AS `company.id`, `company`.`name` AS `company.name` FROM `jobs` AS `job` LEFT OUTER JOIN `companies` AS `company` ON `job`.`companyId` = `company`.`id` ORDER BY `company`.`name` ASC LIMIT 0, 8;
But I can't work out why that included Applicant would fix the problem?

Here, I'm trying to order an associated model and limit it to 10 records. I'm using sequelize version 5.19.1 and sequelize-cli version 5.5.1. I've tried it to order without limiting the records and it worked absolutely fine, while working with the limit it does not order the way it should work. It just limits the records to 10 which is expected while does not order those records simultaneously.
Have tried using subquery: false and separate: true options but it didn't work in this case.
Associations
surgery
surgery.hasMany(models.surgeryDoctors);
surgery.belongsTo(models.species, {
foreignKey: 'speciesId',
targetKey: 'id',
});
surgery.hasMany(models.surgerySpecialities);
surgery doctor
surgeryDoctors.belongsTo(models.doctors, {
foreignKey: 'doctorId',
targetKey: 'id',
});
surgerySpecialities.belongsTo(models.doctorSpecialities, {
foreignKey: 'specialityId',
targetKey: 'id',
});
Query
// here db comprises the sequelize it self.
const { rows, count } = await db.surgery.findAndCountAll({
include: [
{
model: db.surgeryDoctors,
include: [
{
model: db.doctors,
},
],
},
{
model: db.species,
},
{
model: db.surgerySpecialities,
include: [
{
model: db.doctorSpecialities,
},
],
},
],
order: [['species','name','ASC']],
limit: 10,
});
Is there any way to fix it or any better alternative in order to achieve this? (except raw query)

I have a model with an expiration date:
expireAt: {
type: DataTypes.DATE,
allowNull: false
}
I want to create a scope to find the "enabled" records, actually is:
// I add it on associate method, because need to use other model:
QuestionsModel.addScope("enableds", {
where: {
enabled: true // The question is enabled
},
include: [{
model: models.users,
where: {
enabled: true // The author is enabled
}
}]
});
But I don't know how to validate if is expired, I need to discard the expired records (when actual date is higher or equal with expireAt).
Some idea?