I have the following situation:
Table computers, table flags and table computerFlags that links them.
Table computerFlags looks like:
computerName | flagId
computer1 | flag1
computer1 | flag2
computer2 | flag2
computer2 | flag3
computer3 does not have any flags, so it is not in the table
I'm working with sequelize, and I'm trying to execute a query to count all computers without "flag3", in the above example table the answer would be 2 ("computer1" and "computer3").
This is the code I have so far:
import { DataTypes, Sequelize, Op } from "sequelize";
(async () => {
try {
const postgresDB = new Sequelize('postgres://<my-pg-credentials>>', {
logging: true,
});
const computers = postgresDB.define('computer', {
name: {
type: DataTypes.TEXT,
primaryKey: true
},
// Other computer fields
});
const flags = postgresDB.define('flag', {
id: {
type: DataTypes.TEXT,
primaryKey: true
},
name: DataTypes.TEXT,
});
const computerFlags = postgresDB.define('computerFlag', {}, { createdAt: false, updatedAt: false })
computers.belongsToMany(flags, { through: computerFlags });
flags.belongsToMany(computers, { through: computerFlags });
await postgresDB.sync({ alter: true })
const c = await computers.count({
distinct: true,
group: ['computer.name'],
include: [
{
model: flags
},
]
});
} catch (err) { console.log(err) }
})();
I get a half-good result with the next SQL query:
select count("computerName") from "computerFlags"
group by "computerName"
having '2' != all(array_agg("flagId"))
But I can't find a way to produce this in sequelize, and also, for the above table it'll return 1 as 'computer3' is not in the table
To execute this with sequelize, I would like to do something like that:
having: {
[[Sequelize.fn('array_agg', Sequelize.col('flag.id')), 'flagIds']] : {
[Op.all]: {
[Op.ne]: '2'
}
}
}
But there are two problems with that:
I cannot use [[Sequelize.fn ...]] as a left operand
I'm not sure if the way I refer to the flag ID is correct as it should be something like computer->flags->flagId. computer->flags returns an array of flags when I use findAll, each one containing flagId.
I'm really lost and confuse and I'd appreciate your help.
I'm trying to execute a query to count all computers without "flag3"
So use NOT EXISTS. (Some would say, a "semi-anti-join".)
SELECT count(*)
FROM computers c
WHERE NOT EXISTS (
SELECT FROM computerFlags cf
WHERE cf.computerName = c.computerName
AND cf.flagId = 'flag3'
);
Only eliminates computers from the count that actually do have an entry with 'flag3'.
Should perform best.
Aside: CaMeL case names are not ideal for Postgres. See:
Are PostgreSQL column names case-sensitive?
Related
How to include aggregation in if condition ,do I need to use project or condition method in if condition above as well catalogue populate. I need to get the data from the mongo dB in the same order as video Ids array but it's coming in a different order so I decided to use aggregation to get the video in a proper order as in the video Ids array. Please help me to resolve this issue.
let filter = {$match:{
customerId: customerId,
_id: {
$in: _.map(videoIds, id => mongoose.Types.ObjectId(id)) || []
},
_isDeleted: false,
isActive: true
},
$lookup:{
from:'catalogues',
localField:'_isDeleted',
foreignField:'_id',
as:false
}
}
if (!req.isLocationWhitelisted) {
if (req._countryCode) {
$or=
filter.$match['languages.country'] = {
$in: req._countryCode
}
filter.$lookup['languages.country'] = {
$in: req._countryCode
}
,
filter.$match['languages.country.141'] = { $exists: true }
filter.$lookup['languages.country.141'] = { $exists: true }
}
}
let videoList = await Video.aggregate(filter);
I have two models
Units and Filters which are related with M:N association through unit_filter table
this.belongsToMany(Unit, {
through: "unit_filter",
as: "units",
});
this.belongsToMany(Filter, {
through: 'unit_filter',
as: 'filters',
});
The goal is to fetch units which have more than 1 filter associated with and condition.
let result = await Unit.findAll({
include: [
{
model: Filter,
where: {
id: {
[Op.and] : [2,252,4,80]
}
},
as: 'filters',
},
],
});
The above is only working if there is only one id in the array which does not make any sense.
Seqeulize documenation states
Post.findAll({
where: {
[Op.and]: [{ a: 5 }, { b: 6 }], // (a = 5) AND (b = 6)
}
})
So the code I have should work theoritically. I also tried with
where: {
[Op.and] : [{id:2},{id:252},{id:4},{id:80}]
}
which results in getting all the items from the db. It does not even care about the where condition in this case.
Would be of great help if any one points me in right direction.
You need to use Sequelize.literal with a subquery in where option in order to filter units that have more than 1 filter because simply indicating several ids of filters you will get units that have one of indicated filters (from 1 to N).
let result = await Unit.findAll({
where: Sequelize.where(
Sequelize.literal('(SELECT COUNT(*) FROM unit_filter as uf WHERE uf.unit_id=unit.id AND uf.filter_id in ($filter_ids))'),
Op.gt,
'1'),
bind: {
filter_ids: [2,252,4,80]
}
});
I'm new to Aggregation in MongoDB and I'm trying to understand the concepts of it by making examples.
I'm trying to paginate my subdocuments using aggregation but the returned document is always the overall values of all document's specific field.
I want to paginate my following field which contains an array of Object IDs.
I have this User Schema:
const UserSchema = new mongoose.Schema({
username: {
type: String,
unique: true,
required: true
},
firstname: String,
lastname: String,
following: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}],
...
}, { timestamps: true, toJSON: { virtuals: true }, toObject: { getters: true, virtuals: true } });
Without aggregation, I am able to paginate following,
I have this route which gets the user's post by their username
router.get(
'/v1/:username/following',
isAuthenticated,
async (req, res, next) => {
try {
const { username } = req.params;
const { offset: off } = req.query;
let offset = 0;
if (typeof off !== undefined && !isNaN(off)) offset = parseInt(off);
const limit = 2;
const skip = offset * limit;
const user = await User
.findOne({ username })
.populate({
path: 'following',
select: 'profilePicture username fullname',
options: {
skip,
limit,
}
})
res.status(200).send(user.following);
} catch (e) {
console.log(e);
res.status(500).send(e)
}
}
);
And my pagination version using aggregate:
const following = await User.aggregate([
{
$match: { username }
},
{
$lookup: {
'from': User.collection.name,
'let': { 'following': '$following' },
'pipeline': [
{
$project: {
'fullname': 1,
'username': 1,
'profilePicture': 1
}
}
],
'as': 'following'
},
}, {
$project: {
'_id': 0,
'following': {
$slice: ['$following', skip, limit]
}
}
}
]);
Suppose I have this documents:
[
{
_id: '5fdgffdgfdgdsfsdfsf',
username: 'gagi',
following: []
},
{
_id: '5fgjhkljvlkdsjfsldkf',
username: 'kuku',
following: []
},
{
_id: '76jghkdfhasjhfsdkf',
username: 'john',
following: ['5fdgffdgfdgdsfsdfsf', '5fgjhkljvlkdsjfsldkf']
},
]
And when I test my route for user john: /john/following, everything is fine but when I test for different user which doesn't have any following: /gagi/following, the returned result is the same as john's following which aggregate doesn't seem to match user by username.
/john/following | following: 2
/kuku/following | following: 0
Aggregate result:
[
{
_id: '5fdgffdgfdgdsfsdfsf',
username: 'kuku',
...
},
{
_id: '5fgjhkljvlkdsjfsldkf',
username: 'gagi',
...
}
]
I expect /kuku/following to return an empty array [] but the result is same as john's. Actually, all username I test return the same result.
I'm thinking that there must be wrong with my implementation since I've only started exploring aggregation.
Mongoose uses a DBRef to be able to populate the field after it has been retrieved.
DBRefs are only handled on the client side, MongoDB aggregation does not have any operators for handling those.
The reason that aggregation pipeline is returning all of the users is the lookup's pipeline does not have a match stage, so all of the documents in the collection are selected and included in the lookup.
The sample document there is showing an array of strings instead of DBRefs, which wouldn't work with populate.
Essentially, you must decide whether you want to use aggregation or populate to handle the join.
For populate, use the ref as shown in that sample schema.
For aggregate, store an array of ObjectId so you can use lookup to link with the _id field.
I need to create a custom join condition using Sequelize.js with MSSQL. Specifically, I need to join TableB based on a COALESCE value from columns in TableA and TableB and end up with a join condition like this:
LEFT OUTER JOIN [TableB]
ON [TableB].[ColumnB] = COALESCE (
[TableC].[ColumnC],
[TableA].[ColumnA]
)
I'd settle for an OR clause in my join:
LEFT OUTER JOIN [TableB]
ON [TableB].[ColumnB] = [TableA].[ColumnA]
OR [TableB].[ColumnB] = [TableC].[ColumnC]
I read that you can achieve behaviour like this by including required: false in your scope definition. As you can see, I've plastered my scope with it attempting to get this to work.
The best I could get is this (note the AND clause):
LEFT OUTER JOIN [TableB]
ON [TableB].[ColumnB] = [TableA].[ColumnA]
AND [TableB].[ColumnB] = COALESCE (
[TableC].[ColumnC],
[TableA].[ColumnA]
)
If I were using MySQL, I think I could simply use the COALESCE value from the SELECT in the JOIN and be good to go but in my prior research read that it was required to recalculate the value.
I've included a stripped down model definition for TableA:
export default (sequelize, DataTypes) => sequelize.define('TableA',
{
// Attributes omitted..
},
{
classMethods: {
associate ({
TableB,
TableC
}) {
this.belongsTo(TableB, {
foreignKey: 'ColumnA',
targetKey: 'ColumnB'
});
this.belongsTo(TableC, {
foreignKey: 'ColumnA',
targetKey: 'ColumnC'
});
},
attachScope ({
TableB,
TableC
}) {
this.addScope('defaultScope', {
attributes: [
...Object.keys(this.attributes),
[
sequelize.fn(
'COALESCE',
sequelize.col('[TableC].[ColumnC]'),
sequelize.col('[TableA].[ColumnA]')
),
'ColumnA'
]
],
include: [
{
model: TableB,
where: {
ColumnB: sequelize.fn(
'COALESCE',
sequelize.col('[TableC].[ColumnC]'),
sequelize.col('[TableA].[ColumnA]')
)
},
required: false
},
{
model: TableC,
required: false
}
],
required: false
}, { override: true });
}
}
}
);
Any assistance would be greatly appreciated, and if any additional information is required please let me know.
Note: I'm working with a legacy database and unfortunately cannot change the data structure.
Hi I had the same issue and finally I solved using a SQL pure query.
Example:
const query = `SELECT s.*, us.UserId \
FROM UserSections AS us \
RIGHT JOIN Sections AS s ON (s.id = us.SectionId) \
WHERE s.parentId = ${sectionId}`;
return db.query(query, { type: db.QueryTypes.SELECT });
The problem is that this function will return a IMyType instead of IMyTypeInstance
Can it works for you??
I'm using sequelize to get a total count through a relationship. I need it by a customerId that is in a parent table joined through a pivot table. The plain query looks something like this:
SELECT count(p.*) FROM parcels as p
LEFT JOIN orders_parcels as op ON op."parcelId" = p.id
LEFT JOIN orders as o ON op."orderId" = o.id
WHERE o."customerId"=1
This works fine. But not sure how to get the sequelize query.
Parcel.findAndCountAll();
EDIT: OrderParcel
var OrderParcel = service.sequelize.define('OrderParcel', {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
}
}, {
tableName: 'orders_parcels',
freezeTableName: true,
paranoid: true
});
module.exports = OrderParcel;
var Order = require('./Order');
OrderParcel.belongsTo(Order, {
as: 'Order',
foreignKey: 'orderId'
});
var Parcel = require('../parcel/Parcel');
OrderParcel.belongsTo(Parcel, {
as: 'Parcel',
foreignKey: 'parcelId'
});
One way is to use sequelize.query:
As there are often use cases in which it is just easier to execute raw
/ already prepared SQL queries, you can utilize the function
sequelize.query.
var query = "SELECT count(p.*) FROM parcels as p" +
" LEFT JOIN orders_parcels as op ON op."parcelId" = p.id" +
" LEFT JOIN orders as o ON op."orderId" = o.id" +
" WHERE o.customerId=1;";
sequelize.query(query, { type: sequelize.QueryTypes.SELECT}).success(function(count){
console.log(count); // It's show the result of query
res.end();
}).catch(function(error){
res.send('server-error', {error: error});
});
Raw Queries docs
Assuming that you've defined the associations, you can use Model.findAndCountAll. It'd look something like this:
Parcel.findAndCountAll({
include: [{
model: OrderParcel,
required: true,
include: [{
model: Order,
where: {
customerId: idNum
}
}]
}]
}).then(function(result) {
});
I totally agree with Evan Siroky's approach yet the code has to be simplified to work properly:
Parcel.findAndCountAll({
include: [{
model: Order,
where: {
customerId: idNum
},
duplicating: false // Add this line for retrieving all objects
}]
}).then(function(result) {
console.log('Rows: ' + result.rows + ' Count: ' + result.count)
});
Remember to connect your models with belongsToMany method!