I have a raw query like
squelize.query(
`select "order".id, "orderCustomer".id as "orderCustomer.id", "orderCustomer".forename as "orderCustomer.forename"
from orders as "order"
join customers as "orderCustomer" on "orderCustomer".id = "order"."orderCustomerId"
where "orderCustomer".forename ilike '%petra%';`,
{
type: Sequelize.QueryTypes.SELECT,
model: order,
nest: true,
mapToModel: true
})
When I query this in psql I get a correct result set:
-[ RECORD 1 ]----------+------
id | 383
orderCustomer.id | 446
orderCustomer.forename | Petra
-[ RECORD 2 ]----------+------
id | 419
orderCustomer.id | 9
orderCustomer.forename | Petra
The problem is, that Sequelize is apparently not able to form this into an Array of the kind
[
{
id: 383,
orderCustomer: {
id: 446,
forename: 'Petra'
}
},
...
]
Instead I get something like this:
[
{
id: 383,
'orderCustomer.id': 446,
'orderCustomer.forename': 'Petra'
},
...
]
Do I need to include the customer-model in the query's option-object?
UPDATE
I logged the result of my query. There is this property _options on all of the returned order-instances:
_options:{
isNewRecord: false,
_schema: null,
_schemaDelimiter: '',
raw: true, // <--------- possible cause?
attributes: undefined
}
Somehow, I cannot set options.raw to false on my query definition! Maybe that is the reason why …
…sequelize will not try to format the results of the query, or build an instance of a model from the result (see docs > query() > options.raw)
Any ideas?
I'm using your solution: nest: true and it works perfectly!
[err, rows] = await to(sequelize.query(query,{ replacements: { search: '%'+search+'%', limit: limit, offset: skip}, type: sequelize.QueryTypes.SELECT,nest: true}));
Try it without the
model: order,
this solved it for me
It is 2021 and seems that this is still doesn't work with Sequelize 6.6.2. Or rather I can't work out how to make the mapToModel work in conjunction with the nested properties.
I worked around the issue by removing the model: MyModel, mapToModel: true, part so that the query options look like this:
{ type: QueryTypes.SELECT, bind: [mainAssetId], nest: true, raw: false }
I get the object back with all the fields and nested works but it is of course not an instance of the Model. It looks like the model from data perspective so as long as you don't have methods on your model you should be fine.
You could always use something like the class-transformer package to transform it to the model instance.
Assuming you have the models set up properly,
Why not query it using associations, instead of using a raw query.
Something like this should generate a similar result
Model.Order.findAll({
where:{forename:{$iLike:'someString'}},
attributes:[/*Columns to fetch*/]
include:[{model:Model.Customer,attributes:[/*Columns to fetch*/]}]
});
Also IMO using the nest attributes, wont club across multiple rows returned.
I had the same issue and made it work this way:
sequelize.query(
`SELECT q.id, q.title, a.id AS "answers.id"
FROM questions q
JOIN answers a ON a.question_id = q.id`,
model: Question,
mapToModel: true,
nest: true,
raw: true,
type: sequelize.QueryTypes.SELECT }
)
Related
Issue explanation
I want to do a query with pagination that limits to 12 lines each query, to be more specific, that limits to 12 Batches each query. Actually the amount of lines get smaller because a belongsToMany association with a join table i got in this query. The join order to this query is: Offer > hasMany > Batch > belongsToMany > BatchFile > belongsTo > File. The problem is, when i have many registries in the File as 'gallery' association, it brings me duplicated registries of batch.
The query i'm trying to do
const { id } = req.params;
const { page = 1 } = req.query;
const offer = await Offer.findAndCountAll({
attributes: [ 'id', 'name', 'canceled_at'],
where: { id, canceled_at: null },
order: [['offer_batches', 'name', 'ASC']],
include: [
{
/* hasMany association */
model: Batch,
as: 'offer_batches',
attributes: [ 'id', 'name'],
include: [
{ /* Other includes... */ },
{
/* belongsToMany association */
model: File,
as: 'gallery',
attributes: ['id', 'path', 'url'],
},
],
},
],
subQuery: false,
limit: 12,
offset: (page - 1) * 12,
});
Model associations
Offer model
this.hasMany(Batch, { foreignKey: 'offer_id', as: 'offer_batches' });
Batch model
this.belongsToMany(File, { through: models.BatchFile, as: 'gallery' });
BatchFile model (join table)
this.belongsTo(Batch, { foreignKey: 'batch_id', as: 'batch' });
this.belongsTo(File, { foreignKey: 'file_id', as: 'file' });
What i already tried
Giving duplicating: false option to any included Model doesn't worked;
Giving separate: true to the Batch model doesn't worked too;
Giving required: true option to any included model doesn't worked too;
If i remove subQuery: false it doesn't respect the setted limit of lines, and i already tried with all of the above combinations;
I thought sequelize would deal with this situation without problems, maybe i'm doint something wrong.
If helps, here's the raw generated SQL:
SELECT
"Offer"."id",
"Offer"."name",
"Offer"."canceled_at",
"offer_batches"."id"
AS "offer_batches.id", "offer_batches"."name"
AS "offer_batches.name", "offer_batches->gallery"."id"
AS "offer_batches.gallery.id", "offer_batches->gallery"."path"
AS "offer_batches.gallery.path", "offer_batches->gallery->BatchFile"."created_at"
AS "offer_batches.gallery.BatchFile.createdAt", "offer_batches->gallery->BatchFile"."updated_at"
AS "offer_batches.gallery.BatchFile.updatedAt", "offer_batches->gallery->BatchFile"."file_id"
AS "offer_batches.gallery.BatchFile.FileId", "offer_batches->gallery->BatchFile"."batch_id"
AS "offer_batches.gallery.BatchFile.BatchId", "offer_batches->gallery->BatchFile"."batch_id"
AS "offer_batches.gallery.BatchFile.batch_id", "offer_batches->gallery->BatchFile"."file_id"
AS "offer_batches.gallery.BatchFile.file_id"
FROM "offer" AS "Offer"
LEFT OUTER JOIN "batch" AS "offer_batches" ON "Offer"."id" = "offer_batches"."offer_id"
LEFT OUTER JOIN (
"batch_file" AS "offer_batches->gallery->BatchFile"
INNER JOIN "file" AS "offer_batches->gallery"
ON "offer_batches->gallery"."id" = "offer_batches->gallery->BatchFile"."file_id"
)
ON "offer_batches"."id" = "offer_batches->gallery->BatchFile"."batch_id"
WHERE "Offer"."id" = '1' AND "Offer"."canceled_at" IS NULL
ORDER BY "offer_batches"."name"
ASC LIMIT 12 OFFSET 0;
Environment
Node: v14.18.0
package.json dependencies
pg: 8.7.1
pg-hstore: 2.3.4
sequelize: 6.9.0
Trying to find any solution on GitHub issues or stackoverflow, nothing solved this problem. Maybe i'm doing this query wrong, any help would be grateful and welcome :)
Well, i didn't found a quite solution for this, so i resolved to use Lazy loading for this situation instead of Eager loading, as mentioned on Sequelize Docs.
I splitted the query into two new queries.
First one, on the "master" model Offer, with a simple findOne():
const offer = await Offer.findOne({
[/* My attributes */],
where: { /* My conditions */ }
});
And a second one, selecting from model Batch, without subQuery: false option, because is not needed anymore.
const batches = await Batch.findAndCountAll({
attributes: [
'id',
'name',
],
where: { offer_id: offer.id },
order: [/* My ordenation */],
include: [
{
model: File,
as: 'gallery',
attributes: ['id', 'path', 'url'],
},
],
limit: 12,
offset: (page - 1) * 12,
});
select reservation_datetime
from LectureReservation
Inner Join Lecture
On LectureReservation.lecture_id = Lecture.id
Where Lecture.mentor_id = 1
This is my query and I want to change it to sequelize like
if (req.params.id) {
LectureReservation
.findAll({
include: [{
model: Lecture,
where: { mentor_id: req.params.id },
}],
attributes: ['reservation_datetime'],
where: {
lecture_id: Lecture.id,
},
this.. I tried it so hard but can't find solution and my postman keep showing me
"name": "SequelizeEagerLoadingError"
this err..
plz help me to translate query to sequelize..!
Sequelize will do _outer join without required = true.
The errors you have received usually is from association problem.
Try set logging :console.log and check the raw query.
this below Sequelize work fine for me without using order, i'm wondering why i can't use order for root table as posts model? when i use this below code i get this error:
Unhandled rejection Error: 'posts' in order / group clause is not valid association
but that work fine on other models such as channelVideoContainer
models.posts.findAll({
where: {
channelId: 1
},
include: [
{
model: models.channelVideoContainer,
include: [models.fileServerSetting]
}, {
model: models.channelMusicContainer,
include: [models.fileServerSetting]
}, {
model: models.channelImageWithTextContainer,
include: [models.fileServerSetting]
}, {
model: models.channelFilesContainer,
include: [models.fileServerSetting]
},
models.channelPlainTextContainer
], order: [
[{model: models.posts}, 'id', 'DESC'],
], limit: 5
}).then(function (result) {
console.log(JSON.stringify(result));
});
You are getting this error because you are querying the posts table/model, and then sorting by a column on the posts model, however you are specifying a "joined" table in your order. This works for your other models because they are in fact joined (using the include option). Since you are querying the posts model you just need to pass in the name of the column you want to order by. See some of the ORDER examples in the documentation.
// just specify the 'id' column, 'post' is assumed because it is the queried Model
order: [['id', 'DESC']],
As a side note, you may want to specify required: false on your include'd models to perform a LEFT JOIN so that rows come back even if there are no matches in the joined table. If you know that rows will be returned (or they are actually required) then leave it as is.
{
model: models.channelFilesContainer,
include: [models.fileServerSetting],
required: false, // LEFT JOIN the channelFilesContainer model
},
My database model is as follows:
An employee drives one or zero vehicles
A vehicle can be driven by one or more employees
A vehicle has a model type that tells us it's fuel type amongst other things.
I'd like sequelize to fetch me all employees where they don't drive a vehicle, or if they do then the vehicle is not diesel.
So where VehicleID is null OR Vehicle.VehicleModel.IsDiesel = false
My current code is as follows:
var employee = sequelize.define('employee', {
ID: Sequelize.INTEGER,
VehicleID: Sequelize.INTEGER
});
var vehicle = sequelize.define('vehicle', {
ID: Sequelize.INTEGER,
ModelID: Sequelize.INTEGER
});
var vehicleModel = sequelize.define('vehicleModel', {
ID: Sequelize.INTEGER,
IsDiesel: Sequelize.BOOLEAN
});
employee.belongsTo(vehicle);
vehicle.belongsTo(vehicleModel);
If I run the following:
options.include = [{
model: model.Vehicle,
attributes: ['ID', 'ModelID'],
include: [
{
model: model.VehicleModel,
attributes: ['ID', 'IsDiesel']
}]
}];
employee
.findAll(options)
.success(function(results) {
// do stuff
});
Sequelize does a left outer join to get me the included tables. So I get employees who drive vehicles and who don't.
As soon as I add a where to my options:
options.include = [{
model: model.Vehicle,
attributes: ['ID', 'ModelID'],
include: [
{
model: model.VehicleModel,
attributes: ['ID', 'IsDiesel']
where: {
IsDiesel: false
}
}]
}];
Sequelize now does an inner join to get the included tables.
This means that I only get employees who drive a vehicle and the vehicle is not diesel. The employees who don't drive a vehicle are excluded.
Fundamentally, I need a way of telling Sequelize to do a left outer join and at the same time have a where condition that states the column from the joined table is false or null.
EDIT:
It turns out that the solution was to use required: false, as below:
options.include = [{
model: model.Vehicle,
attributes: ['ID', 'ModelID'],
include: [
{
model: model.VehicleModel,
attributes: ['ID', 'IsDiesel']
where: {
IsDiesel: false
},
required: false
}],
required: false
}];
I had already tried putting the first 'required:false' but I missed out on putting the inner one. I thought it wasn't working so I gave up on that approach. Dajalmar Gutierrez's answer made me realise I needed both for it to work.
When you add a where clause, sequelize automatically adds a required: true clause to your code.
Adding required: false to your include segment should solve the problem
Note: you should check this issue iss4019
Eager loading
When you are retrieving data from the database there is a fair chance that you also want to get associations with the same query - this is called eager loading. The basic idea behind that, is the use of the attribute include when you are calling find or findAll.
when you set
required: false
will do
LEFT OUTER JOIN
when
required: true
will do
INNER JOIN
for more detail docs.sequelizejs eager-loading
I'm fairly new to MongoDb / Mongoose, more used to SQL Server or Oracle.
I have a fairly simple Schema for an event.
EventSchema.add({
pkey: { type: String, unique: true },
device: { type: String, required: true },
name: { type: String, required: true },
owner: { type: String, required: true },
description: { type: String, required: true },
});
I was looking at Mongoose Indexes which shows two ways of doing it, I used the field definition.
I also have a very simple API that accepts a POST and calls create on this collection to insert the record.
I wrote a test that checks that the insert of a record with the same pkey should not happen and that the unique:true is functioning. I already have a set of events that I read into an array so I just POST the first of these events again and see what happens, I expected that mongo DB would throw the E11000 duplicate key error, but this did not happen.
var url = 'api/events';
var evt = JSON.parse(JSON.stringify(events[0]));
// POST'ed new record won't have an _id yet
delete evt._id;
api.post(url)
.send(evt)
.end(err, res) {
err.should.exist;
err.code.should.equal(11000);
});
The test fails, there is no error and a duplicate record is inserted.
When I take a look at the collection I can see two records, both with the same pkey (the original record and the copy that I posted for the test). I do notice that the second record has the same creation date as the first but a later modified date.
(does mongo expect me to use the latest modified version record???, the URL is different and so is the ID)
[ { _id: 2,
pkey: '6fea271282eb01467020ce70b5775319',
name: 'Event name 01',
owner: 'Test Owner',
device: 'Device X',
description: 'I have no idea what\'s happening',
__v: 0,
url: '/api/events/2',
modified: '2016-03-23T07:31:18.529Z',
created: '2016-03-23T07:31:18.470Z' },
{ _id: 1,
pkey: '6fea271282eb01467020ce70b5775319',
name: 'Event name 01',
owner: 'Test Owner',
device: 'Device X',
description: 'I have no idea what\'s happening',
__v: 0,
url: '/api/events/1',
modified: '2016-03-23T07:31:18.470Z',
created: '2016-03-23T07:31:18.470Z' }
]
I had assumed that unique: true on the field definition told mongo db that this what you wanted and mongo enforced that for you at save, or maybe I just misunderstood something...
In SQL terms you create a key that can be used in URL lookup but you can build a unique compound index, to prevent duplicate inserts. I need to be able to define what fields in an event make the record unique because on a form data POST the submitter of a form does not have the next available _id value, but use the _id (done by "mongoose-auto-increment") so that the URL's use from other parts of the app are clean, like
/events/1
and not a complete mess of compound values, like
/events/Event%20name%2001%5fDevice%20X%5fTest%20Owner
I'm just about to start coding up the so for now I just wrote a simple test against this single string, but the real schema has a few more fields and will use a combination of them for uniqueness, I really want to get the initial test working before I start adding more tests, more fields and more code.
Is there something that I should be doing to ensure that the second record does not actually get inserted ?
It seems that you have done unique indexing(at schema level) after inserting some records in db.
please follow below steps to avoiding duplicates -
1) drop your db:
$ mongo
> use <db-name>;
> db.dropDatabase();
2) Now do indexing at schema level or db level
var EventSchema = new mongoose.Schema({
pkey: { type: String, unique: true },
device: { type: String, required: true },
name: { type: String, required: true },
owner: { type: String, required: true },
description: { type: String, required: true },
});
It will avoid duplicate record insertion with same pKey value.
and for ensuring the index, use command db.db_name.getIndexes().
I hope it helps.
thank you
OK it looks like it has something to do with the index not having time to update before the second insert is posted (as there is only 9ms between them in my test suite).
need to do something about inserts waiting for "index"
needs to be API side as not all users of the API are web applications
I also found some other SO articles about constraints:
mongoose unique: true not work
Unique index not working with Mongoose / MongoDB
MongoDB/Mongoose unique constraint on Date field
on mongoose.connect add {useCreateIndex: true}
It should look like this
mongoose.connect(uri, {
useNewUrlParser: true,
useUnifiedTopology: true,
useCreateIndex: true
})
add:
EventSchema.index({ pkey: 1 }, { unique: true });
// Rebuild all indexes
await User.syncIndexes();
worked for me.
from https://masteringjs.io/tutorials/mongoose/unique