I want to get the max (most recent) created at date of each group of objects from a table using sequelize.
My entity roughly follows this interface:
class MyEntity {
id,
groupName,
createdAt
}
I want to do something like this:
await MyEntity.findAll({
attributes: [[Sequelize.fn('max', Sequelize.col('created_at')), 'max']],
group: ['group_name']
})
I would interpret this as:
1. For all entites
2. Group by "groupName"
3. And get the max value for each group
However, I get the following error:
SequelizeDatabaseError: column "Template.created_at" must appear in the GROUP BY clause or be used in an aggregate function
I definitely do not want to group by the created_at column as well, as that would be meaningless.
The SQL for this operation seems pretty basic:
SELECT groupName, MAX(createdAt) FROM [MyEntity]
GROUP BY groupName
The issue was caused because I had a default scope applied to the model, that was ordering the results on the created_at column. So, because created_at was not a attribute in my result set, I was unable to sort based on that column and got the error.
I had to unscope my finder prior to performing the aggregation:
await MyEntity.unscoped().findAll({
attributes: [[Sequelize.fn('max', Sequelize.col('created_at')), 'max']],
group: ['group_name']
})
It might be late now. But it might help someone,
I got the same by on sequelize v5
myModel.findAll({
attributes: [
[Sequelize.fn('max', Sequelize.col('date')), 'max'],
'name'
],
group: ['name']
})
Related
I am using Sequelize query() method as follows:
const sequelize = new Sequelize(...);
...
// IMPORTANT: No changed allowed on this query
const queryFromUser = "SELECT table1.colname, table2.colname FROM table1 JOIN table2 ON/*...*/";
const result = await sequelize.query(queryFromUser);
Because I am selecting two columns with identical names (colname), in the result, I am getting something like:
[{ "colname": "val1" }, { "colname": "val2" }...], and this array contains values only from the column table2.colname, as it is overwriting the table1.colname values.
I know that there is an option to use aliases in the SQL query with AS, but I don't have control over this query.
I think it would solve the issue, if there was a way to return the result as a 2D array, instead of the array of objects? Are there any ways to configure the Sequelize query that way?
Im afraid this will not be possible without changes in the library directly connecting to the database and parsing its response.
The reason is:
database returns BOTH values
then in javascript, there is mapping of received rows values to objects
This mapping would looks something like that
// RETURNED VALUE FROM DB: row1 -> fieldName:value&fieldName:value2
// and then javascript code for parsing values from database would look similar to that:
const row = {};
row.fieldName = value;
row.fieldName = value2;
return row;
As you see - unless you change the inner mechanism in the libraries, its impossible to change this (javascript object) behaviour.
UNLESS You are using mysql... If you are using mysql, you might use this https://github.com/mysqljs/mysql#joins-with-overlapping-column-names but there is one catch... Sequelize is not supporting this option, and because of that, you would be forced to maintain usage of both libraries at ones (and both connected)
Behind this line, is older answer (before ,,no change in query'' was added)
Because you use direct sql query (not build by sequelize, but written by hand) you need to alias the columns properly.
So as you saw, one the the colname would be overwritten by the other.
SELECT table1.colname, table2.colname FROM table1 JOIN table2 ON/*...*/
But if you alias then, then that collision will not occur
SELECT table1.colname as colName1, table2.colname as colName2 FROM table1 JOIN table2 ON/*...*/
and you will end up with rows like: {colName1: ..., colName2: ...}
If you use sequelize build in query builder with models - sequelize would alias everything and then return everything with names you wanted.
PS: Here is a link for some basics about aliasing in sql, as you may aliast more than just a column names https://www.w3schools.com/sql/sql_alias.asp
In my case I was using:
const newVal = await sequelize.query(query, {
replacements: [null],
type: QueryTypes.SELECT,
})
I removed type: QueryTypes.SELECT, and it worked fine for me.
I am looking for date range query in QBE. I am trying below query sample
https://Server:port/v1/qbe?format=json&pageLength=10&start=1&directory=/json/&options=search_option_advanced_date&query={"$query":{
"$and":[{"creation_date":{"$le":"2018-12-12T05:40:47.496"}},{"creation_date":{"$ge":"2017-12-12T05:40:47.496"}}],"$filtered":true}}
here i have created path range index on '/Creation_Date_date' & element range index on 'Creation_Date_date' also added below constraint in search_option_advanced_date file in persistent options
<constraint name="creation_date">
<range type="xs:date">
<element name="Creation_Date_date"/>
</range>
</constraint>
Though the results should get fetched i am getting below blanks
{
"snippet-format":"snippet",
"total":0,
"start":1,
"page-length": 10,
"selected": "include-with-ancestors",
"results":[],
"facets": {
"EntityType":{"type":"xs:string", "facetValues":[]},
"Category":{"type":"xs:string", "facetValues":[]},
"Genre":{"type":"xs:string", "facetValues":[]},
"creation_date":{"type":"xs:date", "facetValues":[]}
},
"metrics": {
"query-resolution-time":"PT0.016599S",
"facet-resolution-time":"PT0.000578S",
"extract-resolution-time":null, "total-time":"PT0.017743S"
}
}
To use persistent query options, the QBE query must use the constraint property to specify the query options:
http://docs.marklogic.com/guide/search-dev/qbe#id_32338
That said, if you're using query options, it's usually more straightforward to use a combined query:
http://docs.marklogic.com/guide/rest-dev/search#id_69918
Finally, the constraint is typed as a xs:date value but the query supplies xs:dateTime values.
Hoping that helps,
So I have two Sequelize models with this relationship:
models.Note.belongsToMany(models.Topic, {
through: 'noteTopicRelation'
});
models.Topic.belongsToMany(models.Note, {
through: 'noteTopicRelation'
});
I can make a successful query to the Note model like so, getting all the Notes that belong to the Topic with the id of 2:
models.Note.findAll({
include: [{
model: models.Topic,
through: 'noteTopicRelation',
}]
where: {
'$topics.id$': 2
}
})
However, what if I only want a Note that has multiple specific Topics associated with it (i.e. a Note that is associated with Topics of ids 1, 4, 6)?
I have tried adding this operator on my where:
where: {
'$topics.id$': {$overlap: [1, 4, 6]}
}
But getting an error:
operator does not exist: uuid && text[]
Am I using Op.overlap incorrectly? Is there another way to achieve this result? Thank you!
EDIT: and just to clarify (sorry if this wasn't clear in my original post), I want to get the notes that are strictly associated with all of those Topics. Performing a '$topics.id$': [1, 4, 6] will get me notes that are associated with any of those Topics.
I think you want $in rather than $overlap; the latter maps to the PostgreSQL && operator which is meant for range types, not lists.
So I suggest trying:
where: {
'$topics.id$': [1, 4, 6]
}
The above will get notes which have ANY of the topic IDs (posted before the question was edited to say that only notes with ALL of the provided topic.ids should be returned).
As per the link to the discussion on the Sequelize github issues page in the comments below; one way to get notes with ALL of the topic IDs, you'll need to do something like the following:
var topicIds = [1, 4, 6];
models.NoteTopicRelation
.findAll({
include: [
{model: models.Topic, where: {id: topicIds}}
],
group: ['note_id'],
having: ['COUNT(*) >= ?', topicIds.length]
})
.then((noteTopicItems) => models.Note.find({
where: {id: noteTopicItems.map((item) => item.note_id)}
}))
.then((notes) => {
// do something with notes
});
Note that this method only reliably works if the link table (ie noteTopicRelation in your case) has only unique pairs of note_id & topic_id - ie. there is a unique key of some sort on these fields. Otherwise a topic can be assigned to a note more than once, which will throw up the COUNT(*). I believe the default "through" tables that Sequelize creates have a unique key on both fields; there are legitimate cases where this might not be desired however so I thought it worth mentioning.
Also note that I've made some assumptions about the column/property names of your noteTopicRelation model in the above query so you'll probably need to tweak them.
Another thing to note - the join from NoteTopicRelation to Topic isn't really necessary in the example case; you could achieve the same thing more efficiently using where: {topic_id: topicIds} (which would avoid the join to Topic) if you are only wanting to filter by topic.id. I've left the join there in case you're actually wanting to query on e.g. topic name or include other attributes from the Topic model/table in your where clause (e.g. an enabled attribute or similar).
Let's say you have a queues table, agent_queues, and agents table. An agent can be in many queues, a queue can have many agents. Now let's say you're trying to get a list of queues and the number of agents in those queues. I would expect something like the following to work:
queues.findAll({
include: ['agentQueues'],
group: ['queues.name', 'queues.matcher', 'queues.id'],
attributes: [[Sequelize.fn('count', Sequelize.col('agentQueues.id')), 'agentCount']]
})
Instead it produces something like:
SELECT "queues".*
FROM (SELECT
"queues"."name",
"queues"."matcher",
"queues"."id",
count("agentQueues"."queueId") AS "agentCount"
FROM "queues" AS "queues"
GROUP BY "name", "matcher", "id"
) AS "queues" LEFT OUTER JOIN "agent_queues" AS "agentQueues" ON "queues"."id" = "agentQueues"."queueId";
Where both the group by and count are in the subquery as opposed to the main query. What am I doing wrong here?
An ideal query would look something like this:
SELECT name, matcher, count(agent_queues."queueId") as agentCount FROM queues
LEFT OUTER JOIN agent_queues ON "agent_queues"."queueId" = queues.id
GROUP BY name, matcher;
The result I'm looking for is something like this:
[{ name: 'Some Queue', matcher: '1 = 1', agentCount: 2 }]
It is a bit old question but this problem is also with new versions of sequelize. My best workaround was using a literal instead of fn and count functions.
This code should generate a sql statement you expect.
queues.findAll({
group: ['queues.name', 'queues.matcher'],
attributes: ['name', 'matcher',[literal(`(SELECT count(*) FROM "agentQueues"
WHERE queue."id" = "agentQueues"."queueId"])`, 'agentCount')]]
})
I have a problem when querying mongoDB with nested objects notation:
db.messages.find( { headers : { From: "reservations#marriott.com" } } ).count()
0
db.messages.find( { 'headers.From': "reservations#marriott.com" } ).count()
5
I can't see what I am doing wrong. I am expecting nested object notation to return the same result as the dot notation query. Where am I wrong?
db.messages.find( { headers : { From: "reservations#marriott.com" } } )
This queries for documents where headers equals { From: ... }, i.e. contains no other fields.
db.messages.find( { 'headers.From': "reservations#marriott.com" } )
This only looks at the headers.From field, not affected by other fields contained in, or missing from, headers.
Dot-notation docs
Since there is a lot of confusion about queries MongoDB collection with sub-documents, I thought its worth to explain the above answers with examples:
First I have inserted only two objects in the collection namely: message as:
> db.messages.find().pretty()
{
"_id" : ObjectId("5cce8e417d2e7b3fe9c93c32"),
"headers" : {
"From" : "reservations#marriott.com"
}
}
{
"_id" : ObjectId("5cce8eb97d2e7b3fe9c93c33"),
"headers" : {
"From" : "reservations#marriott.com",
"To" : "kprasad.iitd#gmail.com"
}
}
>
So what is the result of query: db.messages.find({headers: {From: "reservations#marriott.com"} }).count()
It should be one because these queries for documents where headers equal to the object {From: "reservations#marriott.com"}, only i.e. contains no other fields or we should specify the entire sub-document as the value of a field.
So as per the answer from #Edmondo1984
Equality matches within sub-documents select documents if the subdocument matches exactly the specified sub-document, including the field order.
From the above statements, what is the below query result should be?
> db.messages.find({headers: {To: "kprasad.iitd#gmail.com", From: "reservations#marriott.com"} }).count()
0
And what if we will change the order of From and To i.e same as sub-documents of second documents?
> db.messages.find({headers: {From: "reservations#marriott.com", To: "kprasad.iitd#gmail.com"} }).count()
1
so, it matches exactly the specified sub-document, including the field order.
For using dot operator, I think it is very clear for every one. Let's see the result of below query:
> db.messages.find( { 'headers.From': "reservations#marriott.com" } ).count()
2
I hope these explanations with the above example will make someone more clarity on find query with sub-documents.
The two query mechanism work in different ways, as suggested in the docs at the section Subdocuments:
When the field holds an embedded document (i.e, subdocument), you can either specify the entire subdocument as the value of a field, or “reach into” the subdocument using dot notation, to specify values for individual fields in the subdocument:
Equality matches within subdocuments select documents if the subdocument matches exactly the specified subdocument, including the field order.
In the following example, the query matches all documents where the value of the field producer is a subdocument that contains only the field company with the value 'ABC123' and the field address with the value '123 Street', in the exact order:
db.inventory.find( {
producer: {
company: 'ABC123',
address: '123 Street'
}
});