Develop Reference

how to use .populate() in javascript

I have a collection called
CR
what I want to do is query CR like so:
let cre = await CR.find({myid: "xxxxxx"})
and i also have a collection called cla, but that one i need to query based off of the results of cre. The results of cre will return a class_id, in which I need to use to query the cla collection to find the _id. At the end of all of this, I want to ideally merge the two, which I believe you can do through .populate(), and then send it to teh front-end as one.;
I have tried this:
let cre = await cr.find({myid: "xxx"}).populate('loc').populate('desc').populate('dt');
but this isn't working. how can I fix this?

It may be due to schemas, but this is how it's clean and simple to use;
let cre = await cr.find({myid: "xxx"}).populate(['loc','desc','dt']);

Firstly, you can take cla collection "_id" in CR collection schema. In schema of CR collection refer to cla model id like this,
const creSchema = mongoose.Schema({
name: String,
classnId: { type: mongoose.Types.ObjectId, ref: "Cla" }
});
Then you can populate like,
const cres = await CR.find({}).populate({path:'classnId', select:'columnName'});
Hopefully, this will solve your issue.
Note: There in populating you can give multiple column names by space and if you give a minus before a column name like this (-columnName) then that column will not show when you will call the API.

Sequelize how to return result as a 2D array instead of array of objects?

I am using Sequelize query() method as follows:
const sequelize = new Sequelize(...);
...
// IMPORTANT: No changed allowed on this query
const queryFromUser = "SELECT table1.colname, table2.colname FROM table1 JOIN table2 ON/*...*/";
const result = await sequelize.query(queryFromUser);
Because I am selecting two columns with identical names (colname), in the result, I am getting something like:
[{ "colname": "val1" }, { "colname": "val2" }...], and this array contains values only from the column table2.colname, as it is overwriting the table1.colname values.
I know that there is an option to use aliases in the SQL query with AS, but I don't have control over this query.
I think it would solve the issue, if there was a way to return the result as a 2D array, instead of the array of objects? Are there any ways to configure the Sequelize query that way?

Im afraid this will not be possible without changes in the library directly connecting to the database and parsing its response.
The reason is:
database returns BOTH values
then in javascript, there is mapping of received rows values to objects
This mapping would looks something like that
// RETURNED VALUE FROM DB: row1 -> fieldName:value&fieldName:value2
// and then javascript code for parsing values from database would look similar to that:
const row = {};
row.fieldName = value;
row.fieldName = value2;
return row;
As you see - unless you change the inner mechanism in the libraries, its impossible to change this (javascript object) behaviour.
UNLESS You are using mysql... If you are using mysql, you might use this https://github.com/mysqljs/mysql#joins-with-overlapping-column-names but there is one catch... Sequelize is not supporting this option, and because of that, you would be forced to maintain usage of both libraries at ones (and both connected)
Behind this line, is older answer (before ,,no change in query'' was added)
Because you use direct sql query (not build by sequelize, but written by hand) you need to alias the columns properly.
So as you saw, one the the colname would be overwritten by the other.
SELECT table1.colname, table2.colname FROM table1 JOIN table2 ON/*...*/
But if you alias then, then that collision will not occur
SELECT table1.colname as colName1, table2.colname as colName2 FROM table1 JOIN table2 ON/*...*/
and you will end up with rows like: {colName1: ..., colName2: ...}
If you use sequelize build in query builder with models - sequelize would alias everything and then return everything with names you wanted.
PS: Here is a link for some basics about aliasing in sql, as you may aliast more than just a column names https://www.w3schools.com/sql/sql_alias.asp

In my case I was using:
const newVal = await sequelize.query(query, {
replacements: [null],
type: QueryTypes.SELECT,
})
I removed type: QueryTypes.SELECT, and it worked fine for me.

Sequelize - Get max date for a group of objects

I want to get the max (most recent) created at date of each group of objects from a table using sequelize.
My entity roughly follows this interface:
class MyEntity {
id,
groupName,
createdAt
}
I want to do something like this:
await MyEntity.findAll({
attributes: [[Sequelize.fn('max', Sequelize.col('created_at')), 'max']],
group: ['group_name']
})
I would interpret this as:
1. For all entites
2. Group by "groupName"
3. And get the max value for each group
However, I get the following error:
SequelizeDatabaseError: column "Template.created_at" must appear in the GROUP BY clause or be used in an aggregate function
I definitely do not want to group by the created_at column as well, as that would be meaningless.
The SQL for this operation seems pretty basic:
SELECT groupName, MAX(createdAt) FROM [MyEntity]
GROUP BY groupName

The issue was caused because I had a default scope applied to the model, that was ordering the results on the created_at column. So, because created_at was not a attribute in my result set, I was unable to sort based on that column and got the error.
I had to unscope my finder prior to performing the aggregation:
await MyEntity.unscoped().findAll({
attributes: [[Sequelize.fn('max', Sequelize.col('created_at')), 'max']],
group: ['group_name']
})

It might be late now. But it might help someone,
I got the same by on sequelize v5
myModel.findAll({
attributes: [
[Sequelize.fn('max', Sequelize.col('date')), 'max'],
'name'
],
group: ['name']
})

How to merge multiple dynamic queries

I'm using node to communicate with my postgresql database.
I have a list of cars:
[{id:1, areaId: 1},
{id:2, areaId: 2},
{id:3, areaId: 2}]
areaId is a column in another table (say areas), and I want to fetch the area name for each car:
select name from areas
where id = areaId
How can I build a list containing all the cars with the area name attached?
The naive approach is to query the database for each object:
const query = `select name from areas
where id = $1`;
return pg.query(query, car.areaId);
But it seems like a lot of queries to be executed.

You should be able to select multiples Id's at the same time as this.
SELECT name
FROM areas
where ID in (5263, 5625, 5628, 5621)

Yes, you can specify multiple car.ids and fetch car.name and area.name and return them together.
Further you can sort the response by car.id using ORDER BY. I am not aware of any way to enforce the ordering based on the order of the passed in IDS, but you could easily sort the results in memory very quickly with JavaScript.
There are multiple ways to do this in SQL: inner join, full join, subselect...maybe more. I'll demonstrate the "inner join" as it's usually the most efficient, and some SQL query planners will convert subselects to inner joins anyway.
You haven't specified, but assuming you're using node-postgres...here's an answer when carIds is already defined, as an array:
Inner join
const query = `SELECT car.id, car.name, areas.name FROM car INNER JOIN area ON car.areaId WHERE car.id = ANY (${carIds}) ORDER BY car.id`
Finally:
return pq.query(query, carIds)

Querying a Sequelize model to match multiple associations

So I have two Sequelize models with this relationship:
models.Note.belongsToMany(models.Topic, {
through: 'noteTopicRelation'
});
models.Topic.belongsToMany(models.Note, {
through: 'noteTopicRelation'
});
I can make a successful query to the Note model like so, getting all the Notes that belong to the Topic with the id of 2:
models.Note.findAll({
include: [{
model: models.Topic,
through: 'noteTopicRelation',
}]
where: {
'$topics.id$': 2
}
})
However, what if I only want a Note that has multiple specific Topics associated with it (i.e. a Note that is associated with Topics of ids 1, 4, 6)?
I have tried adding this operator on my where:
where: {
'$topics.id$': {$overlap: [1, 4, 6]}
}
But getting an error:
operator does not exist: uuid && text[]
Am I using Op.overlap incorrectly? Is there another way to achieve this result? Thank you!
EDIT: and just to clarify (sorry if this wasn't clear in my original post), I want to get the notes that are strictly associated with all of those Topics. Performing a '$topics.id$': [1, 4, 6] will get me notes that are associated with any of those Topics.

I think you want $in rather than $overlap; the latter maps to the PostgreSQL && operator which is meant for range types, not lists.
So I suggest trying:
where: {
'$topics.id$': [1, 4, 6]
}
The above will get notes which have ANY of the topic IDs (posted before the question was edited to say that only notes with ALL of the provided topic.ids should be returned).
As per the link to the discussion on the Sequelize github issues page in the comments below; one way to get notes with ALL of the topic IDs, you'll need to do something like the following:
var topicIds = [1, 4, 6];
models.NoteTopicRelation
.findAll({
include: [
{model: models.Topic, where: {id: topicIds}}
],
group: ['note_id'],
having: ['COUNT(*) >= ?', topicIds.length]
})
.then((noteTopicItems) => models.Note.find({
where: {id: noteTopicItems.map((item) => item.note_id)}
}))
.then((notes) => {
// do something with notes
});
Note that this method only reliably works if the link table (ie noteTopicRelation in your case) has only unique pairs of note_id & topic_id - ie. there is a unique key of some sort on these fields. Otherwise a topic can be assigned to a note more than once, which will throw up the COUNT(*). I believe the default "through" tables that Sequelize creates have a unique key on both fields; there are legitimate cases where this might not be desired however so I thought it worth mentioning.
Also note that I've made some assumptions about the column/property names of your noteTopicRelation model in the above query so you'll probably need to tweak them.
Another thing to note - the join from NoteTopicRelation to Topic isn't really necessary in the example case; you could achieve the same thing more efficiently using where: {topic_id: topicIds} (which would avoid the join to Topic) if you are only wanting to filter by topic.id. I've left the join there in case you're actually wanting to query on e.g. topic name or include other attributes from the Topic model/table in your where clause (e.g. an enabled attribute or similar).