I have a User model that includes a json data type. I'm using MySQL as my database, but when looking through their documentation for how to query using JSON, there appears to be nothing on MySQL.
const User = db.define("User", {
UserId: {
primaryKey: true,
type: Sequelize.INTEGER
},
colors: {
type: Sequelize.JSON,
allowNull: false
}
}
I have Users that have an id, and an array with their favorite colors in it.
UserId | colors
1 |["black, "blue"]
2 |["blue"]
3 |["black"]
I want to query for all users that have the favorite color blue. I tried querying like this,
User.findAll({ where: {colors: ["blue"]} })
and expected to be returned users 1 and 2, but this doesn't work and I have not found any sources online that shows what to do. Some stackoverflows have solutions for JSON objects, like this Sequelize, MySQL - Filtering rows in table with JSON column values, but I can't find any sources for arrays. Any solutions?
You can use sequelize.fn and sequelize.col methods to specify an SQL function call and a table column, respectively. This way you can create your own where condition.
User.findAll({
where: sequelize.where(sequelize.fn('JSON_CONTAINS', sequelize.col('colors'), sequelize.literal('blue'), sequelize.literal('$')), 1)
});
It corresponds to below mysql query:
// SELECT * FROM Users WHERE JSON_CONTAINS(colors, 'blue', '$') = 1;
The answer should be:
User.findAll({
where: sequelize.where(sequelize.fn('JSON_CONTAINS', sequelize.literal('colors'), '"blue"', '$'), 1)
});
Note:
The 2nd parameter of JSON_CONTAINS should be string.
Because the data of question is an array of string, so we need to add a " for searching keyword (e.g: "blue").
Related
I have a Cheques and a Payees collection, every cheque has its corresponding Payee ID.
What I'm trying to do is to write some queries on cheques, but I need to preform the searching after populating the payee (to get the name)
const search = req.query.search || "";
const cheques = await Cheque
.find({
isCancelled: false,
dueDate: { $gte: sinceDate, $lte: tillDate }
})
.select("_id serial dueDate value payee")
.skip(page * limit)
.limit(limit)
.sort({ dueDate: -1, serial: 1 })
.populate({
path: "payee",
select: "name"
})
I guess what I'm trying do is fit this somewhere in my code,
match: {
name: { $regex: search, $options: "i" }
},
I have tried to put the match within the populate, but then it will still find all cheques even if they don't satisfy the population match but populate as null.
I hate this answer and I hope someone is going to post a better one, but I've surfed the web for that with no luck.
The only method I was able to find is to use the $lookup method in aggregation.
So you'll have to change your code from calling .find() to .aggregate().
It's not the sad news, it's great, stable and no problems at all.
but I hated that because it's going to change some patterns you might be following in your code.
const search = req.query.search || "";
const cheques = await Cheque
.aggregate([
{
$lookup: { // similar to .populate() in mongoose
from: 'payees', // the other collection name
localField: 'payee', // the field referencing the other collection in the curent collection
foreignField: '_id', // the name of the column where the cell in the current collection can be found in the other collection
as: 'payee' // the field you want to place the db response in. this will overwrite payee id with the actual document in the response (it only writes to the response, not on the database, no worries)
},
{ // this is where you'll place your filter object you used to place inside .find()
$match: {
isCancelled: false,
dueDate: { $gte: sinceDate, $lte: tillDate }
'payee.branch': 'YOUR_FILTER', // this is how you access the actual object from the other collection after population, using the dot notation but inside a string.
}
},
{ // this is similar to .select()
$project: {_id: 1, serial: 1, dueDate: 1, value: 1, payee: 1}
},
{
$unwind: '$payee' // this picks the only object in the field payee: [ { payeeDoc } ] --> { payeeDoc }
}
])
.skip(page * limit)
.limit(limit)
.sort({ dueDate: -1, serial: 1 })
Notice how you can no longer chain .select() and .populate() on the model query the way you used to do it on .find(), because now you're using .aggregate() which returns a different class instance in mongoose.
you can call .projcet() instead of doing it inside the aggregation array if you want to, but as far as I know, you can't use the .select() method.
My opinion-based solution to this problem is to include the payee information you need for filtering in the Cheque collection.
In my senario, this happen when I was filtering for the sake of my users-roles and permissions, so someone can not see what another one is seeing.
It's up to you, but this makes it easier later when you want to generate the reports (I assume you're working on a payment service).
The populate() feature provided by mongoose first fetches all the Cheques with given conditions and then makes another query to payee with _ids to populate the fields you wanted.
https://mongoosejs.com/docs/api.html#query_Query-populate
by putting match in populate you're filtering which cheques need to be populated but not the cheques themselves.
A simple solution for this is to filter the cheques which are populated as null and return them for your use.
If you see more queries of this sort and/or the collection is huge, it's better you add the payee name in the Cheques collection itself if that fits your purpose.
I am trying to uniquely store data for each server in a database, I want to be able to store multiple of the same data object. I am only able to store one because if I attempt to store another one it just replaces it, which is the issue here.
Schema
const schema = new mongoose.Schema({
reactionRole: {
type: Object,
required: false,
}
});
Screenshot of the Data
Trying to Accomplish
I want to be able to store the same object here but with different data obviously, would I have to make the schema take a Array and just insert it? I am not really sure how to work around this, thanks!
Example
This is how I want the data to be, the option to be able to add more onto the document instead of replacing it. Should I use an array or what's a solution?
reactionRole: {
<Role.name> config 1: {
Enabled: true,
Added_By: id,
MessageID: msg.id,
Emoji: <emoji>,
Role: id,
},
<Role.name> config 2: {
Enabled: true,
Added_By: id,
MessageID: msg.id,
Emoji: <emoji>,
Role: id,
}
}
I resolved this. I changed the schema to an array and sent the data as objects.
This question is more of a theoretical one, since I have not started implementation yet.
The situation is as follows:
I have an application in which users can upload structured data (like Excel, CSV etc.).
Due to requirements, I want to store them in the database, ideally creating a new table on the fly with the table name set to the file name and columns based on the file itself.
This initialisation is still doable with sequelize, I think.
However, as sequelize relies on models, then I am stuck as I am not sure what type and how many columns there will be, thus creating the need for something off 'Dynamic model' or 'Generic model'.
I am not sure how to do this, and I cannot find anything related when searching. I would appreciate your 2 cents on this approach, and if there are other ideas I am very eager to hear them.
Thanks in advance!
First, you need to add code to map your dynamic columns into Sequelize column definition like:
const columnDefinitions = [
{
field: 'field_name_from_csv',
type: DataTypes.INTEGER,
allowNull: true
},
{
field: 'field_name2_from_csv',
type: DataTypes.INTEGER,
allowNull: true
},
]
So you need to determine the data type of a certain column and create an appropriate column definition.
Second, you need to store all these mapping info in the special table(s) so you know what dynamic tables you have and how to register them in Sequelize.
Once you have a table name and column mappings you can register all tables as models in Sequelize:
// here you need to convert an array of mappings into the object where field names should be keys and column definitions should be values.
const columnMappingsAsObject = ...
const tableName = 'dynamic_table_name'
const dynamicModel = sequelize.define(tableName, columnMappingsAsObject, {
tableName
});
// now you can use it to get records and so on:
const records = await dynamicModel.findAll({})
i have my typeorm column like this, what i want is an array of JSON object which i manage to get.
#Column({
type: 'jsonb',
array: false,
default: () => "'[]'",
nullable: false,
})
tokens!: Array<{ token: string }>;
this is how the field looks, and am fine with it, what i want is to find a document with a particular token, so i came up with the below code, but it returns an empty array.
const user = await User.find({ where: { _id: decoded._id, tokens: { token: token } } });
normally when am using mongooe i can get this working using
const user = await User.findOneBy({_id: decoded._id, "tokens.token": token,}); and this returns a particular user, with the id and token string passed.
i want help on how to find a user using the id and the token string inside in array of object, thanks.
TypeORM does not natively support queries on PostreSQL jsonb columns. Performing a query on the data in a jsonb column would require you to either issue a raw query or write your own WHERE clause in.where or .addWhere of a query builder (doc).
For reference, the jsonb query syntax documentation can be found here.
The structure of the table is:
chats
--> randomId
-->--> participants
-->-->--> 0: 'name1'
-->-->--> 1: 'name2'
-->--> chatItems
etc
What I am trying to do is query the chats table to find all the chats that hold a participant by a passed in username string.
Here is what I have so far:
subscribeChats(username: string) {
return this.af.database.list('chats', {
query: {
orderByChild: 'participants',
equalTo: username, // How to check if participants contain username
}
});
}
Your current data structure is great to look up the participants of a specific chat. It is however not a very good structure for looking up the inverse: the chats that a user participates in.
A few problems here:
you're storing a set as an array
you can only index on fixed paths
Set vs array
A chat can have multiple participants, so you modelled this as an array. But this actually is not the ideal data structure. Likely each participant can only be in the chat once. But by using an array, I could have:
participants: ["puf", "puf"]
That is clearly not what you have in mind, but the data structure allows it. You can try to secure this in code and security rules, but it would be easier if you start with a data structure that implicitly matches your model better.
My rule of thumb: if you find yourself writing array.contains(), you should be using a set.
A set is a structure where each child can be present at most once, so it naturally protects against duplicates. In Firebase you'd model a set as:
participants: {
"puf": true
}
The true here is really just a dummy value: the important thing is that we've moved the name to the key. Now if I'd try to join this chat again, it would be a noop:
participants: {
"puf": true
}
And when you'd join:
participants: {
"john": true,
"puf": true
}
This is the most direct representation of your requirement: a collection that can only contain each participant once.
You can only index known properties
With the above structure, you could query for chats that you are in with:
ref.child("chats").orderByChild("participants/john").equalTo(true)
The problem is that this requires you to define an index on `participants/john":
{
"rules": {
"chats": {
"$chatid": {
"participants": {
".indexOn": ["john", "puf"]
}
}
}
}
}
This will work and perform great. But now each time someone new joins the chat app, you'll need to add another index. That's clearly not a scaleable model. We'll need to change our data structure to allow the query you want.
Invert the index - pull categories up, flattening the tree
Second rule of thumb: model your data to reflect what you show in your app.
Since you are looking to show a list of chat rooms for a user, store the chat rooms for each user:
userChatrooms: {
john: {
chatRoom1: true,
chatRoom2: true
},
puf: {
chatRoom1: true,
chatRoom3: true
}
}
Now you can simply determine your list of chat rooms with:
ref.child("userChatrooms").child("john")
And then loop over the keys to get each room.
You'll like have two relevant lists in your app:
the list of chat rooms for a specific user
the list of participants in a specific chat room
In that case you'll also have both lists in the database.
chatroomUsers
chatroom1
user1: true
user2: true
chatroom2
user1: true
user3: true
userChatrooms
user1:
chatroom1: true
chatroom2: true
user2:
chatroom1: true
user2:
chatroom2: true
I've pulled both lists to the top-level of the tree, since Firebase recommends against nesting data.
Having both lists is completely normal in NoSQL solutions. In the example above we'd refer to userChatrooms as the inverted index of chatroomsUsers.
Cloud Firestore
This is one of the cases where Cloud Firestore has better support for this type of query. Its array-contains operator allows filter documents that have a certain value in an array, while arrayRemove allows you to treat an array as a set. For more on this, see Better Arrays in Cloud Firestore.