I'm trying to insert multiple JSON data into a table called integration_data using the TypeORM. I'm using NodeJS for my app. But somehow, I'm not able to form the query using TypeORM to perform bulk UpSert of my JSON data in Postgres SQL.
Following is my logic:-
let dbConn = await database.getConnection(connectionOptions);
await dbConn.createQueryBuilder()
.insert()
.into(IntegrationData)
.values(rawdata)
.onConflict(`("id") DO UPDATE SET "empData" = excluded."empData"`)
.execute();
Here, I'm passing rawData which is an array of JSON objects.
Entity:-
import { Entity, PrimaryGeneratedColumn, Column } from "typeorm";
#Entity({ name: 'integration_data' })
export class IntegrationData {
#PrimaryGeneratedColumn()
id: number;
#Column({ type: 'bigint', nullable: false, name: "emp_id" })
empId: number;
#Column({ type: 'text', nullable: false, name: "integration_name" })
integrationName: string;
#Column({ type: 'json', nullable: false, name: "emp_data" })
empData: object
}
Here emp_data column in the integration_data table is of JSON. Following is my table on Postgres:-
May I know how can I perform an upsert operation i.e. insert if not exists else update the record in Postgres database based on emp_id which is going to be unique.
Cheers
You can use TypeORMs Repository.save() method to do upserts if the column is set as a primary key. You can simply pass an array of the objects into the method.
From the documentation:
/**
* Saves all given entities in the database.
* If entities do not exist in the database then inserts, otherwise updates.
*/
Related
I am building an education application and I am trying to add/update a field which is an array of objects with addToSet from a javascript array, and if the object already exists (matched with objectId) I want to update the already existing object's array (addToSet) and change another field of that same object.
My model looks like this (simplified):
const course = new Schema(
{
events: [
{
type: Schema.Types.ObjectId,
ref: 'event'
}
],
students: [
{
user: {
type: Schema.Types.ObjectId,
ref: 'user'
},
status: {
type: String,
enum: ['notBooked', 'booked', 'attended', 'completed']
},
events: [
{
type: Schema.Types.ObjectId,
ref: 'event'
}
]
}
],
});
And ideally I would like to use an updateOne query to both addToSet to the course's list of events, while also updating the students list.
Currently I am using this code to accomplish my updates by first finding the course and then using javascript to update it, which works:
const studentsToAdd = this.attendees.map((attendee) => ({
user: attendee._id,
status: 'booked',
events: [this._id]
}));
const studentIds = studentsToAdd.map((student) => student.user);
const course = await courseModel.findById(this.course);
console.log(studentIds);
course.events.push(this._id);
course.students.forEach((student) => {
if (studentIds.some((s) => s.equals(student.user))) {
student.events.push(this._id);
student.status = 'booked';
studentsToAdd.splice(studentsToAdd.indexOf(studentsToAdd.find((s) => s.user.equals(student.user))), 1);
}
});
course.students.push(...studentsToAdd);
course.save();
However I am curious if it is possible to solve this using a single updateOne on the courseModel schema.
This is part of a mongoose middleware function, and "this" references an event object, which has its own _id, attendees of the event, and the course ID (named course).
The goal is to add the student object part of studentsToAdd to the students array of the course IF the student does not exist (signified by user being a reference by objectId), and if the student already exists then I want to add the eventId (this._id) to the events array for that particular student and set status for that student to "booked".
Is this possible? I have tried many iterations using $cond, $elemmatch, "students.$[]" and so forth but I am quite new to mongodb and am unsure how to go about this.
how can I select everything from a table with prisma? (SELECT * FROM application)
const applications = prisma.application.findMany({
// Returns all user fields
include: {
posts: {
select: {
age: true,
about_section: true,
user_id: true
},
},
},
})
console.log(applications.age)
Here is how my schema looks:
model application {
application_id Int #id #default(autoincrement())
age String? #db.VarChar(255)
about_section String? #db.VarChar(255)
user_id Int?
users users? #relation(fields: [user_id], references: [user_id], onDelete: Restrict, onUpdate: Restrict, map: "application_ibfk_1")
##index([user_id], map: "user_id")
}
For rows, findMany() without where will return all rows in table:
Get all Records
The following findMany query returns all User records:
const users = await prisma.user.findMany()
For columns/fields, without specifying include or select, prisma has a default:
By default, when a query returns records (as opposed to a count), the
result includes the default selection set:
All scalar fields defined in the Prisma schema (including enums)
None of the relations
For SELECT * FROM application (does not include users relation):
const applications = await prisma.application.findMany();
To futher include users relation in the model:
const applications = await prisma.application.findMany({
include: {
users: true // will include all fields
}
});
Here is the code below for one of my tables
const { Model, DataTypes } = require('sequelize');
const Algorithm = require('./algorithm');
const Exchange = require('./exchange');
const User = require('./user');
//#JA - This model defines the api keys for each user's exchange
//#JA - For security reasons in case the database gets hacked the keys will be stored using encryption.
module.exports = function(sequelize){
class AlgorithmRule extends Model {}
AlgorithmModel = Algorithm(sequelize);//#JA - Gets a initialized version of Algorithm class
ExchangeModel = Exchange(sequelize);//#JA - Gets initialized version of the Exchange class
UserModel = User(sequelize);//#JA - Gets a initialized version of User class
var AlgorithmRuleFrame = AlgorithmRule.init({
algorithm_id: {
type: DataTypes.INTEGER,
allowNull: false,
references: {
model: AlgorithmModel,
key: 'id',
}
},
exchange_id: {
type: DataTypes.STRING,
references: {
model: ExchangeModel,
key: 'name',
},
},
user_id: {
type: DataTypes.INTEGER,
references: {
model: UserModel,
key: 'id',
},
},
type : { //Partial-Canceled implies that the order was partially filled and then canceled.
type: DataTypes.ENUM('Percent Of Equity','Cash'),
allowNull: false,
defaultValue: 'Percent Of Equity'
},
type_value: { //#JA - This will be either treated as a percentage or 'cash' value for the type chosen for the algorithm.
type: DataTypes.DECIMAL(20,18),
allowNull: false
},
}, {
sequelize,
modelName: 'AlgorithmRule',
indexes: [{ unique: true, fields: ['algorithm_id','exchange_id','user_id'] }]
});
return AlgorithmRuleFrame
};
I'm trying to set this up so that I can allownull:false on algorithm_id and exchange_id and user_id. I want it so there HAS to be values there for any records to be allowed.
I can't even get allowNull:false manually through the database itself. So my first question is, is this even possible?
If it is, how do I do it with sequelize?
I can use the typical hasOne() with foreign key commands because then I can't create a composite unique of the foreign keys. The only way I was able to do this was the way I did using the references: json structure.
How do I allownull:false for a foreignKey reference defined the way I have it?
To be clear something like this will NOT work
Task.belongsTo(User, { foreignKey: { allowNull: false }, onDelete: 'CASCADE' })
This will NOT work because I'm using a composite unique key across 3 foreign keys and in order to do that I need reference to it's name and that is not possible unless it's defined on the table before these commands above our input. Hopefully this makes sense.
Any help is greatly appreciated!
Okay so apparently
algorithm_id: {
type: DataTypes.INTEGER,
allowNull: false,
references: {
model: AlgorithmModel,
key: 'id',
}
},
This code is correct. HOWEVER, if you created the database already and the foreign key was already defined it will NOT change the allowNull via the alter command. You have to COMPLETELY drop the table and THEN it will allow the allowNull:false attribute to work.
This threw me for a loop for a long time, so I help this saves someone else a lot of frustration.
const sequelize = new Sequelize('db', 'userName', 'password',{
dialect: 'postgres',
define: {
underscored: true,
},
query: {
raw: true
}
});
I am using underscored field values for the database and camelCase field values on application level. I am getting the data in correct format using sequelize querying methods but it is causing a problem when I use Raw Sequelize queries with sequelize.query() because it returns underscored field name. Is there any way to change this behavior?
It works.
class TheModel extends Sequelize.Model {
}
TheModel.init({
userId: {type: DataTypes.INTEGER},
label: {type: DataTypes.STRING},
}, {
sequelize: sequelize,
underscored: true,
}
...
TheModel.rawAttributes['userId'].field // 'user_id'
I have a model as below:
const Sequelize = require("sequelize");
const sequelize = new Sequelize({
database: process.env.PGDATABASE,
username: process.env.PGUSER,
password: process.env.PGPASS,
host: process.env.PGHOST,
port: process.env.PGPORT,
dialect: "postgres"
});
var MessageRating = sequelize.define(
"MessageRating",
{
chatId: {
type: Sequelize.STRING,
primaryKey: true,
allowNull: false
},
ratingType: {
type: Sequelize.ENUM,
values: ["NEGATIVE", "POSITIVE", "NEUTRAL", "MIXED"],
allowNull: false
},
rating: {
type: Sequelize.FLOAT,
allowNull: false
}
},
{
indexes: [
{
unique: false,
fields: ["ratingType"]
}
]
}
);
module.exports = MessageRating;
I'm trying to create a table using this model using the following code :
var messageRatingStored = sequelize
.sync()
.then(() =>
MessageRating.create({
chatId: chatId,
ratingType: amazonComprehendResult.ResultList[0].Sentiment,
rating: roundOffRating
})
)
.then(messageRating => {
console.log("MessageRating json", messageRating.toJSON());
});
Whenever I run the code it throws error: "Unhandled rejection SequelizeDatabaseError: invalid input syntax for integer: "0.82""
https://gist.github.com/BikalNepal/51fd28f877b422fcc16825d61cd95847
I tried using a static integer value (82, 10 etc) instead of "roundOffRating" (which gives decimal) and it stores perfectly without errors. Since I'm not using Integer for the type of rating in the model, why is it still expecting integer?
You updated your Sequelize model, but have you updated the database table itself ?
Using sequelize.sync() only creates missing tables. It does not update existing ones. Using sequelize.sync({force: true}) will drop all tables and recreate them, meaning your field will be effectively a float, but you will lose all data present in the database.
You can update your field in the database, without losing data, with the following alter statement:
ALTER TABLE MessageRating ALTER COLUMN rating TYPE float;
Read more about ALTER TABLE.
A useful tip: Sequelize propagates errors from the database as SequelizeDatabaseError errors. When you see this type of error coming from your code, you can be sure the error comes from the database and not sequelize itself.