Since I have given the pool parameters as mentioned in the below code snippet, Is necessary to close the connection after each lambda invocation?
This aws lambda function is supposed to be triggered every minute, is the pool parameter only enough to close the connection?
export const databaseProviders = [
{
provide: 'vendorDB',
useFactory: async (awsSecrets: AwsSecretsService) => {
const { host, port, username, password, database } =
await awsSecrets.getVendorDbConfig();
const sequelize = new Sequelize({
dialect: 'postgres',
host,
port,
username,
password,
database,
logging: true,
pool: {
max: 1,
min: 0,
idle: 1000,
},
});
sequelize.addModels([VendorEntity]);
// await sequelize.sync();
return sequelize;
},
inject: [AwsSecretsService],
},
Related
I'm making a plugin that sends mail in trapi cms using nodemailer and google's oauth2, however my tokens automatically expire after 1 hour. I am trying several ways but my code still not working. Here is my code:
const nodemailer = require("nodemailer");
const { google } = require("googleapis");
const OAuth2 = google.auth.OAuth2;
const oauth2Client = new OAuth2(
`"${process.env.CLIENT_ID }"`,
`"${process.env.CLIENT_SECRET}"`,
"https://developers.google.com/oauthplayground",
);
// oauth2Client.setCredentials({
// refresh_token: `"${process.env.REFRESH_TOKEN }"`,
// });
// const accessToken = oauth2Client.getAccessToken()
module.exports = ({ env }) => ({
// ...
email: {
provider: env(`${process.env.REACT_APP_EMAIL_PROVIDER}`),
providerOptions: {
host: env("EMAIL_SMTP_HOST", "smtp.gmail.com"),
port: env("EMAIL_SMTP_PORT", 587),
auth: {
type: "OAuth2",
user: "hotrodhv58#gmail.com",
clientId: `"${process.env.CLIENT_ID }"`,
clientSecret: `"${process.env.CLIENT_SECRET}"`,
refreshToken: `"${process.env.REFRESH_TOKEN}"`,
accessToken: `"${process.env.ACCESS_TOKEN}"`,
},
},
settings: {
defaultFrom: env(`${process.env.REACT_APP_EMAIL_ADDRESS_FROM}`),
defaultReplyTo: env(`${process.env.REACT_APP_EMAIL_ADDRESS_REPLY}`),
},
},
// ...
});
The error I received:
Error sending email to xxxxssss#gmail.com {"code":"EAUTH","command":"AUTH XOAUTH2"}
If I leave Async Await in the module, I will receive an error:
{"Errno":-4078, "code": "Esocket", "Syncall": "Connect", "Address":"127.0.0.1", "Port": 587, "Command": "Conn"}}
What do I need to do to make them work. Hope to get help from everyone!
I use Sequelize in my nodejs.app to load and save data.
To manange Sequelize I use:
(Sample model)
const { Sequelize, DataTypes } = require("sequelize");
const sequelize = require("../config/mysql");
const login = sequelize.define(
"login_bot",
{
login: {
type: DataTypes.STRING,
allowNull: false,
},
// Model attributes are defined here
password: {
type: DataTypes.STRING,
allowNull: false,
},
team: {
type: DataTypes.STRING,
allowNull: false,
},
},
{
timestamps: false,
paranoid: true,
underscored: true,
freezeTableName: true,
}
);
module.exports = login;
The config for this is:
const Sequelize = require("sequelize");
const sequelize = new Sequelize("xxx", "xxx", "xxx", {
host: "xxx.bplaced.net",
dialect: "mariadb",
pool: {
max: 25,
min: 5,
idle: 20000,
evict: 15000,
acquire: 30000,
},
});
sequelize
.authenticate()
.then(() => {
console.log("Connection to database has been established successfully.");
})
.catch((err) => {
console.error("Unable to connect to database:", err);
});
module.exports = sequelize;
In my code I use: as sample
const logindata = await login.findAll({
where: { team: usersdata[0].team },
attributes: ["login", "password"],
raw: true,
});
All works fine. But after some time I get "too many connections" from the server. So I want to ask whether the workflow is wring and I miss to close the DBConnection some where?
I have in all 4 models. All models with "..config/mysql.js" as reference. Do Sequelize open for each model a connection and keeps this open? Or just one connection for the whole app?
In all I just want to have one connection my app is using all the time.
Or do I have to close Sequelize somewhere at each call? Hope to get some hints here because iam lost at this moment.
When your application needs to retrieve data from the database, it creates a database connection. Creating this connection involves some overhead of time and machine resources for both your application and the database. Many database libraries and ORM's will try to reuse connections when possible, so that they do not incur the overhead of establishing that DB connection over and over again. The pool is the collection of these saved, reusable connections that, in your case, Sequelize pulls from. Your configuration of
pool: {
max: 5,
min: 0,
idle: 10000
}
reflects that your pool should:
Never have more than five open connections (max: 5)
At a minimum, have zero open connections/maintain no minimum number of connections (min: 0)
Remove a connection from the pool after the connection has been idle (not been used) for 10 seconds (idle: 10000)
I'm experiencing below error after upgrading to Node.js v16.4.0 (from Node.js v12).
I'm using knex to connect to db (postgres) and using ORM - objection.js.
KnexTimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?
Please help to resolve!
import Knex from "knex";
const getConnection = () => {
try {
return Knex({
client: 'pg',
connection: {
user: 'user',
password: 'password',
host: 'dbhost',
port: 5432,
database: 'dbname',
},
pool: {
min: 0,
max: 10,
idleTimeoutMillis: 10000,
acquireTimeoutMillis: 30000,
},
acquireConnectionTimeout: 2000,
});
}
catch (err) {
throw err;
}
}
const getTransaction = () => {
const con = getConnection();
const trx = con.transaction();
return trx;
}
Having had the same issue, I traced the problem to an old version of the "pg" package.
Doing
npm i pg#latest
fixed it
I'm trying to assign a different value to sequelize based on whether the app is running locally or on the server. This is my code:
const sequelize = production
? sequelizeHeroku.connect(Sequelize)
: new Sequelize(database, user, password, {
host,
dialect: "mysql",
pool: {
max: 10,
min: 0,
acquire: 30000,
idle: 10000
}
});
This doesn't work locally, as when I log the value of sequelize it is false. However, if I try to assign the variable like this:
const sequelize = new Sequelize(database, user, password, {
host,
dialect: "mysql",
pool: {
max: 10,
min: 0,
acquire: 30000,
idle: 10000
}
});
it works just fine. I also checked that console.log(production == true) prints out false. What am I doing wrong?
It's better to use NODE_ENV environment variable. E.g.
index.ts:
import { Sequelize } from 'sequelize';
import assert from 'assert';
const database = 'sample';
const user = 'testuser';
const password = 'testpass';
const host = '127.0.0.1';
const sequelize =
process.env.NODE_ENV === 'production'
? 'sequelize heroku'
: new Sequelize(database, user, password, {
host,
dialect: 'mysql',
pool: {
max: 10,
min: 0,
acquire: 30000,
idle: 10000,
},
});
assert(process.env.NODE_ENV === 'production', 'NODE_ENV should be "production"');
assert(sequelize === 'sequelize heroku', 'Should use heroku sequelize in production environment');
Execute code in production environment:
NODE_ENV=production npx ts-node ./index.ts
No assertion failed. It works as expected.
I have a MySQL database with a table named 'posts' I'm reading from via FeathersJS and feathers-sequelize. Currently I have a working prototype with the following code which returns the desired result but only to the console, and also the entire contents of the posts table to the /posts route, however I only want to return a specific set of records from the table to /posts.
The table has a column called post_status, in which a post could be 'published' or 'draft'. I only want to return the 'published' records to /posts, but want to achieve this serverside as opposed to /posts?status=published. How can this be achieved?
See below code:
const path = require('path');
const feathers = require('#feathersjs/feathers');
const express = require('#feathersjs/express');
const socketio = require('#feathersjs/socketio');
const Sequelize = require('sequelize');
const service = require('feathers-sequelize');
const sequelize = new Sequelize('sandbox', 'sandbox', 'secretpassword', {
host: 'localhost',
dialect: 'mysql',
pool: {
max: 5,
min: 0,
acquire: 30000,
idle: 10000
},
operatorsAliases: false
});
const Post = sequelize.define('posts', {
post_title: Sequelize.STRING
},
{
timestamps: false,
underscored: true,
});
// Create an Express compatible Feathers application instance.
const app = express(feathers());
// Turn on JSON parser for REST services
app.use(express.json());
// Turn on URL-encoded parser for REST services
app.use(express.urlencoded({ extended: true }));
// Enable REST services
app.configure(express.rest());
// Enable Socket.io services
app.configure(socketio());
app.use(express.errorHandler());
//This works fine to return to the console but not to /posts
Post.findAll({
where: {
post_status: 'published'
}
}) .then(posts => {
console.log(JSON.stringify(posts));
});
//This returns the entire contents of the posts table to /posts
app.use('/posts', service({
Model: Post,
paginate: {
default: 10,
max: 100
},
}));
// Start the server
const port = 3030;
app.listen(port, () => {
console.log(`Feathers server listening on port ${port}`);
});
I tried the 'where' from the findAll method in the service but this did not change the output, or produce any error.
I resolved this by using a where clause in the find method on the service, see below complete code:
const path = require('path');
const feathers = require('#feathersjs/feathers');
const express = require('#feathersjs/express');
const socketio = require('#feathersjs/socketio');
const Sequelize = require('sequelize');
const service = require('feathers-sequelize');
const sequelize = new Sequelize('sandbox', 'sandbox', 'secretpassword', {
host: 'localhost',
dialect: 'mysql',
pool: {
max: 5,
min: 0,
acquire: 30000,
idle: 10000
},
operatorsAliases: false
});
const Post = sequelize.define('posts', {
post_title: Sequelize.STRING,
post_status: Sequelize.STRING
},
{
timestamps: false,
underscored: true,
});
// Create an Express compatible Feathers application instance.
const app = express(feathers());
// Turn on JSON parser for REST services
app.use(express.json());
// Turn on URL-encoded parser for REST services
app.use(express.urlencoded({ extended: true }));
// Enable REST services
app.configure(express.rest());
// Enable Socket.io services
app.configure(socketio());
app.use(express.errorHandler());
const myService = {
find(params) {
let posts = Post.findAll({
where: {
post_status: 'published'
}
});
return Promise.resolve(posts);
},
get(id, params) {},
create(data, params) {},
update(id, data, params) {},
patch(id, data, params) {},
remove(id, params) {},
setup(app, path) {}
}
app.use('/posts', myService);
// Start the server
const port = 3030;
app.listen(port, () => {
console.log(`Feathers server listening on port ${port}`);
});