Github actions Connection error w/ Sequelize - javascript

I have a node js api that connects with sequelize to a mySQL database. I wrote a script to reset the database every time tests are ran. I am building a CI/CD pipeline and whenever the script is ran i get the following message:
throw new SequelizeErrors.ConnectionError(err);
^
ConnectionError [SequelizeConnectionError]: Connection lost: The server closed the connection.
...
parent: Error: Connection lost: The server closed the connection.
...
at TCP.<anonymous> (node:net:709:12) {
fatal: true,
code: 'PROTOCOL_CONNECTION_LOST'
},
original: Error: Connection lost: The server closed the connection.
This is the part of the pipeline that fails:
tests:
runs-on: ubuntu-20.04
env:
MYSQL_DEVELOPMENT_USER: root
MYSQL_DEVELOPMENT_PASSWORD: ${{ secrets.RootPassword }}
MYSQL_DEVELOPMENT_HOST: localhost
MYSQL_DEVELOPMENT_DATABASE: test
...
- uses: mirromutth/mysql-action#v1.1
with:
mysql root password: ${{ secrets.RootPassword }}
mysql database: test
- name: api testing
run: cd api && npm run test
...
This is the npm run test
"test": "npm run resetDevDb && cross-env NODE_ENV=test jest --verbose"
"resetDevDb" : "cross-env NODE_ENV=development node db/scripts/reset-development-db.js"
The reset database Script:
const resetDb = async () => {
await sequelize.authenticate();
await Book.sync({ force: true });
await Author.sync({ force: true });
await User.sync({ force: true });
await Promise.all(
users.map(async (user) => {
await User.create(user);
})
);
await Promise.all(
books.map(async (book) => {
await Book.create(book);
})
);
await Promise.all(
authors.map(async (author) => {
await Author.create(author);
})
);
console.log("Database reseted");
};
The sequelize config:
const developmentConfig = {
dialect: "mysql",
database: process.env.MYSQL_DEVELOPMENT_DATABASE,
username: process.env.MYSQL_DEVELOPMENT_USER,
password: process.env.MYSQL_DEVELOPMENT_PASSWORD,
host: process.env.MYSQL_DEVELOPMENT_HOST,
logging: console.log,
};
And initialization
const { database, username, password, ...options } = require("./config.js");
const { Sequelize } = require("sequelize");
const sequelize = new Sequelize(database, username, password, options);
module.exports = sequelize;
I have ran this exact pipeline with (I believe) the exact same config and was working fine. After changing the remote repository a few times (And adding the secret.RootPassword) to the new repository I started getting this connection error. From some by hand debugging I figure the error ocurrs during the Book.sync({ force: true }) but I'm not sure if it has something to do with the miromutth/mysql action. Thanks!
Note: When I run npm run test locally everything works great, all the tables drop and are created again and filled, and the tests run and pass normally.

Related

How to run node postgres inside Electron?

I'm trying to use postgres (pg) inside electron.
When the window opens, the connection and query works normally, but when the electron window is reloaded, pg connect and queries does not return anything.
<script>
const { Client } = require('pg')
async function query() {
const client = new Client({
host: '192.168.99.100',
port: 5433,
user: 'user',
password: 'password',
database: 'database',
})
console.log(await client.connect())
const res = await client.query('SELECT * from users')
console.log(res)
await client.end()
}
query()
</script>
PS: If i try to use Knex for connect and query the database, it returns this error:
Uncaught (in promise) KnexTimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?
at Client_PG.acquireConnection

How to use Jest global Setup and Teardown in a nodeJS project?

I added tests to my node js project using jest but for each test suite there's a beforeAll method that creates a new test server and connects to a mongo database and an afterAll method that closes both test server and the database. I would like to perform the above tasks globally for all the test suites not one at a time. Below is a sample of my code.
app.js
const express = require("express");
const app = express();
const { connectToDb } = require("./startup/db");
require("./startup/routes")(app);
connectToDb();
...
const port = process.env.PORT || 3000;
if (process.env.NODE_ENV !== "test") {
app.listen(port, () => winston.info(`Listening on port ${port}...`));
}
module.exports = app;
auth.test.js
const request = require("supertest");
const http = require("http");
const { disconnectDb } = require("../../startup/db");
describe("auth middleware", () => {
let server;
beforeAll((done) => {
const app = require("../../app");
server = http.createServer(app);
server.listen(done);
});
afterAll((done) => {
server.close(done);
disconnectDb();
});
it("should return 401 if no token is provided", async () => {
const res = request(server)
.post("/api/genres")
.set("x-auth-token", "")
.send({ name: "genre1" });
expect(res.status).toBe(401);
});
...
jest.config.js
module.exports = {
testEnvironment: "node",
};
Try with this jest.config.js:
module.exports = {
testEnvironment: "node",
globalSetup: '<rootDir>/src/testSetup.ts'
};
And in testSetup.ts you can do:
// testSetup.ts
module.exports = async () => {
const app = require("../../app");
server = http.createServer(app);
server.listen(done);
};
use this config: setupFiles: ['./tests/setup.js']
your setup file should look like this:
// setup.js
(async () => {
const app = require('../app.js')
global.app = app
})()
then you will be able to use app globally in every test suite
I had the same problem, I wanted to make one database connection before all test files and close the connection after all tests in all files.
But....I did not achieve what I wanted and MAYBE we don't need to do this.
I found a solution to launch functions beforeAll(),afterAll() etc... really before ALL TEST FILES and after ALL TEST FILES etc..
So you define these functions once in a certain file and they run for every test file.
To do that, all we need is to create a setupFile.ts and add path to this file in jest.config or in package.json "setupFilesAfterEnv": ["<rootDir>/__tests__/settings/setupTests.ts"],
Here is an example of my jest configuration.
"jest": {
"preset": "ts-jest",
"testEnvironment": "node",
"setupFilesAfterEnv": ["<rootDir>/__tests__/settings/setupTests.ts"],
"rootDir": "src",
"verbose": true,
"clearMocks": true,
"testMatch": [
"**/**/*.test.ts"
]
},
Here is an example of setupFile.ts
import usersCollection from "../../database/user-schema";
import mongoose from "mongoose";
beforeAll(async () => {
try {
await mongoose.connect(process.env.MONGODB_URL!);
await usersCollection.deleteMany({});
} catch (error) {
console.log(error);
}
});
afterAll(async () => {
try {
await mongoose.disconnect();
} catch (error) {
console.log(error);
}
});
It means that we will establish a connection to the database FOR EVERY TEST FILE BEFORE ALL TESTS IN THAT FILE and close connection after all tests in every test file.
What I realized for myself:
In real life we have many test files and not every file needs a connection to a database.
It's perfectly fine to open a connection to a database in files which need a connection and close after all tests in that file, for example integration tests when we test API endpoints.
In other tests to not use real database for many unit tests we can consider to mock(simulate) a database. It's another very interesting topic 😊
If I say something wrong you can correct me
P.S
I also want to mention what is written in the Mongoose documentation
Do not use globalSetup to call mongoose.connect() or
mongoose.createConnection(). Jest runs globalSetup in a separate
environment, so you cannot use any connections you create in
globalSetup in your tests.
https://mongoosejs.com/docs/jest.html

Can't connect Redis server to nodejs, Docker compose

I'm struggling to connect a redis deployment to my nodejs app. Of course locally without the use of docker, it works well, so I'm at odds as to whether this is an issue to do with my code, or the way I've set up my docker compose file
Dockerfile:
FROM node:8
WORKDIR /app
COPY package.json /app
COPY . /app
RUN npm install
CMD ["npm", "start"]
EXPOSE 3000
docker-compose.yml
version: "3"
services:
web:
container_name: web-container
restart: always
depends_on:
- redis
build: .
ports:
- "3000:3000"
links:
- redis
redis:
container_name: redis-container
image: "redis:latest"
ports:
- "6379:6379"
volumes:
- ./data:/data
Redis Connection File (RedisService.js)
const redis = require("redis");
const client = redis.createClient();
const DbUtils = require("../../db_utils");
const {promisify} = require("util");
const getAsync = promisify(client.get).bind(client);
const existsAsync = promisify(client.exists).bind(client);
class RedisCache {
constructor () {
var connected;
// * Initiliase the connection to redis server
client.on("connect", () => {console.log("📒 Redis cache is ready"); connected = true;})
client.on("error", (e) => {console.log("Redis cache error:\n" + e); connected = false;});
}
async setData (id, data) {
// * Stringify data if it's an object
data = data instanceof Object ? JSON.stringify(data) : data;
client.set(id, data);
return true;
}
async getData (key) {
return getAsync(key).then(data => {
data = JSON.parse(data) instanceof Object ? JSON.parse(data) : data;
return data;
})
}
async exists (key) {
return existsAsync(key).then(bool => {
return bool;
})
}
// Returns status of redis cache
async getStatus () {
return this.connected;
}
}
module.exports = new RedisCache();
ERROR
Error: Redis connection to 127.0.0.11:6379 failed - connect ECONNREFUSED 127.0.0.11:6379
When you run your containers via docker-compose they are all connected to a common network. Service name is a DNS name of given container so to access redis container from web you should create the client like :
const client = redis.createClient({
port : 6379,
host : 'redis'
});
You have not configured the host so it uses the default one - 127.0.0.1. But from the point of view of your web container the redis is not running on the localhost. Instead it runs in it's own container which DNS name is redis.
The beginning (docker part) of this tutorial worked for me :
https://medium.com/geekculture/using-redis-with-docker-and-nodejs-express-71dccd495fd3
docker run -d --name <CONTAINER_NAME> -p 127.0.0.1:6379:6379 redis
then in the node server (like in official redis website example) :
const redis = require('redis');
async function start() {
const client = redis.createClient(6379,'127.0.0.1');
await client.connect();
await client.set('mykey', 'Hello from node redis');
const myKeyValue = await client.get('mykey');
console.log(myKeyValue);
}
start();

Cannot connect to MSSQL server using node-mssql

const sql = require("mssql");
var sqlconfig = {
server:"192.168.200.5",
database:"DATA",
user: "labo",
password: "*********",
connectionTimeout: 30000,
port: 1433,
dialect: "mssql",
dialectOptions: {
"instanceName": "SQLEXPRESS"
},
};
new sql.ConnectionPool(sqlconfig).connect().then(pool => {
console.log(pool); return pool.query('select * from data where id = 2')
}).then(result=> {
console.dir(result)
}).catch(err => {
console.log(err);
})
Problem: Return Login failed for user 'Labo' ELOGIN when trying to login to Microsoft SQL Server using node.js.
What I have tried: Grab the example codes on npm to find the avoid extra problems. Error still persisted.
What I suspect: The username I inserted was "labo" but the one returned from err was 'Labo' (with capitalized 'L'). Is this a bug?
This is my code

Running script from Heroku CLI

I have a script that I want to run through the Heroku CLI. It's just a simple script to create a user in a postgressql database with Sequelize.
This is the script:
const argv = require('yargs').argv;
const Sequelize = require('sequelize');
const sqlizr = require('sqlizr');
require('dotenv').load();
// Check params
if (!argv.username) { throw new Error('username is required!'); }
if (!argv.password) { throw new Error('password is required!'); }
if (!argv.clientId) { throw new Error('client id is required!'); }
// Init db connection
const sequelize = new Sequelize(
process.env.DB_DATABASE,
process.env.DB_USER,
process.env.DB_PASS,
{
host: process.env.DB_HOST,
port: 5432,
dialect: 'postgres',
logging: false
}
)
var client_id = argv.clientId;
if(argv.clientId === -1){
client_id = 0;
}
console.log(sequelize)
sqlizr(sequelize, 'api/models/**/*.js');
// Check if user exists
console.log('Check is user exists...');
sequelize.models.USERS.count({
where: {
USERNAME: argv.username
}
})
.then(result => {
if (result > 0) {
console.error('user already exists!');
process.exit(1);
}
})
.then(() => {
console.log('Creating user...');
sequelize.models.USERS.create({
USERNAME: argv.username,
PASSWORD: argv.password,
CLNT_ID: client_id,
EMAIL: 'email#email.com',
PHONE: '123456789'
})
.then(result => {
console.log('User created successfully!');
})
.catch(error => {
console.error('Could not create user!', error);
})
.finally(result => {
process.exit(1);
});
});
Everything goes well if I execute this command locally:
node bin/createUser.js --username admin --password admin --clientId -1
But If i try to run this through the Heroku CLI like this:
heroku run bin/createUser.js --username admin --password admin --clientId -1
I get this in the terminal:
bin/createUser.js: line 4: syntax error near unexpected token `('
bin/createUser.js: line 4: `const yargs = require('yargs');'
I can't figure out what I'm doing wrong here. Hopefully someone can help me and explain why this is happening
You forgot to specify node in the command, so I suspect that Heroku is trying to run createUser.js as if it were a shell script.
You may need to install a node.js buildpack to be able to run the program on Heroku, but try:
heroku run node bin/createUser.js --username admin --password admin --clientId -1

Categories

Resources