I built a TS, MongoDB Client wrapper. for some reason when I call the function that gets the connection, its callback is called twice.
There are 2 calls in total to the get() function, 1 before the export as you can see and another from a mocha test.
I am pretty new to TS and JS in general, but this seems a bit off.
import {Db, MongoClient} from "mongodb";
import {MongoConfig} from '../config/config'
class DbClient {
private cachedDb : Db = null;
private async connectToDatabase() {
console.log('=> connect to database');
let connectionString : string = "mongodb://" + MongoConfig.host + ":" + MongoConfig.port;
return MongoClient.connect(connectionString)
.then(db => {
console.log('=> connected to database');
this.cachedDb = db.db(MongoConfig.database);
return this.cachedDb;
});
}
public async get() {
if (this.cachedDb) {
console.log('=> using cached database instance');
return Promise.resolve(this.cachedDb);
}else{
return this.connectToDatabase();
}
}
}
let client = new DbClient();
client.get();
export = client;
where the console output is:
=> connect to database
=> connected to database
=> connected to database
Any particular reason this is misbehaving?
There are 2 calls in total to the get() function, 1 before the export as you can see and another from a mocha test.
I suspect the output has an additional => connect to database. As I said in the comments: There's a "race condition" where get() could be called multiple times before this.cachedDb is set which would lead to multiple connections/instances of Db being created.
For example:
const a = client.get();
const b = client.get();
// then
a.then(resultA => {
b.then(resultB => {
console.log(resultA !== resultB); // true
});
});
Solution
The problem can be fixed by storing the promise as the cached value (also, no need to have the async keyword on the methods as Randy pointed out, as there's no values being awaited in any of the methods so you can just return the promises):
import {Db, MongoClient} from "mongodb";
import {MongoConfig} from '../config/config'
class DbClient {
private cachedGet: Promise<Db> | undefined;
private connectToDatabase() {
console.log('=> connect to database');
const connectionString = `mongodb://${MongoConfig.host}:${MongoConfig.port}`;
return MongoClient.connect(connectionString);
}
get() {
if (!this.cachedGet) {
this.cachedGet = this.connectToDatabase();
// clear the cached promise on failure so that if a caller
// calls this again, it will try to reconnect
this.cachedGet.catch(() => {
this.cachedGet = undefined;
});
}
return this.cachedGet;
}
}
let client = new DbClient();
client.get();
export = client;
Note: I'm not sure about the best way of using MongoDB (I've never used it), but I suspect connections should not be so long lived as to be cached like this (or should probably only be cached for a short time and then disconnected). You'll need to investigate that though.
Related
I have two cloud functions
One cloud function is for set or updating existing scheduled job
Canceling an existing scheduled job
I am using import * as the schedule from 'node-schedule'; to manage Scheduling a jobs
The problem is cus createJob function is triggered and jobId is returned, but later when I triger cancelJob function all prev scheduled cron jobs do not exist cus node-schedule lives in memory and I can't access the jobs:
this will return empty object: const allJobs = schedule.scheduledJobs;
Does anyone have some solution on this situation?
UTILS this is the main logic that is called when some of my cloud functions are triggered
enter code here
// sendgrid
import * as sgMail from '#sendgrid/mail';
import * as schedule from 'node-schedule';
sgMail.setApiKey(
'apikey',
);
import {
FROM_EMAIL,
EMAIL_TEMPLATE_ID,
MESSAGING_SERVICE_SID,
} from './constants';
export async function updateReminderCronJob(data: any) {
try {
const {
to,
...
} = data;
const message = {
to,
from: FROM_EMAIL,
templateId: EMAIL_TEMPLATE_ID,
};
const jobReferences: any[] = [];
// Stop existing jobs
if (jobIds && jobIds.length > 0) {
jobIds.forEach((j: any) => {
const job = schedule.scheduledJobs[j?.jobId];
if (job) {
job.cancel();
}
});
}
// Create new jobs
timestamps.forEach((date: number) => {
const job = schedule.scheduleJob(date, () => {
if (selectedEmail) {
sgMail.send(message);
}
});
if (job) {
jobReferences.push({
jobId: job.name,
});
}
});
console.warn('jobReferences', jobReferences);
return jobReferences;
} catch (error) {
console.error('Error updateReminderCronJob', error);
return null;
}
}
export async function cancelJobs(jobs: any) {
const allJobs = schedule.scheduledJobs;
jobs.forEach((job: any) => {
if (!allJobs[job?.jobId]) {
return;
}
allJobs[job.jobId].cancel();
});
}
node-schedule will not work effectively in Cloud Functions because it requires that the scheduling and execution all be done on a single machine that stays running without interruption. Cloud Functions does not fully support this behavior, as it will dynamically scale up and down to zero the number of machines servicing requests (even if you set min instances to 1, it may still reduce your active instances to 0 in some cases). You will get unpredictable behavior if you try to schedule this way.
The only way you can get reliable scheduling using Cloud Functions is with pub/sub functions as described in the documentation. Firebase scheduled functions make this a bit easier by managing some of the details. You will not be able to dynamically control repeating jobs, so you will need to build some way to periodically run a job and check to see if it should run at that moment.
I'm using Shopify's Node Api tutorial to create a Redis store. However, the code block provided is in typescript and my entire project is written in javascript (React/nextjs). I've been working for a few hours to try and convert the code to be useable, but am unable to get it to work properly in my project. Seriously struggling with this.
How would I convert the below code block from typescript to javascript?
/* redis-store.ts */
// Import the Session type from the library, along with the Node redis package, and `promisify` from Node
import {Session} from '#shopify/shopify-api/dist/auth/session';
import redis from 'redis';
import {promisify} from 'util';
class RedisStore {
private client: redis.RedisClient;
private getAsync;
private setAsync;
private delAsync;
constructor() {
// Create a new redis client
this.client = redis.createClient();
// Use Node's `promisify` to have redis return a promise from the client methods
this.getAsync = promisify(this.client.get).bind(this.client);
this.setAsync = promisify(this.client.set).bind(this.client);
this.delAsync = promisify(this.client.del).bind(this.client);
}
/*
The storeCallback takes in the Session, and sets a stringified version of it on the redis store
This callback is used for BOTH saving new Sessions and updating existing Sessions.
If the session can be stored, return true
Otherwise, return false
*/
storeCallback = async (session: Session) => {
try {
// Inside our try, we use the `setAsync` method to save our session.
// This method returns a boolean (true if successful, false if not)
return await this.setAsync(session.id, JSON.stringify(session));
} catch (err) {
// throw errors, and handle them gracefully in your application
throw new Error(err);
}
};
/*
The loadCallback takes in the id, and uses the getAsync method to access the session data
If a stored session exists, it's parsed and returned
Otherwise, return undefined
*/
loadCallback = async (id: string) => {
try {
// Inside our try, we use `getAsync` to access the method by id
// If we receive data back, we parse and return it
// If not, we return `undefined`
let reply = await this.getAsync(id);
if (reply) {
return JSON.parse(reply);
} else {
return undefined;
}
} catch (err) {
throw new Error(err);
}
};
/*
The deleteCallback takes in the id, and uses the redis `del` method to delete it from the store
If the session can be deleted, return true
Otherwise, return false
*/
deleteCallback = async (id: string) => {
try {
// Inside our try, we use the `delAsync` method to delete our session.
// This method returns a boolean (true if successful, false if not)
return await this.delAsync(id);
} catch (err) {
throw new Error(err);
}
};
}
// Export the class
export default RedisStore;
Just save all that typescript code in a .ts file (probably redis-store.ts).
then use typescript compiler to convert to your version of javascript by just running tsc command as below
tsc redis-store.ts
for more compiler options, please visit below
https://www.typescriptlang.org/docs/handbook/compiler-options.html
You basically need to get rid of all the types (Session and string) and switch private to #, maybe something like this:
/* redis-store.js */
import redis from 'redis';
import {promisify} from 'util';
class RedisStore {
#client;
#getAsync;
#setAsync;
#delAsync;
constructor() {
// Create a new redis client
this.client = redis.createClient();
this.getAsync = promisify(this.client.get).bind(this.client);
this.setAsync = promisify(this.client.set).bind(this.client);
this.delAsync = promisify(this.client.del).bind(this.client);
}
storeCallback = async (session) => {
try {
// Inside our try, we use the `setAsync` method to save our session.
// This method returns a boolean (true if successful, false if not)
return await this.setAsync(session.id, JSON.stringify(session));
} catch (err) {
// throw errors, and handle them gracefully in your application
throw new Error(err);
}
};
/*
The loadCallback takes in the id, and uses the getAsync method to access the session data
If a stored session exists, it's parsed and returned
Otherwise, return undefined
*/
loadCallback = async (id) => {
try {
// Inside our try, we use `getAsync` to access the method by id
// If we receive data back, we parse and return it
// If not, we return `undefined`
let reply = await this.getAsync(id);
if (reply) {
return JSON.parse(reply);
} else {
return undefined;
}
} catch (err) {
throw new Error(err);
}
};
/*
The deleteCallback takes in the id, and uses the redis `del` method to delete it from the store
If the session can be deleted, return true
Otherwise, return false
*/
deleteCallback = async (id) => {
try {
// Inside our try, we use the `delAsync` method to delete our session.
// This method returns a boolean (true if successful, false if not)
return await this.delAsync(id);
} catch (err) {
throw new Error(err);
}
};
}
// Export the class
export default RedisStore;
I have created a subscriber class to store subscriber details and use a static method to return the instance of the class, but I am not able to set the values using the instance
Here is the subscriber class:
let _instance;
export class Subscriber {
constructor(username, password) {
this._username = username;
this._password = password;
}
setSubscriberId(subscriberId) {
cy.log(subscriberId);
this._subscriberId = subscriberId;
}
setSessionId(sessionId) {
this.sessionId = sessionId;
}
getUserName = () => {
return this._username;
}
getPassword = () => {
return this._password;
}
getSubsciberId() {
return this._subscriberId;
}
getSessionId() {
return this.sessionId;
}
static createSubscriber(username, password) {
if (!_instance) {
_instance = new Subscriber(username, password);
}
return _intance;
}
static getSubscriber() {
return _instance;
}
}
I am creating a instance of the class in before block and accessing the instance in Given block
before("Create a new subscriber before the tests and set local storage", () => {
const username = `TestAutomation${Math.floor(Math.random() * 1000)}#sharklasers.com`;
const password = "test1234";
subscriberHelpers.createSubscriber(username, password, true).then((response) => {
cy.log(response);
Subscriber.createSubscriber(username, password);
Subscriber.getSubscriber().setSubscriberId(response.Subscriber.Id);
Subscriber.getSubscriber().setSessionId(response.SessionId);
}).catch((error) => {
cy.log(error);
});
});
Given(/^I launch selfcare app$/, () => {
cy.launchApp();
});
Given(/^I Set the environemnt for the test$/, () => {
cy.log(Subscriber.getSubscriber());
cy.log(Subscriber.getSubscriber().getSubsciberId());
});
here is the output on the cypress console
Questions:
Why the subscriberID is null even though I am setting it in the before block
if I print the subscriber Object why am I not seeing subscriberID
Here is the output of subscriber object
Properties username and password are defined synchronously in before(), so are present on the object when tested.
But subscriberId is obtained asynchronously, so you will need to wait for completion inside the test, e.g
cy.wrap(Subscriber.getSubscriber()).should(function(subscriber){
expect(subscriber.getSubsciberId()).not.to.be.null
})
Refer to wrap - Objects to see how to handle an object with Cypress commands.
and see should - Differences
When using a callback function with .should() or .and(), on the other hand, there is special logic to rerun the callback function until no assertions throw within it.
In other words, should will retry (up to 5 seconds) until the expect inside the callback does not fail (i.e in your case the async call has completed).
Assume the following class in TypeScript:
class MongoDbContext implements IMongoDbContext {
private connectionString : string;
private databaseName : string;
private database : Db;
public constructor (connectionString : string, databaseName : string) {
this.connectionString = connectionString;
this.databaseName = databaseName;
}
public async initializeAsync () : Promise<MongoDbContext> {
// Create a client that represents a connection with the 'MongoDB' server and get a reference to the database.
var client = await MongoClient.connect(this.connectionString, { useNewUrlParser: true });
this.database = await client.db(this.databaseName);
return this;
}
}
Now, I want to test if an exception is thrown when I'm trying to connect to an unexisting MongoDB server, this is done with the following integration test:
it('Throws when a connection to the database server could not be made.', async () => {
// Arrange.
var exceptionThrowed : boolean = false;
var mongoDbContext = new MongoDbContext('mongodb://127.0.0.1:20000/', 'databaseName');
// Act.
try { await mongoDbContext.initializeAsync(); }
catch (error) { exceptionThrowed = true; }
finally {
// Assert.
expect(exceptionThrowed).to.be.true;
}
}).timeout(5000);
When I run this unit test, my CMD window doesn't print a summary.
It seems that it's hanging somewhere.
What am I'm doing wrong in this case?
Kind regards,
I've managed to find the issue.
It seems that I must close my 'MongoClient' connection for Mocha to quit correctly.
So, I've added an extra method
public async closeAsync () : Promise<void> {
await this.client.close();
}
This method is called after each test.
This is how I connect to a mongoDB using monk(). I'll store it in state.
Assume we want to drop some collections, we call dropDB.
db.js
var state = {
db: null
}
export function connection () {
if (state.db) return
state.db = monk('mongdb://localhost:27017/db')
return state.db
}
export async function dropDB () {
var db = state.db
if (!db) throw Error('Missing database connection')
const Users = db.get('users')
const Content = db.get('content')
await Users.remove({})
await Content.remove({})
}
I'm not quite sure if it is a good approach to use state variable. Maybe someone can comment on that or show an improvement.
Now I want to write a unit test for this function using JestJS:
db.test.js
import monk from 'monk'
import { connection, dropDB } from './db'
jest.mock('monk')
describe('dropDB()', () => {
test('should throw error if db connection is missing', async () => {
expect.assertions(1)
await expect(dropDB()).rejects.toEqual(Error('Missing database connection'))
})
})
This part is easy, but the next part gives me two problems:
How do I mock the remove() methods?
test('should call remove() methods', async () => {
connection() // should set `state.db`, but doesn't work
const remove = jest.fn(() => Promise.resolve({ n: 1, nRemoved: 1, ok: 1 }))
// How do I use this mocked remove()?
expect(remove).toHaveBeenCalledTimes(2)
})
And before that? How do I setup state.db?
Update
As explained by poke the global variable makes the problem. So I switched to a class:
db.js
export class Db {
constructor() {
this.connection = monk('mongdb://localhost:27017/db');
}
async dropDB() {
const Users = this.connection.get('users');
const Content = this.connection.get('content');
await Users.remove({});
await Content.remove({});
}
}
which results in this test file:
db.test.js
import { Db } from './db'
jest.mock('./db')
let db
let remove
describe('DB class', () => {
beforeAll(() => {
const remove = jest.fn(() => Promise.resolve({ n: 1, nRemoved: 1, ok: 1 }))
Db.mockImplementation(() => {
return { dropDB: () => {
// Define this.connection.get() and use remove as a result of it
} }
})
})
describe('dropDB()', () => {
test('should call remove method', () => {
db = new Db()
db.dropDB()
expect(remove).toHaveBeenCalledTimes(2)
})
})
})
How do I mock out any this elements? In this case I need to mock this.connection.get()
Having a global state is definitely the source of your problem here. I would suggest to look for a solution that does not involve global variables at all. As per Global Variables Are Bad, global variables cause tight coupling and make things difficult to test (as you have noticed yourself).
A better solution would be to either pass the database connection explicitly to the dropDB function, so it has the connection as an explicit dependency, or to introduce some stateful object that holds onto the connection and offers the dropDB as a method.
The first option would look like this:
export function openConnection() {
return monk('mongdb://localhost:27017/db');
}
export async function dropDB(connection) {
if (!connection) {
throw Error('Missing database connection');
}
const Users = connection.get('users');
const Content = connection.get('content');
await Users.remove({});
await Content.remove({});
}
This would also make it very easy to test dropDB as you can now just pass a mocked object for it directly.
The other option could look like this:
export class Connection() {
constructor() {
this.connection = monk('mongdb://localhost:27017/db');
}
async dropDB() {
const Users = this.connection.get('users');
const Content = this.connection.get('content');
await Users.remove({});
await Content.remove({});
}
}
A test for the first option could look like this:
test('should call remove() methods', async () => {
const usersRemove = jest.fn().mockReturnValue(Promise.resolve(null));
const contentRemove = jest.fn().mockReturnValue(Promise.resolve(null));
const dbMock = {
get(type) {
if (type === 'users') {
return { remove: usersRemove };
}
else if (type === 'content') {
return { remove: contentRemove };
}
}
};
await dropDB(dbMock);
expect(usersRemove).toHaveBeenCalledTimes(1);
expect(contentRemove).toHaveBeenCalledTimes(1);
});
Basically, the dropDB function expects an object that has a get method which when called returns an object that has a remove method. So you just need to pass something that looks like that, so the function can call those remove methods.
For the class, this is a bit more complicated since the constructor has a dependency on the monk module. One way would be to make that dependency explicit again (just like in the first solution), and pass monk or some other factory there. But we can also use Jest’s manual mocks to simply mock the whole monk module.
Note that we do not want to mock the module containing our Connection type. We want to test that, so we need it in its un-mocked state.
To mock monk, we need to create a mock module of it at __mocks__/monk.js. The manual points out that this __mocks__ folder should be adjacent to the node_modules folder.
In that file, we simply export our custom monk function. This is pretty much the same we already used in the first example, since we only care about getting those remove methods in place:
export default function mockedMonk (url) {
return {
get(type) {
if (type === 'users') {
return { remove: mockedMonk.usersRemove };
}
else if (type === 'content') {
return { remove: mockedMonk.contentRemove };
}
}
};
};
Note that this refers to the functions as mockedMonk.usersRemove and mockedMonk.contentRemove. We’ll use this in the test to configure those function explicitly during the test execution.
Now, in the test function, we need to call jest.mock('monk') to enable Jest to mock the monk module with our mocked module. Then, we can just import it too and set our functions within the test. Basically, just like above:
import { Connection } from './db';
import monk from 'monk';
// enable mock
jest.mock('./monk');
test('should call remove() methods', async () => {
monk.usersRemove = jest.fn().mockReturnValue(Promise.resolve(null));
monk.contentRemove = jest.fn().mockReturnValue(Promise.resolve(null));
const connection = new Connection();
await connection.dropDB();
expect(monk.usersRemove).toHaveBeenCalledTimes(1);
expect(monk.contentRemove).toHaveBeenCalledTimes(1);
});