I am using rxjs 6 and I am executing two async operations where the order is important.
I do have this chunk of code which works perfectly:
dbmsProxy.createDatastores().subscribe(() => {
UsersDAO.insert(users).subscribe(() => {
console.log('FINISHED ALL THE CHAIN');
});
});
But when I try to use concat of rxjs I do have an issue because the second one is executed before the first one finishes:
concat([dbmsProxy.createDatastores(), UsersDAO.insert(users)]).subscribe();
Below the DBMSProxy methods
public createDatastores(): Observable<string> {
const _this: DBMSProxy = this;
const subject = new Subject<string>();
const subscription: Subscription = UsersDAO.createDatastore().subscribe(
onSuccess,
onError,
onFinally
);
return subject;
function onSuccess(datastore: Nedb): void {
console.log(`USERS Datastore Created Successfully`);
_this.db.users = datastore;
subject.next('success');
}
function onError(err: string) {
subject.error('error');
console.error(err);
}
function onFinally() {
subject.complete();
subscription.unsubscribe();
}
}
public insertDocuments(documents: any, datastore: Nedb): Subject<any> {
const subject = new Subject<any>();
datastore.insert(documents, onInsert);
return subject;
function onInsert(err: Error, newDocuments: any) {
if (err) {
subject.error(err);
} else {
// add to the documents to insert the id just created from nedb when inserting the document
documents.forEach((document: any, ind: number) => {
document.id = newDocuments[ind]._id;
});
subject.next(documents);
}
subject.complete();
}
}
And below the UsersDAO methods:
public static createDatastore(): Subject<Nedb | string> {
const subject = new Subject<Nedb | string>();
const datastore = new Nedb({
filename: USERS_DATASTORE_FULL_NAME,
autoload: true,
onload
});
return subject;
function onload(err: Error) {
if (err) {
subject.error(
`Error creating USERS datastore: ${err.name} - ${err.message}`
);
} else {
subject.next(datastore);
}
subject.complete();
}
}
public static insert(users: User[]): Observable<any> {
return DBMSProxy.getInstance()
.insertDocuments(users, DBMSProxy.getInstance().db.users)
.pipe(catchError((val: any) => of('Error inserting the users')));
}
Any idea of what's going on please?
My current solution is to convert the Subject to Observable, create a new Observable with the second one, and remove the square brackets (otherwise I will get back the observables and not the results) and this seems to work:
const operations = concat(
dbmsProxy.createDatastores().asObservable(),
defer(() => UsersDAO.insert(users))
);
operations.subscribe(onSubscribe);
function onSubscribe(result: any) {
console.log('Finished all: ', result);
}
Related
I will explain my app a bit.
1)user makes call
2)validating user request
3)save user info with call status "CREATED",
4)send it to the target user
5)once target user accepts Modify user info with call status "ACCEPTED",
6)Then send event to the sender that user accept the call and rest of logic to connect the user.
All this logic are working fine without any issue,but facing an issue which is very hard to reproduce.
ISSUE
If user A calls User B and user B calls User A at the same time
Both user get calls. I use a locking mechanism to avoid this race condition. Now issue is reduced drastically but let say if user A and user B calls at same time (precisely) The issue still appear.
import * as Redis from 'redis';
export class LockHandler {
private static instance: LockHandler;
private senderUserId: string;
private receiverUserId: string;
private redisClient: any;
private lockSuccessSender: any;
private lockSuccessReciever: any;
private lockKeySender: string;
private lockKeyReciever: string;
private constructor() {
const host = process.env.DISTRIBUTED_REDIS_LOCK_HANDLER_HOST;
let port = Number(process.env.DISTRIBUTED_REDIS_LOCK_HANDLER_PORT);
let password=process.env.DISTRIBUTED_REDIS_LOCK_HANDLER_PASSWORD
port = isNaN(port) ? 6379 : port;
// setTimeout(()=>{
// console.log(">>>>>>>REDIS createClient >>>>>")
// },0.035)
console.table({
port,
host
})
this.redisClient = Redis.createClient({
host,
port,
password
});
}
public static getInstance(): LockHandler {
if (!LockHandler.instance) {
LockHandler.instance = new LockHandler();
}
return LockHandler.instance;
}
public setSenderId(senderId: string): LockHandler {
this.senderUserId = senderId;
return this;
}
public setReceiverId(receiverId: string): LockHandler {
this.receiverUserId = receiverId;
return this;
}
public async acquireLock(): Promise<LockHandler | null> {
this.lockKeySender = `call-lock-${this.senderUserId}-${this.receiverUserId}`;
this.lockKeyReciever = `call-lock-${this.receiverUserId}-${this.senderUserId}`;
const lockValue = `${Date.now()}`;
this.lockSuccessSender = await new Promise((resolve, reject) => {
this.redisClient.set(this.lockKeySender, lockValue, 'NX', 'PX', 5000, (err, result) => {
if (err) {
reject(err);
}
resolve(result === 'OK');
});
});
this.lockSuccessReciever = await new Promise((resolve, reject) => {
this.redisClient.set(this.lockKeyReciever, lockValue, 'NX', 'PX', 5000, (err, result) => {
if (err) {
reject(err);
}
resolve(result === 'OK');
});
});
if (!this.lockSuccessSender || !this.lockSuccessReciever) {
//one of either User is busy
return null;
}
return this;
}
public releaseLock(): null {
if (!this.lockKeyReciever || !this.lockKeySender) return null;
this.redisClient.del(this.lockKeySender, (err) => {
if (err) {
console.error(`Error releasing lock for key ${this.lockKeySender}`);
}
});
this.redisClient.del(this.lockKeyReciever, (err) => {
if (err) {
console.error(`Error releasing lock for key ${this.lockKeyReciever}`);
}
});
return null;
}
}
This is my implementation for locking mechanism with redis.
public async save(entity: CallEntity): Promise<CallEntity> {
if (!entity) return null ;
const lockHandler = LockHandler.getInstance().setSenderId(entity.senderUserId).setReceiverId(entity.receiverUserId);
let lockedCallBuilder = await lockHandler.acquireLock();
if (!lockedCallBuilder) {
//means user is busy
return null;
}
try {
// just create user info with call status "CREATED"
return await this.repository.save(entity)
} catch (error) {
return null ;
} finally {
lockHandler.releaseLock();
}
}
This is how i save the user info.
Can anyone help me to achieve this ?
I am not getting any clue how to mock a method. I have to write a unit test for this function:
index.ts
export async function getTenantExemptionNotes(platform: string) {
return Promise.all([(await getCosmosDbInstance()).getNotes(platform)])
.then(([notes]) => {
return notes;
})
.catch((error) => {
return Promise.reject(error);
});
}
api/CosmosDBAccess.ts
import { Container, CosmosClient, SqlQuerySpec } from "#azure/cosmos";
import { cosmosdbConfig } from "config/Config";
import { Workload } from "config/PlatformConfig";
import { fetchSecret } from "./FetchSecrets";
export class CosmoDbAccess {
private static instance: CosmoDbAccess;
private container: Container;
private constructor(client: CosmosClient) {
this.container = client
.database(cosmosdbConfig.database)
.container(cosmosdbConfig.container);
}
static async getInstance() {
if (!CosmoDbAccess.instance) {
try {
const connectionString = await fetchSecret(
"CosmosDbConnectionString"
);
const client: CosmosClient = new CosmosClient(connectionString);
// Deleting to avoid error: Refused to set unsafe header "user-agent"
delete client["clientContext"].globalEndpointManager.options
.defaultHeaders["User-Agent"];
CosmoDbAccess.instance = new CosmoDbAccess(client);
return CosmoDbAccess.instance;
} catch (error) {
// todo - send to app insights
}
}
return CosmoDbAccess.instance;
}
public async getAllNotesForLastSixMonths() {
const querySpec: SqlQuerySpec = {
// Getting data from past 6 months
query: `SELECT * FROM c
WHERE (udf.convertToDate(c["Date"]) > DateTimeAdd("MM", -6, GetCurrentDateTime()))
AND c.IsArchived != true
ORDER BY c.Date DESC`,
parameters: [],
};
const query = this.container.items.query(querySpec);
const response = await query.fetchAll();
return response.resources;
}
}
export const getCosmosDbInstance = async () => {
const cosmosdb = await CosmoDbAccess.getInstance();
return cosmosdb;
};
index.test.ts
describe("getExemptionNotes()", () => {
beforeEach(() => {
jest.resetAllMocks();
});
it("makes a network call to getKustoResponse which posts to axios and returns what axios returns", async () => {
const mockNotes = [
{
},
];
const cosmosDBInstance = jest
.spyOn(CosmoDbAccess, "getInstance")
.mockReturnValue(Promise.resolve(CosmoDbAccess.instance));
const kustoResponseSpy = jest
.spyOn(CosmoDbAccess.prototype, "getAllNotesForLastSixMonths")
.mockReturnValue(Promise.resolve([mockNotes]));
const actual = await getExemptionNotes();
expect(kustoResponseSpy).toHaveBeenCalledTimes(1);
expect(actual).toEqual(mockNotes);
});
});
I am not able to get instance of CosmosDB or spyOn just the getAllNotesForLastSixMonths method. Please help me code it or give hints. The complexity is because the class is singleton or the methods are static and private
I'm creating a façade for the nats streaming lib as follows:
import nats, { Message, Stan, Subscription, SubscriptionOptions } from 'node-nats-streaming'
class NatsHelper {
private client: Stan | null = null
public connect(url: string, clusterID: string, clientID: string, listener: (...args: any[]) => void, verboseConnection: boolean = true): void {
const clientIDString = `${clientID}-${randomBytes(4).toString('hex')}`
if (verboseConnection) {
console.log(`Connecting to NATS cluster '${clusterID}' with clientID '${clientIDString}' on url '${url}'`)
}
const connectionAttempt = nats.connect(
clusterID,
clientIDString,
{
url
}
)
const setupConnection = (...args: any[]): void => {
this.client = connectionAttempt
this.client.on('close', (): void => {
if (verboseConnection) {
console.log(`Connection with NATS cluster '${clusterID}' with clientID '${clientIDString}' on url '${url}' was closed`)
}
this.client = null
process.exit()
})
process.on('SIGINT', () => this.client?.close())
process.on('SIGTERM', () => this.client?.close())
if (verboseConnection) {
console.log(`Connected to NATS cluster '${clusterID}' with clientID '${clientIDString}' on url '${url}' successfuly`)
}
listener(...args)
}
connectionAttempt.on('connect', setupConnection)
}
}
It happens though that I'm not able to test if the provided listener function is called, because it relies on the Stan 'connect' event to happen and jest finishes the test before it happens.
How can I make jest wait for this event to happen, and then executes the expect function?
You have overcomplicated this. It's perfectly possible to write the test for the original code without modifying it by mocking out the library using jest.mock(), and injecting mock implementations for your on method. Like this:
import nats from "node-nats-streaming";
import { mock } from "jest-mock-extended";
import { NatsHelper } from "./nats";
jest.mock("node-nats-streaming");
describe("NatsHelper", () => {
it("calls listener on connectEvent", () => {
const client = mock<nats.Stan>();
client.on.mockImplementation((name, callback) => {
if (name !== "close") {
callback();
}
return client;
});
jest.mocked(nats).connect.mockReturnValue(client);
const connector = new NatsHelper();
const listener = jest.fn();
connector.connect("foo", "foo", "foo", listener);
expect(listener).toHaveBeenCalled();
});
});
[EDIT] Found the solution I was looking
It happens that we can "convert" an event into a Promise, as follows:
import { randomBytes } from 'crypto'
import nats from 'node-nats-streaming'
export class NullClientError extends Error {
constructor() {
super('Nats client is not connected')
this.name = 'NullClientError'
}
}
export class NatsHelper {
private verboseConnectionString: string
private client: nats.Stan
private connector: nats.Stan
constructor(
private readonly verboseConnection: boolean = true
) { }
public async connect(url: string, clusterID: string, clientID: string, callback: (...args: any[]) => void): Promise<void> {
const clientIDString = `${clientID}-${randomBytes(4).toString('hex')}`
this.verboseConnectionString = `NATS cluster '${clusterID}' with clientID '${clientIDString}' on url '${url}'`
if (this.verboseConnection) {
console.log(`Connecting to ${this.verboseConnectionString}`)
}
this.connector = nats.connect(
clusterID,
clientIDString,
{
url
}
)
this.connector.on('connect', (...args: any[]) => {
const realCallback = this.setupListener(callback)
realCallback(...args)
})
return await new Promise(
resolve => {
if (this.connector) {
this.connector.on('connect', () => {
resolve()
})
}
}
)
}
private setupListener(listener: (...args: any[]) => void): (...args: any[]) => void {
const setupConnection = (...args: any[]): void => {
if (this.connector === undefined) {
throw new NullClientError()
}
this.client = this.connector
if (this.client === undefined) {
throw new NullClientError()
}
this.client.on('close', (): void => {
if (this.verboseConnection) {
console.log(`Connection with ${this.verboseConnectionString} was closed`)
}
process.exit()
})
process.on('SIGINT', () => this.client?.close())
process.on('SIGTERM', () => this.client?.close())
if (this.verboseConnection) {
console.log(`Connected to ${this.verboseConnectionString} successfuly`)
}
listener(...args)
}
return setupConnection
}
}
And then test it with asynchronous tests:
describe('NatsHelper', () => {
test('ensure NatsHelper calls connect with correct values', async () => {
const connectSpy = jest.spyOn(nats, 'connect')
const sut = new NatsHelper(false)
const { url, clusterID, clientID, listener } = makeConnectionParams()
await sut.connect(url, clusterID, clientID, listener)
const clientIDString = connectSpy.mock.calls[0][1]
expect(clientIDString).toContain(clientID)
expect(connectSpy).toHaveBeenCalledWith(clusterID, clientIDString, { url })
})
test('ensure NatsHelper forwards the callback when connected', async () => {
const connectionParms = makeConnectionParams()
const { url, clusterID, clientID } = connectionParms
const listenerSpy = jest.spyOn(connectionParms, 'listener')
const sut = new NatsHelper(false)
await sut.connect(url, clusterID, clientID, connectionParms.listener)
expect(listenerSpy).toHaveBeenCalledTimes(1)
})
}
Let's say I have a simple loop like this:
for (const i=0;i<3;i++) {
to(`This counter is ${i}`)
}
I want to have in my file at the end:
This counter is 0
This counter is 1
This counter is 2
I tried to do that by doing this :
export class S3Output extends Output {
#stream: Readable | null = null
async to(s: string): Promise<void> {
const params = {
Bucket: config.aws.bucketName,
Body: this.#stream,
Key: 'test.json'
}
this.#stream?.pipe(process.stdout)
this.#stream?.push(s)
await S3.upload(params).promise()
return
}
init(): void {
this.#stream = new Readable()
this.#stream._read = function () {};
}
finish(): void {
this.#stream?.push(null)
return
}
}
My init function is called at the beginning of the loop, my to function is called every time I want to push a string in the file and the finish function at the end of the loop. This code doesn't push any data, why?
I actually found what was the problem. I had to send the stream once it was over and not while I was doing it. Also no need of pipe.
export class S3Output extends Output {
#stream: Readable | null = null
async to(s: string): Promise<void> {
this.#stream?.push(s)
return
}
init(): void {
this.#stream = new Readable()
this.#stream._read = function () {}
}
async finish(): Promise<void> {
this.#stream?.push(null)
const params = {
Bucket: config.aws.bucketName,
Body: this.#stream,
Key: 'test.json'
}
await S3.upload(params).promise()
return
}
}
I am trying to create a class that will fetch / cache users from my Firestore database. For some reason, I can't seem to save or expose the previous promise that was created. Here is my class:
export class UserCache {
private cacheTimeMilliseconds: number = 600000;
private userCache: any = {};
public getCacheUser(userid: string): Promise<User> {
return new Promise((resolve, reject) => {
let d = new Date();
d.setTime(d.getTime() - this.cacheTimeMilliseconds);
if (this.userCache[userid] && this.userCache[userid].complete && this.userCache[userid].lastAccess > d.getTime()) {
console.log("User cached");
resolve(this.userCache[userid].user);
}
console.log("Need to cache user");
this.userCache[userid] = {
complete: false
};
this.getSetUserFetchPromise(userid).then((data) => {
let user: User = <User>{ id: data.id, ...data.data() };
this.userCache[userid].user = user;
this.userCache[userid].complete = true;
this.userCache[userid].lastAccess = Date.now();
resolve(user);
});
});
}
private getSetUserFetchPromise(userid: string): Promise<any> {
console.log(this.userCache[userid]);
if (this.userCache[userid] && this.userCache[userid].promise) {
return this.userCache[userid].promise;
} else {
console.log("Creating new user fetch request.");
this.userCache[userid].promise = firestore().collection('users').doc(userid).get();
console.log(this.userCache[userid]);
return this.userCache[userid].promise;
}
}
}
Logs: (there are only 2 unique users, so should only be creating 2 new requests)
In the logs I can see that the promise is getting set in getSetUserFetchPromise, but the next time the function is called, the property is no longer set. I suspect it is either a scope or concurrency issue, but I can't seem to get around it.
I am calling getCacheUser in a consuming class with let oCache = new UserCache() and oCache.getCacheUser('USERID')
Edit following Tuan's answer below
UserCacheProvider.ts
import firestore from '#react-native-firebase/firestore';
import { User } from '../static/models';
class UserCache {
private cacheTimeMilliseconds: number = 600000;
private userCache: any = {};
public getCacheUser(userid: string): Promise<User> {
return new Promise((resolve, reject) => {
let d = new Date();
d.setTime(d.getTime() - this.cacheTimeMilliseconds);
if (this.userCache[userid] && this.userCache[userid].complete && this.userCache[userid].lastAccess > d.getTime()) {
console.log("User cached");
resolve(this.userCache[userid].user);
}
console.log("Need to cache user");
this.userCache[userid] = {
complete: false
};
this.getSetUserFetchPromise(userid).then((data) => {
let user: User = <User>{ id: data.id, ...data.data() };
this.userCache[userid].user = user;
this.userCache[userid].complete = true;
this.userCache[userid].lastAccess = Date.now();
resolve(user);
});
});
}
private getSetUserFetchPromise(userid: string): Promise<any> {
console.log(this.userCache[userid]);
if (this.userCache[userid] && this.userCache[userid].promise) {
return this.userCache[userid].promise;
} else {
console.log("Creating new user fetch request.");
this.userCache[userid].promise = firestore().collection('users').doc(userid).get();
console.log(this.userCache[userid]);
return this.userCache[userid].promise;
}
}
}
const userCache = new UserCache();
export default userCache;
ChatProvider.ts (usage)
let promises = [];
docs.forEach(doc => {
let message: Message = <Message>{ id: doc.id, ...doc.data() };
promises.push(UserCacheProvider.getCacheUser(message.senderid).then((oUser) => {
let conv: GCMessage = {
_id: message.id,
text: message.messagecontent,
createdAt: new Date(message.messagedate),
user: <GCUser>{ _id: oUser.id, avatar: oUser.thumbnail, name: oUser.displayname }
}
if (message.type && message.type == 'info') {
conv.system = true;
}
if (message.messageattachment && message.messageattachment != '') {
conv.image = message.messageattachment;
}
return conv;
}));
});
Promise.all(promises).then((values) => {
resolve(values);
});
Without seeing the calling code, it could be that getCacheUser is called twice before firestore resolves.
As an aside, I think refactoring the class may make debugging easier. I wonder why it caches the user, promise completion status, and the promise itself. Why not just cache the promise, something like:
interface UserCacheRecord {
promise: Promise<User>
lastAccess: number
}
export class UserCache {
private cacheTimeMilliseconds: number = 600000;
private userCache: { [userid: string]: UserCacheRecord } = {};
public async getCacheUser(userid: string): Promise<User> {
let d = new Date();
const cacheExpireTime = d.getTime() - this.cacheTimeMilliseconds
if (this.userCache[userid] && this.userCache[userid].lastAccess > cacheExpireTime) {
console.log("User cached");
return this.userCache[userid].promise
}
console.log("Need to cache user");
this.userCache[userid] = {
promise: this.getUser(userid),
lastAccess: Date.now()
}
return this.userCache[userid].promise
}
private async getUser(userid: string): Promise<User> {
const data = firestore().collection('users').doc(userid).get();
return <User>{ id: data.id, ...data.data() };
}
}
Currently, you create new UserCache everytime you access cache users. You have to export the instance of UserCache class, so just single instance is used for your app.
UserCache.ts
class UserCache {
}
const userCache = new UserCache();
export default userCache;
SomeFile.ts
import UserCache from './UserCache';
UserCache.getCacheUser('USERID')
Update
Added some tests
class UserCache {
userCache = {};
getUser(id) {
return new Promise((resolve, reject) => {
if (this.userCache[id]) {
resolve({
...this.userCache[id],
isCache: true,
});
}
this.requestUser(id).then(data => {
resolve(data);
this.userCache[id] = data;
});
});
}
requestUser(id) {
return Promise.resolve({
id,
});
}
}
const userCache = new UserCache();
export default userCache;
userCache.test.ts
import UserCache from '../test';
describe('Test user cache', () => {
test('User cached successfully', async () => {
const user1: any = await UserCache.getUser('test1');
expect(user1.isCache).toBeUndefined();
const user2: any = await UserCache.getUser('test1');
expect(user2.isCache).toBe(true);
});
});