Develop Reference

Why is JS redis looping more times after succesful write?

I have the following logic for reading and writing redis state
export async function updateRedis() {
let stateName = 'stateName'
try {
let isSuccess = false
while (!isSuccess) {
try {
await redis
.watch(stateName, function (err) {
if (err) {
logger.error(`Error in watch state: ${err}`)
}
redis.get(stateName, function (err, result) {
if (err) {
logger.error(`Error in get state: ${err}`)
}
let state = JSON.parse(result)
// do some processing
redis.multi()
.set(stateName, JSON.stringify(state))
.exec(function (err, result) {
if (err) {
logger.error(`Error in set state: ${err}`)
}
if (result != null) {
isSuccess = true
}
})
console.log(`isSuccess for ${stateName} `, isSuccess)
})
})
} catch (e) {
logger.error(`Error: ${e}`)
}
}
} catch (e) {
logger.error(`Error: ${e}`)
}
return Promise.resolve(true)
}
This will print out
"isSuccess for stateName false"
"isSuccess for stateName true"
"isSuccess for stateName true"
So after the flag changes to true, it will continue for more loops. Sometimes it does more than just once.
Am I doing something wrong?

You cannot mix a synchronous loop (while (!isSuccess) { ... }) with asynchronous functions (redis.watch(stateName, function (err) { ... })).
You can also not await callback-based asynchronous functions. A function must return a promise to be awaitable. Since node-redis gives you promises when you don't pass callbacks to its methods, the key is not to do that (redis.watch(stateName, function (err) { ... }) → redis.watch(stateName)).
Your approach needs to be redone.
Let's make a function that encapsulates a redis transaction with optimistic locking. It takes a connection object, a key, and a value-transforming function, and it returns the result of the .set() operation:
const redisTransaction = async (client, key, transformer) => {
// https://github.com/redis/node-redis/blob/master/docs/isolated-execution.md
return client.executeIsolated(async isolatedClient => {
await isolatedClient.watch(key);
const val = await isolatedClient.get(key);
return isolatedClient.multi()
.set(key, await transformer.call(isolatedClient, val))
.exec();
});
};
Now you can await this function, because it returns a promise. That means we can make a simple infinite loop that exits immediately in case of success (via return), or retries indefinitely.
export async function updateRedis(key, transformer) {
while (true) {
try {
return await redisTransaction(redis, key, transformer);
} catch (err) {
logger.error(`Error for state ${key}: ${err}`);
}
}
}
A transformer function takes a value, and returns a new value. Inside it, the this keyword refers to the isolatedClient from the transaction, which could be useful if your transformation depends on other values from that client.
const result = await updateRedis('stateName', async function (val) {
const state = JSON.parse(val);
const newState = await modifyStateSomehow(state);
return JSON.stringify(newState);
});
The modifyStateSomehow() can itself be an asynchronous (i.e. "promise-returning") function. If it's not, you can make the state transformer a regular function by removing the async and the await.

How to call an API twice if there is an error occurred?

I have an internal API that I would like to post data. Depends on some cases, I am seeing errors. So what I would like to do is to call it again if there is an error occurred.
What I did was to create a counter to pass it to the function and call the function recursively as below. This gives me the error as below:
UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
Here is how I call the function:
....
private RETRY_API = 1;
....
try {
await this.callAPI(request, this.RETRY_API);
} catch (error) {
console.log('error', error);
}
This program never comes to the catch block above.
And here is my actual function that I call the API:
private async callAPI(request, retry) {
return new Promise((resolve, reject) => {
someService.postApiRequest('api/url', request, async(err: any, httpCode: number, data) => {
if (this.RETRY_API == 2) {
return reject(err);
} else if (err) {
this.callAPI(request, retry);
this.RETRY_API++;
} else if ( httpCode !== 200 ) {
this.RETRY_API = 2;
// some stuff
} else {
this.RETRY_API = 2;
// some stuff
return resolve(data);
}
});
})
}
Not sure what I am missing. If there is a better way to call the API twice if an error occurred, that would be great if you let me know.

Let's organize a little differently. First, a promise-wrapper for the api...
private async callAPI(request) {
return new Promise((resolve, reject) => {
someService.postApiRequest('api/url', request,(err: any, httpCode: number, data) => {
err ? reject(err) : resolve(data);
});
});
}
A utility function to use setTimeout with a promise...
async function delay(t) {
return new Promise(resolve => setTimeout(resolve, t));
}
Now, a function that calls and retries with delay...
private async callAPIWithRetry(request, retryCount=2, retryDelay=2000) {
try {
return await callAPI(request);
} catch (error) {
if (retryCount <= 0) throw err;
await delay(retryDelay);
return callAPIWithRetry(request, retryCount-1, retryDelay);
}
}
If you can't force a failure on the api to test the error path some other way, you can at least try this...
private async callAPIWithRetry(request, retryCount=2, retryDelay=2000) {
try {
// I hate to do this, but the only way I can test the error path is to change the code here to throw an error
// return await callAPI(request);
await delay(500);
throw("mock error");
} catch (error) {
if (retryCount <= 0) throw err;
await delay(retryDelay);
return callAPIWithRetry(request, retryCount-1, retryDelay);
}
}

It looks like you need to add return await to the beginning of the line this.callAPI(request, retry); in callAPI function.
Similarly there are some condition blocks that doesn't resolve or reject the promise. While it might work okay, it's considered bad practice. You want to either resolve or reject a promise.

I've accomplished calling an API a second time when I received an error by using axios' interceptors functions.
Here is a code snippet you can review:
axios.interceptors.response.use(
// function called on a successful response 2xx
function (response) {
return response;
},
// function called on an error response ( not 2xx )
async function (error) {
const request = error.config as AxiosRequestConfig;
// request is original API call
// change something about the call and try again
// request.headers['Authorization'] = `Bearer DIFFERENT_TOKEN`;
// return axios(request)
// or Call a different API
// const new_data = await axios.get(...).then(...)
// return new_data
// all else fails return the original error
return Promise.reject(error)
}
);

Try replacing
if (this.RETRY_API == 2)
with
if (this.RETRY_API > 1)

Error "Given transaction number * does not match" in mongodb and nodejs

I want to modify two schema while adding data. For that I used ACID transaction of mongodb with nodejs as follow. But, when I run program it displays the error like
(node:171072) UnhandledPromiseRejectionWarning: MongoError: Given transaction number 3 does not match any in-progress transactions. The active transaction number is 2
at MessageStream.messageHandler (/home/user/Projects/project/node_modules/mongodb/lib/cmap/connection.js:272:20)
at MessageStream.emit (events.js:375:28)
at MessageStream.emit (domain.js:470:12)
addData = async(request: Request, response: Response) => {
const session = await stockSchema.startSession()
try {
const userData = request.body
let data = {}
const transaction = await session.withTransaction(async() => {
try {
userData.products.map(async(item: any) => {
await inventorySchema.findOneAndUpdate({ _id: item.materialID }, { $inc: {qty: -item.qty }}, { session });
});
data = new stockSchema(userData);
await data.save({ session });
} catch (error) {
await session.abortTransaction()
throw new Error("Could not create data. Try again.");
}
});
if (transaction) {
session.endSession()
return returnData(response, data, 'Data created successfully.');
} else {
throw new Error("Could not create data. Try again.");
}
} catch (error: any) {
session.endSession();
return Error(response, error.message, {});
}
}

So you might have figured out the answer to this already, but anyway, after months of frustration, and no clear answer on the internet, i finally figured it out.
The problem with your code above is that you are passing session into a database operation (the .findOneAndUpdate function above) that is running within .map . Meaning, your 'transaction session' is being used concurrently, which is what is causing the error. read this: https://www.mongodb.com/community/forums/t/concurrency-in-mongodb-transactions/14945 (it explains why concurrency with transactions creates a bit of a mess.)
Anyway, instead of .map, use a recursive function that fires each DB operation one after another rather than concurrently, and all your problems will be solved.
You could use a function something like this:
const executeInQueue = async ({
dataAry, //the array that you .map through
callback, //the function that you fire inside .map
idx = 0,
results = [],
}) => {
if (idx === dataAry.length) return results;
//else if idx !== dataAry.length
let d = dataAry[idx];
try {
let result = await callback(d, idx);
results.push(result);
return executeInQueue({
dataAry,
callback,
log,
idx: idx + 1,
results,
});
} catch (err) {
console.log({ err });
return results.push("error");
}
};

Retrying an async function on error, unhandled promise rejection

This function is being called by a parent function, that should return the success object, or the error.
I'm having trouble using async while trying to retry a function 4 times. On success it's all good. On error, I'm getting "Unhandled Promise Rejection".
//create record
const createSFDCRecord = async (object, userData) => {
console.log("trial", retryCount)
return await conn.sobject(object).create(userData, async (err, ret) => {
if (err || !ret.success) {
if (retryCount < retryLimit) {
setTimeout(async () => {
return await createSFDCRecord(object, userData)
}, 2000)
retryCount++
} else {
pagerDutyEvent(
`Failed to send ${userData.Trigger_Code_kcrm__c} form data into ${object} after 5x`,
"error",
err
)
}
}
console.log(`Created ${object} id : ${ret.id}`)
})
}
sobject is from jsforce
http://jsforce.github.io/jsforce/doc/SObject.html

You are mixing promises with callbacks:
sobject(object).create returns a promise, so there is no need to use the callback argument.
return with a value has no sense in a setTimeout callback. That returned value is going nowhere.
The promises resulting from the repeated attempts are not chained: the resolution of the first is not locked into the next one. Instead you have a stand alone promise in a setTimeout callback, whose resolution or rejection is not handled.
The "Unhandled Promise Rejection" happens because you have an await of a promise that can reject, yet you do not capture the exception that will be triggered at that line when this happens.
You should refactor this to something like this (untested code):
const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
const createSFDCRecord = async (object, userData) => {
let retryCount = 1;
while (true) {
console.log("trial", retryCount);
try {
let ret = await conn.sobject(object).create(userData);
if (!ret.success) throw new Error("ret.success is false");
console.log(`Created ${object} id : ${ret.id}`);
return ret.id;
} catch(err) {
if (retryCount >= retryLimit) {
pagerDutyEvent(
`Failed to send ${userData.Trigger_Code_kcrm__c} form data into ${object} after ${retryLimit}x`,
"error",
err
);
return -1; // Or throw an exception... but then deal with it.
}
}
retryCount++;
await delay(2000);
}
}

Async function does not wait for await function to end

i have an async function that do not work as expected, here is the code :
const onCreateCoachSession = async (event, context) => {
const { coachSessionID } = context.params;
let coachSession = event.val();
let opentokSessionId = 'prout';
await opentok.createSession({ mediaMode: 'relayed' }, function(
error,
session
) {
if (error) {
console.log('Error creating session:', error);
} else {
opentokSessionId = session.sessionId;
console.log('opentokSessionIdBefore: ', opentokSessionId);
const sessionId = session.sessionId;
console.log('Session ID: ' + sessionId);
coachSession.tokbox = {
archiving: true,
sessionID: sessionId,
sessionIsCreated: true,
};
db.ref(`coachSessions/${coachSessionID}`).update(coachSession);
}
});
console.log('opentokSessionIdEnd: ', opentokSessionId);
};
My function onCreateCoachSession trigger on a firebase event (it's a cloud function), but it does not end for opentok.createSession to end, i don't understand why as i put an await before.
Can anyone have an idea why my code trigger directly the last console log (opentokSessionIdEnd)
Here is a screenshot on order of console.log :
It's probably a simple problem of async/await that i missed but i cannot see what.
I thanks in advance the community for the help.

You're using createSession in callback mode (you're giving it a callback function), so it doesn't return a Promise, so it can't be awaited.
Two solutions :
1/ Use createSession in Promise mode (if it allows this, see the doc)
let session = null;
try{
session = await opentok.createSession({ mediaMode: 'relayed' })
} catch(err) {
console.log('Error creating session:', error);
}
or 2/ await a Promise
let session;
try {
session = await new Promise((resolve, reject) => {
opentok.createSession({ mediaMode: 'relayed' }, (error, session) => {
if (error) {
return reject(error)
}
resolve(session);
})
})
} catch (err) {
console.log('Error creating session:', err);
throw new Error(err);
}
opentokSessionId = session.sessionId;
console.log('opentokSessionIdBefore: ', opentokSessionId);
// ...

await means it will wait till the promise is resolved. I guess there is no promise returned in this case. you can create your own promise and handle the case