I have the following logic for reading and writing redis state
export async function updateRedis() {
let stateName = 'stateName'
try {
let isSuccess = false
while (!isSuccess) {
try {
await redis
.watch(stateName, function (err) {
if (err) {
logger.error(`Error in watch state: ${err}`)
}
redis.get(stateName, function (err, result) {
if (err) {
logger.error(`Error in get state: ${err}`)
}
let state = JSON.parse(result)
// do some processing
redis.multi()
.set(stateName, JSON.stringify(state))
.exec(function (err, result) {
if (err) {
logger.error(`Error in set state: ${err}`)
}
if (result != null) {
isSuccess = true
}
})
console.log(`isSuccess for ${stateName} `, isSuccess)
})
})
} catch (e) {
logger.error(`Error: ${e}`)
}
}
} catch (e) {
logger.error(`Error: ${e}`)
}
return Promise.resolve(true)
}
This will print out
"isSuccess for stateName false"
"isSuccess for stateName true"
"isSuccess for stateName true"
So after the flag changes to true, it will continue for more loops. Sometimes it does more than just once.
Am I doing something wrong?
You cannot mix a synchronous loop (while (!isSuccess) { ... }) with asynchronous functions (redis.watch(stateName, function (err) { ... })).
You can also not await callback-based asynchronous functions. A function must return a promise to be awaitable. Since node-redis gives you promises when you don't pass callbacks to its methods, the key is not to do that (redis.watch(stateName, function (err) { ... }) → redis.watch(stateName)).
Your approach needs to be redone.
Let's make a function that encapsulates a redis transaction with optimistic locking. It takes a connection object, a key, and a value-transforming function, and it returns the result of the .set() operation:
const redisTransaction = async (client, key, transformer) => {
// https://github.com/redis/node-redis/blob/master/docs/isolated-execution.md
return client.executeIsolated(async isolatedClient => {
await isolatedClient.watch(key);
const val = await isolatedClient.get(key);
return isolatedClient.multi()
.set(key, await transformer.call(isolatedClient, val))
.exec();
});
};
Now you can await this function, because it returns a promise. That means we can make a simple infinite loop that exits immediately in case of success (via return), or retries indefinitely.
export async function updateRedis(key, transformer) {
while (true) {
try {
return await redisTransaction(redis, key, transformer);
} catch (err) {
logger.error(`Error for state ${key}: ${err}`);
}
}
}
A transformer function takes a value, and returns a new value. Inside it, the this keyword refers to the isolatedClient from the transaction, which could be useful if your transformation depends on other values from that client.
const result = await updateRedis('stateName', async function (val) {
const state = JSON.parse(val);
const newState = await modifyStateSomehow(state);
return JSON.stringify(newState);
});
The modifyStateSomehow() can itself be an asynchronous (i.e. "promise-returning") function. If it's not, you can make the state transformer a regular function by removing the async and the await.
Related
I connect to a Postgres database in Node and I get a return value from the database. However, when I try to pass that value to another function, the value is undefined. Check out this code:
async function doDbCall(queryString) {
await client.connect()
.then(async () => {
await client.query(queryString)
.then(dbRes => {
console.log("dbDoCall - dbRes", dbRes.rows[0]);
return dbRes;
})
});
}
Here is the calling function:
async function testTheDb() {
try {
const myReturnVal = await dbConn.doDbCall("SELECT NOW() as now");
console.log("db val:", myReturnVal);
}
catch(ex) {
console.log("db err", ex.message);
}
}
In the console.log in doDbCall() on the query.then success, I get a value in dbRes.rows[0]. The console.log shows dbDoCall - dbRes { now: 2022-09-26T16:47:14.465Z }. So I return that value.
However, in the testTheDb() function, I have another console log where I log the value of myReturnVal. The result of that log shows db val: undefined.
My question is this: Why is myReturnVal null in testTheDb()?
You should not mix async/await syntax with .then() calls. Your problem is that neither the doDbCall function nor the async () => {…} callback do return anything, your only return keyword is nested inside a nested then callback.
You should write either
function doDbCall(queryString) {
return client.connect()
.then(() => {
return client.query(queryString);
})
.then(dbRes => {
console.log("dbDoCall - dbRes", dbRes.rows[0]);
return dbRes;
})
.finally(() => {
client.end();
});
}
or
async function doDbCall(queryString) {
try {
await client.connect()
const dbRes = await client.query(queryString);
console.log("dbDoCall - dbRes", dbRes.rows[0]);
return dbRes;
} finally {
client.end();
}
}
As part of my personal project that makes use of Azure AI APIs and Node.js/Express, I'm trying to respond to a get request to a /viewText route with text and key phrases extracted from an uploaded image/document.
Here's the code that should log that data to the console:
app.get(`/viewText`, async (req, res) => {
const answerReadResult = await readOperation(`${__dirname}\\uploads\\answer`);
const markReadResult = await readOperation(`${__dirname}\\uploads\\mark`);
Promise.all([
readOperation(`${__dirname}\\uploads\\answer`),
keyPhraseExtractor(answerReadResult),
readOperation(`${__dirname}\\uploads\\mark`),
keyPhraseExtractor(markReadResult),
]).then((data) => {
console.log("data from Promise.all: ", data);
});
});
I expected to receive two strings from the readOperation function and an array of strings from the keyPhraseExtractor, but all I get is: data from Promise.all: [ undefined, [], undefined, [] ]
Here's the code for the readOperation function(which takes an image and returns the identifiable text as an array of strings)
const readOperation = async (path) => {
fs.readdir(path, (err, files) => {
if (err) console.log(err);
files.forEach(async (file) => {
try {
const results = await getTextFromImage(
`${__dirname}\\uploads\\answer\\${file}`
);
console.log("data from readOperation: ", results.join(" "));
return results;
} catch (err) {
console.error(err);
}
});
});
};
and the log is: data from readOperation: you must be the change you want to see in the world !
N.B: this logged data is what I'm trying to return from this function (without the join("")) but I'm getting undefined as is apparent from the Promise.all
The keyPhraseExtractor function (shown below) expects an array of strings and should return the key phrases in those strings (also as an array of strings)
const keyPhraseExtractor = async (dataFromReadOperation) => {
let results = dataFromReadOperation;
try {
const data = await keyPhraseExtraction(textAnalyticsClient, results);
console.log("data inside keyPhraseExtractor: ", data);
return data;
} catch (err) {
console.error(err);
}
};
The log is data inside keyPhraseExtractor: []
readOperation is wrong. Within the async function you're kicking off readDir and passing it a callback, but nothing in the async function waits for the callback value (so readDir is likely not done by the time it returns), and there's no return in readOperation, so it absolutely returns undefined.
Yes, you have return results, but that's within the forEach async arrow function. There's nothing that would pass that out to the enclosing promise that readOperation returns.
// This is not async, because it's returning a new Promise anyway.
const readOperation = /* NOT ASYNC */ (path) => {
return new Promise((resolve, reject) => {
fs.readdir(path, (err, files) => {
if (err) {
console.log(err);
reject(err); // Outer. Don't forget to stop here.
}
// Resolve the outer promise with an inner promise
// from Promise.all, so getTextFromImage can run in
// parallel. Also use `map` instead of `forEach` so the
// return value is an array of Promises (due to the async).
resolve(Promise.all(files.map(async (file) => {
try {
const results = await getTextFromImage(
`${__dirname}\\uploads\\answer\\${file}`
);
console.log("data from readOperation: ", results.join(" "));
return results.join(" "); // each file's return should
// be a string, right?
} catch (err) {
console.error(err); // Inner.
throw err; // Re-throw so it rejects the promise.
}
})));
});
});
};
That said, I'd suggest using the fs Promises API so you're not mixing err-first callback APIs and async functions.
// This is async, because you can await the Promises inside.
const readOperation = async (path) => {
let files;
try {
files = await fsPromises.readdir(path);
} catch (err) {
console.log(err); // Outer.
throw err;
}
// Still use Promise.all here, so the mapping can happen in parallel.
// You could await this, but async functions that return Promises will
// unwrap the results anyway: it only affects where rejections come from.
return Promise.all(files.map(async (file) => {
try {
const results = await getTextFromImage(
`${__dirname}\\uploads\\answer\\${file}`
);
console.log("data from readOperation: ", results.join(" "));
return results.join(" ");
} catch (err) {
console.error(err); // Inner.
throw err; // Re-throw so it rejects the promise.
}
});
};
I'm trying to build an object based on the responses from multiple promises, but I'm noticed where only the first promise is actually being worked on while the second is being ignored. What is the best practice to make this work?
if (goldenListKeys[0].name === 'date') {
const date = moment('07-01-2019', 'YYYY-MM-DD').format('MM/DD/YYYY');
_.assign(tmpObj, { inputData: { [goldenListKeys[0].name]: date } });
try {
await this.plansApi
.compileFields({ tmpObj, carrier, benefitQuery })
.catch(error => {
value = error.response['invalid-selection'];
console.log(`One: ${value}`);
});
} catch (err) {}
}
if (goldenListKeys[1].name === 'state') {
console.log('Here');
_.assign(tmpObj, {
inputData: { ...tmpObj, [goldenListKeys[1].name]: 'NC' },
});
try {
await this.plansApi
.compileFields({ tmpObj, carrier, benefitQuery })
.catch(error => {
value = error.response['invalid-selection'];
_.assign(goldenListKeys, { filler: value });
console.log(`Two: ${value}`);
});
} catch (err) {}
}
It appears that you missed one of the fundamental features of async/await.
An async function can contain an await expression that pauses the execution of the async function and waits for the passed Promise's resolution, and then resumes the async function's execution and returns the resolved value.
The key part being is that the execution inside the function call is paused. So your next if statement won't be considered until after the first promise has resolved.
from https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
This is usually where I like to use the Promise.all function MDN Link
This is how I would modify your code (I know it doesn't use async/await but, it should accomplish your goal)
function yourFunction(){
let myPromiseArray = [];
if (goldenListKeys[0].name === 'date') {
const date = moment('07-01-2019', 'YYYY-MM-DD').format('MM/DD/YYYY');
_.assign(tmpObj, { inputData: { [goldenListKeys[0].name]: date } });
myPromiseArray.push(his.plansApi
.compileFields({ tmpObj, carrier, benefitQuery }))
}
if (goldenListKeys[1].name === 'state') {
_.assign(tmpObj, {
inputData: { ...tmpObj, [goldenListKeys[1].name]: 'NC' },
});
myPromiseArray.push(this.plansApi
.compileFields({ tmpObj, carrier, benefitQuery }))
}
Promise.all(myPromiseArray).then((resultArray)=>{
//do something with results
}).catch(errorArray => {
//handle Errors
})
}
I want to test the following storeCache function.
The critical part I want to test is the callback function in Meteor.call(...).
I want to mock Meteor.call(...) but it is wrapped by a Promise and the callback itself also relies on the wrapping Promise.
export async function storeCache(cache) {
// do something
return new Promise((resolve, reject) => {
Meteor.call("transferCache", cache, async (error, result) => {
if (error) {
reject(error);
} else {
try {
const result = await persistCache();
resolve(result)
} catch (e) {
reject(e);
}
}
});
});
}
What is the best way to test the defined callback function via Jest?
Or is there a better way to structure the Code to make it easiert to test?
it should be as simple as
// in your imports
import { Meteor } from 'meteor/meteor'
jest.mock('meteor/meteor', () => {
return {
Meteor: {
call: jest.fn()
}
}
})
//in your test case
Meteor.call.mockImplementation((eventName, cache, callback) => {
// your assertions
// you can call the callback here
// you probably want to mock persistCache too
})
So here's my solution:
The Meteor.call with its promise is in a separate function called helper.transferData().
This removed the necessity for a callback. I can put the logic directly in
storeCache() and mock helper.transferData().
export async function transferData(cache) {
return new Promise((resolve, reject) => {
Meteor.call("transferCache", cache, async (error, result) => {
if (error) {
reject(error);
} else {
resolve(result);
}
});
});
}
export async function storeCache(cache) {
// do something
try{
// call meteor method
const transferResult = await helper.transferData(cache);
// callback logic
const result = await persistCache();
// return Promise for compability reasons
return Promise.Resolve(result);
} catch (e) {
return Promise.Reject(result);
}
I want that my loop waits for the two async calls performAction and checkAssertion before starting evaluating a new performAction continuing its loop. I read I should use the for-of loop, but from the logs I've something like this:
1. action which violates the assertion
2. action from another bot
3. found and assertion violated during checkAssertion
4. found and assertion violated during checkAssertion
How can I force my for-loop to wait?
public async start(duration: number) : Promise<void> {
// init
await this.updateStatus();
// for each round, let bots perform actions
let bots = this._bots;
for (let currentStep = 0; currentStep < duration; currentStep++){
Logger.info({"label": "step", "stepNumber": currentStep});
console.log("Step: " + currentStep);
for (const bot of bots) {
try {
let transactionResult = await bot.performAction();
await this.checkAssertion(transactionResult);
} catch (error) {
let err = (error as Error);
// handle error here
}
}
}
}
Edit:
protected async checkAssertion(transactionResult: any) : Promise<void> {
if (transactionResult != null) {
this.checkTotalBalance().then(result => {
assert(result, "Total balance not equal to totalSupply");
assert(!this.checkSuccessfulOverflow(transactionResult), "Successful transaction execution with overflow");
}).catch( async (error) => {
Logger.info({"label": "assertionFail", "error": error});
await this.updateStatus();
throw error;
});
}
}
I think it's because you're using then(...) and catch(...) inside the function checkAssertion (instead of try ... catch). So, it do its action after all performAction() calls completed. So, rewrite this function as follows:
protected async checkAssertion(transactionResult: any) : Promise<void> {
if (transactionResult != null) {
try {
const result = await this.checkTotalBalance();
assert(result, "Total balance not equal to totalSupply");
assert(!this.checkSuccessfulOverflow(transactionResult), "Successful transaction execution with overflow");
} catch (error) {
Logger.info({ "label": "assertionFail", "error": error });
await this.updateStatus();
throw error;
}
}
}