// The Below code already contains the suggestions from the answers and hence works :)
within the below script I tried to fully execute the 'createDatabase' function before the .then call at the end starts to run. Unfortunately I couldn't figure out a solution to achieve just that.
In generell the flow should be as followed:
GetFiles - Fully Execute it
CreateDatabase - Fully Execute it (while awaiting each .map call to finish before starting the next)
Exit the script within the .then call
Thanks a lot for any advise :)
const db = require("../database")
const fsp = require("fs").promises
const root = "./database/migrations/"
const getFiles = async () => {
let fileNames = await fsp.readdir(root)
return fileNames.map(fileName => Number(fileName.split(".")[0]))
}
const createDatabase = async fileNumbers => {
fileNumbers.sort((a, b) => a - b)
for (let fileNumber of fileNumbers) {
const length = fileNumber.toString().length
const x = require(`.${root}${fileNumber.toString()}.js`)
await x.create()
}
}
const run = async () => {
let fileNumbers = await getFiles()
await createDatabase(fileNumbers)
}
run()
.then(() => {
console.log("Database setup successfully!")
db.end()
process.exitCode = 0
})
.catch(err => {
console.log("Error creating Database!", err)
})
The x.create code looks as follows:
const dbQ = (query, message) => {
return new Promise((resolve, reject) => {
db.query(query, (err, result) => {
if (err) {
console.log(`Error: ${err.sqlMessage}`)
return reject()
}
console.log(`Success: ${message}!`)
return resolve()
})
})
}
x.create = async () => {
const query = `
CREATE TABLE IF NOT EXISTS Country (
Code CHAR(2) UNIQUE NOT NULL,
Flag VARCHAR(1024),
Name_de VARCHAR(64) NOT NULL,
Name_en VARCHAR(64) NOT NULL,
Primary Key (Code)
)`
const result = await dbQ(query, "Created Table COUNTRY")
return result
}
If you want each x.create to fully execute before the next one starts, i.e. this is what I interpret where you say while awaiting each .map call to finish before starting the next - then you could use async/await with a for loop as follows:
const createDatabase = async fileNumbers => {
fileNumbers.sort((a, b) => a - b);
for (let fileNumber of fileNumbers) {
const x = require(`.${root}${fileNumber.toString()}.js`);
await x.create();
})
}
However, this also assumes that x.create() returns a Promise - as you've not shown what is the typical content of .${root}${fileNumber.toString()}.js file is, then I'm only speculating
The other interpretation of your question would simply require you to change createDatabase so that promises is actually an array of promises (at the moment, it's an array of undefined
const createDatabase = async fileNumbers => {
fileNumbers.sort((a, b) => a - b);
const promises = fileNumbers.map(fileNumber => {
const x = require(`.${root}${fileNumber.toString()}.js`);
return x.create();
})
await Promise.all(promises);
}
Now all .create() run in "parallel", but createDatabase only resolves onces all promises returned by x.create() resolve - again, assumes that x.create() returns a promise
Related
I am currently designing a todo app with react and firebase without any node js server code. Everything was fine when inserting data and authenticating, but when I tried to query all the tasks where uid equals a certain value(that means all the tasks that belongs to a particular user), somehow the query gets skipped and returns null. But then, after a second or so, the array got retrieved and printed out.
The code:
function retrieveTasksOfUser(uid) {
// get all tasks where the uid equals the given value
return db.collection("tasks").where("uid", "==", uid).get();
}
function retrieveTasks() {
let res = [];
retrieveTasksOfUser(currentUser["uid"])
.then((snapshot) => {
snapshot.forEach((doc) => {
// include the doc id in each returned object
let buffer = doc.data();
buffer["id"] = doc.id;
res.push(buffer);
});
console.log(res);
return res;
})
.catch((err) => {
console.error("Error retrieving tasks: ", err);
return res;
});
}
let tasks = retrieveTasks();
console.log({ tasks });
EDIT:
Inspired by Frank's answer, I modified my code and got:
async function retrieveTasks() {
let res = [];
const snapshot = await retrieveTasksOfUser(currentUser["uid"]);
snapshot.forEach((doc) => {
let buffer = doc.data();
buffer["id"] = doc.id;
res.push(buffer);
});
return res;
}
let tasks = [];
let promise = retrieveTasks();
promise.then((res) => {
console.log("Tasks are");
tasks = res;
console.log(res);
});
console.log(tasks);
But the result turns out to be an empty array
Thanks a bunch in advance.
You're not returning anything from the top-level code in retrieveTasks, so that means that tasks will always be undefined.
The simplest fix is to return the result of the query, which then means that your return res will be bubbled up:
return retrieveTasksOfUser(currentUser["uid"])
...
But this means you're returning a promise, as the data is loaded asynchronously. So in the calling code, you have to wait for that promise to resolve with then:
retrieveTasks().then((tasks) => {
console.log({ tasks });
})
You can make all of this a lot more readable (although it'll function exactly the same) by using the async and await keywords.
With those, the code would become:
async function retrieveTasks() {
let res = [];
const snapshot = await retrieveTasksOfUser(currentUser["uid"]);
snapshot.forEach((doc) => {
let buffer = doc.data();
buffer["id"] = doc.id;
res.push(buffer);
});
return res;
}
let tasks = await retrieveTasks();
console.log({ tasks });
Or with a bit more modern JavaScript magic, we can make it even shorter:
async function retrieveTasks() {
const snapshot = await retrieveTasksOfUser(currentUser["uid"]);
return snapshot.docs.map((doc) => { ...doc.data, id: doc.id });
}
let tasks = await retrieveTasks();
console.log({ tasks });
This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Closed 2 years ago.
My code looks something like this:
(async () => {
try {
const results = await heavyCalculation();
saveResultsToFiles(results);
} catch (e) {
handleError(e);
} finally {
process.exit(0);
}
})();
const saveResultsToFiles = (results) => {
results.forEach(result => {
(async () => await saveResultFile(result));
})
}
const saveResultFile = (result) => {
return promiseToPreprocess(result)
.then(processedResult => saveToFile(processedResult))
}
const promiseToPreprocess = async (result) => {
// this function returns a promise to preprocess the data
}
const saveToFile = (data) => {
// this function synchronously saves data to a file
}
I thought this code would
perform calculations
wait for each piece of the results to be preprocessed and saved to a file
then exit
The first step works, as the program seems to await the heavy calculation results. However, it appears the finally clause is entered before the promise within the forEach-loop is resolved, causing the program to exit early. What am I doing wrong?
You have two problems here:
Your forEach loop in saveResultsToFiles does not return anything so you have no way to make other parts of your code "wait" for each item's promise to resolve.
saveResultFile returns a promise but this promise is not awaited in your try block.
The result of both of these issues is that the try block only "starts" the process of saving to files but doesn't wait for it to finish before yieling to the finally block.
Here are solutions you could try.
You need to be able to await each of the saveResultFile calls and for that you need to access the array of promises instanciated in saveResultsToFiles. With .map you will actually get an array of results (as opposed to .forEach):
const saveResultsToFiles = (results) => {
return results.map(result => saveResultFile(result));
}
Now that saveResultsToFiles actually returns an array of promises, you should await all of them before proceeding. This is exactly what Promise.all is for:
try {
const results = await heavyCalculation();
await Promise.all(saveResultsToFiles(results));
}
You are not awaiting saveResultsToFiles(results);
Try:
(async () => {
try {
const results = await heavyCalculation();
saveResultsToFiles(results);
} catch (e) {
handleError(e);
} finally {
process.exit(0);
}
})();
const saveResultsToFiles = async (results) => {
results.forEach(result => {
await saveResultFile(result);
})
}
const saveResultFile = (result) => {
return promiseToPreprocess(result)
.then(processedResult => saveToFile(processedResult))
}
const promiseToPreprocess = async (result) => {
// this function returns a promise to preprocess the data
}
const saveToFile = (data) => {
// this function synchronously saves data to a file
}
I have a Firebase Function which sends back data from databases. The problem is sometimes I have to return data all of 3 collections, sometimes only need from 1 and sometimes 2 of them. But this is an antipattern. How can I improve my code?
Right now I'm creating a function, which returns a promise, in which I'm using await for getting db values and this is wrapped in try{} block.
module.exports.getList = (uid, listType) => new Promise(async (resolve, reject) => {
let returnValue = [];
try {
if (listType.contains("a")) {
const block = await db.collection('alist').doc(uid).get();
returnValue.push(block);
}
if (listType.contains("b")) {
const like = await db.collection('blist').doc(uid).get();
returnValue.push(like);
}
if (listType.contains("c")) {
const match = await db.collection('clist').doc(uid).get();
returnValue.push(match);
}
} catch (e) {
return reject(e);
}
return resolve(returnValue);});
How should I modify this snippet in order to not be an antipattern? Or is it not because of the try-catch block?
You can make the getList function async instead, without new Promise or try/catch:
module.exports.getList = async (uid, listType) => {
const returnValue = [];
if (listType.contains("a")) {
const block = await db.collection('alist').doc(uid).get();
returnValue.push(block);
}
if (listType.contains("b")) {
const like = await db.collection('blist').doc(uid).get();
returnValue.push(like);
}
if (listType.contains("c")) {
const match = await db.collection('clist').doc(uid).get();
returnValue.push(match);
}
return returnValue;
};
Calling it will return a Promise that rejects with an error if there's an asynchronous error, or it will resolve to the desired array.
Note that unless there's a good reason to await each call in serial, you can use Promise.all instead, so that the requests go out in parallel, and make the code a lot more concise in the process:
module.exports.getList = (uid, listType) => Promise.all(
['alist', 'blist', 'clist']
.filter(name => listType.contains(name[0]))
.map(name => db.collection(name).doc(uid).get())
);
The function below calls several asynchronous functions in a for loop. It's parsing different CSV files to build a single JavaScript object. I'd like to return the object after the for loop is done. Its returning the empty object right away while it does the asynchronous tasks. Makes sense, however I have tried various Promise / async /await combinations hopes of running something once the for loop has completed. I am clearly not understanding what is going on. Is there a better pattern to follow for something like this or am I thinking about it incorrectly?
async function createFormConfig(files: string[]): Promise<object>
return new Promise(resolve => {
const retConfig: any = {};
for (const file of files) {
file.match(matchFilesForFormConfigMap.get('FIELD')) ?
parseCsv(file).then(parsedData => {
retConfig.fields = parsedData.data;
})
: file.match(matchFilesForFormConfigMap.get('FORM'))
? parseCsv(file).then(parsedData => retConfig.formProperties = parsedData.data[0])
: file.match(matchFilesForFormConfigMap.get('PDF'))
? parseCsv(file).then(parsedData => retConfig.jsPdfProperties = parsedData.data[0])
: file.match(matchFilesForFormConfigMap.get('META'))
? parseCsv(file).then(parsedData => {
retConfig.name = parsedData.data[0].name;
retConfig.imgType = parsedData.data[0].imgType;
// console.log(retConfig); <- THIS CONSOLE WILL OUTPUT RETCONFIG LOOKING LIKE I WANT IT
})
: file.match(matchFilesForFormConfigMap.get('PAGES'))
? parseCsv(file).then(parsedData => retConfig.pages = parsedData.data)
: console.log('there is an extra file: ' + file);
}
resolve(retConfig); // <- THIS RETURNS: {}
});
This is the code I'm using to call the function in hopes of getting my 'retConfig' filled with the CSV data.
getFilesFromDirectory(`${clOptions.directory}/**/*.csv`)
.then(async (files) => {
const config = await createFormConfig(files);
console.log(config);
})
.catch(err => console.error(err));
};
First, an async function returns a Promise, so you dont have to return one explicitely.Here is how you can simplify your code:
async function createFormConfig(files: string[]): Promise<object> {
// return new Promise(resolve => { <-- remove
const retConfig: any = {};
// ...
// The value returned by an async function is the one you get
// in the callback passed to the function `.then`
return retConfig;
// }); <-- remove
}
Then, your function createFormConfig returns the config before it has finished to compute it. Here is how you can have it computed before returning it:
async function createFormConfig(files: string[]): Promise<object> {
const retConfig: any = {};
// Return a Promise for each file that have to be parsed
const parsingCsv = files.map(async file => {
if (file.match(matchFilesForFormConfigMap.get('FIELD'))) {
const { data } = await parseCsv(file);
retConfig.fields = data;
} else if (file.match(matchFilesForFormConfigMap.get('FORM'))) {
const { data } = await parseCsv(file);
retConfig.formProperties = data[0];
} else if (file.match(matchFilesForFormConfigMap.get('PDF'))) {
const { data } = await parseCsv(file);
retConfig.jsPdfProperties = data[0];
} else if (file.match(matchFilesForFormConfigMap.get('META'))) {
const { data } = await parseCsv(file);
retConfig.name = data[0].name;
retConfig.imgType = data[0].imgType;
} else if (file.match(matchFilesForFormConfigMap.get('PAGES'))) {
const { data } = await parseCsv(file);
retConfig.pages = data;
} else {
console.log('there is an extra file: ' + file);
}
});
// Wait for the Promises to resolve
await Promise.all(parsingCsv)
return retConfig;
}
async functions already return promises, you don't need to wrap the code in a new one. Just return a value from the function and the caller will receive a promise that resolves to the returned value.
Also, you have made an async function, but you're not actually using await anywhere. So the for loop runs through the whole loop before any of your promises resolve. This is why none of the data is making it into your object.
It will really simplify your code to only use await and get rid of the then() calls. For example you can do this:
async function createFormConfig(files: string[]): Promise<object> {
const retConfig: any = {};
for (const file of files) {
if (file.match(matchFilesForFormConfigMap.get('FIELD')){
// no need for the then here
let parsedData = await parseCsv(file)
retConfig.field = parsedData.data
}
// ...etc
At the end you can just return the value:
return retConfig
I am trying to use await-async without try-catch for this:
const getUsers = async (reject, time) => (
new Promise((resolve, reject) => {
setTimeout(() => {
if (reject) {
reject(....)
}
resolve(.....);
}, time);
})
);
module.exports = {
getUsers ,
};
With try-catch block it looks like this:
const { getUsers } = require('./users');
const users = async () => {
try {
const value = await getUsers(1000, false);
.....
} catch (error) {
.....
}
}
users();
How can I write the same code without using the try-catch block?
Using the promise functions then-catch to make the process simpler I use this utils :
// utils.js
const utils = promise => (
promise
.then(data => ({ data, error: null }))
.catch(error => ({ error, data: null }))
);
module.exports = utils;
And then
const { getUsers } = require('./api');
const utils = require('./utils');
const users = async () => {
const { error, data } = await utils(getUsers(2000, false));
if (!error) {
console.info(data);
return;
}
console.error(error);
}
users();
Without using the try-catch block I got the same output, this way makes it better to understand the code.
In Extension to L Y E S - C H I O U K H's Answer:
The Utils Function is actually correct but, make sure to add the return keyword before the promise as shown down below:
// utils.js
const utils = promise => (
return promise
.then(data => { [data, null]; })
.catch(error => { [null, error]; });
);
module.exports = utils;
When Calling in Main Code:
let resonse, error; // any variable name is fine make sure there is one for error and the response
[response, error] = await utils(any_function()); // Make sure that inside the tuple, response is first and error is last like: [response, error].
if (error) console.log(error);
// -- Do Whatever with the Response -- //
Using My Method Would Give you Benefits like:
Your Own Variable Names.
Not Running into Type Safety issues when using Typescript.
Good Reason to Strong Type your code.
Personally, I have been using this in my code lately, and has reduced some many headaches, my code is cleaner, I don't have to stick with the same variable names, especially when working on a large codebase.
Happy Coding :)
See Ya!
If you have a valid default for the error case you can use the catch method on the getUsers promise and then await a promise whose error will be handled
const users = async () => {
const value = await getUsers(1000, false).catch(e => null);
}
While this approach should work it should be noted that this may mask the case when getUsers returns null vs when it raises an error, and you will still need to check for the null or get a null access error. All in all I would stick with the try { .. } catch (e) { ... } for most casses
A package I found called await-to-js can also help it.
import to from 'await-to-js';
const [err, users] = await to(getUsers());
if(err) doSomething();
The idea is like Lyes CHIOUKH's method, just a wrapper. Copied the source code here.
/**
* #param { Promise } promise
* #param { Object= } errorExt - Additional Information you can pass to the err object
* #return { Promise }
*/
export function to<T, U = Error> (
promise: Promise<T>,
errorExt?: object
): Promise<[U | null, T | undefined]> {
return promise
.then<[null, T]>((data: T) => [null, data])
.catch<[U, undefined]>((err: U) => {
if (errorExt) {
Object.assign(err, errorExt);
}
return [err, undefined];
});
}
export default to;
If you have such above single line async/await function, then this would have been clean code for you:
const getUsers = async (time, shouldReject=false) => {
return new Promise((resolve, reject) => {
setTimeout(() => {
if (shouldReject) {
reject(Error('Rejected...'));
} else {
resolve(["User1", "User2"]);
}
}, time);
});
}
const userOperation = users => {
console.log("Operating user", users);
}
// Example 1, pass
getUsers(100)
.then(users => userOperation(users))
.catch(e => console.log(e.message));
// Example 2, rejected
getUsers(100, true)
.then(users => userOperation(users))
.catch(e => console.log(e.message));
And for multiple await in a single async function, it would good to have try/catch block as below:
const users = async () => {
try {
const value = await getUsers(1000, false);
const value1 = await getUsers2(1000, false);
...
} catch (error) {
...
}
}