Develop Reference

Understanding promises async/await using parseCSV, what am I missing?

I feel like I a missing something fundamental here. I simply want to ensure "data" holds the parsed data from parseCSV before I continue to clean/edit it.
What am I missing? I fundamentally understand async / await but I suppose there is something I don't understand when it comes to callbacks and async / await.
const csv = require("csv");
const fs = require("fs");
(async function start() {
const file = importCSV("src/tg.csv");
const data = await parseCSV(file);
console.log(await parseCSV(file)); // why does this print a parser and not the parsed data?
})();
function importCSV(path) {
return fs.readFileSync(path, "utf8");
}
async function parseCSV(file) {
return await csv.parse(file, { columns: true }, async (err, data) => {
if (err) return err;
//console.log(data);
return data;
});
}

Because you are not returning the data. the csv.parse is not a promise so basically the data is being returned to the parent function of the anonymous callback function.
Just think of the implementation of a function who uses callback, how do you implement that?
const a = (cb) => {
const data = getdatafromsomewhere();
cb(data);
}
so if your function is not specifically returning to the parent function the returned data is not referenced from anywhere.
So the easiest way would be to promisify the csv.parse
const util = require('util');
const promiseParse = util. promisify(csv.parse);
function parseCSV(file) {
return promiseParse(file, { columns: true })
}

C# async Task vs Js async Task (node js)

Ok, so I know how to program in C# fairly well and I have started programming in JS recently (node js). To be honest I was in a bot of shock from async calls.
Let's say I have this code in C#:
var t_1 = SomeAsyncTask();
var t_2 = SomeOtherAsyncTask();
Task.WaitAll(t_1, t_2);
var res_1 = t_1.Result;
var res_2 = t_2.Result;
Is there a JS equivalent of this? So far I have managed this:
In User.js:
var express = require("express");
var router = express.Router();
var sqlDo = require("../../js_help/DatabasReq/sqlDo.js");
router.get("/", async function(req, res){
var json = sqlDo.ExecCommand("select * from Users");
res.send(json); //json.recordset
});
module.exports = router;
In sqlDo.js:
module.exports = {
ExecCommand: function(command){
// sql and config are defined before.
sql.connect(config, function () {
var request = new sql.Request();
request.query(command, function (err, recordset) {
if (err) console.log(err)
console.log(recordset.recordset);
return recordset;
});
});
}
};
My problem is that this code is running async. I have tried putting await to different places but nothing worked. So when I start my server it returns nothing. I can tell that it is completing a call because I let it read results into console.
Thanks for any help!
Btw: I have tried googling/stackoverlow-ing,.. But I was not able to find anything that would look like C# equivalent. Is it even possible to write it like in c#? Again thanks for every answer...

To make your ExecCommand function async, you have to make it return a Promise. Read about Promises for instance here
module.exports = {
ExecCommand: function(command){
return new Promise((resolve, reject) => { //return a Promise from the function
sql.connect(config, function () {
var request = new sql.Request();
request.query(command, function (err, recordset) {
if (err) {
reject(err); //if there is an error, reject the Promise
} else {
resolve(recordset); //if no error, resolve the Promise with the result
}
});
});
});
}
};
Depending on your SQL library, it may also support promises already, instead of callbacks
module.exports = {
ExecCommand: function(command) {
return sql.connect(config)
.then(() => {
return new sql.Request().query(command);
})
}
};
or with async/await
module.exports = {
ExecCommand: async function(command) {
await sql.connect(config);
return await new sql.Request().query(command);
}
};
Then you can call this function in the requesthandler either like this
router.get("/", async function(req, res){
try {
var json = await sqlDo.ExecCommand("select * from Users");
res.send(json);
catch (err) {
console.log(err);
res.sendStatus(500);
}
});
or like this
router.get("/", function(req, res){
sqlDo.ExecCommand("select * from Users")
.then(json => { //the promise resolved
res.send(json);
})
.catch(err => { //the promise rejected
res.sendStatus(500);
console.log(err);
});
});
I prefer the second variant. But that may be just my personal opinion ...

Get async result in async.filter() array nodejs

Need to parse some XML files from mass array with file_path values.
Try to use async, fs, xml2js.
When use single string file_path all works perfect. But when I use aync.filter() with array I can't understand how I can return result from xml.parseString()
const fs = require('fs');
const xml2js = require('xml2js');
const async = require('async');
var mass=['/file1.xml','/fil2.xml','/file3.xml',...]
async.filter(mass, async function(file_path, callback){
if(fs.statSync(file_path)['size']>0){
fs.readFileSync(file_path, 'utf8', function(err, data) {
xml.parseString(data, function (err, result) {
console.log(Object.keys(result)[0]);
return result; //need get this result to results array
})
})
}
}, function(err, results) {
console.log(results)
});
Who can understand how it works and what I need to change in my code.
Thanks a lot!

You are trying to map and filter at the same time. Since your filter condition is synchronously available, use the array filter method for that, and then pass that to async.map.
You should then call the callback function, that async.map provides to you, passing it the result. So don't return it, but call the callback.
The readFileSync method does not take a callback like its asynchronous counterpart. It just returns the data.
Also, drop the async keyword, as you are not using the await keyword at all.
async.map(mass.filter((file_path) => fs.statSync(file_path).size > 0),
function(file_path, callback){
var data = fs.readFileSync(file_path, 'utf8');
xml.parseString(data, function (err, result) {
console.log(Object.keys(result)[0]);
callback(null, result);
})
}, function(err, results) {
console.log(results)
});
It should be noted however, that since Node now comes with the Promise API, and even the async/await extension to that, the async module has become much less interesting. Consider using Promises.
const promises = mass.filter(file_path => {
return fs.statSync(file_path).size > 0
}).map(function(file_path) {
return new Promise(resolve => {
const data = fs.readFileSync(file_path, 'utf8');
xml.parseString(data, function (err, result) {
console.log(Object.keys(result)[0]);
resolve(result);
});
});
});
Promise.all(promises).then(results => {
console.log(results);
});

Store fs.stat while looping through files into an array, in Node JS

I'm looping through files in a directory and storing the file details to an array data. The following code populates the array if I don't attempt to run fs.stat to get things like the file create/edit date:
fs.readdir('../src/templates', function (err, files) {
if (err) {
throw err;
}
var data = [];
files
.forEach(function (file) {
try {
fs.stat('../src/templates/'+file,(error,stats) => {
data.push({ Name : file,Path : path.join(query, file) });
});
} catch(e) {
console.log(e);
}
});
res.json(data);
});
});
If I move the data.push(...) outside the fs.stat the array returns with the file data. Inside the fs.stat it returns empty. I assume this is an asynchronous issue in that the for loop is running and finishing before fs.stat runs.
I'm thinking I need to use a promise here but unsure.

If you want or need to be asynchronous:
const fs = require("fs");
const path = require("path");
const { promisify } = require("util");
const asyncStat = promisify(fs.stat);
fs.readdir('../src/templates', async function(err, files) {
if (err) {
throw err;
}
const data = await Promise.all(files.map(async function(file) {
try {
const stats = await asyncStat('../src/templates/' + file);
return { Name: file, Path: path.join(query, file), stats };
} catch (e) {
console.log(e);
}
}));
res.json(data);
});
Note that I used map instead of forEach and then awaited all Promises (async makes function return a promise).
I also needed to change fs.stat to use promise with util.promisify.

You're right about the issue being in the asynchronous call. You could use a promise, or you could use fs.statSync(...), which returns a fs.Stats object and operates synchonously.
files.forEach(function (file) {
try {
var fileStats = fs.statSync('../src/templates/' + file);
data.push({
Name : file,
Path : path.join(query, file)
});
} catch(e) {
console.log(e);
}
});

Returning a value after writing to a file is complete

Hi I am writing a function in Node JS for which I have to return a filepath. My problem is in that function I am writing to a file and I want after writing to a file is finished then my return should work. Before looking into code, I know this can be duplicate and I have really did a research on this but I am not just being able to get there. I have tried using callback but the problem is I want to return a value which is already defined. So, before making any judgement calls for duplicate or lack of research, please read the code.
Also, tried to return value in fs.append callback but still did not solved.
My function:
const fs = require('fs');
const path = require('path');
module.exports.createDownloadFile = (request) => {
let filePath;
if (request) {
const userID = xyz;
filePath = path.join(__dirname, userID.concat('.txt'));
fs.open(filePath, 'w', (err) => {
if (err) throw new Error('FILE_NOT_PRESENT');
fs.appendFile(filePath, 'content to write');
});
}
return filePath;
};
I am getting the filePath where I am calling function, it's just at that time file is empty that is why I want to return after file is written completely.

Promises allow you to structure code and return values more like traditional synchronous code. util.promisify can help promisify regular node callback functions.
const fs = require('fs')
const path = require('path')
const fsAppendFileAsync = util.promisify(fs.appendFile)
const fsOpenAsync = util.promisify(fs.open)
module.exports.createDownloadFile = async (request) => {
if (!request) throw new Error('nope')
const userID = xyz
let filePath = path.join(__dirname, userID.concat('.txt'))
let fd = await fsOpenAsync(filePath, 'w')
await fsAppendFileAsync(fd, 'content to write')
return filePath
};
Note that async/await are ES2017 and require Node.js 7.6+ or Babel.
Opening a file with w creates or truncates the file and promises will reject on errors that are thrown so I've left the error handler out. You can use try {} catch (e) {} blocks to handle specific errors.
The Bluebird promise library is helpful too, especially Promise.promisifyAll which creates the promisified Async methods for you:
const Promise = require('bluebird')
const fs = Promise.promisifyAll(require('fs'))
fs.appendFileAsync('file', 'content to write')

use promises like this :
const fs = require('fs');
const path = require('path');
module.exports.createDownloadFile = (request) => {
return new Promise((resolve, reject) => {
let filePath;
if (request) {
const userID = xyz;
filePath = path.join(__dirname, userID.concat('.txt'));
fs.open(filePath, 'w', (err) => {
if (err) reject(err);
else
fs.appendFile(filePath, 'content to write', (err) => {
if (err)
reject(err)
else
resolve(filePath)
});
});
}
});
};
and call it like this :
createDownloadFile(requeset).then(filePath => {
console.log(filePath)
})
or use sync functions without Promises:
module.exports.createDownloadFile = (request) => {
let filePath;
if (request) {
const userID = xyz;
filePath = path.join(__dirname, userID.concat('.txt'));
fs.openSync(filePath,"w");
fs.appendFileSync(filePath, 'content to write');
}
return filePath;
};