I feel like I a missing something fundamental here. I simply want to ensure "data" holds the parsed data from parseCSV before I continue to clean/edit it.
What am I missing? I fundamentally understand async / await but I suppose there is something I don't understand when it comes to callbacks and async / await.
const csv = require("csv");
const fs = require("fs");
(async function start() {
const file = importCSV("src/tg.csv");
const data = await parseCSV(file);
console.log(await parseCSV(file)); // why does this print a parser and not the parsed data?
})();
function importCSV(path) {
return fs.readFileSync(path, "utf8");
}
async function parseCSV(file) {
return await csv.parse(file, { columns: true }, async (err, data) => {
if (err) return err;
//console.log(data);
return data;
});
}
Because you are not returning the data. the csv.parse is not a promise so basically the data is being returned to the parent function of the anonymous callback function.
Just think of the implementation of a function who uses callback, how do you implement that?
const a = (cb) => {
const data = getdatafromsomewhere();
cb(data);
}
so if your function is not specifically returning to the parent function the returned data is not referenced from anywhere.
So the easiest way would be to promisify the csv.parse
const util = require('util');
const promiseParse = util. promisify(csv.parse);
function parseCSV(file) {
return promiseParse(file, { columns: true })
}
Related
In node.js I am trying to get a list of bid and ask prices from a trade exchange website (within an async function). Within the foreach statement I can console.info() the object data(on each iteration) but when I put all of this into an array and then return it to another function it passes as 'undefined'.
const symbolPrice = async() => {
let symbolObj = {}
let symbolList = []
await bookTickers((error, ticker) => {
ticker.forEach(symbol => {
if (symbol.symbol.toUpperCase().startsWith(starts.toUpperCase())) {
symbolObj = {
symbol: symbol.symbol,
bid: symbol.bidPrice,
ask: symbol.askPrice
}
console.info(symbolObj);
}
symbolList.push(symbolObj)
});
const results = Promise.all(symbolList)
return results;
});
}
const symbolPriceTest = async() => {
const res = await symbolPrice(null, 'ETH', true);
console.log(res)
}
I have tried pretty much everything I can find on the internet like different awaits/Promise.all()'s. I do admit I am not as familiar with async coding as I would like to be.
So, if the basic problem here is to call bookTickers(), retrieve the asynchronous result from that call, process it and then get that processed result as the resolved value from calling symbolPrice(), then you can do that like this:
const { promisify } = require('util');
const bookTickersP = promisify(bookTickers);
async function symbolPrice(/* declare arguments here */) {
let symbolList = [];
const ticker = await bookTickersP(/* fill in arguments here */);
for (let symbol of ticker) {
if (symbol.symbol.toUpperCase().startsWith(starts.toUpperCase())) {
symbolList.push({
symbol: symbol.symbol,
bid: symbol.bidPrice,
ask: symbol.askPrice
});
}
}
return symbolList;
}
async function symbolPriceTest() {
const res = await symbolPrice(null, 'ETH', true);
console.log(res)
}
Things to learn from your original attempt:
Only use await when you are awaiting a promise.
Only use Promise.all() when you are passing it an array of promises (or an array of a mixture of values and promises).
Don't mix plain callback asynchronous functions with promises. If you don't have a promise-returning version of your asynchronous function, then promisify it so you do (as shown in my code above with bookTickersP().
Do not guess with async and await and just throw it somewhere hoping it will do something useful. You MUST know that you're awaiting a promise that is connected to the result you're after.
Don't reuse variables in a loop.
Your original implementation of symbolPrice() had no return value at the top level of the function (the only return value was inside a callback so that just returns from the callback, not from the main function). That's why symbolPrice() didn't return anything. Now, because you were using an asynchronous callback, you couldn't actually directly return the results anyway so other things had to be redone.
Just a few thoughts on organization that might be reused in other contexts...
Promisify book tickers (with a library, or pure js using the following pattern). This is just the api made modern:
async function bookTickersP() {
return new Promise((resolve, reject) => {
bookTickers((error, ticker) => {
error ? reject(error) : resolve(ticker);
})
});
}
Use that to shape data in the way that the app needs. This is your app's async model getter:
// resolves to [{ symbol, bid, ask }, ...]
async function getSymbols() {
const result = await bookTickersP();
return result.map(({ symbol, bidPrice, askPrice }) => {
return { symbol: symbol.toUpperCase(), bid: bidPrice, ask: askPrice }
});
}
Write business logic that does things with the model, like ask about particular symbols:
async function symbolPriceTest(search) {
const prefix = search.toUpperCase();
const symbols = await getSymbols();
return symbols.filter(s => s.symbol.startsWith(prefix));
}
symbolPriceTest('ETH').then(console.log);
I used Papa Parse to parse a .csv file, and pushed the result to an empty array called parsed_data. I am able to use console.log(parsed_data), and view the arrays produced. However, when I try to index the data, for example, console.log(parsed_data[0]), the result is undefined. Not sure what's going wrong here.
Example code:
let parsed_data = [];
const data_url = "acdata.csv";
async function getData() {
const response = await fetch(data_url);
const blob = await response.blob();
const data = Papa.parse(blob, {
complete: function(results) {
//console.log("Finished:", results.data);
parsed_data.push(results.data);
}
});
};
console.log(parsed_data);
getData();
Since Papa parse complete callback is called asynchronously, you'll need to wait for that to complete - however, papa parse doesn't seem to use Promises, so, you can "promisify" the parse function like so
const papaParsePromise = blob => new Promise(resolve => Papa.parse(blob, { complete: resolve }));
Another way of looking at that function, if you don't understand the => notation, is
function papaParsePromise(blob) {
return new Promise(function(resolve) {
Papa.parse(blob, {
complete: function(data) {
resolve(data);
}
);
});
}
that returns a promise that resolves to the data that is passed to the complete callback
Your code would also need to wait for the promise returned by getData before it can use anything in that data. Unless your code is inside another async function, you'll need to use promise .then method, as below
const data_url = "acdata.csv";
async function getData() {
// create a function that returns a Promise that resolves when papa parse complete is called
const papaParsePromise = blob => new Promise(resolve => Papa.parse(blob, { complete: resolve }));
const response = await fetch(data_url);
const blob = await response.blob();
const data = await papaParsePromise(blob);
return data;
};
getData()
.then(parse_data => {
console.log(parsed_data);
console.log(parsed_data[0]);
// i.e. do what you need with parsed_data here
});
if, however, you are calling getData inside an async function - along with the changes to getData above, you can simply use await getData() to wait for the value - i.e.
async function someFunction() {
const parsed_data = await getData();
// do what you need with parsed_data here
}
I read this article from node
https://nodejs.org/en/docs/guides/blocking-vs-non-blocking/
It says the code below is a process blocker:
const fs = require('fs');
const data = fs.readFileSync('/file.md'); // blocks here until file is read
console.log(data);
// moreWork(); will run after console.log
what if I add await?
will the code above becomes non-blocking or it will stay in its true nature?
Example code:
const fs = require('fs');
const data = await fs.readFileSync('/file.md'); // no more blocking
console.log(data);
Thank you
No, the code can't run since await must use in async function.
And await should use for function that return promise.
the code means:
// await new Promise(...)
// console.log(...)
new Promise().then((...) => console.log(...))
If you should non-block function, you should use fs.readFile instead.
Blocking means that the whole application is blocked.
So, all SetInterval, Promise, Events or whatever async callback are paused until that sync function ends its execution.
It's the same of what you get when you use a for..loop.
NodeJS provides you some non-blocking file system methods with callbacks, just change your code like this.
const fs = require('fs');
fs.readFile('/file.md', (err, data) => {
if (err) throw err;
console.log(data)
});
There is the only way.
await operator wait a promise, and wrapped in async function
you should code like this
const fs = require("fs");
function readFile(fileName) {
return new Promise((resolve, reject) => {
fs.readFile(fileName, (err, data) => {
if (err) reject(err);
resolve(data);
});
});
}
async function f1() {
try {
var x = await readFile("foo.json");
console.log(x);
} catch (e) {
console.log(e); // 30
}
}
f1();
I am trying to understand async calls using async/await and try/catch.
In the example below, how can I save my successful response to a variable that can be utilized throughout the rest of the code?
const axios = require('axios');
const users = 'http://localhost:3000/users';
const asyncExample = async () =>{
try {
const data = await axios(users);
console.log(data); //200
}
catch (err) {
console.log(err);
}
};
//Save response on a variable
const globalData = asyncExample();
console.log(globalData) //Promise { <pending> }
1) Return something from your asyncExample function
const asyncExample = async () => {
const result = await axios(users)
return result
}
2) Call that function and handle its returned Promise:
;(async () => {
const users = await asyncExample()
console.log(users)
})()
Here's why should you handle it like this:
You can't do top-level await (there's a proposal for it though);
await must exist within an async function.
However I must point out that your original example doesn't need async/await
at all; Since axios already returns a Promise you can simply do:
const asyncExample = () => {
return axios(users)
}
const users = await asyncExample()
try..catch creates a new block scope. Use let to define data before try..catch instead of const, return data from asyncExample function call
(async() => {
const users = 123;
const asyncExample = async() => {
let data;
try {
data = await Promise.resolve(users);
} catch (err) {
console.log(err);
}
return data;
};
//Save response on a variable
const globalData = await asyncExample();
console.log(globalData);
// return globalData;
})();
I had same issue with you and found this post. After 2 days of trying I finally found a simple solution.
According to the document of JS, an async function will only return a Promise object instead of value. To access the response of Promise, you have to use .then()method or await which can return the resulting object of Promise is instead of Promise itself.
To change variables from await, you have access and change the variable you want to assign within the async function instead of return from it.
//Save response on a variable
var globalData;
const asyncExample = async () =>{
try {
const data = await axios(users);
globalData = data; // this will change globalData
console.log(data); //200
}
catch (err) {
console.log(err);
}
};
asyncExample();
But if you do this, you may get an undefined output.
asyncExample();
console.log(globalData) //undefined
Since asyncExample() is an async function, when console.log is called, asyncExample() has not finished yet, so globalData is still not assigned. The following code will call console.log after asyncExample() was done.
const show = async () => {
await asyncExample();
console.log(globalData);
}
show();
Because the events are happening asynchronously you need to tie in a callback/promise. I'm going to assume it returns a promise.
const axios = require('axios');
const users = 'http://localhost:3000/users';
const asyncExample = async () =>{
try {
const data = await axios(users);
console.log(data); //200
}
catch (err) {
console.log(err);
}
};
//Save response on a variable
const globalData = asyncExample().then( (success, err) => {
if (err) { console.error(err); }
console.log(success)
}
Just use a callback/promise (cascading programming):
axios(users).then(function(response) {
const globalData = response;
console.log(globalData)
});
I've node project.
Root file is index.js and file helper.js, here I've some helper functions and it imported to index.js.
I'm trying to get some data, using function in helper.js, but when I calling it in index.js it returning undefined.
But in helper.js everething is OK, console.log showing data that I need.
How I can fix this problem?
index.js file content:
const helper = require('./helper');
let data = helper.getData();
console.log(data); // undefined
helper.js file content:
const fs = require('fs');
module.exports = {
getData: () => {
fs.readFile('data.json', 'utf8', (err, data) => {
const allData = JSON.parse(data);
console.log(allData); // IS OK!
return allData;
});
}
}
You can use Promise:
const fs = require('fs');
module.exports = {
getData: () => {
return new Promise(function(resolve, reject){
fs.readFile('data.json', 'utf8', (err, data) => {
if(err){
reject(err);
} else {
try {
resolve(JSON.parse(data));
} catch(ex){
reject(ex);
}
}
});
});
}
}
and then:
helper.getData().then(function(data){
console.log(data);
}, function(err){
// here something failed
});
The problem is that fs.readFile method is asynchronous and will not give you as result any data check the documentation here.
So one option is to use a Promise as I did or to use a callback as suggested in the answer of #Tatsuyuki Ishi, you can check the docs about callback implementation.
The problem is that fs.readFile is an asynchronous function and so doesn't return anything.
If you really need it to return something you can use the synchronous version, fs.readFileSync.
Otherwise - and a better way to do it - would be to have getData return a promise that you can then resolve with allData.
readFile is an asynchronous function, which accepts a callback. You have two options:
1 . Get a callback as parameter in getData().
getData: (callback) => {
fs.readFile('data.json', 'utf8', (err, data) => {
const allData = JSON.parse(data);
console.log(allData); // IS OK!
callback(allData);
});
}
2 . Use the synchronous version.
getData: () => {
var data = fs.readFileSync('data.json', 'utf8');
const allData = JSON.parse(data);
console.log(allData); // IS OK!
return allData;
}
Of course, you can use Promise which is more beautiful on chaining things, but it's often used with dependencies like Bluebird.
The problem is, you are returning allData from the callback function, not the getData function. And since getData has no explicit return, your helper.getData() function will return undefined and this value would printed instead of what you wanted.
I suggest using Promise to return the data properly, as in #sand's answer.