Develop Reference

JS async function returning undefined although calling it inside an asynchronous function

I am using nodeJs Express and I have an async function which is used in another function.
here is my code:
/* get user info */
const getUserInfo = async (req) => {
let userID = req.cookies.userId;
if(!userID) return null;
await connection.query('SELECT * FROM users WHERE id = ' + userID,
function (err, rows, fields) {
if (err) throw err
// if user not found
if (rows.length === 0) {
return null;
}
// if user found
else {
return rows[0];
}
});
}
/* display dashboard page */
router.get("/", async function (req, res, next) {
let userInfo = await getUserInfo(req);
console.log(userInfo) // It's result is undefined
if (userInfo) {
res.render('dashboard/profile', {
fullName: userInfo.fname + " " + userInfo.lname,
email: userInfo.email,
phone: userInfo.phone,
});
}
else {
res.render('authentication/register', {
title: 'ثبت نام',
});
}
});
how to resolve this problem? I need userInfo retun me some data.

await is only useful if the value on the right-hand side is a promise.
connection.query is taking a callback function, which implies it isn't returning a promise.
You need to either:
Find out how to make the API you are using return a promise (if that is possible)
Wrap the callback in a promise
Replace the API with one that supports promises natively
You also need getUserInfo to have a return statement of its own.

You have to return some value from your getUserInfo function
If connection query doesn't support promise you should wrap it like this
/* get user info */
const getUserInfo = async(req) => {
let userID = req.cookies.userId;
if (!userID) return null;
return new Promise((resolve, reject) => {
connection.query('SELECT * FROM users WHERE id = ' + userID,
function(err, rows, fields) {
if (err) {
reject(err)
return;
}
// if user not found
if (rows.length === 0) {
resolve(null)
}
// if user found
else {
resolve(rows[0]);
}
});
});
}

s3.getObject not working from dev environment

This is my code, which works fine if i run it from my local using local aws account , but it doesn't work from my dev environment. S3.getobject api doesnt get executed and code prints the next log skipping the getobject call :
const unzipFromS3 = (key) => {
return new Promise(async (resolve, reject) => {
log.info("inside unzipfroms3");
var zlib = require('zlib');
// let fileName = _.replace(key, 'Root/', '');
let options = {
'Bucket': config.bucketName,
'Key': "Root/" + key,
}
log.info("Key:", options);
await s3.getObject(options).on('error', error => {
log.error(error) }).promise().then((res) => {
yauzl.fromBuffer(res.body, { lazyEntries: true }, function (err, zipfile) {
log.info("Inside Yauzl")
if (err) throw err;
zipfile.readEntry();
zipfile.on("entry", function (entry) {
if (/\/$/.test(entry.fileName)) {
zipfile.readEntry();
} else {
zipfile.openReadStream(entry, function (err, readStream) {
if (err) throw err;
// readStream.pipe(fs.createWriteStream(`result/${entry.fileName}`));
readStream
.pipe(uploadFromStream(s3));
function uploadFromStream(s3) {
log.info("Inside uploadFromStream")
var pass = new Stream.PassThrough();
let options = {
'Bucket': config.bucketName,
'Key': entry.fileName,
}
var params = { ...options, Body: pass };
s3.upload(params, function (err, data) {
log.error(err, data);
});
return pass;
}
readStream.on("end", function () {
zipfile.readEntry();
});
});
}
});
});
});
});
};

In order to use await, i.e. the promised based version of S3.getObject(), you must add the promise() method to your method call as explained in the Using JavaScript Promises chapter of the AWS SDK developer guide. Moreover, there is also an Using async/await chapter that you can look into.
In your case, the code can be modified to something like:
await s3.getObject(options).promise()
.then((res) => {
yauzl.fromBuffer(/* more code */);
});

Why would the same exact Firebase Function take 10x as long to run using a schedule/onRun trigger than a HTTP onRequest trigger?

I have a 2 identical Firebase functions that batch write data to Firestore. One is wrapped in a scheduled/onRun trigger, and the other is a HTTP onRequest trigger.
Both functions work fine and throw no errors.
They have the same amount of memory and timeout as well.
When invoking the http trigger, the function runs through and completes in about 30 seconds.
When invoking the scheduled onRun trigger, the function takes 5+ minutes to complete.
Is there something different about the runtimes that is not documented or something?
Edit: It works now - I made processMentions await totalMentions and return null.
processMentions does not have to return a promise, only a value because the actual scheduledPull/onRun function is returning the processMentions async function, which resolves the promise by returning a value.
Cheers for the help #dougstevenson
Triggers:
/**
* Get manual mentions
*/
exports.get = functions.https.onRequest((req, res) => {
const topic = 'topic'
const query = 'queryString'
processMentions(res, query, topic)
})
/**
* Get schedule mentions
*/
exports.scheduledPull = functions.pubsub.schedule('every day 1:00').onRun((context) => {
const topic = 'topic'
const query = 'queryString'
return processMentions('sched', query, topic)
})
Logic:
const functions = require('firebase-functions')
const admin = require('firebase-admin')
admin.initializeApp()
const db = admin.firestore()
const axios = require('axios')
const moment = require('moment')
// Globals
const auth = 'token'
const url = 'https://apiurl.com/'
async function totalMentions(nextPage, start, end, query) {
try {
let config = {
headers: {
Authorization: auth,
Accept: 'text/html',
}
}
const response = await axios.get(url, config)
const total = response.data.results.total
const loops = Math.ceil(total / 500)
return loops
} catch (error) {
console.log('error 1', error)
}
}
async function allMentions(nextPage, start, end, query) {
try {
let config = {
headers: {
Authorization: auth,
Accept: 'text/html',
},
}
const response = await axios.get(url, config)
return response
} catch (error) {
console.log('error 2', error)
}
}
async function saveData(response, end, topic) {
try {
let data = await response.data.results.clips
let batch = db.batch()
data.forEach((c) => {
delete c.localTime
let reff = db.collection(collection).doc(date).collection(collection).doc(c.id.toString())
batch.set(reff, c)
})
let batches = await batch.commit()
return batches
} catch (error) {
console.log('error3 ', error)
}
}
async function processMentions(res, query, topic) {
try {
totalMentions(1, start, end, query)
.then(async (loops) => {
let endbatch = 0
for (let i = 1; i <= loops; i++) {
await allMentions(i, start, end, query)
.then(async (response) => {
await saveData(response, end, topic)
return ++endbatch
})
.catch((err) => {
console.log('error 4 ' + err)
})
if (endbatch === loops) {
if (res !== 'sched') {
console.log('http trigger finished')
return res.status(200).end()
} else {
return console.log('schedule finished')
}
}
}
})
.catch((err) => {
console.log('error5 ' + err)
})
} catch (error) {
console.log('error6 ' + error)
}
}

For the pubsub trigger to work correctly, processMentions needs to return a promise that resovles when all of the async work is complete. Right now, it's returning nothing, which (since it's declared async) translates into a promise that's resolved immediately with no value. Calling then/catch on a promise isn't doing what you expect - you need to return a promise chain from your async work.
I'm not sure why you have it declared async, without also using await inside of it to manage the promises much more easily.

How to achieve recursive Promise calls in Node.js

I am calling an API where I can only fetch 1000 records per request,
I was able to achieve this using recursion.
I am now trying to achieve the same using promises, I am fairly new to Node.js and JavaScript too.
I tried adding the recursion code in an if else block but failed
var requestP = require('request-promise');
const option = {
url: 'rest/api/2/search',
json: true,
qs: {
//jql: "project in (FLAGPS)",
}
}
const callback = (body) => {
// some code
.
.
.//saving records to file
.
//some code
if (totlExtractedRecords < total) {
requestP(option, callback).auth('api-reader', token, true)
.then(callback)
.catch((err) => {
console.log('Error Observed ' + err)
})
}
}
requestP(option).auth('api-reader', token, true)
.then(callback)
.catch((err) => {
console.log('Error Observed ' + err)
})
I want to execute the method using promise and in a synchronous way,
i.e. I want to wait until the records are all exported to a file and continue with my code

I think its better to create your own promise and simply resolve it when your done with your recursion. Here's a simply example just for you to understand the approach
async function myRecursiveLogic(resolveMethod, ctr = 0) {
// This is where you do the logic
await new Promise((res) => setTimeout(res, 1000)); // wait - just for example
ctr++;
console.log('counter:', ctr);
if (ctr === 5) {
resolveMethod(); // Work done, resolve the promise
} else {
await myRecursiveLogic(resolveMethod, ctr); // recursion - continue work
}
}
// Run the method with a single promise
new Promise((res) => myRecursiveLogic(res)).then(r => console.log('done'));

Here's a clean and nice solution using the latest NodeJS features.
The recursive function will continue executing until a specific condition is met (in this example asynchronously getting some data).
const sleep = require('util').promisify(setTimeout)
const recursive = async () => {
await sleep(1000)
const data = await getDataViaPromise() // you can replace this with request-promise
if (!data) {
return recursive() // call the function again
}
return data // job done, return the data
}
The recursive function can be used as follows:
const main = async () => {
const data = await recursive()
// do something here with the data
}

Using your code, I'd refactored it as shown below. I hope it helps.
const requestP = require('request-promise');
const option = {
url: 'rest/api/2/search',
json: true,
qs: {
//jql: "project in (FLAGPS)",
}
};
/*
NOTE: Add async to the function so you can udse await inside the function
*/
const callback = async (body) => {
// some code
//saving records to file
//some code
try {
const result = await requestP(option, callback).auth('api-reader', token, true);
if (totlExtractedRecords < total) {
return callback(result);
}
return result;
} catch (error) {
console.log('Error Observed ' + err);
return error;
}
}

Created this code using feed back from Amir Popovich
const rp = require('Request-Promise')
const fs = require('fs')
const pageSize = 200
const options = {
url: 'https://jira.xyz.com/rest/api/2/search',
json: true,
qs: {
jql: "project in (PROJECT_XYZ)",
maxResults: pageSize,
startAt: 0,
fields: '*all'
},
auth: {
user: 'api-reader',
pass: '<token>',
sendImmediately: true
}
}
const updateCSV = (elment) => {
//fs.writeFileSync('issuedata.json', JSON.stringify(elment.body, undefined, 4))
}
async function getPageinatedData(resolve, reject, ctr = 0) {
var total = 0
await rp(options).then((body) => {
let a = body.issues
console.log(a)
a.forEach(element => {
console.log(element)
//updateCSV(element)
});
total = body.total
}).catch((error) => {
reject(error)
return
})
ctr = ctr + pageSize
options.qs.startAt = ctr
if (ctr >= total) {
resolve();
} else {
await getPageinatedData(resolve, reject, ctr);
}
}
new Promise((resolve, reject) => getPageinatedData(resolve, reject))
.then(() => console.log('DONE'))
.catch((error) => console.log('Error observed - ' + error.name + '\n' + 'Error Code - ' + error.statusCode));

Function not returning pg-pool results

I'm trying to use pg-pool to query a Postgresql database. And then export this data to a Node.js express rest API.
Here's my exports function.
exports.getDashboard = function(req, response, next) {
let returnData = fetchData();
return response.status(200).json(returnData);
};
Which calls fetchData();
fetchData = async function() {
let returnData = [];
returnData.languages = await getLatestLanguages();
return returnData;
};
Which calls getLatestLanguages()
getLatestLanguages = function() {
pgPool.pool.query(
'SELECT * FROM "WordLanguage" ORDER BY id DESC LIMIT 30 ;',
(error, results) => {
if (error) {
throw error;
}
return results.rows;
}
);
}
If i place a console.log(results.rows) before getLatestLanguages() returns results.rows, then I get the data logged to the console.
However the object isn't being returned to fetchData. I tested this by logging the returnData to console before it is returned to exports.getDashboard();
I believe my problem is something to do with the async nature of pg-pool so I tried making my function async with an await but that didn't help.
What am I doing wrong?

you need getLatestLanguages to return a Promise, so you can await it from the caller
getLatestLanguages = function() {
return new Promise((resolve, reject) => {
pgPool.pool.query(
'SELECT * FROM "WordLanguage" ORDER BY id DESC LIMIT 30 ;',
(error, results) => {
if (error) {
reject(error);
}
resolve(results.rows);
}
);
})
}
you also need to await fetchData(), therefore getDashboard should be async
exports.getDashboard = async function(req, response, next) {
let returnData = await fetchData();
return response.status(200).json(returnData);
};

getLatestLanguages() should return a promise. For example
getLatestLanguages = function() {
return new Promise((resolve, reject) => {
pgPool.pool.query(
'SELECT * FROM "WordLanguage" ORDER BY id DESC LIMIT 30 ;',
(error, results) => {
if (error) {
reject(error);
}
resolve(results.rows);
}
);
});
};
fetchData() is async and therefore should be awaited
exports.getDashboard = async function(req, response, next) {
let returnData = await fetchData();
return response.status(200).json({ languages: returnData.languages });
};
Also make sure that you return returnData.languages in the correct format as above instead of ...json(returnData);