Develop Reference

How to access a variable used to hold an array of data in a async function

//service class
async importExcel(userId, file){
const business = await this.getBusinessByUserId(Types.ObjectId(userId));
let errorData = [];
readXlsxFile(file, {schema}).then(async ({rows, errors}) => {
if(errors.length === 0){
await this.excelUploadQueue.add({
rows,
business
})
}
errors.forEach((error) => {
const errors = {
column: error.column,
value: error.value,
reason: error.reason,
}
errorData.push(errors)
});
console.log(errorData); can access the errorData here
})
console.log(errorData); cannot access it here, i want to access it here so that i
return errorData; // can return it here to my controller
}
//controller
#UseGuards(JwtAuthGuard)
#UseInterceptors(FileInterceptor('file'
, {
storage: diskStorage({
destination: './csv',
filename: (req, file, cb) => {
const uniqueSuffix = Date.now() + '-' + Math.round(Math.random() * 1E9)
cb(null, file.fieldname + '-' + uniqueSuffix + "-" + file.originalname)
}
})
}
))
#Post('/import-excel/employee')
#ApiOperation({summary: 'upload exployee data via excel'})
async uploadExcel(
#UploadedFile() file: Express.Multer.File,
#Request() req
){
const { id: userId } = req.user;
return await this.businessService.importExcel(userId, file.path);
}
How can i go abt it,been on it for some hours....When i apply a settimeout like the below code
setTimeout(() => {
console.log(errorData) // i could access it but its not returning to my controller
}, 100);

Don't mix async/await syntax with .then() calls. Just write
async importExcel(userId, file) {
const business = await this.getBusinessByUserId(Types.ObjectId(userId));
const {rows, errors} = await readXlsxFile(file, {schema});
// ^^^^^
if (errors.length === 0) {
await this.excelUploadQueue.add({
rows,
business
})
}
const errorData = errors.map(error => ({
column: error.column,
value: error.value,
reason: error.reason,
}));
console.log(errorData); // can access the errorData here
return errorData; // and can return it here to my controller
}
The problem is not even related to the forEach loop, you simply were trying to access errorData outside of the then callback. And btw, never use an async callback to forEach - easily fixed in your case since the callback didn't actually do anything asynchronous (though I still wonder why you need this at all - you're creating new objects with exactly the same properties as the errors, maybe just return errors directly?).

Node.js promise fails intermittently, even when handled

I'm learning to use MongoDB by creating a simple blog app. However, a portion of my code that saves a given post seems to give problems with promises occasionally, but not always, and whether the code succeeds simply seems to be luck.
Each post in my database is stored with the following schema:
{
title: String,
author: String,
body: String,
slug: String,
baseSlug: String,
published: { type: Boolean, default: false }
}
The slug defines the link used to access the blog post, and is automatically generated based upon the title of the blog post. However, if article titles are duplicates, the slug will have a number added to the end to differentiate it from similar articles, while the baseSlug will remain the same. For example:
I create the post "My first post", and it is assigned the baseSlug of "my-first-post". Because no other posts have the same baseSlug, the slug is also set to be "my-first-post".
I create another post called "My first post", and it is assigned the baseSlug of "my-first-post". However, because another post has the same baseSlug, it is assigned the slug "my-first-post-1".
To create this behavior, I wrote the following addpost route in Express:
app.post("/addpost", (req, res) => {
let postInfo = req.body;
for (key of Object.keys(postInfo)) {
if (postInfo[key] == "true") postInfo[key] = true;
}
let slug = postInfo.title
.toLowerCase()
.split(" ")
.filter(hasNumber) // return /\d/.test(str);
.slice(0, 5)
.join("-");
postInfo.slug = slug;
var postData;
Post.find({ baseSlug: postInfo.slug }, (error, documents) => {
if (documents.length > 0) {
let largestSlugSuffix = 0;
for (let document of documents) {
var fullSlug = document.slug.split("-");
var suffix = fullSlug[fullSlug.length - 1];
if (!isNaN(suffix)) {
if (parseInt(suffix) > largestSlugSuffix) {
largestSlugSuffix = suffix;
}
}
}
largestSlugSuffix++;
postInfo.baseSlug = postInfo.slug;
postInfo.slug += "-" + largestSlugSuffix;
} else {
postInfo.baseSlug = postInfo.slug;
}
postData = new Post(postInfo);
})
.then(() => {
postData
.save()
.then(result => {
res.redirect("/");
})
.catch(err => {
console.log(err);
res.status(400).send("Unable to save data");
});
})
.catch(err => {
console.log(err);
res.status(400).send("Unable to save data");
});
});
This code seems to work most of the time, but sometimes it fails, and outputs the following:
TypeError: Cannot read property 'save' of undefined
at C:\Users\User\BlogTest\app.js:94:18
at processTicksAndRejections (internal/process/task_queues.js:94:5)
(For reference, line 94 in my file is postData.save())
I suspect it is because the main body of the function takes longer than it should to execute, and the postData variable is not yet defined. However, postData.save() should not be executed until the promise finishes, because of the .then() callback function.
Why is my code behaving like this? Is there any way to fix it?

The issue is that you are mixing promises with callbacks and closures. That's not how this is intended to work.
When you chain promises, whatever you return in the first promise handler will be added as an input to the next one. And if you return a promise, that promise will be resolved first before being sent to the next thenable.
So you need to return promises from your promises, like this:
app.post("/addpost", (req, res) => {
let postInfo = req.body;
for (key of Object.keys(postInfo)) {
if (postInfo[key] == "true") postInfo[key] = true;
}
let slug = postInfo.title
.toLowerCase()
.split(" ")
.filter(hasNumber) // return /\d/.test(str);
.slice(0, 5)
.join("-");
postInfo.slug = slug;
// var postData; <-- Don't do that
Post.find({ baseSlug: postInfo.slug })
.then((documents) => {
if (documents.length > 0) {
let largestSlugSuffix = 0;
for (let document of documents) {
var fullSlug = document.slug.split("-");
var suffix = fullSlug[fullSlug.length - 1];
if (!isNaN(suffix)) {
if (parseInt(suffix) > largestSlugSuffix) {
largestSlugSuffix = suffix;
}
}
}
largestSlugSuffix++;
postInfo.baseSlug = postInfo.slug;
postInfo.slug += "-" + largestSlugSuffix;
} else {
postInfo.baseSlug = postInfo.slug;
}
return new Post(postInfo);
// We could actually have called postData.save() in this method,
// but I wanted to return it to exemplify what I'm talking about
})
// It is important to return the promise generated by postData.save().
// This way it will be resolved first, before invoking the next .then method
.then( (postData) => { return postData.save(); })
// This method will wait postData.save() to complete
.then( () => { res.redirect("/"); })
.catch( (err) => {
console.log(err);
res.status(400).send("Unable to save data");
});
});
It can be greatly simplified with async/await:
app.post("/addpost", async (req, res) => {
try {
let postInfo = req.body;
for (key of Object.keys(postInfo)) {
if (postInfo[key] == "true") postInfo[key] = true;
}
let slug = postInfo.title
.toLowerCase()
.split(" ")
.filter(hasNumber)
.slice(0, 5)
.join("-");
postInfo.slug = slug;
let documents = await Post.find({ baseSlug: postInfo.slug });
if (documents.length > 0) {
let largestSlugSuffix = 0;
for (let document of documents) {
var fullSlug = document.slug.split("-");
var suffix = fullSlug[fullSlug.length - 1];
if (!isNaN(suffix)) {
if (parseInt(suffix) > largestSlugSuffix) {
largestSlugSuffix = suffix;
}
}
}
largestSlugSuffix++;
postInfo.baseSlug = postInfo.slug;
postInfo.slug += "-" + largestSlugSuffix;
} else {
postInfo.baseSlug = postInfo.slug;
}
let postData = new Post(postInfo);
await postData.save();
res.redirect("/");
} catch (err) {
console.log(err);
res.status(400).send("Unable to save data");
};
});

You are mixing callbacks and promises and while it may do something, I'm not sure what it will do exactly. You should pick one or the other and not mix them as much as possible. I would recommend picking promises if you are using a language that supports async/await, otherwise callbacks.
So for example your outter handler could be an async function
app.post("/addpost", async (req, res) => {
//...
})
Your real bug is in handling Post.find you are handling it somewhat with a callback and somewhat with a promise, and probably whats happening is that its random which one will get called first the callback or the promise resolution. Instead of both you should just do this now that you have an async function:
try {
const posts = await Post.find({ baseSlug: postInfo.slug });
// stuff you were doing in the callback
const post = new Post(postInfo)
// Now the promise code
await post.save()
// success!
res.redirect("/");
} catch (err) {
// With an async function you can just catch errors like normal
console.log(err);
res.status(400).send("Unable to save data");
}
If you're not using webpack or typescript and cannot target es7 then and thus cannot use async/await then I would recommend just using callbacks, do not use .then or .catch and that would look more like:
function error(err) {
console.log(err)
res.status(400).send("Unable to save data")
}
Post.find({ baseSlug: postInfo.slug }, (err, documents) => {
if (err) return error(err)
// stuff you're doing in the callback now
const post = new Post(postInfo)
post.save((err) => {
if (err) return error(err)
// success!
res.redirect("/");
})
})

How to achieve recursive Promise calls in Node.js

I am calling an API where I can only fetch 1000 records per request,
I was able to achieve this using recursion.
I am now trying to achieve the same using promises, I am fairly new to Node.js and JavaScript too.
I tried adding the recursion code in an if else block but failed
var requestP = require('request-promise');
const option = {
url: 'rest/api/2/search',
json: true,
qs: {
//jql: "project in (FLAGPS)",
}
}
const callback = (body) => {
// some code
.
.
.//saving records to file
.
//some code
if (totlExtractedRecords < total) {
requestP(option, callback).auth('api-reader', token, true)
.then(callback)
.catch((err) => {
console.log('Error Observed ' + err)
})
}
}
requestP(option).auth('api-reader', token, true)
.then(callback)
.catch((err) => {
console.log('Error Observed ' + err)
})
I want to execute the method using promise and in a synchronous way,
i.e. I want to wait until the records are all exported to a file and continue with my code

I think its better to create your own promise and simply resolve it when your done with your recursion. Here's a simply example just for you to understand the approach
async function myRecursiveLogic(resolveMethod, ctr = 0) {
// This is where you do the logic
await new Promise((res) => setTimeout(res, 1000)); // wait - just for example
ctr++;
console.log('counter:', ctr);
if (ctr === 5) {
resolveMethod(); // Work done, resolve the promise
} else {
await myRecursiveLogic(resolveMethod, ctr); // recursion - continue work
}
}
// Run the method with a single promise
new Promise((res) => myRecursiveLogic(res)).then(r => console.log('done'));

Here's a clean and nice solution using the latest NodeJS features.
The recursive function will continue executing until a specific condition is met (in this example asynchronously getting some data).
const sleep = require('util').promisify(setTimeout)
const recursive = async () => {
await sleep(1000)
const data = await getDataViaPromise() // you can replace this with request-promise
if (!data) {
return recursive() // call the function again
}
return data // job done, return the data
}
The recursive function can be used as follows:
const main = async () => {
const data = await recursive()
// do something here with the data
}

Using your code, I'd refactored it as shown below. I hope it helps.
const requestP = require('request-promise');
const option = {
url: 'rest/api/2/search',
json: true,
qs: {
//jql: "project in (FLAGPS)",
}
};
/*
NOTE: Add async to the function so you can udse await inside the function
*/
const callback = async (body) => {
// some code
//saving records to file
//some code
try {
const result = await requestP(option, callback).auth('api-reader', token, true);
if (totlExtractedRecords < total) {
return callback(result);
}
return result;
} catch (error) {
console.log('Error Observed ' + err);
return error;
}
}

Created this code using feed back from Amir Popovich
const rp = require('Request-Promise')
const fs = require('fs')
const pageSize = 200
const options = {
url: 'https://jira.xyz.com/rest/api/2/search',
json: true,
qs: {
jql: "project in (PROJECT_XYZ)",
maxResults: pageSize,
startAt: 0,
fields: '*all'
},
auth: {
user: 'api-reader',
pass: '<token>',
sendImmediately: true
}
}
const updateCSV = (elment) => {
//fs.writeFileSync('issuedata.json', JSON.stringify(elment.body, undefined, 4))
}
async function getPageinatedData(resolve, reject, ctr = 0) {
var total = 0
await rp(options).then((body) => {
let a = body.issues
console.log(a)
a.forEach(element => {
console.log(element)
//updateCSV(element)
});
total = body.total
}).catch((error) => {
reject(error)
return
})
ctr = ctr + pageSize
options.qs.startAt = ctr
if (ctr >= total) {
resolve();
} else {
await getPageinatedData(resolve, reject, ctr);
}
}
new Promise((resolve, reject) => getPageinatedData(resolve, reject))
.then(() => console.log('DONE'))
.catch((error) => console.log('Error observed - ' + error.name + '\n' + 'Error Code - ' + error.statusCode));

function inside function is not waiting for promise in javascript

Sorry if my title is not very explicit I dont know how to explain this properly.
I am trying to use the distinct function for my app that uses loopback 3 and mongodb. It seems to work right but my endpoint wont hit the return inside my function.
This is my code
const distinctUsers = await sellerCollection.distinct('userId',{
hostId : host.id,
eventId:{
"$ne" : eventId
}
}, async function (err, userIds) {;
if(!userIds || userIds.length ==0)
return [];
const filter = {
where:{
id: {
inq: userIds
}
}
};
console.log("should be last")
return await BPUser.find(filter);
});
console.log(distinctUsers);
console.log("wtf??");
//return [];
If I uncomment the return [] it will send the return and later it will show the should be last, so even when I dont have the return it seems to finish. It is now waiting for the response. I dont like the way my code looks so any pointer of how to make this look better I will take it.

It looks like the sellerCollection.distinct takes a callback as one of it's parameters, therefore, you cannot use async/await with a callback-style function, since it's not a promise.
I would suggest turning this call into a promise if you'd like to use async/await:
function findDistinct(hostId, eventId) {
return new Promise((resolve, reject) => {
sellerCollection.distinct(
'userId',
{ hostId, eventId: { "$ne": eventId } },
function (error, userIds) {
if (error) {
reject(error);
return;
}
if (!userIds || userIds.length === 0) {
resolve([]);
return;
}
resolve(userIds);
}
)
})
}
Then, you can use this new function with async/await like such:
async function getDistinctUsers() {
try {
const hostId = ...
const eventId = ...
const distinctUsers = await findDistinct(hostId, eventId)
if (distinctUsers.length === 0) {
return
}
const filter = {
where: {
id: { inq: userIds }
}
}
const bpUsers = await BPUser.find(filter) // assuming it's a promise
console.log(bpUsers)
} catch (error) {
// handle error
}
}

Alexa Skill error

I'm trying to call a 3rd party API in my Alexa Skill, and I'm getting a "Session ended with reason: ERROR" in CloudWatch log. The issue appears to be in my NumberIntentHandler or my httpGet function, but I'm not sure where.
UPDATED CODE
-- Handler that's getting fired --
const NumberIntentHandler = {
canHandle(handlerInput) {
return handlerInput.requestEnvelope.request.type === 'IntentRequest'
&& handlerInput.requestEnvelope.request.intent.name === 'NumberIntent';
},
handle(handlerInput) {
let slotNum = handlerInput.requestEnvelope.request.intent.slots.number.value;
//var myRequest = parseInt(slotNum);
const myRequest = parseInt(slotNum);
console.log('NumberIntentHandler myRequest: ', myRequest);
var options = `http://numbersapi.com/${myRequest}`;
console.log('NumberIntentHandler options: ', options);
// Use the async function
const myResult = httpGet(options);
console.log("sent : " + options);
console.log("received : " + myResult);
const speechText = myResult;
console.log('speechText: ', speechText); // Print the speechText */
return handlerInput.responseBuilder
.speak(speechText)
.withSimpleCard('Here is your fact: ', speechText)
.getResponse();
},
};
-- Function that's getting called from the Handler --
async function httpGet(options) {
// return new pending promise
console.log(`~~~~~~~~~ httpGet ~~~~~~~~~`);
console.log(`~~~~~${JSON.stringify(options)}~~~~~`);
return new Promise((resolve, reject) => {
const request = http.get(options, (response) => {
// handle http errors
if (response < 200 || response > 299) {
reject(new Error('Failed to load page, status code: ' + response));
}// temporary data holder
const body = [];
// on every content chunk, push it to the data array
response.on('data', (chunk) => body.push(chunk));
// we are done, resolve promise with those joined chunks
response.on('end', () => resolve(body.join('')));
console.log('body: ', body[0]);
});
// handle connection errors of the request
request.on('error', (err) => reject(err));
request.end();
});
}
Updated Code - Eliminated async/await/promise
-- Handler --
const NumberIntentHandler = {
canHandle(handlerInput) {
return handlerInput.requestEnvelope.request.type === 'IntentRequest'
&& handlerInput.requestEnvelope.request.intent.name === 'NumberIntent';
},
handle(handlerInput) {
let slotNum = handlerInput.requestEnvelope.request.intent.slots.number.value;
//var myRequest = parseInt(slotNum);
const myRequest = parseInt(slotNum);
console.log('NumberIntentHandler myRequest: ', myRequest);
var options = `http://numbersapi.com/${myRequest}`;
console.log('NumberIntentHandler options: ', options);
// Use the async function
//const myResult = httpGet(options);
const myResult = httpGet(options, res => {
console.log("sent : " + options);
console.log("received : " + myResult);
const speechText = myResult;
console.log('speechText: ', speechText); // Print the speechText */
return handlerInput.responseBuilder
.speak(speechText)
.withSimpleCard('Here is your fact: ', speechText)
.getResponse();
});
},
};
-- Function --
function httpGet(options, cb) {
http.get(options, res => {
console.log(`~~~~~${JSON.stringify(options)}~~~~~`);
// simplified version without error handling
let output = [];
res.on('data', d => output.push(d)); // or concat to a string instead?
res.on('end', () => cb(output));
console.log('output: ', output[0]);
});
}

I believe you will need to call resolve with your response in httpGet.
As a side note (unrelated to your problem) - I can recommend using request-promise, it implements a very nice promise api around http and would simplify your code in this case. (I know I know, async/await are new and fun tools but in this case I would go with "simpler" :) ).
Also, if I remember correctly, the callback for http.get is being invoked with only one argument.
edit, after changes:
you could get rid of the promise and async to simplify your code.
Just a note on async/await - if the awaited expression isn't a promise then it is cast into one automatically. In your current code you either need to use it like a promise (chain a .then() for example) or await it.
Anyways, here is an example that is just using a callback:
function httpGet(options, cb) {
http.get(options, res => {
// simplified version without error handling
let output = [];
res.on('data', d => output.push(d)); // or concat to a string instead?
res.on('end', () => cb(output));
});
}
httpGet(options, res => {
// building the alexa response, all your intent handler code that needs the response from your request
})