Develop Reference

Socket hangup due to wrong handling of promises

I have script to move data from one platform to another. The source db allows only 100 records to be fetched in a single request. So I created a routine to fetch by batches of 100 which works fine I guess.
Now I try to process each records of 100 and do the necessary transformations (which involves axios call to get certain data) and create a record in firebase firestore.
Now when I run this migration in firebase express node, I get socket hang up ECONNRESET.
I know this is caused by wrong handling of promises.
Here is what my code looks like:
import { scrollByBatches } from "../helpers/migrations/apiScroll";
import { createServiceLocation } from "../helpers/locations";
const mapServiceLocationData = async (serviceLocation: any, env: string) => {
try {
const migratedServiceLocation: any = {
isMigrated: true,
id: serviceLocation._id,
};
if (serviceLocation.list?.length) {
await Promise.all(serviceLocation.ids.map(async (id: string) => {
const { data } = await dbEndPoint.priceMultiplier({ id }); // error says socket hangup on this call
let multiplierUnit;
let serviceType;
if (data.response._id) {
multiplierUnit = data.response;
const result = await dbEndPoint.serviceType({ id: multiplierUnit.service_custom_service_type }); // error says socket hangup on this call
if (result.data.response._id) {
serviceType = result.data.response.type_text;
migratedServiceLocation.logs = [...multiplierUnit.history_list_text, ...migratedServiceLocation.logs];
}
}
}));
}
await createServiceLocation(migratedServiceLocation); // create record in destination db
} catch (error) {
console.log("Error serviceLocation: ", serviceLocation._id, JSON.stringify(error));
}
return null; // is this even necessary?
};
export const up = async () => {
try {
// get 100 docs from source db => process it.. => fetch next 100 => so on...
await scrollByBatches(dbEndPoint.serviceLocation, async (serviceLocations: any) => {
await Promise.all(
serviceLocations.map(async (serviceLocation: any) => {
await mapServiceLocationData(serviceLocation);
})
);
}, 100);
} catch (error) {
console.log("Error", JSON.stringify(error));
}
return null; // is this even necessary?
};
The error I get in firebase functions console is:
For clarity on how the fetch by batches looks like:
const iterateInBatches = async (endPoint: any, limit: number, cursor: number, callback: any, resolve: any, reject: any) => {
try {
const result = await endPoint({ limit, cursor });
const { results, remaining }: any = result.data.response;
if (remaining >= 0) {
await callback(results);
}
if ((remaining)) {
setTimeout(() => {
iterateInBatches(endPoint, limit, (cursor + limit), callback, resolve, reject);
}, 1000); // wait a second
} else {
resolve();
}
} catch (err) {
reject(err);
}
};
export const scrollByBatches = async (endPoint: any, callback: any, limit: number, cursor: number = 0) => {
return new Promise((resolve, reject) => {
iterateInBatches(endPoint, limit, cursor, callback, resolve, reject);
});
};
What am I doing wrong? I have added comments in the code sections for readability.
Thanks.

There are two cases when socket hang up gets thrown:
When you are a client
When you, as a client, send a request to a remote server, and receive no timely response. Your socket is ended which throws this error. You should catch this error and decide how to handle it: whether to retry the request, queue it for later, etc.
When you are a server/proxy
When you, as a server, perhaps a proxy server, receive a request from a client, then start acting upon it (or relay the request to the upstream server), and before you have prepared the response, the client decides to cancel/abort the request.
I would suggest a number of possibilities for you to try and test that might help you solve your issue of ECONNRESET :
If you have access to the source database, you could try looking
there for some logs or metrics. Perhaps you are overloading the
service.
Quick and dirty solution for development: Use longjohn, you get long
stack traces that will contain the async operations. Clean and
correct solution: Technically, in node, whenever you emit an 'error'
event and no one listens to it, it will throw the error. To make it
not throw, put a listener on it and handle it yourself. That way you
can log the error with more information.
You can also set NODE_DEBUG=net or use strace. They both provide you
what the node is doing internally.
You could restart your server and run the connection again, maybe
your server crashed or refused the connection most likely blocked by
the User Agent.
You could also try running this code locally, instead of in cloud
functions to see if there is a different result. It's possible that
the RSG/google network is interfering somehow.
You can also have a look at this GitHub issue and stackoverflow
thread to see the common fixes for the ECONNRESET issue and see if
those help resolve the issue.

How to get error line using Nestjs avoiding duplicate code

I made a controller on nestjs as below.
#Post()
public async addConfig(#Res() res, #Body() createConfigDto: CreateConfigDto) {
let config = null;
try {
config = await this.configService.create(createConfigDto);
} catch (err) {
return res.status(HttpStatus.INTERNAL_SERVER_ERROR).json({
status: 500,
message: 'Error: Config not created!',
});
}
if (!config) {
return res.status(HttpStatus.NOT_FOUND).json({
status: 404,
message: 'Not Found',
});
}
return res.status(HttpStatus.OK).json({
message: 'Config has been created successfully',
config,
});
}
And there is a service in other file.
public async create(createConfigDto: CreateConfigDto): Promise<IConfig> {
do something 1
do something 2
do something 3
return result;
}
If an internal server error occurs among "do somehting 1,2 or 3" when I request to this api, it will gives me response 500.
But I want to know in which line the error happend.
Therefore I made catch line function.
const getStackTrace = () => {
const obj = {};
Error.captureStackTrace(obj, getStackTrace);
return obj.stack;
};
And I wrap the code with this function, like this
public async create(createConfigDto: CreateConfigDto): Promise<IConfig> {
try{
do something 1
}catch(error){
console.log(error)
getTrace()
}
try{
do something 2
}catch(error){
console.log(error)
getTrace()
}
try{
do something 3
}catch(error){
console.log(error)
getTrace()
}
return result;
}
But the problem is the service code will grow a lot with this trace function.
I want to know whether there is more efficient way avoiding duplicate code using nestJs for this case.
I think interceptor or execption filter help this.
But I don't know how to code for this case , since I 'm about to start using nestjs.
Thank you for reading my question.

Set timeout inside an async function with try and catch does not catch error

I have an Ionic 3 App where I use async and await of ES6 features for syntactic sugar of promises. I know this is just a little bit basic question or because I am just new in using async and await feature.
Now the problem is I had a setTimeout inside my async function of signIn method to authenticate the user.
This is my code below:
async signIn() {
try {
this.loader = true // Present a loader
this.backgroundLoader = this.loadingCtrl.create({ content: 'Signing you in...' });
this.backgroundLoader.present();
// Response from server
const response: any = await this.authService.loginDriver(this.loginForm.value)
console.log(response)
if (response.status == 200) { // If success response
const token = await this.storage.set('token', response.data.token)
console.log('token:', token)
this.events.publish('driver:authenticated') // Broadcast an event that the driver has authenticated
setTimeout(async () => { // Dismiss the loader after successfull authentication
try {
const meDetails = await this.driverMe.getDriverMeInfo() // Get the profile information of the driver if
console.log(meDetails)
} catch (err) { console.log(err) }
this.backgroundLoader.dismiss();
this.loader = false
this.navCtrl.setRoot("HomePage")
this.menu.swipeEnable(true);
}, 1500);
}
} catch(err) { // If something goes wrong or an error occured
console.log(err)
this.backgroundLoader.dismiss();
this.loader = false
err.status == 401 || 422 // Something wrong in your authentication credentials
? this.alertMessage.alertMessage('Incorrect email or password', null)
: this.alertMessage.alertMessage('Something went wrong.', 'Please try again.') // Other errors not related to the data entry to be authenticated
}
}
The function does a basic authentication to send a post request to an API server and get the response token and put it on a Storage and use that token for every request that needs authentication middleware in the backend.
Actually there is no error in my code it works great. But if you look at the try and catch inside the setTimeout method. It looks uglier. The purpose of the try and catch is to catch an error for every promises but I am redeclaring it again inside the setTimeout function.
I'm not sure but I think this is happening because of the callback function on the setTimeout is a new async and await that's why it wont catch the error outside.
So my question really is there a way to handle this? To avoid redeclarations of try and catch inside a setTimeout method.
Appreciate if someone could help.
Thanks in advance.
Is there anyway to handle this using one try and catch

Dialogflow api call works, but chatbot shuts off

In Dialogflow, i use the free edition (V2) with the blaze plan from Firebase.
I have an Intent that works on the word "test". When i enter "test" in the simulator, the chatbot gives an non response and leaves the chat. It is suppose to make an call to my API and retrieves information.
The weird part is, there is a console.log that prints out the body and that returns the JSON from the API. So that means the API call works fine, but there is still an error somewhere within the bot.
I found this question: Dialogflow v2 error “MalformedResponse 'final_response' must be set”
It looks alot like my problem, yet i cant seem to figure out what i should change to make mine work.
Thanks in advance for your time.
The Fulfullment:
function testcommand(agent) {
callNPApi().then((output) => {
agent.add(output);
}).catch(() => {
agent.add("That went wrong!");
});
}
function callNPApi() {
return new Promise((resolve, reject) => {
request2(url, function (error, response2, body){
//The substring is too ensure it doesnt crash for the character limit yet
body = body.substring(1,10);
console.log('Api errors: ' + JSON.stringify(error));
console.log('Api body: ' + JSON.stringify(body));
if (error) {
reject();
}
resolve('api call returned: ');
});
});
}
The Response in the console:
{
"responseMetadata": {
"status": {
"code": 10,
"message": "Failed to parse Dialogflow response into AppResponse because of empty speech response",
"details": [
{
"#type": "type.googleapis.com/google.protobuf.Value",
"value": "{\"id\":\"bca7bd81-58f1-40e7-a5d5-e36b60986b66\",\"timestamp\":\"2018-09-06T12:45:26.718Z\",\"lang\":\"nl\",\"result\":{},\"alternateResult\":{},\"status\":{\"code\":200,\"errorType\":\"success\"},\"sessionId\":\"ABwppHFav_2zx7FWHNQn7d0uw8B_I06cY91SKfn1eJnVNFa3q_Y6CrE_OAJPV-ajaZXl7o2ZHfdlVAZwXw\"}"
}
]
}
}
}
The Error in the console:
MalformedResponse
'final_response' must be set.

Yup, this is the same problem.
The issue is that you're returning a Promise from callNPApi(), but your event handler (which I assume is testcommand()) isn't also returning a Promise. If you are doing async calls anywhere in your handler, you must use a Promise, and if you are using a Promise you must also return that Promise from the handler.
In your case, this should be a simple change. Simply add "return" to your handler. So it might look something like this
function testcommand(agent) {
return callNPApi().then((output) => {
agent.add(output);
}).catch(() => {
agent.add("That went wrong!");
});
}

Returning a response immediately with res.json

I have an express route which takes in some parameters, queries the database, and then returns some response.
I am using sequelize to query the db:
router.get('/query', function(req,res) {
var name = req.params.name;
var gid = req.params.gid;
// Query the db
models.user.find({ where: { name: name }}).then(function(user) {
models.group.find({ where: { id: gid }}).then(function(group) {
// if user found, return data to client
if (user) {
res.json({"user": user, "group": group});
}
});
}).catch(function(error) {
// catch any errors from db query
res.status(500).json({"error":error});
});
// Return a server error for any other reason
// This causes ERROR
res.status(500).json({"error":"Something went wrong. Check your input."});
});
But I keep getting the error on the last line:
Can't set headers after they are sent
It seems like the last line is ALWAYS run, even if it finds a user (which should return data to the client and be done).
Why doesn't res.json(..) immediately return to the client when a user is found? Since headers were already set, when the last line runs, it throws that error.

You need to only conditionally return an error. The line:
res.status(500).json({"error":"Something went wrong. Check your input."});
is always getting executed. The reason for this is that the function you pass to the find method is only called later in the event loop after the db responds. This means that when that call back is called you have already set the error on the response.
Your should either remove that line or decide when you want to return an error but don't return an error every time.

Remember javascript is asynchronous.
As soon you call this function
models.user.find({ where: { name: name }})
That last line is executed:
res.status(500).json({"error":"Something went wrong. Check your input."});
It seems you are trying to cater for 2 scenarios:
Bad request data from client - i.e. no gid given
Internal server errors - i.e. error with the database
I would recommend changing your catch function to something like this:
.catch(function(error) {
// catch any errors from db query
if (err === "Unable to connect to database") {
return res.status(500).json({ error: "There was an internal error"})
}
res.status(400).json({"error": "Bad input, please ensure you sent all required data" });
});
Have a read up on the list of standard HTTP status codes:
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

What #bhspencer said is right. You have to remove that last line.
That line probably gets executed before any query in the database.

You need to implement a return in
models.user.find({ where: { name: name }}).then(function(user) {
models.group.find({ where: { id: gid }}).then(function(group) {
// if user found, return data to client
if (user) {
res.json({"user": user, "group": group});
return;
}
});
}).catch(function(error) {
// catch any errors from db query
res.status(500).json({"error":error});
return;
});
Actually res.json( does not end the processing of node.js code execution without return statement.