socket in nodejs will not close no matter what - javascript

So I'm trying to make a script that will connect a bot to a kahoot. However I can only use a certain name one time otherwise I will get a duplicate name error, even after closing all sockets, terminating sockets, removing listeners, etc. Is there something I'm missing?
Edit: added the cleanup on process end that still doesn't work
var Kahoot = require("kahoot.js-latest");
var colors = require('colors');
var bot = new Kahoot;
function cleanup() {
bot.leave();
bot.removeAllListeners();
bot.socket.close();
bot.socket.terminate();
}
bot.join(process.argv[2], "a")
.then(() => {
console.log("Joining...".cyan);
})
.catch((err) => {
console.log(err);
cleanup();
});
[`exit`, `SIGINT`, `SIGUSR1`, `SIGUSR2`, `SIGTERM`].forEach((eventType) => {
process.on(eventType, (code) => {
for (n in bots) {
cleanup();
}
console.log("Exiting Process with code: ".yellow + code);
})
})
After terminating the process and trying to run it again with the same name, I get a duplicate name error even though surely that connection should be destroyed by now
Edit: the error:
{
description: 'Duplicate name',
type: 'loginResponse',
error: 'USER_INPUT'
}

Related

Socket hangup due to wrong handling of promises

I have script to move data from one platform to another. The source db allows only 100 records to be fetched in a single request. So I created a routine to fetch by batches of 100 which works fine I guess.
Now I try to process each records of 100 and do the necessary transformations (which involves axios call to get certain data) and create a record in firebase firestore.
Now when I run this migration in firebase express node, I get socket hang up ECONNRESET.
I know this is caused by wrong handling of promises.
Here is what my code looks like:
import { scrollByBatches } from "../helpers/migrations/apiScroll";
import { createServiceLocation } from "../helpers/locations";
const mapServiceLocationData = async (serviceLocation: any, env: string) => {
try {
const migratedServiceLocation: any = {
isMigrated: true,
id: serviceLocation._id,
};
if (serviceLocation.list?.length) {
await Promise.all(serviceLocation.ids.map(async (id: string) => {
const { data } = await dbEndPoint.priceMultiplier({ id }); // error says socket hangup on this call
let multiplierUnit;
let serviceType;
if (data.response._id) {
multiplierUnit = data.response;
const result = await dbEndPoint.serviceType({ id: multiplierUnit.service_custom_service_type }); // error says socket hangup on this call
if (result.data.response._id) {
serviceType = result.data.response.type_text;
migratedServiceLocation.logs = [...multiplierUnit.history_list_text, ...migratedServiceLocation.logs];
}
}
}));
}
await createServiceLocation(migratedServiceLocation); // create record in destination db
} catch (error) {
console.log("Error serviceLocation: ", serviceLocation._id, JSON.stringify(error));
}
return null; // is this even necessary?
};
export const up = async () => {
try {
// get 100 docs from source db => process it.. => fetch next 100 => so on...
await scrollByBatches(dbEndPoint.serviceLocation, async (serviceLocations: any) => {
await Promise.all(
serviceLocations.map(async (serviceLocation: any) => {
await mapServiceLocationData(serviceLocation);
})
);
}, 100);
} catch (error) {
console.log("Error", JSON.stringify(error));
}
return null; // is this even necessary?
};
The error I get in firebase functions console is:
For clarity on how the fetch by batches looks like:
const iterateInBatches = async (endPoint: any, limit: number, cursor: number, callback: any, resolve: any, reject: any) => {
try {
const result = await endPoint({ limit, cursor });
const { results, remaining }: any = result.data.response;
if (remaining >= 0) {
await callback(results);
}
if ((remaining)) {
setTimeout(() => {
iterateInBatches(endPoint, limit, (cursor + limit), callback, resolve, reject);
}, 1000); // wait a second
} else {
resolve();
}
} catch (err) {
reject(err);
}
};
export const scrollByBatches = async (endPoint: any, callback: any, limit: number, cursor: number = 0) => {
return new Promise((resolve, reject) => {
iterateInBatches(endPoint, limit, cursor, callback, resolve, reject);
});
};
What am I doing wrong? I have added comments in the code sections for readability.
Thanks.
There are two cases when socket hang up gets thrown:
When you are a client
When you, as a client, send a request to a remote server, and receive no timely response. Your socket is ended which throws this error. You should catch this error and decide how to handle it: whether to retry the request, queue it for later, etc.
When you are a server/proxy
When you, as a server, perhaps a proxy server, receive a request from a client, then start acting upon it (or relay the request to the upstream server), and before you have prepared the response, the client decides to cancel/abort the request.
I would suggest a number of possibilities for you to try and test that might help you solve your issue of ECONNRESET :
If you have access to the source database, you could try looking
there for some logs or metrics. Perhaps you are overloading the
service.
Quick and dirty solution for development: Use longjohn, you get long
stack traces that will contain the async operations. Clean and
correct solution: Technically, in node, whenever you emit an 'error'
event and no one listens to it, it will throw the error. To make it
not throw, put a listener on it and handle it yourself. That way you
can log the error with more information.
You can also set NODE_DEBUG=net or use strace. They both provide you
what the node is doing internally.
You could restart your server and run the connection again, maybe
your server crashed or refused the connection most likely blocked by
the User Agent.
You could also try running this code locally, instead of in cloud
functions to see if there is a different result. It's possible that
the RSG/google network is interfering somehow.
You can also have a look at this GitHub issue and stackoverflow
thread to see the common fixes for the ECONNRESET issue and see if
those help resolve the issue.

Getting cal cancelled error in firebase cloud functions

Firebase cloud functions logs "Error: 1 CANCELLED: Call cancelled" sometimes. am trying to send the push notifications one day before the timestamp, everything works till getting the fcm tokens but not able to send notification here is my code.
const NotificationLive_dayBefore = async () => {
try {
const path = models.payments.firebasePath;
let imageURL = null;
let tokens = [];
let subscribedUsers = await getSubscribedUsers().catch(e => { console.log(e) });
if (subscribedUsers && subscribedUsers.length > 0) {
for(const subscriber of subscribedUsers){
const userDoc = db
.collection('Payments').doc('v1').collection('users')
.doc(subscriber)
.collection('subscriptions').where('deleted_at', '==', null)
.get() // <------------here is the error according to logs
.then(async (snapshot) => {
if (!snapshot.empty) {
console.log('snapshot.empty', snapshot.empty);
for(const doc of snapshot.docs ){
// let friend_doc_id = doc.data().friend_doc_id
console.log("Friend id", doc.id);
if (doc.exists) {
let oneDay = new Date().getTime() + (24 * 60 * 60 * 1000);
let oneDayMinus1 = new Date().getTime() + (23 * 60 * 60 * 1000);
console.log(oneDayMinus1);
console.log(oneDay);
try {
let liveLesson = await db
.collection('Lessons').doc('v1').collection('friends')
.doc(doc.id)
.collection('live')
.where('start_time', '>', new Date(oneDayMinus1))
.where('start_time', '<', new Date(oneDay))
.where('deleted_at', '==', null)
.get();
for( const liveSnap of liveLesson.docs){
console.log("liveSnapid", '=>', liveSnap.data());
console.log('Private lesson exists', liveSnap.exists);
if (liveSnap.exists) {
// time is equal send notification
console.log("Subscriber ID", subscriber);
const Users = db
.collection(models.notification.firebasePath)
.where('deleted_at', '==', null)
.where('__name__', '==', subscriber)
.get()
.then(async (UserSnapshot) => {
for( const userdoc of UserSnapshot.docs){
const userdocument = userdoc.data();
if (userdocument.fcm_token) {
tokens = userdocument.fcm_token;
}
console.log('tokens',tokens);
if (tokens.length>0) {
try {
let live_lessons_images = await db
.collection('Accounts').doc('v1').collection('friends')
.doc(doc.id)
.get();
if (!live_lessons_images.data().image_url) {
imageURL = null;
console.log("Image not found");
} else {
imageURL = live_lessons_images.data().image_url;
}
} catch (error) {
console.log('That did not go well.', error)
}
console.log("notification sent live lesson 24 hr before");
await sendNotificationNew(
"test",
test notification,
imageURL,
tokens
).catch(error => { console.error("promise 1", error) });; //send notification to users as a remainder for live lesson before one day
}
}
}).catch(error => { console.error("promise 1", error) });
}
}
} catch (error) {
console.log('That did not go well.', error)
}
} else {
console.log("friend_doc_id is not present");
}
}
}
}).catch(error => { console.error("promise error", error) });
} //end foreach of subscribed user
}
return Promise.all('success');
}
catch (err) {
// Handle error here
// This will return error from api
console.log("Exceptions: ", err);
}}
and my sendNotificationNew function is
const sendNotificationNew = async (title, body, image, tokens) => {
console.log("title", title);
if (tokens.length > 0) {
if(tokens.length>1){
tokens = tokens[tokens.length-1];
}
try {
let message = {
notification: {
title: title,
body: body
},
token: tokens
};
if (image) {
message.notification.image = image;
}
admin.messaging().send(message)
.then((response) => {
// Response is a message ID string.
console.log('Successfully sent message:', response);
})
.catch((error) => {
console.log('Error sending message:', error);
console.log('Error sending message:', tokens);
});
} catch (err) {
console.log(err);
}
} else {
console.log("Token not available");
}
};
i was able to see the logs printing "title" of the notification, but not "Successfully sent message" or "Error sending message:".
Error message i got
Error: 1 CANCELLED: Call cancelled
at Object.callErrorFromStatus (/workspace/node_modules/#grpc/grpc-js/build/src/call.js:31)
at Object.onReceiveStatus (/workspace/node_modules/#grpc/grpc-js/build/src/client.js:327)
at Object.onReceiveStatus (/workspace/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:299)
at (/workspace/node_modules/#grpc/grpc-js/build/src/call-stream.js:145)
at processTicksAndRejections (internal/process/task_queues.js:79)
And this
Error: 9 FAILED_PRECONDITION: The requested snapshot version is too old.
at Object.callErrorFromStatus (/workspace/node_modules/#grpc/grpc-js/build/src/call.js:31)
at Object.onReceiveStatus (/workspace/node_modules/#grpc/grpc-js/build/src/client.js:327)
at Object.onReceiveStatus (/workspace/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:299)
at (/workspace/node_modules/#grpc/grpc-js/build/src/call-stream.js:145)
at processTicksAndRejections (internal/process/task_queues.js:79)
3rd Edit: Finally I've found the solution to this problem. A same error occurred in 2 cloud functions and I solved both of them by following the same process.
I'm not deleting the previous wrong answers, which didn't work for me, so that you also could know what won't work.
This error occurs because the Firestore has hit a limit.
I was trying to do lots of writes and updates in Firestore in parallel.
Example: Here all the functions are called all together. This is very fast as all the tasks are done in parallel. And recommended by Firebase.
But this might hit a limit in firestore and give error.
exports.botsCompletingLectures = functions.region('asia-south1')
.database.ref('triggerCloudFunctions/botsCompletingLectures')
.onUpdate(async(change, context)=>{
const promises = [];
promises.push(doSomeTaskInFirestore());
promises.push(doAnotherTaskInFirestore());
promises.push(doSomeMoreTaskInFirestore());
function doSomeTaskInFirestore(){
//Write to lots of documents in a collection
}
async function doAnotherTaskInFirestore(){
//Update lots of documents in firestore
}
async function doSomeMoreTaskInFirestore(){
//Do more tasks in firestore
}
return Promise.all(promises);
});
Solution: Here only one task will execute at a time and so it will take little more time. But will not give any error.
exports.botsCompletingLectures = functions.region('asia-south1')
.database.ref('triggerCloudFunctions/botsCompletingLectures')
.onUpdate(async(change, context)=>{
await doSomeTaskInFirestore()
await doAnotherTaskInFirestore()
return doSomeMoreTaskInFirestore()
function doSomeTaskInFirestore(){
//Write to lots of documents in a collection
}
async function doAnotherTaskInFirestore(){
//Update lots of documents in firestore
}
async function doSomeMoreTaskInFirestore(){
//Do more tasks in firestore
}
});
2nd Edit: The Cloud Function worked well for 4-5 days and then started giving errors again.
So this time I've given up on trying to fix it and instead enabled Retry on failure in Cloud Functions.
In my case as the Cloud Function is running correctly for some days and gives error on other days, due to some temperory problems, like network issue or cold start, so I can enable it here.
We should not enable it, if the error is permanent for eg a Bug in the code, else the function will keep on retrying for 7 days.
You can learn about Enabling Retry in Cloud Functions from this video.
https://www.youtube.com/watch?v=Pwsy8XR7HNE
1st Edit: The error appeared again the next day
So after reading and searching about it, I found out this problem occurs because of Cold Starting of a function after a long time and some Network problem and memory leak and most of the people (including me), who are getting this error, are getting it in PubSub Cloud Function and while doing some tasks in Firestore.
So I used a workaround. I don't know if it is recommended or not, but I'm tired of these errors, so I just did it.
I created a PubSub Cloud Function that updates a field value in Realtime Database. And this update in the field will trigger another function, that will do the task in firestore.
exports.triggerTheMainFunction = functions.pubsub.schedule('40 11 * * *').onRun(async(context)=> {
return admin.database().ref()
.child('triggerOtherFunction')
.child('doSomeTaskInFirestore')
.set(admin.database.ServerValue.increment(1))
.catch((error)=>{
console.log('Error incrementing the value', error);
});
});
And converted the Actual Function for Firestore from PubSub to Event Triggered. And since then I'm not getting any errors.
exports.doSomeTaskInCloudFirestore = functions
.database.ref('triggerOtherFunction/doSomeTaskInFirestore')
.onUpdate(async(change, context)=>{
//Do the task that was needed to be done in firestore.
});
If I get any errors in future, then I'll update this answer.
First Answer
I also got a similar error. I don't know what was causing the problem. But I solved it by installing the latest version.
So first I saved a copy of index.js on desktop and reinstalled everything.
I was using NodeJS 14 version. So I uninstalled it from control panel and downloaded the nodeJS 16 version from the nodejs website. And installed it.
ran in terminal.
npm install -g npm
then
npm install npm#latest -g
Then
firebase init
then
npm install -g firebase-tools
And then redeployed the same cloud function, without making any changes. And test run the function. And the error disappeared.
I resolved this issue by adding indexes in the firestore and proper catch blocks to all the promises to avoid unhandled rejection errors.

Using Discord.js to get all of the messages sent since the bot went offline

So I am making a bot that will store all of the messages sent in a channel into a txt file on my computer. Since I don't have my computer on all the time and I run the bot on my computer it leaves gaps in messages stored in the .txt file. I am trying to make it so that my bot will get all the messages sent since it went offline and store all of them in my file. I created a txt file that saves the message id of every message, it will overwrite the last message so its basically just a file with one message id in it. When my bot starts a boolean variable is instantiated to false and will be set to true once the startup procedure has been completed. This is what I have so far:
var latest;
var beenRunning = false;
bot.on('message', msg => {
latest = msg.id;
if(beenRunning == false) {
msg.channel.messages.fetch(latest, {limit: 100})
.then(messages =>
//code to get messages since last online
)
.catch(console.error);
beenRunning = true;
}
else {
messageTest(msg);
}
fs.writeFile('lastMsg.txt', latest, (err) => {
if (err) throw err;
})
});
I am not sure how to get every message since going offline using the fetch() method. How would I do this?
Not sure how you are structuring your lastMsg.txt, so I just used json.
Might be a bit slow since it'll be a bigger file and its using a lot of array higher functions
let logged = require("./lastMsg.json");
let beenRunning;
bot.on('message', msg => {
if (!beenRunning) {
msg.channel.messages.fetch({ limit: 100 })
.then(messages => {
const reduced = messages
.map(e => ({ id: e.id, content: e.content }))
.filter(e => {
return logged.find(l => l.id === e.id && l.content === e.content);
});
logged = logged.concat(reduced.array());
beenRunning = true;
})
.catch(console.error);
} else {
messageTest(msg);
}
fs.writeFile('lastMsg.json', JSON.stringify(logged), (err) => {
if (err) throw err;
})
});
Initalize your ./lastMsg.json to
[]

Why Is An Error Returning A Whole Document

I am trying to get a document using findOne, I have a simple error handler (if (err) console.log(err);), but is returning a whole document. I can't get past this if statement. If I remove the if (err), then it gets caught by another statement which says the document does not exist.
Using Mongoose 5.4.0, have tried removing the if statement but just gets caught by the others saying it doesn't exist. I'm also using discord.js, so wherever it says message.reply or message.channel.send - it just means that it will send a message to a channel - this has nothing to do with the error.
guildModel.findOne({"GuildName": GuildSearch}).then((err, result) => {
if (result) {
let guildEmbed = new Discord.RichEmbed()
.setTitle(GuildSearch)
.setColor("00ff65")
.setDescription(result.GuildDescription);
return message.channel.send(guildEmbed);
} else {
let NoDoc = new Discord.RichEmbed()
.setTitle("Oops!")
.setDescription(`<#!${message.author.id}>, There is no Server with the name ${GuildSearch} recorded with me.`)
.setColor("ff7f00")
.setFooter("Developed By William#8495");
return message.channel.send(NoDoc);
};
}).catch(err => {
return message.reply("Error: " + err);
});
It should just send a field of the document named GuildDescription, but it sends Error: and the whole document.
You are using "Promise style" query so the first argument should be result, not err:
guildModel.findOne({ "GuildName": GuildSearch }).then(result => {
}).catch(err => {
console.log(err)
})
Or you could use the other syntax:
guildModel.findOne({ "GuildName": GuildSearch }, (err, result) => {
if (err) console.log(err)
});
Mongoose documentation

Nodejs fs.createReadStream event error

I am trying to read data from a file very quickly (data from an ADC). The conversion is started on opening the file, and finished at close. I need to wait opening the file again, and wait for the currently conversion to complete.
My problem is, that when i am opening the file very quickly nodejs wont caught the expected events. Any clue how to fix this?
node.on('input', function(){
readStream = fs.createReadStream(path.location,{encoding: 'utf8'});
if (readyFlag == 1) {
readStream.on('data',(data) => {
data = {payload: data/1000};
node.send(data);
console.log(`data: ${data}`);
})
}
readStream.on('open', () => {
console.log("file opened");
readyFlag = 0;
})
readStream.on('close', () => {
console.log("file closed");
readyFlag = 1;
})
readStream.on('error', (err) => {
console.log(err);
})
})
I end up with the file being opened all the time.
So looks to me like the problem here is the context of readStream - it's not local to the input callback meaning each message will overwrite the reference to the previous stream.
The fact this starts to fail when you increase the frequency of the messages seems logical to me because the stream hasn't had a chance to finish so readyFlag is never reset.
The fix for this should literally a one liner, make the stream a local var
const readStream = ...
I think I have solved it. The readyFlag has to be global to the input scope, otherwise it would not work. I moved the 'open' event inside the the if statement, which is checking for the readyFlag. That seemed to do the problem. Now the system dont hang, and the file is only being opened, when its done reading and have closed the previously open section.
Here is the code:
var readyFlag = 1;
node.on('input', function(msg){
const readStream = fs.createReadStream(path.location,{encoding: 'utf8'});
if (readyFlag == 1) {
readStream.on('data',(data) => {
data = {payload: data/1000};
node.send(data);
console.log('data read');
readyFlag = 0;
})
readStream.on('open', () => {
console.log("file opened");
})
}
readStream.on('close', () => {
console.log("file closed");
readyFlag = 1;
})
readStream.on('error', (err) => {
console.log(err);
})
})

Categories

Resources