Too many requests from nodejs server causes "connect EADDRNOTAVAIL" - javascript

In my Node server (running on Mac), I'm requesting data from an external API using websockets and sending the result I get to my cloud API.
The websocket is sending a lot of data (1000+ obj/s). When I try to POST this to my cloud server, I get error on client side (my node server). This only happens during high load, I get no errors when load is lower (say 300/s). I'm not POSTing to my local server so I don't know what this error means.
One thing I noticed is that the ports are not reused. When I have high load, I go up to 15000+ used ports on my local machine and then I start getting these errors.
I have tried setting "agent: false" in the HTTP header without success. I have also tried "connection": "keep-alive" without success (found these two suggestions on SO).
This is the code that is POSTing the data I get from websocket:
function PostCode(codestring, apipathstring) {
return new Promise((resolve,reject)=>{
// An object of options to indicate where to post to
var post_options = {
host: 'hostname', //
port: 'port',
path: apipathstring,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(post_data),
}
};
// Set up the request
var post_req = https.request(post_options,function(res){
let data = '';
res.setEncoding('utf8');
// A chunk of data has been recieved.
res.on('data', (chunk) => {
data += chunk;
});
// The whole response has been received. Print out the result.
res.on('end', () => {
resolve(data);
});
// Error event received
}).on("error", (err) => {
reject("POSTING Error: " + err);
});
post_req.write(post_data);
post_req.end();
})
}
Optimally, I would stream even high traffic data without getting these connection errors.
Errors below:
Error: connect EADDRNOTAVAIL [my_cloud_IP] - Local (0.0.0.0:0)
Error: getaddrinfo ENOTFOUND [my_cloud_IP] [my_cloud_IP]:[my_cloud_port]

You can create your own "keep alive" agent, and use that one specific agent to send all the HTTP requests:
const http = require('http');
const keepAliveAgent = new http.Agent({ keepAlive: true });
options.agent = keepAliveAgent;
http.request(options, onResponseCallback);
According to the document:
keepAlive Keep sockets around even when there are no outstanding requests, so they can be used for future requests without having to reestablish a TCP connection.

Related

How to connect to a TCP server and pass a Javascript to it

I'm definitely a newbie with JS and node. I have telescope management software called SkyX Pro, and it has the ability to run a TCP Server on port 3040. I can connect to it using Netcat and hand it a Javascript starting with //* Javascript *// this works and allows me to startup cameras and other equipment and send commands for taking pictures etc. The issue is it needs to be run from a batch file which makes getting any information back to an HTML page tough (Like Camera, focuser and filter wheel status and temperatures).
The NC call looks like "NC localhost 3040 < Javascript-file.js
To get around the browser to local machine security issues I want to run this from node.js with maybe socket.io-client if possible, but I don't know the proper syntax for it.
I have seen plenty of client syntax sending hello's etc. but nothing send javascript and allowing for two-way connectivity that I can understand.
I have tried using:
var socket = io.connect('http://localhost');`enter code here`
socket.on('httpServer', function (data) {
console.log(data);
document.write(data + "\r\n");
socket.emit('tcp', "For TCP");
});
const net = require('net');
const client = new net.Socket();
client.connect({ port: 3040, host: process.argv[2] });
client.on('data', (data) => {
console.log(data.toString('utf-8'));
But I do not understand it well enough to troubleshoot why it's not working.
Any help would be wonderful, and please treat me like a baby that needs its step by step.
Cheer
Peter
Reading [1], We can assume socket-io isn't the perfect fit for you, because that Server you have sound like a typical tcp-socket server, not a socket.io server ( which requires special headers ) or a web-socket server.
So you only needs "net" library to do the job.
const net = require('net');
// module to send a message to TCP-socket server and wait for the response from socket-server
const sendAndReceive = async (client, message) => {
client.write(message);
let response = null
await ( new Promise( (resolve, reject) => {
client.on('data', function(data) {
response = data;
resolve()
});
}))
return response;
}
// send a single message to the socket-server and print the response
const sendJSCode = (message) => {
// create socket-client
const client = new net.Socket();
client.connect(3040, 'localhost', async function() {
console.log('Connected');
// send message and receive response
const response = await sendAndReceive(client, message)
// parse and print repsonse string
const stringifiedResponse = Buffer.from(response).toString()
console.log('from server: ', stringifiedResponse)
// clean up connection
client.destroy()
});
}
sendJSCode('var Out; \n Out="TheSky Build=" + Application.build \n\r')
This script will:
Initiate a socket client
on connection successfully, client sends a message
client receives back response from that message
client prints response to terminal
Note that TheSkyX has a limitation of 4096 bytes for each message[2], any more than that and we will need to chunk the message. So you may want to keep the js-code short and precise.
that snippet I gave is minimal, it doesn't handle errors from server. If you want, you can add client.on("error", .. ) to handle it.
Your point of connecting to the socket server directly from browser is very intriguing, unfortunately it is not allowed by modern browsers natively due to security concerns 3
[1] https://socket.io/docs/#What-Socket-IO-is-not:~:text=That%20is%20why%20a%20WebSocket%20client,to%20a%20plain%20WebSocket%20server%20either.
[2] https://www.bisque.com/wp-content/scripttheskyx/scriptOverSocket.html#MSearchField:~:text=set%20to%204096%20bytes

Avoid 'Remote request only' error Message when Node Server access domain

Using Node.js, I am trying to send a GET request to server and I've used the following code for that.
const https = require('https');
https.get('https://example-domain-name/', (resp) => {
let data = '';
// A chunk of data has been recieved.
resp.on('data', (chunk) => {
data += chunk;
});
// The whole response has been received. Print out the result.
resp.on('end', () => {
console.log(data);
});
}).on("error", (err) => {
console.log("Error: " + err.message);
});
The problem is that, the server is responding statusCode 500 (Runtime error).
An application error occurred on the server. The current custom error
settings for this application prevent the details of the application
error from being viewed remotely (for security reasons). It could,
however, be viewed by browsers running on the local server machine.
But browser is displaying the page normally, but using node js request, i get the something following response data(text).
How to avoid this and normally request and response data like browser using node.js?
Please Help... StackOverFlow is only my basis of help now!!

Struggling to perform a "simple" HTTP GET using Node.js - Destination Server Using TLS 1.0

I'm struggling to perform a "simple" HTTP GET using Node.js
I've done some digging, and I feel the error may stem from the Server Using TLS 1.0?
Below is a simple block of code to replicate the issue.
I'm using Node v10.9.0
const https = require('https');
https.get('https://supplychain.2go.com.ph', (resp) => {
let data = '';
resp.on('data', (chunk) => {
data += chunk;
});
resp.on('end', () => {
console.log(data);
});
}).on("error", (err) => {
console.log("Error: " + err.message);
});
The error returned is:
Client network socket disconnected before secure TLS connection was established
I have tried a number of things such as adding options:
rejectUnauthorized:false
I've also tried the same thing with axios and the request library. Both yield the same negative results.
I have no control over the sever end, so I cannot alter the TLS setting
Thank you for taking the time to read - Any help would be greatly appreciated.

Dialogflow Webhook (Webhook call failed. Error: 500 Internal Server Error)

I've followed this tutorial's code (https://dialogflow.com/docs/getting-started/basic-fulfillment-conversation) to return results of an API to dialog flow. However my webhook keeps failing. Can someone help me figure out why?
Here's one of the failed conversations:
Here's my code:
'use strict';
const http = require('http');
exports.Hadoop = (req, res) => {
// Get name node server from the request
let nameNodeServer = req.body.queryResult.parameters['nameNodeServer']; // nameNodeServer is a required param
// Call the Hadoop API
getNameNodeInfo(nameNodeServer).then(function(output) {
res.json({ 'fulfillmentText': output }); // Return the results to Dialogflow
}).catch(() => {
res.json({ 'fulfillmentText': 'getNameNodeInfo() Error'- });
});
};
function getNameNodeInfo (nameNodeServer) {
return new Promise((resolve, reject) => {
// Create url for the HTTP request to get the name node info
let url = 'http://' + nameNodeServer + '[rest of url]';
// Make the HTTP request to get the name node info
http.get(url, (res) => {
let body = ''; // var to store the response chunks
res.on('data', (chunk) => {body += chunk; });
res.on('end', () => {
// After all the data has been received, parse the JSON for desired data
let response = JSON.parse(body);
let beans = response['beans'][0];
// Create response
let output = `Percent Used: ${beans['PercentUsed']}`;
// Resolve the promise with the output text
console.log(output);
resolve(output);
});
res.on('error', (error) => {
console.log(`Error calling the Hadoop API: ${error}`);
reject();
});
});
});
}
I believe the getNameNodeInfo function and the retrieval of the name node server are correct, as they logged the correct output in debugging.
Diagnostic Info:
I contacted someone at Dialogflow and this was their response.
Thank you for providing all the information. I have observed in your
code that you have used http requests instead of https. The service
must use HTTPS and the URL must be publicly accessible in order for
the fulfillment to function. Dialogflow does not support self-signed
SSL certs. For information on SSL setup, please refer to this :
https://developers.google.com/web/fundamentals/security/encrypt-in-transit/enable-https
We've had a somewhat different, but related, issue:
Internal Server Error when running an agent.
“status”: {
“code”: 500,
“errorType”: “internal_server_error”,
“errorDetails”: “Internal Server Error”
},
This error was not caused by any changes we introduced. We are using that agent in a dev version of an app and one morning it stopped working.
We tested by creating a .zip and restoring into a new agent. The new agent would work properly, but we would continue to get the 500 error on the agent hooked into our dev app. We submitted a help request and overnight the error got resolved. We suspect that DialogFlow team had to manually reboot the server or something similar.

Client side unable to hit its own server endpoint

I've a AngularFullStack project in which I'm trying to hit my project's server endpoint using the following code, however the problem is the call isn't going through from angular client side to the nodejs server. Since it's my first project, I've little to no idea what might be going wrong that might be causing this weird conflict.
ClientSide:
$http({
method: 'POST',
url: '/api/soapAPIs/soapAPI',
data: request,
headers: {
'Content-Type': 'application/json'
}
})
.then(function(response) {
console.log("response from server is: ", response);
})
.catch(function(error) {
console.log("error in calling server is: ", error);
});
I have installed CORS on server side and wrote the following code in app.js but still it doesn't work.
Server App.js
// Setup server
var app = express();
var cors = require('cors');
//add cors to do the cross site requests
app.use(cors());
Server Endpoint Code:
export function soapAPI(req, res) {
console.log("SERVER SIDE ", req.body);
res.status(statusCode).json({"status":"success"});
res.send();
}
Following are the problems:
If I try to hit nodejs endpoint like this /api/soapAPIs/soapAPI, the browser shows pending request and it never goes to the server.
When I add the full classified url for the endpoint like this http:localhost:9000/api/soapAPIs/soapAPI, it hits the endpoint but then the client never receives response from the server. I don't want to provide the full classified URL on the client side for obvious reasons.
How can I resolve this issue? Please let me know if you need any other code/information.
EDIT:
By using full classified path, the server endpoint gets hit but its response is never received by the client side. I get this error in browser:
Possibly unhandled rejection: {"data":null,"status":-1,"config":{"method":"GET","transformRequest":[null],"transformResponse":[null],"jsonpCallbackParam":"callback","url":"/api/things","headers":{"Accept":"application/json, text/plain, */*"}},"statusText":""}
Uncomment the following line:
/*mongoose.connect(config.mongo.uri, config.mongo.options);
mongoose.connection.on('error', function(err) {
console.error(`MongoDB connection error: ${err}`);
process.exit(-1); // eslint-disable-line no-process-exit
});
*/
Basically this is causing your controller to not return any response because your mongoose models are being referenced but the connection to mongo fails.

Categories

Resources