I am using request to access data from an api. sometimes, when the api Url is down, node throws error and shuts down with the following error
`at ClientRequest.emit (events.js:315:20)`
`at TLSSocket.socketErrorListener (_http_client.js:426:9)`
`at TLSSocket.emit (events.js:315:20)`
`at emitErrorNT (internal/streams/destroy.js:92:8)`
`at processTicksAndRejections (internal/process/task_queues.js:84:21)
errno: 'ENOTFOUND',
code: 'ENOTFOUND',
syscall: 'getaddrinfo',
hostname: APIURL
is there any way to handle this and prevent node from exiting?
my code is
const JSONdata= ()=>{
request({url:API_URL2,json:true},(error,response,body)=>{
const data=[]
var jsondata=body
data.push(jsondata)
const data2=JSON.stringify(data)
fs.writeFile('sample.txt',data2,(err) => { if(error){console.log(error} })})
having a try-catch will prevent failure or system crash, also you get process.uncaughtException() to do something when the system crashes
check below link for more details
https://nodejs.org/api/process.html#process_event_uncaughtexception
Related
I want my program to continue running after catching ECONNRESET. How do I do that? I'm using basic-ftp for my ftp library. Here's the error below:
Node.js v17.7.2
node:events:505
throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at TCP.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on Socket instance at:
at emitErrorNT (node:internal/streams/destroy:164:8)
at emitErrorCloseNT (node:internal/streams/destroy:129:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -4077,
code: 'ECONNRESET',
syscall: 'read'
}
How do I catch it and have the program continue running? I've already tried try/catch blocks shown below:
async connect(port,ip_address){
try{
await this.client.access({
port: port,
host: ip_address,
secure: false
})
}
catch{
this.num_of_files_read=0;
this.files_completed=0;
return 1
}
return 0
}
getting error:While implementing ethereum blockchain code ,
error that i am getting is :Invalid JSON RPC response while using INFURA
Attaching error log below kindly help
Error: Invalid JSON RPC response: ""
at Object.InvalidResponse (/home/joshi/Desktop/inbox/node_modules/truffle-hdwallet-provider/node_modules/web3/lib/web3/errors.js:35:16)
at XMLHttpRequest.request.onreadystatechange (/home/joshi/Desktop/inbox/node_modules/truffle-hdwallet-provider/node_modules/web3/lib/web3/httpprovider.js:115:32)
at XMLHttpRequestEventTarget.dispatchEvent (/home/joshi/Desktop/inbox/node_modules/xhr2/lib/xhr2.js:64:18)
at XMLHttpRequest._setReadyState (/home/joshi/Desktop/inbox/node_modules/xhr2/lib/xhr2.js:354:12)
at XMLHttpRequest._onHttpRequestError (/home/joshi/Desktop/inbox/node_modules/xhr2/lib/xhr2.js:544:12)
at ClientRequest.<anonymous> (/home/joshi/Desktop/inbox/node_modules/xhr2/lib/xhr2.js:414:24)
at emitOne (events.js:116:13)
at ClientRequest.emit (events.js:211:7)
at TLSSocket.socketErrorListener (_http_client.js:387:9)
at emitOne (events.js:116:13)
at TLSSocket.emit (events.js:211:7)
at emitErrorNT (internal/streams/destroy.js:64:8)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9)
If you don't post more information, such as a little piece of code is difficult to understand what is happening.
What you should try is this:
In every link to infura api remove the "/v3" part from the url.
From this:
const provider = new HDWalletProvider(mnemonic,
'https://rinkeby.infura.io/v3/project_id');
To this:
const provider = new HDWalletProvider(mnemonic,
'https://rinkeby.infura.io/project_id');
Hope this helps.
Don't use www and /v3 in infura API URI and it's required to use https in the URI. eg.
https://rinkeby.infura.io/project_id
I'm posting some sample data to my Smee endpoint and I am also running a local instance of my ProBot app. I get a bad request in my bot logs and I see this is the Smee response:
{ Error: cannot POST / (400)
at Response.toError (/usr/local/lib/node_modules/smee-client/node_modules/superagent/lib/node/response.js:94:15)
at ResponseBase._setStatusProperties (/usr/local/lib/node_modules/smee-client/node_modules/superagent/lib/response-base.js:123:16)
at new Response (/usr/local/lib/node_modules/smee-client/node_modules/superagent/lib/node/response.js:41:8)
at Request._emitResponse (/usr/local/lib/node_modules/smee-client/node_modules/superagent/lib/node/index.js:752:20)
at IncomingMessage.parser (/usr/local/lib/node_modules/smee-client/node_modules/superagent/lib/node/index.js:916:38)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1055:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9)
status: 400,
text: 'Required headers missing: x-github-event, x-github-delivery',
method: 'POST',
path: '/' },
How can I get these header values to test with in my app? Not sure if it's me, but the documentation doesn't note this anywhere
I used https://www.freeformatter.com/hmac-generator.html#ad-output to throw in the payload as the body and my secret and the hash was valid!
im using heroku to host my site however it keeps spitting out the same error which is
2018-05-06T19:28:52.212104+00:00 app[web.1]:AssertionError [ERR_ASSERTION]: false == true
2018-05-06T19:28:52.212106+00:00 app[web.1]:at Object.exports.connect (_tls_wrap.js:1036:3)
2018-05-06T19:28:52.212107+00:00 app[web.1]: at Socket. (/app/node_modules/pg/lib/connection.js:94:23)
2018-05-06T19:28:52.212109+00:00 app[web.1]:at Object.onceWrapper (events.js:315:30)
2018-05-06T19:28:52.212110+00:00 app[web.1]:at emitOne (events.js:116:13)
2018-05-06T19:28:52.212112+00:00 app[web.1]: at Socket.emit (events.js:211:7)
2018-05-06T19:28:52.212113+00:00 app[web.1]:at addChunk (_stream_readable.js:263:12)
2018-05-06T19:28:52.212115+00:00 app[web.1]:at readableAddChunk (_stream_readable.js:250:11)
2018-05-06T19:28:52.212116+00:00 app[web.1]:at Socket.Readable.push (_stream_readable.js:208:10)
2018-05-06T19:28:52.212118+00:00 app[web.1]: at TCP.onread >>(net.js:607:20)
this is how im connecting (taken straight off of herokus website)
const { Client } = require("pg");
const router = express.Router();
const pool = new Client({
connectionString: process.env.DATABASE_URL,
ssl: true,
});
pool.connect();
I found out what the error was:
the error was that i did not have a db create. i decided to use elephantSQL and from their i connected to elephantSQL with pgadmin and recreated my database. it works perfect now
to see how this works see : https://www.elephantsql.com/docs/pgadmin.html
im more than happy to answer some questions
OK, this isn't really a solution but it might help you out. This happens to me on node 8 (and 10) but when I downgraded to node 7 it seems to work fine. I'll be watching this post to see if someone comes up with a better fix.
I set up a very basic web scraper to check stock of a specific item on Costco.com for my grandfather. It's working great locally, but when I run it through Heroku it fails (seemingly 50% of the time). Here's the code for the scraper
const task = () => {
// toggle so doesn't send message multiple times if continuously available
let alreadyAvailable = false;
let url = 'http://www.costco.com/Kirkland-Signature-Four-Piece-Urethane-Cover-Golf-Ball,-2-dozen.product.100310467.html';
request(url, function(error, response, html){
let $ = cheerio.load(html);
if(error){
throw new Error(error);
}
if ( $('#product-page #product-details #ctas #add-to-cart input[type="button"]')['0'].attribs.value === 'Out of Stock') {
alreadyAvailable = false;
console.log("still out of stock");
} else {
if (alreadyAvailable === false) {
sendMessage();
alreadyAvailable = true;
}
}
});
};
and here are the logs
2016-12-25T03:48:39.675549+00:00 heroku[scheduler.5440]: Starting process with command `node scraper.js`
2016-12-25T03:48:40.262503+00:00 heroku[scheduler.5440]: State changed from starting to up
2016-12-25T03:48:41.509416+00:00 app[scheduler.5440]: /app/scraper.js:34
2016-12-25T03:48:41.509432+00:00 app[scheduler.5440]: if ( $('#product-page #product-details #ctas #add-to-cart input[type="button"]')['0'].attribs.value === 'Out of Stock') {
2016-12-25T03:48:41.509433+00:00 app[scheduler.5440]: ^
2016-12-25T03:48:41.509433+00:00 app[scheduler.5440]:
2016-12-25T03:48:41.509434+00:00 app[scheduler.5440]: TypeError: Cannot read property 'attribs' of undefined
2016-12-25T03:48:41.509434+00:00 app[scheduler.5440]: at Request._callback (/app/scraper.js:34:90)
2016-12-25T03:48:41.509435+00:00 app[scheduler.5440]: at Request.self.callback (/app/node_modules/request/request.js:186:22)
2016-12-25T03:48:41.509436+00:00 app[scheduler.5440]: at emitTwo (events.js:106:13)
2016-12-25T03:48:41.509436+00:00 app[scheduler.5440]: at Request.emit (events.js:191:7)
2016-12-25T03:48:41.509436+00:00 app[scheduler.5440]: at Request.<anonymous> (/app/node_modules/request/request.js:1081:10)
2016-12-25T03:48:41.509437+00:00 app[scheduler.5440]: at emitOne (events.js:96:13)
2016-12-25T03:48:41.509437+00:00 app[scheduler.5440]: at Request.emit (events.js:188:7)
2016-12-25T03:48:41.509438+00:00 app[scheduler.5440]: at IncomingMessage.<anonymous> (/app/node_modules/request/request.js:1001:12)
2016-12-25T03:48:41.509438+00:00 app[scheduler.5440]: at IncomingMessage.g (events.js:291:16)
2016-12-25T03:48:41.509439+00:00 app[scheduler.5440]: at emitNone (events.js:91:20)
2016-12-25T03:48:41.509439+00:00 app[scheduler.5440]: at IncomingMessage.emit (events.js:185:7)
2016-12-25T03:48:41.509439+00:00 app[scheduler.5440]: at endReadableNT (_stream_readable.js:974:12)
2016-12-25T03:48:41.509440+00:00 app[scheduler.5440]: at _combinedTickCallback (internal/process/next_tick.js:74:11)
2016-12-25T03:48:41.509440+00:00 app[scheduler.5440]: at process._tickCallback (internal/process/next_tick.js:98:9)
2016-12-25T03:48:41.560539+00:00 heroku[scheduler.5440]: State changed from up to complete
2016-12-25T03:48:41.550655+00:00 heroku[scheduler.5440]: Process exited with status 1
2016-12-25T03:58:42.438807+00:00 app[api]: Starting process with command `node scraper.js` by user scheduler#addons.heroku.com
2016-12-25T03:58:43.701468+00:00 heroku[scheduler.5038]: Starting process with command `node scraper.js`
2016-12-25T03:58:44.312279+00:00 heroku[scheduler.5038]: State changed from starting to up
2016-12-25T03:58:45.769564+00:00 app[scheduler.5038]: still out of stock
2016-12-25T03:58:45.827867+00:00 heroku[scheduler.5038]: State changed from up to complete
2016-12-25T03:58:45.814921+00:00 heroku[scheduler.5038]: Process exited with status 0
You can see that sometimes I get the console log inside of the if-block and others I get a type error because it's trying to read attributes from an html element that don't exist. I was thinking that this might be a async issue, but I'm not sure how to go about fixing it. I assumed Request wasn't running the callback until it got all the html.
The problem here is what Costco's website is returning.
Your code is failing when parsing the DOM via Cheerio. What this means in your case is that the particular HTML you're attempting to scrape doesn't actually exist (that's what the error is saying).
This could be caused by a few possible things:
Costco is rendering a DIFFERENT page than what you expect (maybe it thinks you're a bot, or is doing some throttling).
You are receiving a redirect or some other sort of non-error HTTP status code, and the HTML you're looking for doesn't exist there.
Costco's website changes the HTML dynamically to prevent people from scraping.
What I would do if I were you is this:
Have your process log all the page's HTML when the task runs.
The next time your process fails, copy the HTML from the Heroku logs into a local editor, and see what it returned.
I'm willing to bet you will be surprised =)