I am making a call from my react app to a node server that needs to run a command on some physical hardware. The hardware takes about three minutes to complete its routine however the request always times out in my react app after only two. The hardware is connected over serial and will print "done" to the command line when finished so I need some way to see when that happens in my node server.
Ive tried increasing the setTimeout on my node server but to my understanding that only impacts the request it makes to other servers and not the requests made to it. I've also tried spawning a child process rather than running exec however I'm not sure how to access it in a later api call if it is scoped into the initial calls function.
I have a function run in my server that looks like this:
function run(command) {
return new Promise((resolve, reject) => {
exec(command, (err, stdout, stderr) => {
if(err){
reject(stderr);
}
resolve(stdout);
});
});
}
My api endpoint exec uses run like so:
app.post('/api/exec', (req, res) => {
req.setTimeout(0); //some commands take a long time, so we shouldnt time out the response
if(req.body.command){
run(req.body.command)
.then(data => {
return res.status(201).send({
message: data
});
}).catch(err => {
return res.status(500).send({
message: err
});
});
} else {
return res.status(400).send({
message: "exec requires a command member"
})
}
});
Finally, I am calling the api from my react app with the following:
execute = (c) => {
fetch(`/api/exec`, {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
command: c
})
}).then (resp => {
... do stuff with my app ...
I expect that the server would run my command until completion and then respond with the stdout from running the command, however my client application times out after ~two minutes of calling the execute function.
Several questions already have good answers regarding how to decrease the timeout time for an api call by wrapping in a timeout promise, however this method does not allow me to exceed the two minute default timeout time.
Related
I am using a simple JS script to query a Postgres DB. I simply want to write a simple query to the webpage, but every time I think I'm calling the function, it calls twice
// query the db
rate = () => pool
.query(rateQuery)
.then(res => {return res.rows[0]})
.catch(err => console.log('error: ', err.stack))
const app = http.createServer((request, response) => {
// set response header
response.writeHead(200, { 'Content-Type': 'text/html' });
// get result of promise
r = rate()
.then(res => response.write(JSON.stringify(res), () => {console.log("DONE"); response.end()}))
.catch(err => console.log('error: ', err.stack))
});
app.listen(3000);
When the page is refreshed, it prints DONE twice but I only want it once, any help is much appreciated - thanks.
If you add console.log(request.url) in your request handler, you will likely see that that the second request is when the browser asks your server for the /favicon.ico that represents the site. Browsers do this. When a user types in a URL for a site and hits enter, the browser requests that URL from the target site and then it also asks for /favicon.ico if there wasn't a previously cached icon already for that site.
In general, you should not have an http request handler like this that pays no attention to the request URL path because then you will process anything that the browser or a search crawler or anything sends you. Instead, you should look for a specific URL path and only do your work when it's the desired path and return a 404 response for all other paths.
I'd suggest you change your request handler to this:
const app = http.createServer((request, response) => {
// set response header
if (request.url === "/") {
// get result of promise
rate().then(res => {
response.writeHead(200, { 'Content-Type': 'text/plain' });
response.write(JSON.stringify(res), () => {
response.end();
console.log("DONE");
});
}).catch(err => {
console.log('error: ', err.stack);
response.writeHead(500);
response.end();
});
} else {
response.writeHead(404, "unknown URL");
response.end();
}
});
Note: I change the content-type to text/plain. You had it as text/html, but you weren't sending html at all. You could perhaps make the content-type application/json, but in any case when you're sending JSON, it's not HTML.
This also incorporates cleaner error handling where all error paths send an error response.
P.S. Writing out this simple request handler using the plain http.createServer() request handler reminds me of how much simpler it is to use Express for request handler implementations. It handles so much of this for you while still giving you full control. It's lightweight and simpler.
FYI, here's the program written using Express:
const express = require('express');
const app = express();
app.get("/", (req, res) => {
rate().then(res => {
res.json(res);
}).catch(err => {
console.log(err);
res.sendStatus(500);
});
});
// if we get here, then no other request handler handled the incoming request
// so we send back a 404
app.use((req, res) => {
res.sendStatus(404);
});
app.listen(3000);
There is one possibility. rate already is a function which you defined as
// query the db
rate = () => pool
.query(..........
Therefore, I don't think you would need parenthesis when you call it. Change from this
// get result of promise
r = rate()
.then .........
to this
// get result of promise
r = rate
.then .........
I am accessing an API that streams (RFC2616 for HTTP/1.1) the response. The stream never ends, but from time to time it throughs an error requiring the program to restart the request. I can't figure out how in NodeJS to restart the request without restarting the program. I've tried loops to check if an error occurred, but I end up creating a blocking code that stops the request methods. The code below works until an error occurs.
const request = require('request');
const options = {
method: 'GET',
url: 'https://api.tradestation.com/v3/marketdata/stream/quotes/IWM',
headers: {Authorization: 'Bearer ' + api.token,},
};
request(options)
.on('data', function (data) {
console.log(data.toString());
//Process data here
})
.on('error', function (err) {
console.error(err);
});
If it helps, the documentation to the API for streaming is here, and the API call is here.
On a side note, I tried using Axios but it doesn't seem to like endless streams (tried onDownloadProgress).
I would stick your request inside a function, you can then call it again on error.. I would also put a slight delay on the retry..
eg.
function callAPI() {
request(options)
.on('data', function (data) {
console.log(data.toString());
//Process data here
})
.on('error', function (err) {
console.error(err);
setTimeout(callAPI, 1000); //wait a second and retry.
});
}
Problem
I have a node app deployed on Heroku and I'm trying to use Heroku Scheduler to execute tasks since node-cron doesn't work with Heroku.
The scheduler runs but it always exits with code 0. A picture of the error message is below.
I've been working on this for 4 hours already with no luck so if anyone has any suggestions that would be amazing!
Code
Here's the code in the file I'm using with Heroku Scheduler. I have it scheduled to run every 10 minutes. The getActiveComps function calls a function from another file in the node app that makes calls to an external API and then writes data to MongoDB via Mongoose.
#!/app/.heroku/node/bin/node
const getActiveComps = require('../api/functions/schedulerFunctions.js');
console.log('scheduler ran');
(async () => {
console.log('before');
await getActiveComps();
console.log('after');
})();
I never did solve the problem of running async function in a scheduled job, but I did a workaround. What I did was to make an internal API that ran the function, and make the scheduled job call the internal API to run the task. This may or may not work for you.
Defined a new API in my router:
router.get('/refresh', (req, res, next) => {
//this is the function that I needed the scheduler to run.
utils.getData().then(response => {
res.status(200).json(response);
}).catch(err => {
res.status(500).json({
error: err
});
});
})
scheduled_job.js:
const fetch = require('node-fetch');
async function run() {
try {
let res = await fetch(process.env.REFRESH_PATH);
result = await res.json();
console.log(result);
} catch (err) {
console.log(err);
}
}
run();
Of course, you should set your API path inside the heroku config vars.
Hope this helps you.
I have a Node.js system that uploads a large number of objects to MongoDB and creates folders in dropbox for each object. This takes around 0.5 seconds per object. In situations therefore where i have many objects this could take up to around a minute. What i currently do is notify the client that the array of objects has been accepted using a 202 response code. However how do i then notify the client of completion a minute later.
app.post('/BulkAdd', function (req, res) {
issues = []
console.log(req.body)
res.status(202).send({response:"Processing"});
api_functions.bulkAdd(req.body).then( (failed, issues, success) => {
console.log('done')
})
});
bulkAdd: async function (req, callback) {
let failed = []
let issues = []
let success = []
i = 1
await req.reduce((promise, audit) => {
// return promise.then(_ => dropbox_functions.createFolder(audit.scanner_ui)
let globalData;
return promise.then(_ => this.add(audit)
.then((data)=> {globalData = data; return dropbox_functions.createFolder(data.ui, data)}, (error)=> {failed.push({audit: audit, error: 'There was an error adding this case to the database'}); console.log(error)})
.then((data)=>{console.log(data, globalData);return dropbox_functions.checkScannerFolderExists(audit.scanner_ui)},(error)=>{issues.push({audit: globalData, error: 'There was an error creating the case folder in dropbox'})})
.then((data)=>{return dropbox_functions.moveFolder(audit.scanner_ui, globalData.ui)},(error)=>{issues.push({audit: globalData, error: 'No data folder was found so an empty one was created'}); return dropbox_functions.createDataFolder(globalData.ui)})
.then(()=>success.push({audit:globalData}), issues.push({audit: globalData, error: 'Scanner folder found but items not moved'}))
);
}, Promise.resolve()).catch(error => {console.log(error)});
return(failed, issues, success)
},
Well the problem with making client request wait, is it will timeout after certain period or sometimes will show error with no response received.
What you can do is
- Make client request to server to initiate the task, and return 200OK and keep doing your task on server.
- Now write a file on server after insertion of every object as status.
- Read the file from client every 5-10 sec to check if server has completed creating objects or not.
- Mean while your task is not completed on server, show status with completion percentage or some animation.
Or simply implement WebHook or WebSockets to maintain communication.
I am trying to run a node js child process in the backend which is taking at least 10 min time to complete, and I want to send the response after 10 min. In node js, i have increased the server timeout to 10 min and its fine. But in the browser, it's still a problem. Its saying timeout error after 2 min of waiting.
As I am using axios module from npm, it has the functionality of timeout but that not working.
Could someone help me how to stalled browser for more than 10 min to send the message from the server and then render the UI?
UI Code:
action created code->
export const executeFile = (fileName) => {
return ((dispatch)=>{
axios.post(`/execute`,{fileName},{timeout:1000000}).then(
res=> dispatch({
type:'EXECUTE_FILE',
payload:res
})
).catch(err=>console.log(err))
})
}
backend code->
app.post('/execute',(req,res)=>{
console.log(req.body.fileName,"aaaaaaaaa");
// handler_function.deleteFolderRecursive()
exec('sleep 200 && ls', (err, stdout, stderr) => {
if (err) {
console.error(err,"here");
return;
}
console.log(stdout,"in here sucess");
res.send(stdout)
});
});