Problem
I have a node app deployed on Heroku and I'm trying to use Heroku Scheduler to execute tasks since node-cron doesn't work with Heroku.
The scheduler runs but it always exits with code 0. A picture of the error message is below.
I've been working on this for 4 hours already with no luck so if anyone has any suggestions that would be amazing!
Code
Here's the code in the file I'm using with Heroku Scheduler. I have it scheduled to run every 10 minutes. The getActiveComps function calls a function from another file in the node app that makes calls to an external API and then writes data to MongoDB via Mongoose.
#!/app/.heroku/node/bin/node
const getActiveComps = require('../api/functions/schedulerFunctions.js');
console.log('scheduler ran');
(async () => {
console.log('before');
await getActiveComps();
console.log('after');
})();
I never did solve the problem of running async function in a scheduled job, but I did a workaround. What I did was to make an internal API that ran the function, and make the scheduled job call the internal API to run the task. This may or may not work for you.
Defined a new API in my router:
router.get('/refresh', (req, res, next) => {
//this is the function that I needed the scheduler to run.
utils.getData().then(response => {
res.status(200).json(response);
}).catch(err => {
res.status(500).json({
error: err
});
});
})
scheduled_job.js:
const fetch = require('node-fetch');
async function run() {
try {
let res = await fetch(process.env.REFRESH_PATH);
result = await res.json();
console.log(result);
} catch (err) {
console.log(err);
}
}
run();
Of course, you should set your API path inside the heroku config vars.
Hope this helps you.
Related
Recently, I decided to try out vercel for a small project and ran into issues using sendgrid with vercel. I've implemented sendgrid into nodejs/express projects before on heroku with no issues, but for some reason Vercel just doesn't work.
Even worse, sendgrid gives no errors whatsoever so its been difficult to troubleshoot. The promise just hangs forever. Everything works fine locally when testing and I've confirmed that env variables are correct and loading.
Here's my code:
const sgMail = require('#sendgrid/mail');
const { magicLinkTemplate } = require('./templates');
const sendEmail = async (msg) => {
sgMail.setApiKey(process.env.SENDGRID_API_KEY);
try {
await sgMail.send(msg)
} catch (error) {
console.error(error);
if (error.response) {
console.error(error.response.body)
}
}
}
I've tried a number of iterations, and failed every time. Has anyone else run into issues using vercel with sendgrid?
Alright, so I believe I solved my own problem.
The issue lies in how vercel handles serverless functions:
If you're on free tier, each request must resolve in 10 seconds (for pro its 60)
Vercel seems to instantly terminate a serverless function once a response is returned, aka side effects that are not finished will cease to run.
Previously, I was running the email sends as a side effect and the api response would return without waiting for sendgrid to fullfil its promise. Below is a watered down version of my original code.
router.post('/', checkUser)
async function checkUser(req, res, next) {
const { email } = req.body;
const magicLink = generateMagicLink()
sendMagicLink(email, magicLink)
res.status(200).json({})
}
The sendMagicLink function is async and would continue to run locally, but in vercel it would cease once a status was returned from the api.
New code looks like this...
router.post('/', checkUser)
async function checkUser(req, res, next) {
const { email } = req.body;
const magicLink = generateMagicLink()
await sendMagicLink(email, magicLink)
res.status(200).json({})
}
Now the serverless function stays alive because it waits for the async process to finish.
This whole thing tripped me up because if you have a persistent server, there would be no issues with the original version. Hope this helps someone!
I have a middleware error handler that is working great but next(err) and return and return next(err) seems to not stop execution when dealing with promises. What is the proper way to stop execution of my code when an error is found?
For reference: err is a standard Error class in this case.
I don't think you need the code in userProvider.fetchFriends to help with this but if that's wrong please let me know.
const uid = req.query.steamUserId;
//Use the userProvider to get steam friend data
const friendData = await userProvider.fetchFriends(uid)
.catch(err => {
return next(err); //Does not stop execution. Why? just next(err) doesn't either.
});
//Putting this after every catch works but seems very stupid. How do I avoid this?
if(res.writableEnded)
return;
...Other code that runs but causes errors
}
You've got two problems here.
First: next() is explicitly continuing to the next middleware or endpoint.
To finish dealing with the request, send a response.
const middleware = (req, res, next) => {
if (something(req)) {
next();
} else {
res.status(500).json({ ok: false, msg: "some error message" });
}
}
Second: You need to watch your asynchronous logic carefully.
You can't:
trigger something asynchronous
send a response
send a different response when the asynchronous function is complete
You can only send one response to a request.
Either:
Don't call next or res.something in the catch block and just log the error internally or
Don't call next or res.something outside the promise handling and move it to a then handler instead (you might want to switch to using async/await to make your logic easier to follow)
The issue was that I was mixing async/await with .then/.catch. Needed to use try/catch.
ty #jonsharpe
export const getSteamFriends = async (req, res, next) => {
try{
const uid = req.query.steamUserId;
//Use the userProvider to get steam friend data
const friendData = await userProvider.fetchFriends(uid);
//more code in the middle
} catch(e) {
return next(e);
}
};
I am making a backend api with node/express and am running into an error. I've tried many different approaches as well as looked at many different questions on here, but none seem to fix my problem. As far as I can tell my routes are structured properly, but they simply don't work. Perhaps I'm missing something super obvious, but I'm at a loss.
Bellow is a simplified version of my index.js where I start my node server:
const movies = require("./routes/movies");
app.use("/api/movies/", movies);
const port = process.env.PORT || 5000;
app.listen(port, () => console.log(`Listening on port ${port}...`));
And then in my movies.js I have several endpoints:
// DOES WORK - returns movies with the specifed object
router.get("/genre/:id", async (req, res) => {
const movies = await Movie.find({ genres: req.params.id });
if (!movies)
return res
.status(400)
.send(`No movies with genre of id ${req.params.id} could be found.`);
res.send(movies);
});
// DOESN'T WORK - returns main html page
router.get("/tmdbid/:id", async (req, res) => {
const movie = await Movie.findOne({
tmdbID: req.params.id,
});
if (!movie)
return res
.status(400)
.send(`No movies with tmdbID of ${req.params.id} could be found.`);
if (res.status(404)) res.send("Route not found");
res.send(movie);
});
// DOESN'T WORK - returns Cannot GET /api/movies/title/{title}
router.get("/title/:title", async (req, res) => {
const movie = await Movie.find({
title: { $regex: new RegExp(req.params.title, "i")
},});
if (!movie)
return res
.status(400)
.send(`The movie with name of ${req.params.title} could not be found.`);
res.send(movie);
});
I'm not sure what needs to be done here. All of these share similar syntax, but each result in a different response. None of them log any errors to the terminal either. I know for sure general refactoring should be done, but as far as the problem at hand goes, I can't wrap my head around why the last two aren't working. Lastly, I have several other endpoints in the this file, all of which are working fine. Just these two have issues.
Add a console.log statement to check whether the endpoint is hit or not. The code logic in your code sample above sounds right. However, I would try formatting the code batter.
if (!movie){
return res.status(400).send(`No movies with genre of id ${req.params.id} could be found.`);
}
Also, make sure you do not forget to restart your node js server.
What are those return statements doing there? and you could improve your if statements as well like such ->
try {
const movie = await Movie.findOne({
tmdbID: req.params.id,
});
if (!movie) {
res
.status(400)
.send(`No movie(s) could be found.`);
} else {
res
.status(200)
.send(movie);
}
} catch(err) {
res.send(500);
}
I eventually found my answer and it was simply a mistake I had in the url when calling the endpoint, not in the endpoint's code itself. Rather than calling to localhost:5000, I was calling to localhost:3000, which is probably why it was returning the index.html of my react app.
So, my code works as is. Perhaps not the best structure ever, but as long as it works, I'm happy.
Still not sure why the /title/:title endpoint wasn't working and then suddenly was. That remains a mystery.
I am making a call from my react app to a node server that needs to run a command on some physical hardware. The hardware takes about three minutes to complete its routine however the request always times out in my react app after only two. The hardware is connected over serial and will print "done" to the command line when finished so I need some way to see when that happens in my node server.
Ive tried increasing the setTimeout on my node server but to my understanding that only impacts the request it makes to other servers and not the requests made to it. I've also tried spawning a child process rather than running exec however I'm not sure how to access it in a later api call if it is scoped into the initial calls function.
I have a function run in my server that looks like this:
function run(command) {
return new Promise((resolve, reject) => {
exec(command, (err, stdout, stderr) => {
if(err){
reject(stderr);
}
resolve(stdout);
});
});
}
My api endpoint exec uses run like so:
app.post('/api/exec', (req, res) => {
req.setTimeout(0); //some commands take a long time, so we shouldnt time out the response
if(req.body.command){
run(req.body.command)
.then(data => {
return res.status(201).send({
message: data
});
}).catch(err => {
return res.status(500).send({
message: err
});
});
} else {
return res.status(400).send({
message: "exec requires a command member"
})
}
});
Finally, I am calling the api from my react app with the following:
execute = (c) => {
fetch(`/api/exec`, {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
command: c
})
}).then (resp => {
... do stuff with my app ...
I expect that the server would run my command until completion and then respond with the stdout from running the command, however my client application times out after ~two minutes of calling the execute function.
Several questions already have good answers regarding how to decrease the timeout time for an api call by wrapping in a timeout promise, however this method does not allow me to exceed the two minute default timeout time.
For some reason, I cannot get testcafe's RequestLogger to log any of the API calls that I am making. I've read nearly all of the articles and questions on the RequestLogger and everything is pointing to what appears to be the same as the below code as working. Not sure what I'm doing wrong, any help would be great.
References:
https://devexpress.github.io/testcafe/documentation/test-api/intercepting-http-requests/logging-http-requests.html
Cannot intercept outgoing AJAX request from page using Testcafe
How to log Google Analytics calls in Testcafe?
I am running locally and hitting an API that is running locally as well, front-end on port 3000 and backend on port 8080, API is at: 8080/api/admin. I can see the logger as injected into the test but nothing updates it, its just a bland object with initial props and will error out after the t.expect statement.
I wonder if the beforeEach is breaking something but I need it in order to fire any API calls because the user needs to be authenticated. I can see the API request being called when debugging, that I am trying to intercept, but no luck
testcafe version: 1.0.0 || 0.23.3
Test code
// have tried several urls, from exact to generic to ports.
const logger = RequestLogger("/api/", {
logRequestHeaders: true,
logRequestBody: true
});
const url = 'localhost:3000/reports';
fixture `Report`
.page(url)
.requestHooks(logger)
.beforeEach(async (t: TestController) => {
await loginAndNavToReports({ t });
});
test("Reports", async (t: TestController) => {
// this fires an api call through the /api/ path
await t.click(".test-reportLaborSummary");
// have tried several comparisons here, all fail or expect falsey to be truthy errors
await t.expect(logger.count(() => true)).ok();
}
I suspect that TestCafe runs faster than the code that calls the api.
Before using the logger object you should wait that it has received at least one call.
To check if the logger has received a call, I suggest to do it this way:
await wait_for_first_request();
const receivedCalls = logger.requests.length;
if (receivedCalls === 0) {
throw new Error('api has not been called')
}
async function wait_for_first_request() {
for (let i = 0; i < 50; i++) {
await t.wait(100);
if (logger.requests.length > 0 ) {
return;
}
}
}