I'm trying to pass a variable to EJS a second time in my code and am running into trouble. Here is my code:
axios.get(yearURL)
.then(function (res) {
let data = res.data.MRData.SeasonTable.Seasons;
years = data.map(d => d.season);
app.get('/', function (req, res) {
res.render('index.ejs', {
years: years
});
app.post('/down', function(req, res) {
let year = req.body;
res.redirect('/');
axios.get(`http://ergast.com/api/f1/${year.year}/drivers.json`)
.then(function (res) {
let data = res.data.MRData.DriverTable.Drivers;
drivers = data.map(d => `${d.givenName} ${d.familyName}`);
})
.catch(function (err) {
console.log(err);
})
res.render('index.ejs', {
drivers: drivers,
years: years
});
Whenever I run this however, I receive an error that I cannot set headers after they are sent to the client. I've also read elsewhere that apparently you can not call res.render twice. So my question is, how can I pass another set of data to EJS after I have already called res.render once before?
Here it is as pseudocode. It's good to start your program with this level of logical structure, and then implement it:
Define ready = false, errored = false, and data = undefined
variables.
Get the data from the remote API, in the then branch, set
ready = true, assign result to data. In the error branch, set errored
= true. Should we retry on error?
Define the / GET route.
If not ready, check errored. If not errored, we are still waiting for the data. In this case, do we wait for the call to resolve, or return something to the client to let them know?
If not ready, and errored, tell the client that there was an error.
If ready == true, then we have data to render a response to the client.
Define the /down route. It needs to take a year parameter, and we need to make an async call in the route handler to get the data.
Can we cache the data, so that subsequent calls for the same year return data that we fetched previously? If we can, use an object as a lookup dictionary. If the object has a key for that year, use the cached data to render the response. If not, make the call, and in the then branch, add the response to the cache object, and use the data to render the response.
Related
The Nodejs functions return an error from try/catch scope, such as the one below if the user doesn't exist of if a database is not reachable:
router.delete('/delete/:email', async (req, res) => {
var email = req.params.email;
try {
let result = await User.remove({"email": email});
res.status(204).send(email);
} catch (err) {
res.status(400).send(err);
}
});
I can also return the Error from Nodejs server by myself:
return res.status(400).send(new Error(`The user with email ${email} doesn't exist.`));
The first problem is that I can't find the error message that is embedded somewhere deep in the body the returned Error object. It is stored in one of its 100+ attributes. Where should I look for it so I could display in on a screen for the end user to read it?
Then, the err object generated by the try/catch scope has a set of different attributes comparing to the Error object created with new Error("Here is my error message"). Is there a way to normalize the returned Errors so they all have the same or similar attributes?
You don't need to return the whole error object from the server, and arguably shouldn't since error messages can expose internals about your code and infrastructure.
One way you could handle this is to format and return an error message from the server yourself. Assuming you're using express this would look something like:
return res.status(400).json({ message: `The user with email ${email} doesn't exist.` });
Alternatively you could use an error handling middleware like strong-error-handler found here: https://github.com/strongloop/strong-error-handler which automatically builds a json formatted message that's easier to parse, but keep in mind that the content of the message differs depending on whether you set debug mode to true or no.
If you want to develop a secure web application with nice error handling, i will suggest you the following structure.
Step 1. At front end divide your api calls in four main operations for e.g. inset,update,query and filter.
now whenever your page loads and you want to show some data fetched from server then your api call must be like 'https://domainname.tld/server/query' and send some payload with this api call according to need of your data requirements to be fetched.
At backend probably at Server.js handle like this :
app.all("/server/query", function (req, res) {
try {
console.log(a);
// some database or io blocking process
} catch (error) {
// error handling
var err = writeCustomError(error.message || error.errmsg || error.stack);
res.status(417).json(err).end();
}
});
function writeCustomError(message) {
var errorObject = {};
errorObject.message = message;
errorObject.code = 10001; // as you want
errorObject.status = "failed";
return errorObject;
}
in try block you can also handle logical errors using same function i.e writeCustomError
So if you use this approach you can also implement end-to-end encryption and send only eP('encrypted payload') and eK('encryption Key'),by doing this end users and bad end users even can not evaluate your serve API calls.
If you are thinking how will you route different paths at server then simplest solution is send uri in payload from client to server for e.g
User wants to reset password :-
then
call api like this
https://domain.tld/server/execute and send Json object in payload like this {uri:"reset-password",old:"",new:""}.
at backend
use
app.all("/server/execute", function (req, res) {
try {
// decrypt payload
req.url = payload.uri;
next();
} catch (error) {
// error handling
var err = writeCustomError(error.message || error.errmsg || error.stack);
res.status(417).json(err).end();
}
});
app.all("/reset-password", function (req, res) {
try {
// reset logic
} catch (error) {
// error handling
var err = writeCustomError(error.message || error.errmsg || error.stack);
res.status(417).json(err).end();
}
});
so in this way only developer know where password reset logic and how it can called and what parameters are required.
I will also suggest you to create different router files for express like QueryRouter,InsertRouter etc.
Also try to implement end-to-end encryption.Any query regarding post,kindly comment it.
I'm creating an app with SignalR Core on Azure. Now I need to match the invoked request with the returned value and callback. I would like to use the invocationId to make sure the method that will be executed is the latest invoked request. To avoid as happens in the attached image.
var _this.latestInvokeId = ??
connection.invoke('getData', _this.ID)
.then(function (data) {
if(this.invocationId === _this.latestInvokeId)
console.log('Use the data', data)
})
.catch(function (error) {
console.log(error);
});
When the getData is invoked multiple times in a short interval I would like to be sure to use the right response. So the response with invocationId 8 needs to be ignored and only the response with invocationId 9 is used.
I've implemented a new id that is send on the request and can be validated when the value is returned.
I have a Node.js system that uploads a large number of objects to MongoDB and creates folders in dropbox for each object. This takes around 0.5 seconds per object. In situations therefore where i have many objects this could take up to around a minute. What i currently do is notify the client that the array of objects has been accepted using a 202 response code. However how do i then notify the client of completion a minute later.
app.post('/BulkAdd', function (req, res) {
issues = []
console.log(req.body)
res.status(202).send({response:"Processing"});
api_functions.bulkAdd(req.body).then( (failed, issues, success) => {
console.log('done')
})
});
bulkAdd: async function (req, callback) {
let failed = []
let issues = []
let success = []
i = 1
await req.reduce((promise, audit) => {
// return promise.then(_ => dropbox_functions.createFolder(audit.scanner_ui)
let globalData;
return promise.then(_ => this.add(audit)
.then((data)=> {globalData = data; return dropbox_functions.createFolder(data.ui, data)}, (error)=> {failed.push({audit: audit, error: 'There was an error adding this case to the database'}); console.log(error)})
.then((data)=>{console.log(data, globalData);return dropbox_functions.checkScannerFolderExists(audit.scanner_ui)},(error)=>{issues.push({audit: globalData, error: 'There was an error creating the case folder in dropbox'})})
.then((data)=>{return dropbox_functions.moveFolder(audit.scanner_ui, globalData.ui)},(error)=>{issues.push({audit: globalData, error: 'No data folder was found so an empty one was created'}); return dropbox_functions.createDataFolder(globalData.ui)})
.then(()=>success.push({audit:globalData}), issues.push({audit: globalData, error: 'Scanner folder found but items not moved'}))
);
}, Promise.resolve()).catch(error => {console.log(error)});
return(failed, issues, success)
},
Well the problem with making client request wait, is it will timeout after certain period or sometimes will show error with no response received.
What you can do is
- Make client request to server to initiate the task, and return 200OK and keep doing your task on server.
- Now write a file on server after insertion of every object as status.
- Read the file from client every 5-10 sec to check if server has completed creating objects or not.
- Mean while your task is not completed on server, show status with completion percentage or some animation.
Or simply implement WebHook or WebSockets to maintain communication.
I'm trying to use cron to trigger my cloud function in order to look at my database every couple hours. I can trigger the function automatically, but the output is not what I was expecting. I'm just a tad bit confused on why I am not able to retrieve anything from the database, meaning I can't log out my console.log("refund"). I currently have 1 document in the request collection and has 1 file with the replied field that satisfies replied == false. So I'm just confused on how to go about this correctly since I believe it should've logged that once?
exports.daily_job = functions.https.onRequest((req, res) => {
const key = req.query.key;
// Exit if the keys don't match.
if (!secureCompare(key, functions.config().cron.key)) {
console.log('The key provided in the request does not match the key set in the environment. Check that', key,
'matches the cron.key attribute in `firebase env:get`');
res.status(403).send('Security key does not match. Make sure your "key" URL query parameter matches the ' +
'cron.key environment variable.');
return null;
}
let db = admin.firestore()
let request = db.collection('request')
.where('replied', '==', false)
.get().then(function(querySnapshot){
querySnapshot.forEach(function(doc) {
console.log("refunded")
})
})
.catch(function (error) {
console.log('Error getting documents: ', error)
res.send('error');
})
res.send('finished refund');
return null;
});
You're not waiting on the promise returned by get(), which is asynchronous. As your code is now, the entire function sends a response "finished refund" immediately after the query is made, before it has time to finish. Once you send a response, the function is terminated.
You need to send the client response only after all the async work is complete in your function, which would be in BOTH your then() and catch() callbacks.
I have a Node.js server which queries MySQL database. It serves as an api end point where it returns JSON and also backend server for my Express application where it returns the retrieved list as an object to the view.
I am looking into implementing flat-cache for increasing the response time. Below is the code snippet.
const flatCache = require('flat-cache');
var cache = flatCache.load('productsCache');
//get all products for the given customer id
router.get('/all/:customer_id', flatCacheMiddleware, function(req, res){
var customerId = req.params.customer_id;
//implemented custom handler for querying
queryHandler.queryRecordsWithParam('select * from products where idCustomers = ? order by CreatedDateTime DESC', customerId, function(err, rows){
if(err) {
res.status(500).send(err.message);
return;
}
res.status(200).send(rows);
});
});
//caching middleware
function flatCacheMiddleware(req, res, next) {
var key = '__express__' + req.originalUrl || req.url;
var cacheContent = cache.getKey(key);
if(cacheContent){
res.send(cacheContent);
} else{
res.sendResponse = res.send;
res.send = (body) => {
cache.setKey(key,body);
cache.save();
res.sendResponse(body)
}
next();
}
}
I ran the node.js server locally and the caching has indeed greatly reduced the response time.
However there are two issues I am facing that I need your help with.
Before putting that flatCacheMiddleware middleware, I received the response in JSON, now when I test, it sends me an HTML. I am not too well versed with JS strict mode (planning to learn it soon), but I am sure the answer lies in the flatCacheMiddleware function.
So what do I modify in the flatCacheMiddleware function so it would send me JSON?
I manually added a new row to the products table for that customer and when I called the end point, it still showed me the old rows. So at what point do I clear the cache?
In a web app it would ideally be when the user logs out, but if I am using this as an api endpoint (or even on webapp there is no guarantee that the user will log out the traditional way), how do I determine if new records have been added and the cache needs to be cleared.
Appreciate the help. If there are any other node.js caching related suggestions you all can give, it would be truly helpful.
I found a solution to the issue by parsing the content to JSON format.
Change line:
res.send(cacheContent);
To:
res.send(JSON.parse(cacheContent));
I created cache 'brute force' invalidation method. Calling clear method will clear both cache file and data stored in memory. You have to call it after db change. You can also try delete specified key using cache.removeKey('key');.
function clear(req, res, next) {
try {
cache.destroy()
} catch (err) {
logger.error(`cache invalidation error ${JSON.stringify(err)}`);
res.status(500).json({
'message' : 'cache invalidation error',
'error' : JSON.stringify(err)
});
} finally {
res.status(200).json({'message' : 'cache invalidated'})
}
}
Notice, that calling the cache.save() function will remove other cached API function. Change it into cache.save(true) will 'prevent the removal of non visited keys' (like mentioned in comment in the flat-cache documentation.