Develop Reference

Node.js flat-cache, when to clear caches

I have a Node.js server which queries MySQL database. It serves as an api end point where it returns JSON and also backend server for my Express application where it returns the retrieved list as an object to the view.
I am looking into implementing flat-cache for increasing the response time. Below is the code snippet.
const flatCache = require('flat-cache');
var cache = flatCache.load('productsCache');
//get all products for the given customer id
router.get('/all/:customer_id', flatCacheMiddleware, function(req, res){
var customerId = req.params.customer_id;
//implemented custom handler for querying
queryHandler.queryRecordsWithParam('select * from products where idCustomers = ? order by CreatedDateTime DESC', customerId, function(err, rows){
if(err) {
res.status(500).send(err.message);
return;
}
res.status(200).send(rows);
});
});
//caching middleware
function flatCacheMiddleware(req, res, next) {
var key = '__express__' + req.originalUrl || req.url;
var cacheContent = cache.getKey(key);
if(cacheContent){
res.send(cacheContent);
} else{
res.sendResponse = res.send;
res.send = (body) => {
cache.setKey(key,body);
cache.save();
res.sendResponse(body)
}
next();
}
}
I ran the node.js server locally and the caching has indeed greatly reduced the response time.
However there are two issues I am facing that I need your help with.
Before putting that flatCacheMiddleware middleware, I received the response in JSON, now when I test, it sends me an HTML. I am not too well versed with JS strict mode (planning to learn it soon), but I am sure the answer lies in the flatCacheMiddleware function.
So what do I modify in the flatCacheMiddleware function so it would send me JSON?
I manually added a new row to the products table for that customer and when I called the end point, it still showed me the old rows. So at what point do I clear the cache?
In a web app it would ideally be when the user logs out, but if I am using this as an api endpoint (or even on webapp there is no guarantee that the user will log out the traditional way), how do I determine if new records have been added and the cache needs to be cleared.
Appreciate the help. If there are any other node.js caching related suggestions you all can give, it would be truly helpful.

I found a solution to the issue by parsing the content to JSON format.
Change line:
res.send(cacheContent);
To:
res.send(JSON.parse(cacheContent));
I created cache 'brute force' invalidation method. Calling clear method will clear both cache file and data stored in memory. You have to call it after db change. You can also try delete specified key using cache.removeKey('key');.
function clear(req, res, next) {
try {
cache.destroy()
} catch (err) {
logger.error(`cache invalidation error ${JSON.stringify(err)}`);
res.status(500).json({
'message' : 'cache invalidation error',
'error' : JSON.stringify(err)
});
} finally {
res.status(200).json({'message' : 'cache invalidated'})
}
}
Notice, that calling the cache.save() function will remove other cached API function. Change it into cache.save(true) will 'prevent the removal of non visited keys' (like mentioned in comment in the flat-cache documentation.

403 error in calling DerivativeApi of Autodesk Forge

Autodesk Forge's DerivativeApi is not working with my client id and secret. Apis returns unauthorized error, { statusCode: 403, statusMessage: 'Unauthorized’ }.
But they works with the sample's client id/secret (from https://github.com/Autodesk-Forge/forge-api-nodejs-client/blob/master/samples/dmSample.js).
Is there any limitation for calling DerivativeApi (for translating) with free account? or should I do something?
Here is the sample code...
var ForgeSDK = require('forge-apis');
// TODO - insert your CLIENT_ID and CLIENT_SECRET
// Below id/secret from the sample are working, but mine is not.
var CLIENT_ID = 'wmizntnpzCJxPGF9lxsIiTZGbGO2cJqw',
CLIENT_SECRET = 'g5IPJwvhOHcrdbFy';
var derivativesApi = new ForgeSDK.DerivativesApi();
// Initialize the 2-legged oauth2 client
var oAuth2TwoLegged = new ForgeSDK.AuthClientTwoLegged(CLIENT_ID, CLIENT_SECRET,
['data:write', 'data:read', 'bucket:read', 'bucket:update', 'bucket:create'], true);
function defaultHandleError(err) {
console.error('\x1b[31m Error:', err, '\x1b[0m');
}
oAuth2TwoLegged.authenticate().then(function (credentials) {
console.log("**** Got Credentials", credentials);
derivativesApi.getFormats({}, oAuth2TwoLegged, oAuth2TwoLegged.getCredentials()).then(res => {
console.log(res);
}, defaultHandleError);
}, defaultHandleError);
Thanks

First, and extremely important, NEVER share your ID & Secret. If the above is your correct one, please create a new secret (to invalidate this).
Now the ID & Secret should work for any sample, just make sure you activated the APIs for it. On your app, select Model Derivative API on the screen, if not available, please review this article.
Finally, the code sample above is not actually calling Model Derivative. If so, note that files uploaded into one account are not accessible from other accounts, and URNs are unique.

Changing req object within api server?

I recently picked up a client who has a dev team. I am working on a website that has already been developed and am running into some things that are slightly strange to me.
I've always though it was essentially bad to mess with the request object within route handling (I could be completely wrong here).
The following code reaaaaally confuses me as I am not sure why they are assigning the req.query.msg to something instead of just creating a variable and passing it through on the ejs page render.
/********************************************************
* *
* CHANGE PASSWORD ROUTE THAT POSTS THE NEW PASSWORD TO *
* DATABASE. *
* *
********************************************************/
app.post('/client/password', function (req, res) {
var url = URLS.ClientChangePW;
if(req.session.securityquestions[0].SSN !== "null" || req.session.securityquestions[0].SSN !== "undefined"){
if(req.body.pwd !== req.body.pwdconf){
res.redirect('/client/changePassword' + config.PWD_MISMATCH);
} else {
var ssn = req.session.securityquestions[0].SSN;
while(ssn.length < 9){
ssn = "0" + ssn;
}
url = url.replace("#ssn", ssn);
url = url.replace("#newpw", req.body.pwd);
}
request.put(url, function (err, xres, body) {
var data = JSON.parse(body);
if(data.status === 200){
email(req.session.securityquestions[0].EMAIL, "none", "forgotpw", function(result){
if(result){
req.query.msg = "Your password has been reset.";
} else {
req.query.msg = "Request unsuccessful. Please call number here for assistance.";
}
res.render('pages/login', {
session: req.session,
msg: req.query.msg
});
});
} else {
req.query.msg = "Request unsuccessful. Please call number here for assistance.";
res.render('pages/login', {
session: req.session,
msg: req.query.msg
});
}
});
}
});
Again, I have never really messed with the req object so I could be wrong. I always thought the client sets up the request and we use that to send a response.

I am not sure why they are assigning the req.query.msg to something instead of just creating a variable and passing it through on the ejs page render.
There does not appear to be any reason to be assigning to the req.query.msg property here. If this were my code, I'd be using a separate local variable for it.
Again, I have never really messed with the req object so I could be wrong. I always thought the client sets up the request and we use that to send a response.
Though this is not what is happening here, it is common in Express development to use middleware that sets state on the req object for request handlers further down the routing stack to use. The req object is the canonical object where you keep request-specific state while processing the request. If you only have one request handler function working on the request, then there's no reason to put state on the req object as you can just use local variables in that one request handler function. But, if you're using middleware whose job it is to set things up before ultimately getting to a request handler, then the req object is where that setup state is usually put. You'll notice that req.session is also used in this code. That .session property was put there by some middleware upstream in the request processing.
So, it IS common to add state to the req object when using middleware. But, in the .msg property example in the code you show, there is no particular reason to put that on the req object as its value is only needed in the local function so it can just as easily (and I would argue more clearly) be in a local variable.

Parse Server Node.js SDK: Alternative to Parse.User.become?

I want to completely dissociate my client app from Parse server, to ease the switch to other Baas/custom backend in the future. As such, all client request will point to a node.js server who will make the request to Parse on behalf of the user.
Client <--> Node.js Server <--> Parse Server
As such, I need the node.js server to be able to switch between users so I can keep the context of their authentification.
I know how to authentificate, then keep the sessionToken of the user, and I ve seen during my research than the "accepted" solution to this problem was to call Parse.User.disableUnsafeCurrentUser, then using Parse.User.become() to switch the current user to the one making a request.
But that feels hackish, and I m pretty sure it will, sooner or later, lead to a race condition where the current user is switched before the request is made to Parse.
Another solution I found was to not care about Parse.User, and use the masterKey to save everything by the server, but that would make the server responsible of the ACL.
Is there a way to make request from different user other than thoses two?

Any request to the backend (query.find(), object.save(), etc) takes an optional options parameter as the final argument. This lets you specify extra permissions levels, such as forcing the master key or using a specific session token.
If you have the session token, your server code can make a request on behalf of that user, preserving ACL permissions.
Let's assume you have a table of Item objects, where we rely on ACLs to ensure that a user can only retrieve his own Items. The following code would use an explicit session token and only return the Items the user can see:
// fetch items visible to the user associate with `token`
fetchItems(token) {
new Parse.Query('Item')
.find({ sessionToken: token })
.then((results) => {
// do something with the items
});
}
become() was really designed for the Parse Cloud Code environment, where each request lives in a sandbox, and you can rely on a global current user for each request. It doesn't really make sense in a Node.js app, and we'll probably deprecate it.

I recently wrote a NodeJS application and had the same problem. I found that the combination of Parse.User.disableUnsafeCurrentUser and Parse.User.become() was not only hackish, but also caused several other problems I wasn't able to anticipate.
So here's what I did: I used
Parse.Cloud.useMasterKey(); and then loaded the current user by session ID as if it was a regular user object. It looked something like this:
module.exports = function(req, res, next) {
var Parse = req.app.locals.parse, query;
res.locals.parse = Parse;
if (req.session.userid === undefined) {
res.locals.user = undefined;
return next();
}
Parse.Cloud.useMasterKey();
query = new Parse.Query(Parse.User);
query.equalTo("objectId", req.session.userid);
query.first().then(function(result) {
res.locals.user = result;
return next();
}, function(err) {
res.locals.user = undefined;
console.error("error recovering user " + req.session.userid);
return next();
});
};
This code can obviously be optimized, but you can see the general idea. Upside: It works! Downside: No more use of Parse.User.current(), and the need to take special care in the backend that no conditions occur where someone overwrites data without permission.

Call specific functions with Websockets?

I'm trying to deploy an app with Heroku but I guess they don't allow use of Socket.IO so I have to use basic WebSockets. How do I send data to specific functions e.g.
With sockets it would be:
// Server
socket.on('testFunction', function(data) {
console.log(data);
});
// Client
socket.emit('testFunction', 'Hello!');
But with Websockets from what I've googled all I can find is
// Server
ws.onmessage = function(data) {};
// Client
ws.send('This is a string, what do I do with myself');
Any information would be great. Thanks!

See here for some documentation and examples.
Here are some rough equivalents to your Socket.IO examples:
// Server
ws.onmessage = function(event) {
var msg = JSON.parse(event.data);
switch(msg.type) {
case "testFunction":
console.log(msg.text);
break;
}
};
// Client
ws.send(JSON.stringify({type: "testFunction", text: "Hello!"}));

The solution to your problem depends on your server side code.
The way i solved the function calling problem, is by sending my data to server as an object( stringified), containing controller and action properties.
e.g in pseudo code
var request = {controller: "users", action: "login", params: {username: "loginuser", password: "333"} };
socket.emit(JSON.stringify( request) );
and on server side i get the controller value, and instantiate a new controller based on it...eg new users_controller(), and call the action on that controller with the params sent from emit.
Of course you need to test if controllers and actions exists, handle errors etc.