Node.js flat-cache, when to clear caches - javascript

I have a Node.js server which queries MySQL database. It serves as an api end point where it returns JSON and also backend server for my Express application where it returns the retrieved list as an object to the view.
I am looking into implementing flat-cache for increasing the response time. Below is the code snippet.
const flatCache = require('flat-cache');
var cache = flatCache.load('productsCache');
//get all products for the given customer id
router.get('/all/:customer_id', flatCacheMiddleware, function(req, res){
var customerId = req.params.customer_id;
//implemented custom handler for querying
queryHandler.queryRecordsWithParam('select * from products where idCustomers = ? order by CreatedDateTime DESC', customerId, function(err, rows){
if(err) {
res.status(500).send(err.message);
return;
}
res.status(200).send(rows);
});
});
//caching middleware
function flatCacheMiddleware(req, res, next) {
var key = '__express__' + req.originalUrl || req.url;
var cacheContent = cache.getKey(key);
if(cacheContent){
res.send(cacheContent);
} else{
res.sendResponse = res.send;
res.send = (body) => {
cache.setKey(key,body);
cache.save();
res.sendResponse(body)
}
next();
}
}
I ran the node.js server locally and the caching has indeed greatly reduced the response time.
However there are two issues I am facing that I need your help with.
Before putting that flatCacheMiddleware middleware, I received the response in JSON, now when I test, it sends me an HTML. I am not too well versed with JS strict mode (planning to learn it soon), but I am sure the answer lies in the flatCacheMiddleware function.
So what do I modify in the flatCacheMiddleware function so it would send me JSON?
I manually added a new row to the products table for that customer and when I called the end point, it still showed me the old rows. So at what point do I clear the cache?
In a web app it would ideally be when the user logs out, but if I am using this as an api endpoint (or even on webapp there is no guarantee that the user will log out the traditional way), how do I determine if new records have been added and the cache needs to be cleared.
Appreciate the help. If there are any other node.js caching related suggestions you all can give, it would be truly helpful.

I found a solution to the issue by parsing the content to JSON format.
Change line:
res.send(cacheContent);
To:
res.send(JSON.parse(cacheContent));
I created cache 'brute force' invalidation method. Calling clear method will clear both cache file and data stored in memory. You have to call it after db change. You can also try delete specified key using cache.removeKey('key');.
function clear(req, res, next) {
try {
cache.destroy()
} catch (err) {
logger.error(`cache invalidation error ${JSON.stringify(err)}`);
res.status(500).json({
'message' : 'cache invalidation error',
'error' : JSON.stringify(err)
});
} finally {
res.status(200).json({'message' : 'cache invalidated'})
}
}
Notice, that calling the cache.save() function will remove other cached API function. Change it into cache.save(true) will 'prevent the removal of non visited keys' (like mentioned in comment in the flat-cache documentation.

Related

How do I search for a data in the database with Node JS, Postgres, and dust js

I'm making a webpage with Node JS with dustjs and PostgreSQL. How do I make a search query in the html, so I can pass the value to the app.get
Do I need to use JQuery?
app.get('/teachers', function(req, res){
pool.connect(function(err, client, done){
if(err) {
return console.error("error", err);
}
client.query('SELECT * FROM teachers', function(err, result){
if(err){
return console.error('error running query', err)
}
res.render('teacherindex', {teachers: result.rows});
done();
});
});
});
app.get('/teachers/:str', (req,res)=>{
pool.connect((err, client, done) => {
if (err) throw err
client.query('SELECT * FROM teachers WHERE name = $1', [req.query.namesearch], (err, result) => {
done()
if (err) {
console.log(err.stack)
} else {
res.render('teacherindex', {teachers: result.rows});
}
})
})
})
This is my JQuery
$("#myBtn").click(function(){
var str = $("#myInput").val();
var url = '/teachers/'+str;
if(confirm('Search Record?')){
$.ajax({
url: url,
type: 'put',
success: function(result){
console.log('Searching');
window.location.href='/teachers';
},
error: function(err){
console.log(err);
}
});
}
});
My HTML
<input type="text" id="myInput" data-id="namesearch">
<button type="button" id="myBtn">Show Value</button>
Thank you!
FINAL ANSWER:
Ok so it turns out the issue you were having was something completely different. You are trying to use server side rendering for this, and I was showing you how to render the retrieved data on the client side.
I have forked, and updated your repo - which can be found at the link below..
Please review my changes and let me know if you have any questions.
Working repo: https://github.com/oze4/hanstanawi.github.io
Demo Video: https://raw.githubusercontent.com/oze4/hanstanawi.github.io/master/fake_uni_demo.mp4
EDIT:
I went ahead and built a repository to try and help you grasp these concepts. You can find the repo here - I tried to keep things as simple and understandable as possible, but let me know if you have any questions.
I had to make some minor changes to the paths, which I have commented explanations on the code in the repo.
I am using a "mock" database (just a JSON object in a different file) but the logic remains the same.
The index.js is the main entry point and contains all route data.
The index.html file is what gets sent to the user, and is the main HTML file, which contains the jQuery code.
If you download/fork/test out the code in that repo, open up your browsers developer tools, go to the network tab, and check out the differences.
Using req.params
Using req.query
ORIGINAL ANSWER:
So there are a couple of things wrong with your code and why you are unable to see the value of the textbox server side.
You are sending a PUT request but your server is expecting a GET request
You are looking for the value in req.query when you should be looking for it in req.params
You are looking for the incorrect variable name in your route (on top of using query when you should be using params) req.query.namesearch needs to be req.params.str
See here for more on req.query vs req.params
More detailed examples below.
In your route you are specifying app.get - in other words, you are expecting a GET request to be sent to your server.. but your are sending a PUT request..
If you were sending your AJAX to your server by using something like /teachers?str=someName then you would use req.query.str - or if you wanted to use namesearch you would do: /teachers?namesearch=someName and then to get the value: req.query.namesearch
If you send your AJAX to your server by using the something like /teachers/someName then you should be using req.params.str
// ||
// \/ Server is expecting a GET request
app.get('/teachers/:str', (req, res) => {
// GET THE CORRECT VALUE
let namesearch = req.params.str;
pool.connect((err, client, done) => {
// ... other code here
client.query(
'SELECT * FROM teachers WHERE name = $1',
// SPECIFY THE CORRECT VALUE
namesearch,
(err, result) => {
// ... other code here
})
})
});
But in your AJAX request, you are specifying PUT.. (should be GET)
By default, AJAX will send GET requests, so you really don't have to specify any type here, but I personally like to specify GET in type, just for the sake of brevity - just more succinct in my opinion.
Again, specifying GET in type is not needed since AJAX sends GET by default, specifying GET in type is a matter of preference.
$("#myBtn").click(function () {
// ... other code here
let textboxValue = $("#myTextbox").val();
let theURL = "/teachers/" + textboxValue;
// OR if you wanted to use `req.query.str` server side
// let theURL = "/teachers?str=" + textboxValue;
if (confirm('Search Record?')) {
$.ajax({
url: theURL,
// ||
// \/ You are sending a PUT request, not a GET request
type: 'put', // EITHER CHANGE THIS TO GET OR JUST REMOVE type
// ... other code here
});
}
});
It appears you are grabbing the value correctly from the textbox, you just need to make sure your server is accepting the same type that you are sending.

Parse Server Node.js SDK: Alternative to Parse.User.become?

I want to completely dissociate my client app from Parse server, to ease the switch to other Baas/custom backend in the future. As such, all client request will point to a node.js server who will make the request to Parse on behalf of the user.
Client <--> Node.js Server <--> Parse Server
As such, I need the node.js server to be able to switch between users so I can keep the context of their authentification.
I know how to authentificate, then keep the sessionToken of the user, and I ve seen during my research than the "accepted" solution to this problem was to call Parse.User.disableUnsafeCurrentUser, then using Parse.User.become() to switch the current user to the one making a request.
But that feels hackish, and I m pretty sure it will, sooner or later, lead to a race condition where the current user is switched before the request is made to Parse.
Another solution I found was to not care about Parse.User, and use the masterKey to save everything by the server, but that would make the server responsible of the ACL.
Is there a way to make request from different user other than thoses two?
Any request to the backend (query.find(), object.save(), etc) takes an optional options parameter as the final argument. This lets you specify extra permissions levels, such as forcing the master key or using a specific session token.
If you have the session token, your server code can make a request on behalf of that user, preserving ACL permissions.
Let's assume you have a table of Item objects, where we rely on ACLs to ensure that a user can only retrieve his own Items. The following code would use an explicit session token and only return the Items the user can see:
// fetch items visible to the user associate with `token`
fetchItems(token) {
new Parse.Query('Item')
.find({ sessionToken: token })
.then((results) => {
// do something with the items
});
}
become() was really designed for the Parse Cloud Code environment, where each request lives in a sandbox, and you can rely on a global current user for each request. It doesn't really make sense in a Node.js app, and we'll probably deprecate it.
I recently wrote a NodeJS application and had the same problem. I found that the combination of Parse.User.disableUnsafeCurrentUser and Parse.User.become() was not only hackish, but also caused several other problems I wasn't able to anticipate.
So here's what I did: I used
Parse.Cloud.useMasterKey(); and then loaded the current user by session ID as if it was a regular user object. It looked something like this:
module.exports = function(req, res, next) {
var Parse = req.app.locals.parse, query;
res.locals.parse = Parse;
if (req.session.userid === undefined) {
res.locals.user = undefined;
return next();
}
Parse.Cloud.useMasterKey();
query = new Parse.Query(Parse.User);
query.equalTo("objectId", req.session.userid);
query.first().then(function(result) {
res.locals.user = result;
return next();
}, function(err) {
res.locals.user = undefined;
console.error("error recovering user " + req.session.userid);
return next();
});
};
This code can obviously be optimized, but you can see the general idea. Upside: It works! Downside: No more use of Parse.User.current(), and the need to take special care in the backend that no conditions occur where someone overwrites data without permission.

Update a JSON file in AngularJS

I've got some data from a JSON file, which I use in my HTML getting it first from AngularJS like this:
$http.get('js/data.json').success(function(data) {
$scope.data = data;
});
And I want to update this JSON file after clicking a button in the HTML:
<button ng-click="postData(id)">Post</button>
You cannot write on files via JavaScript only (AngularJS).
You are to go via server side and point your "post" request to a server side script (i.e: PHP) and make that script do the job.
This sort of thing won't work. The file you are trying to write to would be on a server; and as it is right now, it would be a static resource. I'd suggest reading up on Angular resources, here. You can set up your server-side code to perform CRUD operations on the json file, but an actually database would be best. If you prefer to use a json format, Mongodb is your best choice; here is a link to Mongodb University, which offers free courses. I've done it in the past, and it's been great.
Now, for some actually help in your situation:
You can perform a GET request on your json file because it's seen as a static resource. The POST request, however, needs server-side scripting to do anything.
$http.get('api/YOUR_RESOURCE').success(function(data) {
$scope.database = data;
});
$http.post('api/YOUR_RESOURCE', {
data_key: data_value,
data_key2: data_value2
}).success(function(data) {
data[id].available = false;
});
This may be further ahead on your path to learning Angular, but here is a snippet of Node.js server code, with a Mongo database and Mongoose to handle the 'Schema', to help you get an idea of how this works:
var mongoose = require('mongoose'),
YOUR_RESOURCE = mongoose.model('YOUR_RESOURCE');
app.route('/api/YOUR_RESOURCE')
// This should be your GET request; 'api/
.get(
// Get all docs in resource
YOUR_RESOURCE.find().exec(function (err, data) {
if (err) {
return res.status(400).send({
message: SOME_ERROR_HANDLER
});
} else {
res.json(data); // return list of all docs found
}
});)
// Add new doc to database
.post(function (req, res) {
// The keys of the object sent from your Angular app should match
// those of the model
var your_resource = new YOUR_RESOURCE(req.body);
your_resource.save(function (err) {
if (err) {
return res.status(400).send({
message: SOME_ERROR_HANDLER
});
} else {
// returns newly created doc to Angular after successful save
res.json(your_resource);
}
});
);
Here is an SO page with a list of resources on getting started with Node; I recommend Node because of it's ease of use and the fact that it is written in JS. The Mongo University lessons also go through setting up you server for use with the database; you can choose between several flavors, such as Java, .NET, Python or Node.
There is a bit left out in the examples above, such as the Mongoose model and Node setup, but those will be covered in the resources I've linked to on the page, if you choose to read them. Hope this helps :)

How do I specify MongoDB key/value parameters in an $http.get with an AngularJS controller?

I'm using the MEAN stack for a web app. In my controller.js, I have the following:
var refresh3 = function() {
$http.get('/users').success(function(response) {
console.log("I got the data I requested");
$scope.users = response;
$scope.contact3 = "";
});
};
refresh3();
This pulls back every object in the "users" collection from my MongoDB. How could I add parameters to this to bring back, for example, only objects where "name" : "Bob" ? I have tried adding parameters in the '/users' parentheses using:
$http.get('/users', {params:{"name":"Bob"}})
I've also tried variants of that, but no luck. Any help would be appreciated!
If your server is receiving the data
(and it should, as $http.get('/users', {params:{"name":"Bob"}}) is correct)
On server side, make use of the query string:
req.query
like so:
app.get('/users', function(req,res){
if(req.query.name){
db.users.find({"name":req.query.name},function (err, docs) { console.log(docs); res.json(docs); });
}
else{
db.users.find(function (err, docs) { console.log(docs); res.json(docs); });
}
});
WHAT WAS THE ISSUE?
You hinted in your comments that your server was set to respond to the app.get('/users') GET request like so:
db.users.find(function (err, docs) {
// docs is an array of all the documents in users collection
console.log(docs); res.json(docs); }); });
So I believe that your angularjs $http get is correct, and your server is receiving the parameters {"name":"Bob"} as it should;
it just doesn't know what to do with them:
all it is set to do is to return the whole collection in the particular case of a app.get('/users') GET request.
HOW TO CONFIGURE YOUR SERVER FOR REST
You do not have to re-invent the wheel on the server.
Rather, you could consider using a middleware to automate the task (in the present case, the task is to issue a proper MongoDb request when you receive a get query with parameters from the client)
e.g. express-restify-mongoose middleware

Using the PUT method with Express.js

I'm trying to implement update functionality to an Express.js app, and I'd like to use a PUT request to send the new data, but I keep getting errors using PUT. From everything I've read, it's just a matter of using app.put, but that isn't working. I've got the following in my routes file:
send = function(req, res) {
req.send(res.locals.content);
};
app.put('/api/:company', function(res,req) {
res.send('this is an update');
}, send);
When I use postman to make a PUT request, I get a "cannot PUT /api/petshop" as an error. I don't understand why I can't PUT, or what's going wrong.
You may be lacking the actual update function. You have the put path returning the result back to the client but missing the part when you tell the database to update the data.
If you're using MongoDB and ExpressJS, you could write something like this :
app.put('/api/:company', function (req, res) {
var company = req.company;
company = _.extend(company, req.body);
company.save(function(err) {
if (err) {
return res.send('/company', {
errors: err.errors,
company: company
});
} else {
res.jsonp(company);
}
})
});
This mean stack project may help you as it covers this CRUD functionality which I just used here swapping their articles for your companies. same same.
Your callback function has the arguments in the wrong order.
Change the order of callback to function(req, res).
Don't use function(res, req).
Also if you want to redirect in put or delete (to get adress), you can't use normal res.redirect('/path'), you should use res.redirect(303, '/path') instead. (source)
If not, you'll get Cannot PUT error.
Have you been checking out your headers information?
Because header should be header['content-type'] = 'application/json'; then only you will get the update object in server side (node-express), otherwise if you have content type plain 'text/htm' like that you will get empty req.body in your node app.

Categories

Resources