I have a JavaScript script to check if there is a new post in a certain subreddit. Reddit magicaly provides a JSON endpoint on every link. In this case I have the following endpoint:
https://www.reddit.com/r/webdev/new.json?limit=1
I then used request module for node.js to get the specefic data I need eg domain, selftext, author and domain. However this changes every single time a new post is published and I therefore use a setInterval function to check every few seconds if a new post is released (Its like some sort of polling). I am saving every posts unique id to mongo to prevent double posting since that would assume every post is new on every request.
So the structure is something like:
setInterval(function () {
request({
uri: redditEndpoint,
json: true
}, function (error, response, body) {}) });
// I then save the data sent to mongo as unique to prevent double posting
// I consume the data here
}, 1000);
I would like to move away from this method and move over to websockets. However I am not sure how to correctly implement a websocket on such an endpoint, preferably with socket.io
That websocket endpoint would need to be provided by the server - in this case from the reddit server. You can't create a websocket connection to a random server which does not offer websocket support.
The only thing you could do is build a proxy server, which polls the reddit server for new events (like what you are currently doing) and which then offers a websocket endpoint itself for other clients.
Related
I am current using Microsoft Graph API (beta) to get Presence Status e.g. Online, away etc. in an spfx webpart (using React) using GraphClient:
this.props.context.msGraphClientFactory.getClient().then(async (client: MSGraphClient) => {
let response = await client
.api('/communications/getPresencesByUserId')
.version('beta')
.post(postData)
console.log("Communication API Response: "+ response);
this.usersWithPresence = response.value;
});
This is working fine, but to get updated status of a user, I have to refresh the page so another API call is made and updated presence status. I want to do it like this happens in 'Skype'.
What I need is suggestions about a mechanism that I can apply to get
real time updates in user's presence status, so as soon as user
updates the status this is reflected in my webpart. I know I can use
setInterval or setTimeout functions to request for presence status
after specific intervals but for learning purposes i don't want to
request API this way again and again but rather getting updated
message from server like this happens using web sockets. How a web
socket sort of stuff can be applied with this API?
your suggestions are welcome.
Today this API doesn't support any kind of subscription mechanism to when the status changes. There is a uservoice entry you can upvote for it. That means the only way to get any changes is to periodically poll it. As for socket io, the only support today is for SharePoint lists, for any other resource you need to stand up your own infrastructure to relay the message.
Currently I am making a page that display's data gathered from an API. Most of the data is updated on the server side every 4 hours but some of it is updated whenever a client requests the index route. As a result, there is a delay in the index file being sent since the data needs to be updated. I want to gather the updated data after the page has been requested and sent so there is no delay. My first idea was to make the request on the client side which will handle updating the display after the data is gathered but from my knowledge, I don't know how to do that without giving them the API key. Should I approach the problem this way or is there a better way to do it? I'm using Express for the back-end, Axios is used to make the get requests, and EJS is the template engine.
Here is the code:
// This is called before the data is send in a for loop
data.gameData[i].player_count = await SteamModule.liveGetCurrentPlayers(data.gameData[i].appid);
res.render('index', {data: data});
// This is the function that is called
liveGetCurrentPlayers: async (id) => {
const res = await axios.get(`${base}/ISteamUserStats/GetNumberOfCurrentPlayers/v1/?key=${key}&appid=${id}`, {timeout: 1000}).catch(err => {
Logger.error("Error getting updated user data per request");
return 'Error';
});
if(res.data) {
return res.data.response.player_count;
} else {
return 'Error';
}
}
Here's a bit of a drawing to explain what I've said in comments.....
(the code you showed should constantly update - without other info I can't help with whatever the other issue was, but this is the overall idea....)
Where:
Client requests data from you (your server)
Your server sends html and css to show a 'frame' of the page (no data, just something for them to see and feel like something is happening...)
Your server requests data from the API server (all the various "20-ish" things you said you wanted to serve....)
As the data is updated (or you may have it already), you send the data to the client, updating their 'frame' page with current data.
You can maintain your keys on the server side and add restriction that those API's can only be accessed by your client side URL. So you would access the API and it will maintain your session and handle the authorized KEY part as well.
Anything and everything on the client side is accessible if it running in your browser.
You can add security measures on the server but not on client side for protecting your key.
I have an app that has a REST api. I want it so that the only requests that can be made to the REST api are ones originating from the app itself. How can I do that?
I am using a node.js+express server too.
EDIT: the app is fully a public web app.
Simply define the header in your request, what this does is, it allows requests only from a certain domain, and instantly rejects any other domain.
response.set('Access-Control-Allow-Origin', 'domain.tld');
EDIT: IF you're really keen against web scraping stuff, you could make a function to double check client's origin.
function checkOrigin (origin) {
if (origin === "your.domain.tld") {
return true;
} else {
return false;
}
}
/* Handling it in response */
if (checkOrigin(response.headers.origin)) {
// Let client get the thing from API
} else {
response.write("Send them error that they're not allowed to use the API");
response.end();
}
Above example should work for the default HTTP/HTTPS module, and should also work for Express, if I'm not mistaken.
EDIT 2: To back my claim up that it should also work for Express, I found this quotation at their documentation;
The req (request) and res (response) are the exact same objects that Node provides, so you can invoke req.pipe(), req.on('data', callback), and anything else you would do without Express involved.
I would recommend using an API key from the client. CORS filters are too easy to circumvent.
A simple approach for securing a How to implement a secure REST API with node.js
Overview from above post:
Because users can CREATE resources (aka POST/PUT actions) you need to secure your api. You can use oauth or you can build your own solution but keep in mind that all the solutions can be broken if the password it's really easy to discover. The basic idea is to authenticate users using the username, password and a token, aka the apitoken. This apitoken can be generated using node-uuid and the password can be hashed using pbkdf2
Then, you need to save the session somewhere. If you save it in memory in a plain object, if you kill the server and reboot it again the session will be destroyed. Also, this is not scalable. If you use haproxy to load balance between machines or if you simply use workers, this session state will be stored in a single process so if the same user is redirected to another process/machine it will need to authenticate again. Therefore you need to store the session in a common place. This is typically done using redis.
When the user is authenticated (username+password+apitoken) generate another token for the session, aka accesstoken. Again, with node-uuid. Send to the user the accesstoken and the userid. The userid (key) and the accesstoken (value) are stored in redis with and expire time, e.g. 1h.
Now, every time the user does any operation using the rest api it will need to send the userid and the accesstoken.
I got an old website on a webserver. It is to big and not structured well enough but needs to be improved by e.g. Account management. As it is (in my opinion) at its end of lifetime, we do not want to put more effort in it but instead migrate to new technology. For that, we want to use node.js and AngularJS, because the whole project is more a webapp than it was at the beginning. As a migration concept, we want to include the old stuff via a kind of routing through the node.js server and replace it step by step. For that I looked into the "request" library without getting the right grip.
Goal is, to route some requests after authorization check to the old server, without leaving the new server (gate). For that I need to check and parse the gets and posts. Some other requests have to response by the node.js server itself.
As I think that I am not the only one with that approach I am asking for experience in that matter.
I had to do something similar, because we made a new API which was not compatible with the first version and some features were not implemented in the newer API so we had to do like a bridge. Authentication was happening in the first server, and then we were routing the request to the old API and then returning to the user.
The approach I took was using a module like request to make the call in the old server.
Assuming you are using expres for your new API, you can do something similar
var request = require('request');
app.get('/test', function(req, res) {
//authenticate stuff
var options = {
url: 'http://oldendpoint.com/test',
headers: {
//headers for authenticate in the old endpoint
}
};
request(options, function (error, response, body) {
if (!error && response.statusCode == 200) {
res.send(body); //this will send it back to your client
}
});
});
Basically you get a request to your new API (node.js app) in the /test endpoint and this, after auth or after whatever check, will forward the request to the old system, and then it will return some data which is forwarded to the client who made the request in the first place.
I'm developing an hybrid mobile app using HTML/CSS/JS, I'm going over security with login information, the system I have set up creates an hash after a user logs in, this hash has a time limit and is set via localStorage
Essentially, I would have something like this is localstorage:
hash
5f4a09cfec2a6d8f306eecb3844e33e9
hash_expiration
1373012945
password
*encryted user password*
This hash is sent to my server for validation in the header of all my AJAX requests (accompanied by the user id for database matching)
I'm mostly opening this topic to discuss best practices on how to deal with recreating hash keys, I need to figure out a way to refresh a users hash key.
Considering my experience with AJAX and JS is still rather limited, I thought about using the AJAX setup to check for a new hash, like so:
$.ajaxSetup({
beforeSend: function(xhr, settings) {
var time = new Date().getTime(); //unix time
var hash_time = localStorage.getItem("hash_expiration");
if(time>hash_time){
//ajax request to fetch new hash, async: false to make sure this completes before continuing with other AJAX calls
}
}
});
I would send the user id and his encrypted password to verify him and return a new hash.
Should I be sending AJAX requests in the ajaxSetup's beforeSend? How would this conflict with other beforeSends across my application?
Basicallly on the clients side you shouldnt have anything except hash. On the server side this hash must be associated with user it belongs to, expire time and anything else you need.
Send this hash with each request, and on server side validate it. When it expires you have to send (server) appropriate headers like 401 - Unauthorized. Client have to understand that response and try to exchange hash to new one. And finally when client gets new valid hash it can resume sending requests.
... and you shouldnt check expire time at client, this job for server.
thanks.