My question is simple and general: when making calls to RESTFUL APIs, whether they be mine or external ones, is it common practice/ok to have the token exposed on the front end? For instance, in the documentation for the Google Maps api, they suggest the following code:
<script src="https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY&callback=initMap"
async defer></script>
the fact that your API key is exposed on the front end for all to see is ok then? I guess google has has the option to restrict access, so that can solve that, but what about other services that don't give that option?
Is it better to keep my API calls on the backend to protect my tokens? Having them on the backend, I would think, would not be preferred, because they I cannot get the data asynchronously
In short , yes it's possible to use someone else API keys in your site if you can spoof http referer header on client system .In fact you can even saturate his quota completely if you have either of these
Full control over your clients browser (to manipulate HTTP headers it might send to google servers)
You own huge collection of unique IP adresses (try only if you're freakingly rich and only motive you're left with in life is to empty his quota xD)
Idk atm will tell if i learn something more or someone comments something
Related
Where to save security information when I want to send a Web API call to DialogFlow from Browser? I use Angular.
I have a backend of the application, but I don't want to send the request through my backend (don't want to pay for the additional traffic).
Is it even possible to make a secure call from a browser?
Are there any techniques I could apply to achieve this? As I know DialogFlow uses Google Cloud (GCP) behind the scene, are there any recommendations from GCP (so far can't find any)?
Anything sensitive you need to store client side is out of your control, and therefore you need to assume it will be compromised.
If the API can be called from your server instead of the browser then you can store the token on the server to protect it, but this depends on what the API is used for.
It appears their documentation on accessing the API can be found here: https://dialogflow.com/docs/reference/v2-auth-setup#getting_the_service_account_key
I am using Google Cloud Storage. To upload to cloud storage I have looked at different methods. The method I find most common is that the file is sent to the server, and from there it is sent to Google Cloud storage.
I want to move the file directly from the user's web browser to Google Cloud Storage. I can't find any tutorials related to this. I have read through the Google API Client SDK for JavaScript.
Going through the Google API reference, it states that files can be transferred using a HTTP request. But I am confused about how to do it using the API client library for JavaScript.
People here would require to share some code. But I haven't written any code, I have failed in finding a method to do the job.
EDIT 1: Untested Sample Code
So I got really interested in this, and had a few minutes to throw some code together. I decided to build a tiny Express server to get the access token, but still do the upload from the client. I used fetch to do the upload instead of the client library.
I don't have a Google cloud account, and thus have not tested this, so I can't confirm that it works, but I can't see why it shouldn't. Code is on my GitHub here.
Please read through it and make the necessary changes before attempting to run it. Most notably, you need to specify the location of the private key file, as well as ensure that it's there, and you need to set the bucket name in index.html.
End of edit 1
Disclaimer: I've only ever used the Node.js Google client library for sending emails, but I think I have a basic grasp of Google's APIs.
In order to use any Google service, we need access tokens to verify our identity; however, since we are looking to allow any user to upload to our own Cloud Storage bucket, we do not need to go through the standard OAuth process.
Google provides what they call a service account, which is an account that we use to identify instances of our own apps accessing our own resources. Whereas in a standard OAuth process we'd need to identify our app to the service, have the user consent to using our app (and thus grant us permission), get an access token for that specific user, and then make requests to the service; with a service account, we can skip the user consent process, since we are, in a sense, our own user. Using a service account enables us to simply use our credentials generated from the Google API console to generate a JWT (JSON web token), which we then use to get an access token, which we use to make requests to the cloud storage service. See here for Google's guide on this process.
In the past, I've used packages like this one to generate JWT's, but I couldn't find any client libraries for encoding JWT's; mostly because they are generated almost exclusively on servers. However, I found this tutorial, which, at a cursory glance, seems sufficient enough to write our own encoding algorithm.
I'd like to point out here that opening an app to allow the public free access to your Google resources may prove detrimental to you or your organization in the future, as I'm sure you've considered. This is a major security risk, which is why all the tutorials you've seen so far have implemented two consecutive uploads.
If it were me, I would at least do the first part of the authentication process on my server: when the user is ready to upload, I would send a request to my server to generate the access token for Google services using my service account's credentials, and then I would send each user a new access token that my server generated. This way, I have an added layer of security between the outside world and my Google account, as the burden of the authentication lies with my server, and only the uploading gets done by the client.
Anyways, once we have the access token, we can utilize the CORS feature that Google provides to upload files to our bucket. This feature allows us to use standard XHR 2 requests to use Google's services, and is essentially designed to be used in place of the JavaScript client library. I would prefer to use the CORS feature over the client library only because I think it's a little more straightforward, and slightly more flexible in its implementation. (I haven't tested this, but I think fetch would work here just as well as XHR 2.).
From here, we'd need to get the file from the user, as well as any information we want from them regarding the file (read: file name), and then make a POST request to https://www.googleapis.com/upload/storage/v1/b/<BUCKET_NAME_HERE>/o (replacing with the name of your bucket, of course) with the access token added to the URL as per the Making authenticated requests section of the CORS feature page and whatever other parameters in the body/query string that you wish to include, as per the Cloud Storage API documentation on inserting an object. An API listing for the Cloud Storage service can be found here for reference.
As I've never done this before, and I don't have the ability to test this out, I don't have any sample code to include with my answer, but I hope that my post is clear enough that putting together the code should be relatively straightforward from here.
Just to set the record straight, I've always found OAuth to be pretty confusing, and have generally shied away from playing with it due to my fear of its unknowns. However, I think I've finally mastered it, especially after this post, so I can't wait to get a free hour to play around with it.
Please let me know if anything I said is not clear or coherent.
I have developed an API for my new service and am in process of developing SDKs (php, ruby and JavaScript) for this API.
Some of the calls to API are open to public, but some require API key and API secret. My question is, how do I make sure that people can hide their key and secret from world while using JavaScript API.
I would Imagine the call something like:
jQuery.ajax({
url:'http://api.domain.com/v1/display/',
data: {offset:0, limit:0, apiKey:'apikeynotlikelogin',apiSecret:'apisecretlikepassword'},
success: function(data){
alert(data);
}
});
But almost everyone these days know that if they open firebug or even simplier ctrl+shift+j in chrome, they can see the code together with all information above. I have considered many options, but it seems to me like there is just no way how to hide apiKey and apiSecret on front-end.
Any suggestions? There must be a way Im sure.
Thanks!
EDIT:
Thanks everyone for answers and trying to solve the issue. Here are some of the answers and why is still not what I need:
Using domain name in the request to make sure its from the correct client.
This is valid suggestion, but still could be quite easily faked
Generating unique key for each call
This seems to be more advanced, but again I found it not usable for my case. I need to authorize the "App" (thats what people register in system in order to get credentials and authorize to API) because users will have different levels of privacy set and according to those clients will be served with result.
So If I cam make client to first call "handshake" to get the session unique key, but then again (either in handshake or next request), client has to send his apiKey and apiSecret in order to authorize to API and get the correct result (according to policies etc.) So at the end, it is exactly the same as without the handshake request, anyone who knows the apiKey and apiSecret could first call handshake and then the authorization.
Hope it makes sense
You can make it harder, but you can't really secure it.
You can have them register the pages from which they are going to make the requests, and check that the request originates from an authorized origin when checking authentication. Not perfect, as this can be faked, but harder for a casual user.
Or they can proxy through a server, but this does not help much since they need to secure access to their proxy.
I am designing a new service that would enable 'customers' to register and pay a per-use type fee for particular searches they perform. This service would be exposed using a RESTFul and SOAP interface. Typically the web service would integrate with the customer's website and then be exposed to the 'public' where anyone would be able to use the customer's website and take advantage of my web service features (which the customer would pay for but have full control of moderating the requests so they don't get charged too much).
I want to design the service that optimises the integration to make it as simple as possible. The web service API will change so creating an internal proxy to expose the web service to the public in some cases is too much of a detractor for customers. So the issue as I see it is creating a web service that balances authentication, security and integration.
Ideal
Not use OAuth
Avoid forcing the customer to create an internal proxy which re-exposes the same web service API I have already.
Be secure (token username/pass whatever and ssl)
Embed a javascript library in customer website - This would be a client Javascript library to make integration steps even easier.
The Javascript library would need to be secure enough so that the public wouldn't be able to simply grab credentials and re-purpose it themselves
Not be too hacky, if possible, so the web service doesn't have to be re-built if Firefox 87 comes out (to be released in as many minutes) and decides to fubar it.
It seems that some kinda of 3-way authentication process is needed for this to work, i.e. authenticates a particular client (in the public), the web service (the customer) and the web service.
Has anyone implemented something kind of similar and how did they tackle a situation like this?
I also understand there is a balance between what can be done, and what would violate cross-domain security, so perhaps the entire web service might be exposed by another GET only interface which would return JSONP data.
/** Addendum **/
I have since discovered a web service that does what I'm looking after. However, I am not confident I understand the implementation details entirely. So perhaps someone could also elaborate on my thinking.
The web service I discovered seems to host the Javascript on the service side. The customer would then integrate their website with the service side by including the Javascript in a script tag, but supplies a key to do so i.e.
Somehow if I add the script to my website it doesn't work. So somewhere along the line the token must be registered to a particular customer domain, and the 'client-lib.js' is actually a servlet or something similar which can somehow detect that the user from the 'public' coming in has actually originated from the 'customer' domain.
Is my thinking right? Is there some kind of http header that can be used this way? Is that safe?
Cheers
First of all - let me provide you a link to another SO question which I answered yesterday - as it gives a pretty extensive answer to a similar question-set.
I am assuming that you are going to charge the owner of the site from which the search is made, and not care so much who the individual user is who makes the search. If that's incorrect, please clarify and I will update my answer accordingly.
Clearly, in any such case, the first and foremost thing you need to do is to make sure you know which client this is on each request. And - as you said, you also want to make sure you're protecting yourself from cross-site attacks and people stealing your user's keys.
What you might consider would be the following:
Create a private key on your side - which only your service knows.
Whenever a new consumer site creates an account with you, create a new shared key which only you and they will know. I suggest creating this key by using your private key as a password, and encrypting some kind of identifier which will let you identify this particular user.
As part of your registration process, make the consumer site tell you what URI they will be using your scripts on.
Now - the way that you both do your tracking and authentication becomes fairly simple.
You mentioned providing a JS library which won't need to update every time FF updates. I suggest building that library using jQuery, or another, similarly supported cross-browser JS foundational library - and letting that wrap your AJAX.
When the client site requests your script, however, have them provide you something like:
http://www.yourdomain.com/scripts/library.js?key={shared key}
On your side, when you receive this request, check the following:
When you decrypt their shared key using your private key, you should not get gibberish. If you do - it's because their key has been altered in some way - and is not valid. This should result in a 401: Unauthorized error.
Once you decrypt the key and know which client site this is (because that's what the key contains) - check to make sure that the request is coming from the same URI that client registered with. This now protects you from someone stealing their key and injecting it into a different website.
As long as the above matches, let them download the file.
Now - when you serve the JS file, you can do so in a way that injects the key into that file - and therefore it has access to their shared key. On each of your AJAX requests, include that key so that you can identify which client this request is coming from again. In a RESTful environment, there shouldn't really be sessions - so you need this level of authentication on each post. I suggest including it as a cookie.
On your server-side - simply repeat the checks of their key on each subsequent request - and voila - you've built yourself some fairly tight security without a lot of overhead.
That said - if you expect a lot of traffic - you may want to come back to this and explore more deep security processes in the future, as rolling your own security matrix can leave unexpected holes. However - it is a good start and will get you off the ground.
Feel free to ask any questions if you need, and I will try to update my answer accordingly.
The best way to go about it is something like this (taking that you want to use javascript hosted on your server and make the include part as simple as it can be):
*user registers on your website and he receives a token for his domain
*the user can include a js file pointing to your server
the js file will be something like:
<script type="text/javascript" src="http://your.server.com/js.php?token=###&widget=xxx"></script>
or
<script type="text/javascript" src="http://your.server.com/js.js?token=###&widget=xxx"></script>
if you will use a .htaccess to redirect
*in the php file check if the token matches the requests domain, if yes echo out the js lib, if not throw a error or something
*in the js you will need to build some ajax calls to your service and stuff to manipulate the HTML (create a widget holder,show some data, etc.)
*also all the calls should have the token, and again you can use the same logic to check if token==server address
EDIT:
The REFERER is sent by the client's browser as part of the HTTP protocol, and is therefore unreliable indeed.
If you want to verify if a request is coming from your site, well you can't, but you can verify the user has been to your site and/or is authenticated. Cookies are sent in AJAX requests so you can rely on that. But this means you need to use something like oAuth
If you want to use this method, you should still check the referrer as well to prevent CSRF en.wikipedia.org/wiki/Cross-site_request_forgery
Ideally you should use a unique token per session per user (per request if you're paranoid) to prevent CSRF attacks. Checking the referrer is just security by obfuscation and not quite a real solution.
This is probably a generic security question, but I thought I'd ask in the realm of what I'm developing.
The scenario is: A web service (WCF Web Api) that uses an API Key to validate and tell me who the user is, and a mix of jQuery and application on the front ends.
On the one hand, the traffic can be https so it cannot be inspected, but if I use the same key per user (say a guid), and I am using it in both then there's the chance it could be taken and someone could impersonate the user.
If I implement something akin to OAuth, then a user and a per-app key is generated, and that could work - but still for the jQuery side I would need the app API key in the javascript.
This would only be a problem if someone was on the actual computer and did a view-source.
What should I do?
md5 or encrypt the key somehow?
Put the key in a session variable, then when using ajax retrieve it?
Get over it, it's not that big a deal/problem.
I'm sure it's probably a common problem - so any pointers would be welcome.
To make this clearer - this is my API I have written that I am querying against, not a google, etc. So I can do per session tokens, etc, I'm just trying to work out the best way to secure the client side tokens/keys that I would use.
I'm being a bit overly cautious here, but just using this to learn.
(I suggest tagging this post "security".)
First, you should be clear about what you're protecting against. Can you trust the client at all? A crafty user could stick a Greasemonkey script on your page and call exactly the code that your UI calls to send requests. Hiding everything in a Javascript closure only means you need a debugger; it doesn't make an attack impossible. Firebug can trace HTTPS requests. Also consider a compromised client: is there a keylogger installed? Is the entire system secretly running virtualized so that an attacker can inspect any part of memory at any time at their leisure? Security when you're as exposed as a webapp is is really tricky.
Nonetheless, here are a few things for you to consider:
Consider not actually using keys but rather HMAC hashes of, e.g., a token you give immediately upon authentication.
DOM storage can be a bit harder to poke at than cookies.
Have a look at Google's implementation of OAuth 2 for an example security model. Basically you use tokens that are only valid for a limited time (and perhaps for a single IP address). That way even if the token is intercepted or cloned, it's only valid for a short length of time. Of course you need to be careful about what you do when the token runs out; could an attacker just do the same thing your code does and get a new valid token?
Don't neglect server-side security: even if your client should have checked before submitting the request, check again on the server if the user actually has permission to do what they're asking. In fact, this advice may obviate most of the above.
It depends on how the API key is used. API keys like that provided by Google are tied to the URL of the site originating the request; if you try and use the key on a site with an alternate URL then the service throws and error thus removing the need to protect the key on the client side.
Some basic API's however are tied to a client and can be used across multiple domains, so in this instance I have previously gone with the practice of wrapping this API in server side code and placing some restrictions on how the client can communicate with the local service and protecting the service.
My overall recommendation however would be to apply restrictions on the Web API around how keys can be used and thus removes the complications and necessity of trying to protect them on the client.
How about using jQuery to call server side code that handles communication with the API. If you are using MVC you can call a controller action that can contain the code and API key to hit your service and return a partial view (or even JSON) to your UX. If you are using web forms you could create an aspx page that will do the API communication in the code behind and then write content to the response stream for your UX to consume. Then your UX code can just contain some $.post() or $.load() calls to your server side code and both your API key and endpoint would be protected.
Generally in cases like this though you proxy requests through the server using 'AJAX' which verifies the browser making requests is authorized to do so. If you want to call the service directly from JavaScript, then you need some kind of token system like JSON Web Tokens (JWT) and you'll have to work out cross-domain issues if the service is located somewhere other than the current domain.
see http://blogs.msdn.com/b/rjacobs/archive/2010/06/14/how-to-do-api-key-verification-for-rest-services-in-net-4.aspx for more information
(How to do API Key Verification for REST Services in .NET 4)