I would like to host files on a private AWS S3 bucket which can only be accessed by users who are authenticated to my web application. The links to these file downloads must be static.
Simple proxy method:
I know this could be done using a proxy service. In this case the static links would point to the service and the service would handle validating the requesting users session, if it were valid the service would respond with the file contents from S3.
Presigned URL proxy method:
However rather than implement a proxy to gate access to the files, I was wondering if I could use presigned URLs somehow instead?
https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html
In this case, the role of the proxy is to just return a presigned URL to the user rather than the actual payload of the file from S3. The end user could then use this presigned URL to download the file directly from S3. What I'm not clear on is how this flow is manage in the browser, I am assuming I would need to write JavaScript to the following:
Request presigned URL from proxy service
Wait for response
Use the presigned URL provided in the response (the presigned URL) to download the actual file
Am I on the right track here?
Simply return a 307 redirect from your server to the presigned URL. E.g. the client requests:
GET /the/file HTTP/1.1
And the server generates a presigned URL and responds with:
HTTP/1.1 307 Temporary Redirect
Location: https://s3.aws..../the/file?...
That's a valid approach.
Beware of expiring credentials. Signed URLs will be good for the lesser of the time until the access credentials used to sign them expire, or their expiry time (which you control, within limits) happens. In the case that you're already using temporary credentials (which is very good!) you might want to use AssumeRole explicitly to control the expiry time (you can assume a role from a role to get new temporary credentials with a new time limit).
There's another option too: Amazon Cognito. This can bridge the gap between your user accounts and then issue per-user short-term credentials to your users' browser environments directly. They can then make API calls to S3 with their own credentials. This has some benefit (you can better express user permissions in their profile, rather than checking them yourself before they generate URLs ) and some complexity (can I DoS your account with my user creds, or do you control what APIs I can call? Least Privilege really matters when IAM your only auth tier) On the other hand, IAM calls are free and you don't pay for the servers to host them, so this alo sounds cost effective if you are using federated identity - user pools, not so much.
Related
I'm using google calendar API to add the event to calendar. I was wondering if it would be security issue as I'm using client Id and API code in JS which can be exposed to someone using the application?
Also, if it is the case how to secure those keys?
PS- Docs that I'm following- https://developers.google.com/calendar/quickstart/js
Have you noticed the redirect uri? THe redirect uri tells googles auth server where to return the access token.
Even if I grab your client id and client secret. I cant use it because the server is going to send the access token to the redirect uri to your server.
This is why you should not set localhost as a redirect uri. 😉
RFC oauth2 redirection endpoint
After completing its interaction with the resource owner, the
authorization server directs the resource owner's user-agent back to
the client. The authorization server redirects the user-agent to the
client's redirection endpoint previously established with the
authorization server during the client registration process or when
making the authorization request.
That being said you should try to keep these keys private you should not share them or add them to open source projects. However Client sided applications like javascript is a gray area.
If a web app uploads data to an amazon s3 bucket is it possible to restrict access to the data for the specific ip address that the web app initiated the upload from and / or can the web app obtain some sort of token that it uses to access the data prior to uploading it?
For example if Hans in Holland uploads 235325.json and Tina in Germany uploads 3453453.json, the web application client that Hans is running cannot see Tinas 3453453.json file and vice versa. Access to the file upload is only accessible by the user that uploaded it and 100% off access to the rest of the world.
There is no way to achieve this natively (like something you will just configure). You are better off implementing permission control at application level.
But, you can enhance your security mechanisms by generating pre-signed URLs that includes a policy that checks for an IP condition.
In this case:
Hans gets a S3 URL for an object that he uploaded and can legitimately download.
Hans sends the S3 URL to Tina via email.
Tina tries to open the link.
The link will fail for Tina since the allowed IP address included in the S3 URL won't match.
For more information, take a look at Creating a Signed URL Using a Custom Policy. (Scroll to "Creating a Policy Statement for a Signed URL That Uses a Custom Policy".)
Per my review of how to setup secure access to amazon s3 buckets it looks like we first generate an IAM user and then tie a security policy allowing s3 access to that user. After that we can generate API keys for the bucket, which can authenticate request for bucket access. That's my understanding at this point, please correct me if I missed something.
I assume the API keys should be server side only (The Secret Access Key). In other words it's not safe to place these directly inside the webapp? Hence we would first have to send the data to our server, and then once there we can send it to the bucket using the API key?
Is there any way to secure access directly from a web app to an amazon s3 bucket?
Approach Summary
Per the discussion with #CaesarKabalan it sounds like the approach that would allow this is:
1) Create an IAM user that can create identities that can be authenticated via Amazon Cognito - Lets call the credentials assigned from this step Cognito Credentials.
2) The user signs in to the webapp with for example Google
3) The webapp makes a request to the webapp's server (Could be a lambda function) to signup the user with Amazon Cognito
4) The webapp now obtains credentials for the user directly from Amazon Cognito and uses these to send the data to the s3 bucket.
I think that's where we are conceptually. Now it's time to test!
From your question I'm not sure what portions of your application are in AWS nor your security policies but you basically have three options:
(Bad) Store your keys on the client. Depending on the scope of your deployment this might be ok. For example if each client has it's own dedicated user and bucket there probably isn't much risk, especially if this is for a private organization where you control all aspects of the access. This is the easiest but less secure. You should not use this if your app is multi-tenant. Probably move along...
(Great) Use an API endpoint to move this data into your bucket. This would involve some sort of infrastructure to receive the file securely from the client then move it into S3 with the security keys stored locally. This would be similar to a traditional web app doing IO into a database. All data into S3 goes through this tier of your app. Downsides are you have to write that service, host it, and pay for bandwidth costs.
(Best) Use Amazon Cognito to assign each app/user their own access key. I haven't done this personally but my understanding is you can provision each entity their own short-lived access credentials that can be renewed and you can give them access to write data straight to S3. The hard part here will be structuring your S3 buckets and properly designing the IAM credentials for your app users to ONLY be able to do exactly what you want. The upside here is the users write directly to S3 bucket, you're using all native AWS services and writing very little custom code. This I would consider the best, most secure, and enterprise class solution. Here is an example: Amazon S3: Allows Amazon Cognito Users to Access Objects in Their Bucket
Happy to answer any more questions or clarify.
Is there any ways to secure the api response with jwt or any other method but without authentication (login page) so that only owner site can have access to the api.
All the methods and tutorials I saw on google was based on login system with jwt.
As an example, if I have rest api like:
router.get('/api/posts', (req, res) => {
var body = ... // get some via database
res.json(body);
})
Then I want to consume this only by my site: example.com. And most importantly without authentication (login system)
First of all, every API request must go through https.
Then you can "secure" user-specific APIs by giving each user a unique token which must be sent at every request.
It is as well possible to check the host or useragent of the user which requests the API and allow only specific custom useragents (depending on your needs).
Other than that:
If you need a JSON response while the user is logged in on the same server, you can check if a given cookie or session is set and can be related to that one specific user.
If you do server to server requests for that API, you could check if the server hostname is valid and matches the one(s) who are allowed to have access.
You can as well use encryption to secure your API response (here as well: depending on your needs). If this is true, you can use a private/public key encryption similar to GPG/PGP. Of course, only the one who should have access to the API should be allowed to decrypt the response.
GUID (Globally Unique Identifier) may be an option if you don't care if anyone could find out the path to your API. GUID URLs could look like this:
example.com/api/v1/c9a646d3-9c61-4cb7-bfcd-ee2522c8f633
Is there a way to use Fine Uploader to upload to an Amazon S3 bucket by providing the already signed policy document along with the key and the other credentials all at once by overriding the policy post request with our own XML api call?
Our company API returns all the credentials including signed policy for the file in one response and is already well established so setting up a signing page is not an option.
This may work for non-chunked uploads since Fine Uploader will only make one request for the signed policy document. However, when uploading files in chunks, the S3 REST API must be used. In that case, a policy document is not used. Instead, a long string of relevant headers for each request must be signed. This signature is then included with the REST call. The headers change with each request, therefore requiring a new signature.
If you want to support chunking and concurrent chunking to S3, you'll need to ensure each request is signed separately via a signature server, or make use of an identity provider to handle this client-side, as demonstrated in our documentation at http://docs.fineuploader.com/branch/master/features/no-server-uploads.html.