Per my review of how to setup secure access to amazon s3 buckets it looks like we first generate an IAM user and then tie a security policy allowing s3 access to that user. After that we can generate API keys for the bucket, which can authenticate request for bucket access. That's my understanding at this point, please correct me if I missed something.
I assume the API keys should be server side only (The Secret Access Key). In other words it's not safe to place these directly inside the webapp? Hence we would first have to send the data to our server, and then once there we can send it to the bucket using the API key?
Is there any way to secure access directly from a web app to an amazon s3 bucket?
Approach Summary
Per the discussion with #CaesarKabalan it sounds like the approach that would allow this is:
1) Create an IAM user that can create identities that can be authenticated via Amazon Cognito - Lets call the credentials assigned from this step Cognito Credentials.
2) The user signs in to the webapp with for example Google
3) The webapp makes a request to the webapp's server (Could be a lambda function) to signup the user with Amazon Cognito
4) The webapp now obtains credentials for the user directly from Amazon Cognito and uses these to send the data to the s3 bucket.
I think that's where we are conceptually. Now it's time to test!
From your question I'm not sure what portions of your application are in AWS nor your security policies but you basically have three options:
(Bad) Store your keys on the client. Depending on the scope of your deployment this might be ok. For example if each client has it's own dedicated user and bucket there probably isn't much risk, especially if this is for a private organization where you control all aspects of the access. This is the easiest but less secure. You should not use this if your app is multi-tenant. Probably move along...
(Great) Use an API endpoint to move this data into your bucket. This would involve some sort of infrastructure to receive the file securely from the client then move it into S3 with the security keys stored locally. This would be similar to a traditional web app doing IO into a database. All data into S3 goes through this tier of your app. Downsides are you have to write that service, host it, and pay for bandwidth costs.
(Best) Use Amazon Cognito to assign each app/user their own access key. I haven't done this personally but my understanding is you can provision each entity their own short-lived access credentials that can be renewed and you can give them access to write data straight to S3. The hard part here will be structuring your S3 buckets and properly designing the IAM credentials for your app users to ONLY be able to do exactly what you want. The upside here is the users write directly to S3 bucket, you're using all native AWS services and writing very little custom code. This I would consider the best, most secure, and enterprise class solution. Here is an example: Amazon S3: Allows Amazon Cognito Users to Access Objects in Their Bucket
Happy to answer any more questions or clarify.
Related
I am building a node.JS server which interacts with Microsoft APIs. The server deals with multiple applications created in Azure where each application will have a Client ID and Client secret configured. I have a logic that will switch between these credentials and query the appropriate Microsoft Application based on the organization of requesting user. My concern is about storing the Client id and Client secrets.
Initially, I had planned to store these in a YAML configuration file and read it, but then there can be Applications configured when server is running and also this method can't be efficient when the number of Applications configured is high.
My second option is to store these credentials in a database so that I can query them whenever needed, but this way, the person adding a new client might have to go through a learning curve to use database plus I do not want to expose my database passwords.
Is there a common practice to store a large number of sensitive application credentials like this?
You should look into dedicated secret storage. In Azure you can use the Azure Vault service for example. AWS has Secret Storage service, Google Cloud has Secret Manager.
If you use kubernetes for your deployment it has a built in cluster secret store.
You can also use a dedicated third party secret manager server / module such as Hashicorp Vault or Keywhiz from Square.
I would like to host files on a private AWS S3 bucket which can only be accessed by users who are authenticated to my web application. The links to these file downloads must be static.
Simple proxy method:
I know this could be done using a proxy service. In this case the static links would point to the service and the service would handle validating the requesting users session, if it were valid the service would respond with the file contents from S3.
Presigned URL proxy method:
However rather than implement a proxy to gate access to the files, I was wondering if I could use presigned URLs somehow instead?
https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html
In this case, the role of the proxy is to just return a presigned URL to the user rather than the actual payload of the file from S3. The end user could then use this presigned URL to download the file directly from S3. What I'm not clear on is how this flow is manage in the browser, I am assuming I would need to write JavaScript to the following:
Request presigned URL from proxy service
Wait for response
Use the presigned URL provided in the response (the presigned URL) to download the actual file
Am I on the right track here?
Simply return a 307 redirect from your server to the presigned URL. E.g. the client requests:
GET /the/file HTTP/1.1
And the server generates a presigned URL and responds with:
HTTP/1.1 307 Temporary Redirect
Location: https://s3.aws..../the/file?...
That's a valid approach.
Beware of expiring credentials. Signed URLs will be good for the lesser of the time until the access credentials used to sign them expire, or their expiry time (which you control, within limits) happens. In the case that you're already using temporary credentials (which is very good!) you might want to use AssumeRole explicitly to control the expiry time (you can assume a role from a role to get new temporary credentials with a new time limit).
There's another option too: Amazon Cognito. This can bridge the gap between your user accounts and then issue per-user short-term credentials to your users' browser environments directly. They can then make API calls to S3 with their own credentials. This has some benefit (you can better express user permissions in their profile, rather than checking them yourself before they generate URLs ) and some complexity (can I DoS your account with my user creds, or do you control what APIs I can call? Least Privilege really matters when IAM your only auth tier) On the other hand, IAM calls are free and you don't pay for the servers to host them, so this alo sounds cost effective if you are using federated identity - user pools, not so much.
I've read through quite a few pages of documentation and other StackOverflow questions/answers but can't seem to come across anything that can help me with my scenario.
I'm hosting a public, static site in an S3 bucket. This site makes some calls to an API that I have hosted in an EC2-instance in a VPC. Because of this, my API can only be called by other instances and services in the same VPC.
But I'm not sure how to allow the S3 Bucket site access to my API.
I've tried creating VPC Endpoints and going down that route, but all that did was restrict access to my S3 site from only the instances within my VPC (which is not what I want).
I would appreciate any help with this, thank you so much.
Hopefully my question is clear.
No, S3 Static Websites are 100% client side code. So it's basically just html + css + javascript being delivered, as-is from S3. If you want to get dynamic content into your website, you need to look at calling an API accessible from your user's browser, i.e. from the internet.
AWS API Gateway with Private Integrations could be used to accept the incoming REST call and send it on to your EC2 Server in your VPC.
My preferred solution to adding dynamic data to S3 Static Websites is using API Gateway with AWS Lambda to create a serverless website. This minimises running costs, maintenance, and allows for quick deployments. See The Serverless Framework for getting up and running with this solution.
A static site doesn't run on a server. It runs entirely in the web browser of each site visitor. The computer it is running on would be the laptop of your end-user. None of your code runs in the S3 bucket. The S3 bucket simply stores the files and serves them to the end-user's browser which then runs the code. The route you are going down to attempt to give the S3 bucket access to the VPC resource is not going to work.
You will need to make your API publicly accessible in order for code running in your static site (running in web browsers, on end-user's laptops/tablets/phones/etc.) to be able to access it. You should look into something like API keys or JWT tokens to provide security for your public API.
I have built an app in React that uses the Dropbox API & will be stored on AWS S3 & CloudFront. The app accesses the Dropbox folder using a token. At the moment this token is client-side and obviously is completely accessible.
I have tried reaching out to Dropbox, looked into cookies & HTML5 web storage but can't seem to find a simple explanation.
What would be the simplest way of securing this token on the server?
There are few pros and cons of storing an access token in the server.
However, the most secure way of storing it on your server is sending it to the server via https link.
One major disadvantage of storing an access token in the server is that you, as the owner of the service, is bound to take the responsibility of securing the token. If your server is ever compromised, the hacker gets access to all the data of all the users by having simple access to all their access tokens.
You can always store the dropbox access token on the client side as a storage variable. Each storage is accessible only to the scripts served from the same domain.
~Edit~
If the Dropbox account is owned by the owner of the website and it should be hidden from the end user, you need to operate the Dropbox account from within the server. This DropBox accessing microservice has to be utilized as a proxy for accessing files.
I would to ask a few questions to better understand some procedures. I'm trying to write a web api project which will be a backend for both web and mobile clients.
The problem that i've in mind is about security. I don't want to use Identity or any other providers. I want to use my own database user and role structures.
Only authenticated client applications should be consuming my application. So that anonymous applications should not consume it.
So what should be the approach ? I 've written a custom AuthorizationAttribute and check some custom headers like "AppID","AppSecurity" key which i store in my own database and if the client sends the right appId and the key it means the app is authenticated to consume the API which does not sound very secure to me.
Another issue is that ; Lets say i've developed a javascript web application and i've to first authenticate the application itself before making GET/POST/PUT/DELETE etc requests which means i've to add some kind of authentication data like username, appkey, password in one of the js files for sending the "AppID" and the "AppSecurity" keys in the header. A client who knows how to use some developer tools or fiddler can easily capture my header values that are being sent to the server side? Even if i pass authentication values on the body of my json request it still can be found on the js files that are sent to the client. I'm also confused about that tooç
So basically i want to build a server side api that will serve the data and get data from the authenticated client applications only. What i need is a simple example for that without using any identity providers.