What is the current best practice if I want to allow a user to have a local html/js website that can upload and download files from an Amazon S3 bucket, as an anonymous user?
I.E. I would like to load a local version of index.html in my browser, and access a previously set up (public?) s3 bucket, and not require the user to be logged in to Amazon, or any other identity service. (I am aware of the pitfalls of using a public s3 bucket)
Is my only option to use the AWS javascript SDK, and an Unauthenticated user as mentioned here?
https://aws.amazon.com/blogs/developer/authentication-with-amazon-cognito-in-the-browser/
That is the only reasonably safe, reasonably viable solution... assuming it will work with a local file.
It's technically possible to write to a bucket using ordinary HTTP PUT requests by setting bucket policy accordingly, but... here be dragons. If your bucket allows anonymous access, then it's relatively trivial for anyone to upload objects to your bucket that you cannot access, at all, other than to delete them.
Usually, when someone asks about anonymous access, they're trying to save some effort... but the effort saved is not likely to be worth the potential cost.
Related
I've read through quite a few pages of documentation and other StackOverflow questions/answers but can't seem to come across anything that can help me with my scenario.
I'm hosting a public, static site in an S3 bucket. This site makes some calls to an API that I have hosted in an EC2-instance in a VPC. Because of this, my API can only be called by other instances and services in the same VPC.
But I'm not sure how to allow the S3 Bucket site access to my API.
I've tried creating VPC Endpoints and going down that route, but all that did was restrict access to my S3 site from only the instances within my VPC (which is not what I want).
I would appreciate any help with this, thank you so much.
Hopefully my question is clear.
No, S3 Static Websites are 100% client side code. So it's basically just html + css + javascript being delivered, as-is from S3. If you want to get dynamic content into your website, you need to look at calling an API accessible from your user's browser, i.e. from the internet.
AWS API Gateway with Private Integrations could be used to accept the incoming REST call and send it on to your EC2 Server in your VPC.
My preferred solution to adding dynamic data to S3 Static Websites is using API Gateway with AWS Lambda to create a serverless website. This minimises running costs, maintenance, and allows for quick deployments. See The Serverless Framework for getting up and running with this solution.
A static site doesn't run on a server. It runs entirely in the web browser of each site visitor. The computer it is running on would be the laptop of your end-user. None of your code runs in the S3 bucket. The S3 bucket simply stores the files and serves them to the end-user's browser which then runs the code. The route you are going down to attempt to give the S3 bucket access to the VPC resource is not going to work.
You will need to make your API publicly accessible in order for code running in your static site (running in web browsers, on end-user's laptops/tablets/phones/etc.) to be able to access it. You should look into something like API keys or JWT tokens to provide security for your public API.
Per my review of how to setup secure access to amazon s3 buckets it looks like we first generate an IAM user and then tie a security policy allowing s3 access to that user. After that we can generate API keys for the bucket, which can authenticate request for bucket access. That's my understanding at this point, please correct me if I missed something.
I assume the API keys should be server side only (The Secret Access Key). In other words it's not safe to place these directly inside the webapp? Hence we would first have to send the data to our server, and then once there we can send it to the bucket using the API key?
Is there any way to secure access directly from a web app to an amazon s3 bucket?
Approach Summary
Per the discussion with #CaesarKabalan it sounds like the approach that would allow this is:
1) Create an IAM user that can create identities that can be authenticated via Amazon Cognito - Lets call the credentials assigned from this step Cognito Credentials.
2) The user signs in to the webapp with for example Google
3) The webapp makes a request to the webapp's server (Could be a lambda function) to signup the user with Amazon Cognito
4) The webapp now obtains credentials for the user directly from Amazon Cognito and uses these to send the data to the s3 bucket.
I think that's where we are conceptually. Now it's time to test!
From your question I'm not sure what portions of your application are in AWS nor your security policies but you basically have three options:
(Bad) Store your keys on the client. Depending on the scope of your deployment this might be ok. For example if each client has it's own dedicated user and bucket there probably isn't much risk, especially if this is for a private organization where you control all aspects of the access. This is the easiest but less secure. You should not use this if your app is multi-tenant. Probably move along...
(Great) Use an API endpoint to move this data into your bucket. This would involve some sort of infrastructure to receive the file securely from the client then move it into S3 with the security keys stored locally. This would be similar to a traditional web app doing IO into a database. All data into S3 goes through this tier of your app. Downsides are you have to write that service, host it, and pay for bandwidth costs.
(Best) Use Amazon Cognito to assign each app/user their own access key. I haven't done this personally but my understanding is you can provision each entity their own short-lived access credentials that can be renewed and you can give them access to write data straight to S3. The hard part here will be structuring your S3 buckets and properly designing the IAM credentials for your app users to ONLY be able to do exactly what you want. The upside here is the users write directly to S3 bucket, you're using all native AWS services and writing very little custom code. This I would consider the best, most secure, and enterprise class solution. Here is an example: Amazon S3: Allows Amazon Cognito Users to Access Objects in Their Bucket
Happy to answer any more questions or clarify.
I'm creating an application using processing.js and hosting the result via dropbox public folder, the idea is to use a .txt file generated via that platform to comunicate certain data to a local 3d modelling enviroment (rhinoceros/grasshopper), is it possible to write to a .txt file hosted in the public folder in dropbox directly from the sketch running in the web?
I mean, using: saveStrings("test.txt","this is an example");
the html containing the sketch, the .txt file and the sketch file itself are all stored in the same public folder in dropbox, you can see the site here: https://dl.dropboxusercontent.com/u/97841548/kinetica%20App/KineticaAppHTML.html
thanks in advance
You can't directly access your local filesystem from a web page like that, and you don't get write access to your Dropbox account just by virtue of the KineticaAppHTML.html page being hosted on Dropbox.
One way to do this however would be to use the Dropbox API. You could either proxy the file writes to a server you control and then have that server make the API calls to Dropbox, or you could use the Dropbox API directly from JavaScript on your KineticaAppHTML.html page.
Dropbox offers a JavaScript SDK you can use:
https://www.dropbox.com/developers/datastore/sdks/js
There's a tutorial here, though it focuses on datastores functionality and not files:
https://www.dropbox.com/developers/datastore/tutorial/js
The basics for authenticating the user are relevant though.
Once authenticated, you can use this method to write new contents to the file in your Dropbox:
https://www.dropbox.com/developers/datastore/docs/js#Dropbox.Client.writeFile
Note however that this strategy only results in you yourself, i.e., in your own browser, being able to make Dropbox API calls. If you want other users to use this to, this setup would have them connecting to their own Dropbox accounts. If you need them to connect to only your own account, you'd need to host and use your own actual web app, like I mentioned earlier, where you could safely make API calls to your own account. (You could technically avoid this by embedding an access token in your web page, but this is highly discouraged due the security implications there.)
Short answer: no.
Webpages in your browser do not have read/write access to your file system. Only the web.
Long answer: yes, but not the way you describe.
If you have a server running with a RESTful API that you can call by URL (like any other API on the web), then you can use any "ajax" approach there is to communicate with that, giving it the data it needs to generate those files, and making it responsible for putting them in the right dropbox folder.
I have a pre signed url that allows you to PUT documents objects into S3.
http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadObjectPreSignedURLDotNetSDK.html
How do I put a file from the browser to S3 using javascript? I am using angular, but I am open to using any javascript library.
I believe I could POST to my server, and then PUT the object on the amazon server, but I would prefer to do it from the browser.
I have changed the CORS settings on S3 to allow PUTs.
I was planning to use angular file upload, but it is hard coded to POST not PUT.
https://github.com/danialfarid/angular-file-upload
Amazon has a guide (here) that describes how to POST-upload a file into your S3 bucket. It relies on an input <form> signed with your AWS private key. You can specify restrictions on the target directory, as well as file-size restrictions.
It's a bit annoying to use, because you have to duplicate most of the fields in both the <form> and the signed policy, but it seems to work.
After the POST, S3 will redirect the browser to a URL you specify in the form (with parameters specifying the name of the uploaded file, etc.). This isn't ideal for Angular web sites, which tend to be "applications" rather than a set of discrete pages, but you could probably work with it.
On my Angular site, I did the POST in Javascript using $http.post() and passed all the appropriate form data. Unfortunately, I was always getting "cancelled" errors, even though the uploads were actually successful. In my case, I just double-checked by downloading the file with $http.get() and comparing it to the original data... but this was only a viable solution because my files were only a couple of KB.
I'm looking for a front-end solution for uploading files to amazon s3 (that is, not passing them through my server.
The solution I have found is
https://code.google.com/p/swfupload/
It might do the job, but it requier flash and this is the first sentence of the project description is:
SWFUpload has not been under active development for several years.
Here are my desired features, though none of them are nessesary
No plugins
Built with Amazon S3 in mind, handling buckets authentification etc.
Some way to see file upload-progress
Support from IE8+
kgu87 is correct, this article pretty much explains the entire process to upload files directly to S3 without passing them trough your own server.
You can also check out the AWS docs related to this on:
http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingHTTPPOST.html
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html
If you're looking for an upload tool that supports HTML5 uploads directly to S3, check out Plupload
They have a great article that explains how to set it up:
https://github.com/moxiecode/plupload/wiki/Upload-to-Amazon-S3
The documentation describes a PHP service that's used to generate a policy and signature (both are required for S3 to accept your download) but you can use any language to generate those. Also, in certain use cases, you can just generate a one-time policy with a very high expiration time and hard code it into your upload form.
You could start by using this tutorial as a baseline , if you are asking about uploading from your web app -
http://aws.amazon.com/articles/1434
You could use this tool: http://aws.amazon.com/customerapps/Amazon-S3/Consumers/2069 free, and works in most browsers; doesn't require a server.
S3 Browser is a free client interface for users of the Amazon S3 service.
It provides an easy to use GUI to manage your storage buckets and allows you to:
- Browse, create, delete Amazon S3 buckets
- Upload and download files to and from Amazon S3
- Create public URLs to share the files.
- Keep your files backed up on a multiple data centers.
- Set Access Control on buckets and files.
S3 Browser is free for non-commercial use.