I'm building a proof-of-concept UI served as a static JS application (just free static hosting on Netlify), and I'd like to be able to read and write data to a NoSQL database.
As it's a PoC, I need to keep it 100% free. The request rate will be super low; only a few people will ever see it. DynamoDB seems like a good choice for size and cost ($0).
So what's the simplest possible way to read/write data from DynamoDB from a browser client? It's hard to tell what's absolutely necessary. Ideally I'd connect directly to the database (no API gateway, no lambdas), while maintaining some kind of baseline security.
You can query dynamodb from your browser using aws sdk for browser,
you will need to carefully create a IAM user with limited access (eg. no updateTable perms etc. )
Keep in mind that anyone could hijack your access keys and increase the throughput and generate cost ( if they have updateTable priv. ) or query your tables outside of your website and consume all the available capacity.
I would still go with API gateway / lambda
There is no need for API gateway / lambda for simple DynamoDB requests. I would suggest to use AWS Cognito with the AWS Amplify Auth Library to get access to the DynamoDB. Then use a client like Dynamo-Easy which has some nice abstraction, so you can jump start your dev without learning the low level of the DynamoDB JS client from AWS.
Related
I am having a Node.js and Vue.js project, where a user is going to provide his AWS credentials, a pointer to some online resource (which stores a large amount of data), and some algorithm on this data is going to be run on user's AWS account that he/she provided.
For this purpose, I am having two difficulties and I would like to ask for some help.
Firstly, I want to deploy some simple javascript code in the cloud, to test that everything works. What is the easiest way to do that? How can the npm packages aws-sdk and aws-lambda help me? Do I necessarily need to give my debit card details to make use of AWS just for quick testing purpose?
The second thing is, is there an authorization library/tool that AWS offers, like Facebook, for example, so the user just needs to input his username and password into a window, and he/she is automatically authorized (with OAuth, probably that's what they are using).
In addition, I would appreciate any general advice on how to approach this problem, how can I run code on huge amount of data on cloud user accounts? Maybe another cloud platform is more appropriate? Thank you!
This is a big question. so I'll provide some pointers for you to do further reading on:
to start with, decide if you want your webapp to be server-based (EC2, Node.js, and Express) or serverless (CloudFront, API Gateway, Lambda, and S3)
learn how to use Cognito as a way to get AWS credentials associated with a social provider login (such as Facebook or Google)
to operate in another user's AWS account, you should leverage cross-account IAM roles (they create a role and give you permission to assume it)
on the question of running code against large amounts of data, the repository for this data will typically be S3 or perhaps Redshift in some situations, and the compute environment could be any one of Lambda (short lifetime, serverless), EMR (clustered Hadoop, Spark etc.), EC2 (vanilla VM), Athena (SQL queries over content in S3), or ECS (containers). You haven't given enough information to help decide which might be more suitable.
The simplest option to test things out is likely to be S3 (for storage) and EC2 (use t2.micro instances in Free Tier, deploy your web application just like you would on any other Linux environment).
Yes, to use AWS you need an account and to get an account you need to supply a credit card. There is a substantial Free Tier in your first year, however.
I have a tiny table (100 records) that I keep in my Dynamodb backend. My web app will scan table at the beginning of the session. And then users will query this data with different parameters. My intention is not to go my backend for each query and do it on client side(front end) for better performance.
But because I don't want anyone to see my tiny table's data I would like to encrypt it while sending and decrypt it on browser side after arrival. I'm using Nodejs, dynamodb and API gateways as backend (AWS serverless architecture).
I'm a newbie and was wondering if it is possible and what the best practices are.
I'll give an example to describe my concern better. Imagine skyscanner keeps all airline-flight-ticketprice data in one table. they will have 2 options to let everbody to search publicly. First they can let users to query the table everytime they search (which will be slow). Second they can scan the table's data and send it to browser and users can search flights much faste on front end (with arrays etc..). I want to implement the 2nd approach but I also want to keep my data encrypted so nobody can copy my data and create a very similar website :)
Thanks.
Using Cognito Identity Pools
You can achieve this with authentication using AWS Cognito Identity Pool(Granting who can access which DynamoDB Table and even which Partition key) and using AWS JavaScript SDK which uses SSL to encrypt the communication between the browser and DynamoDB.
Note: You can also use AWS Cognito UserPools if you don't have a user store which can be connected to Cognito Identity Pool.
Using API Gateway and Lambda endpoint for Temporary Access Credentials
If you already have an existing authentication mechanism you can use a API Gateway and Lambda endpoint where the Lambda function will have the code to assume an IAM Role and send the temporary access credentials to the browser. Then in the browser you can use AWS SDK for JavaScript to access DynamoDB.
Here's a demo app that does specifically what you're looking for...once logged in, the user has access to his own "row" in dynamoDB...
https://github.com/awslabs/aws-cognito-angular-quickstart
Here's what you get by running the install script (it creates all of the resources for you):
http://cognito.budilov.com
I'm working on a React-Redux web-app which integrates with AWS Cognito for user authentication/data storage and with the Shopify API so users can buy items through our site.
With both SDKs (Cognito, Shopify), I've run into an issue: Their core functionality attaches data behind the scenes to localStorage, requiring both SDKs to be run client-side.
But running this code entirely client-side means that the API tokens which both APIs require are completely insecure, such that someone could just grab them from my bundle and then authenticate/fill a cart/see inventory/whatever from anywhere (right?).
I wrote issues on both repos to point this out. Here's the more recent one, on Shopify. I've looked at similar questions on SO, but nothing I found addresses these custom SDKs/ingrained localStorage usage directly, and I'm starting to wonder if I'm missing/misunderstanding something about client-side security, so I figured I should just ask people who know more about this.
What I'm interested in is whether, abstractly, there's a good way to secure a client-side SDK like this. Some thoughts:
Originally, I tried to proxy all requests through the server, but then the localStorage functionality didn't work, and I had to fake it out post-request and add a whole bunch of code that the SDK is designed to take care of. This proved prohibitively difficult/messy, especially with Cognito.
I'm also considering creating a server-side endpoint that simply returns the credentials and blocks requests from outside the domain. In that case, the creds wouldn't be in the bundle, but wouldn't they be eventually scannable by someone on the site once that request for credentials has been made?
Is the idea that these secret keys don't actually need to be secure, because adding to a Shopify cart or registering a user with an application don't need to be secure actions? I'm just worried that I obviously don't know the full scope of actions that a user could take with these credentials, and it feels like an obvious best practice to keep them secret.
Thanks!
Can't you just put the keys and such in a .env file? This way nobody can see what keys you've got stored in there. You can then access your keys through process.env.YOUR_VAR
For Cognito you could store stuff like user pool id, app client id, identity pool id in a .env file.
NPM package for dotenv can be found here: NPM dotenv
Furthermore, what supersecret stuff are you currently storing that you're worried about? By "API tokens", do you mean the OpenId token which you get after authenticating to Cognito?
I can respond to the Cognito portion for this. Your AWS Secret Key and Access Key are not stored in the client. For your React.js app, you only need the Cognito User Pool Id and the App Client Id in your app. Those are the only keys that are exposed to the user.
I cover this in detail in a comprehensive tutorial here - http://serverless-stack.com/chapters/login-with-aws-cognito.html
I'm currently working on a WebGL sketch drawing project where users can draw arbitrary objects on an html canvas. The javascript libraries and files are all stored on a node.js server which is currently being started up locally every time the software has to be run. Essentially all of the functionality for saving all of the drawn objects on the page has been implemented where the drawings are being written as JSON objects, but the next step is to persist these objects to a database where they can be mapped to a user id. I will also need to implement a login system where users will login and be able to select previously drawn objects to edit from the database.
If this was just a normal website, I would probably just use express.js or something similar, but as the views are essentially rendered entirely in WebGL, I wouldn't think that frameworks would work well with this construct.
Given that I currently just need to create a login system and implement a feature for persisting the JSON object to the DB, are there any frameworks or existing software that accommodates the specified needs of the system?
With regard to authentication, I would recommend taking a look at OAuth and using existing identity providers (e.g. Google, Facebook, etc). You can still retain profiles for your users but you don't have to deal with all of the intricacies of authentication, authorization, security, etc.
There are a ton of JavaScript libraries out there for handling OAuth/OAuth2 interactions. Some even have built-in identity providers. Here are a couple links that returned all sorts of potentially useful libraries:
https://www.npmjs.com/search?q=oauth2
https://www.google.com/search?q=javascript%20oauth2%20library
As for a database, you have a lot of options for storing raw JSON. Some that I've used recently for my JavaScript projects are PostgreSQL, MongoDB, and ArangoDB. You can find well written JS libraries for interacting with any of those.
Another thing to think about is if you want to install the database on your server or use a hosted solution such as RDS or DynamoDB (available from Amazon).
Regardless of the exact authentication and persistence options you choose you will likely use a pattern similar to this:
Your Node.js server is deployed somewhere accessible on the internet, where it exposes the endpoints for your WebGL application, authentication, and saving/loading sketches.
When the user navigates to the WebGL application endpoint on your Node.js server they are required to authenticate (which will utilize your authentication endpoints on the Node.js server).
When the user requests a "save" in your WebGL application you will submit the JSON representation of their sketch (and their authorization token) to the "save" endpoint of your Node.js server.
This "save" endpoint validates the user's authorization token and inserts the JSON into whatever database you've chosen.
The "load" endpoint works just like the "save" endpoint but in reverse. The user asks for a specific sketch. The identity of the sketch (id, name, etc) is sent along with their authorization token. The "load" endpoint on your Node.js server validates their authorization token and then queries the database for the sketch.
The key pattern to notice here is that users don't send requests to your database directly. Your WebGL application should communicate back to your Node.js server and it should commmunicate with your database. This is essential for controlling security, performance, and future updates.
Hopefully this gives you an idea of where to go next.
EDIT based on comments:
I searched around for a Node.js + PostgreSQL guide but didn't find anything I would feel comfortable recommending. I did find this JS library though, which I would check out:
https://github.com/vitaly-t/pg-promise
For MongoDB I would just use their official guide:
https://docs.mongodb.org/getting-started/node/introduction/
I'm trying to create a simple in-browser web app to display the contents on a given rethink table with some nice formatting. I'm having trouble finding a way to actually connect to rethink without having to use node.js. All I want to do is get the data out and then run it through some styling/layout stuff. Node + dependencies are overkill for a tiny browser-only app.
Unfortunately, you're going to need a server. It might be node.js or it might be another language, but you'll need a server.
RethinkDB is not Firebase. It can't be queried from your browser. If you absolutely need browser side querying and can't have a server, you should use Firbase.
If you want to use RethinkDB, you can just have a very thin server that just redirects your queries to RethinkDB. This can be done over HTTP or over WebSockets.
Why
Ultimately, the reason why you don't want to query your database from the browser is security. RethinkDB has no users or read only accounts. That means that if your database is accessible from your browsers, anyone can come and delete all your databases (including your system tables) with a simple query.
For example:
r.db('rethinkdb').tableList().forEach(function (tableName) {
return r.db('rethinkdb').tableDrop(tableName);
});
And now, all your database is gone :).
Keep in mind that this is something the RethinkDB team is aware of and working on.
https://github.com/rethinkdb/rethinkdb/issues/218