Develop Reference

Managing two different Azure resources under the same domain name with domain routing

I am trying to deploy a React app with a simple content management function in Microsoft Azure.
When users access static content on the website, the app simply reads html code from a database, and display it.
Given that the html code itself is static, I need to host files like images as static resources in Azure blob storage. But since I already assigned my custom domain name to the app, I am not able to use the same domain name with blob storage.
How do I integrate blob storage in my app so that when the browser tries to access files hosting under the route i.e. "/assets", that it looks up the path name and file name in the corresponding folder in Azure blob storage?
For example, if the html code wants to access "/assets/img/1.jpg", it will get "/img/1.jpg" from my Azure Blob Storage folder?

• You can integrate azure blob storage in your react app by setting the resource name in ‘src/azure-storage-blob.ts’ as follows: -
‘const storageAccountName = process.env.storageresourcename || "fileuploaddemo";’
• Generate a SAS token with the below parameters: -
Property Value
Allowed services Blob
Allowed resource types Service, Container, Object
Allowed permissions Read, write, delete, list, add, create
Enable deletions of version Checked
Start and expiry date/time Accept the start date/time and set the end date time 24 hours in the future. Your SAS token is only good for 24 hours.
HTTPS only Selected
Preferred routing tier Basic
Signing Key key1 selected
• Set the SAS token in the ‘src/azure-storage-blob.ts’ as follows. Don’t add the ‘?’ in the SAS token value when adding the SAS token in the code.
// remove ? if it is first character of token
const sasToken = process.env.storagesastoken || "SAS token";
• Now, configure CORS to allow the resource to connect to the app with custom domain name and save it. Thus, you will be able to upload the images in the azure blob storage by opening the app URL in the browser locally.
• Once the images are uploaded, they can be accessed and arranged through the storage explorer in a hierarchy and similarly the path of those images can be mapped to the Azure blob storage and accessed through a browser.
Please refer the below links for more information: -
https://learn.microsoft.com/en-us/azure/developer/javascript/tutorial/browser-file-upload-azure-storage-blob
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-custom-domain-name?tabs=azure-portal
Thanking you,

how can I implement a WebSocket using Tornado on a live server instead of localhost

I'm new to using WebSockets, and have so far been limited to using them via localhost and developing everything locally. I have followed this tutorial for live data visualisation:
https://medium.com/#benjaminmbrown/real-time-data-visualization-with-d3-crossfilter-and-websockets-in-python-tutorial-dba5255e7f0e
I worked through the tutorial (all of the scripts are available from the GitHub link in the article) and it works perfectly via localhost. However, I have some website space on which I am able to put the main index script which works when the WebSocket script is running on the same local machine that is displaying the website in the browser (in this case listening to localhost 8001), but I am struggling to figure out how to use a WebSocket with a live server rather than just via the local host method in this article! I would like to be able to access the webpage that displays the live data from any computer through my website, i.e there is one computer running the WebSocket that is unrelated to the computer(s) which displays the website in their browser. Any help would be greatly appreciated!
Edit: I think that I specifically need help with rewriting the websocket.py script such that the information is being fed to a server, i.e www.examplewebsite.com, which I can then access via my index file page at www.examplewebsite.com/index.html using the js in my index file:
var connection = new WebSocket('wss://examplewebsite.com/blank:8001/websocket');
or something similar, as opposed to
var connection = new WebSocket('ws://localhost:8001/websocket');
in the original version, but I am unsure how to establish this when setting up the WebSocket using the existing code in order to listen there in the index file. When running the WebSocket on my local machine, the live webpage successfully displays the live updating data, however, if someone else (who is not running the WebSocket) goes to the same page they will not see live data.
The websocket.py script is as follows:
import time
import random
import json
import datetime
import os
from tornado import websocket, web, ioloop
from datetime import timedelta
from random import randint
paymentTypes = ["cash", "tab", "visa","mastercard","bitcoin"]
namesArray = ['Ben', 'Jarrod', 'Vijay', 'Aziz']
class WebSocketHandler(websocket.WebSocketHandler):
# Addition for Tornado as of 2017, need the following method
# per: http://stackoverflow.com/questions/24851207/tornado-403-get-warning-when-opening-websocket/25071488#25071488
def check_origin(self, origin):
return True
#on open of this socket
def open(self):
print ('Connection established.')
#ioloop to wait for 3 seconds before starting to send data
ioloop.IOLoop.instance().add_timeout(datetime.timedelta(seconds=3), self.send_data)
#close connection
def on_close(self):
print ('Connection closed.')
def check_origin(self, origin):
return True
# Our function to send new (random) data for charts
def send_data(self):
print ("Sending Data")
#create a bunch of random data for various dimensions we want
qty = random.randrange(1,4)
total = random.randrange(30,1000)
tip = random.randrange(10, 100)
payType = paymentTypes[random.randrange(0,4)]
name = namesArray[random.randrange(0,4)]
spent = random.randrange(1,150);
year = random.randrange(2012,2016)
#create a new data point
point_data = {
'quantity': qty,
'total' : total,
'tip': tip,
'payType': payType,
'Name': name,
'Spent': spent,
'Year' : year,
'x': time.time()
}
print (point_data)
#write the json object to the socket
self.write_message(json.dumps(point_data))
#create new ioloop instance to intermittently publish data
ioloop.IOLoop.instance().add_timeout(datetime.timedelta(seconds=1), self.send_data)
if __name__ == "__main__":
#create new web app w/ websocket endpoint available at /websocket
print ("Starting websocket server program. Awaiting client requests to open websocket ...")
application = web.Application([(r'/static/(.*)', web.StaticFileHandler, {'path': os.path.dirname(__file__)}),
(r'/websocket', WebSocketHandler)])
application.listen(8001)
ioloop.IOLoop.instance().start()
The Github link to all the tutorial scripts, I am using index.html and websocket.py: https://github.com/benjaminmbrown/real-time-data-viz-d3-crossfilter-websocket-tutorial/tree/master/rt-data-viz
NOTE: In my version, I have replaced some of the randomly generated numbers in the websocket.py script with numbers generated by an external device so the WebSocket must be established in python on the machine connected to said external device.

Your web socket server should run on a system that can be found via DNS by a hostname or IP address and listen there on port 8001 for connections.
Let’s say this system can be found (ping, nslookup) at local.host.home, then change
var connection = new WebSocket('ws://local.host.home:8001/websocket');
Now the server should feed any client that connects.

is it okay grant public read access to some objects in my Amazon S3 bucket?

I store profile pictures in the s3 bucket and I store the aws URL in my database. when I need the profile picture I set the URL in the database into image tag.for this I set s3 bucket policy as public for read access.is this is a good idea or is there any other way to do this?

One way of going around making a bucket publicly accessible is to:
put all your image files under one 'folder'
create a CloudFront Distribution that serves only from that folder
only allow read access to the Identity that will be generated by the CF wizard
On the back end you should be able to infer the final location of the assets, as you should know the CF endpoint at this point.
Note: you should set the CF endpoint as an env var for your backend and not hardcode it.

Browser-based upload - AWS S3 Signed POST - Verify correct file is uploaded

I'm trying to understand how can I inform the backend that the user has uploaded a file to s3, its key and preventing the user from tampering with the key.
The files to be uploaded are private, meaning I'm generating signed GET URLs to download the files when the user wants them.
I'm using boto3 to create the presigned POST urls, so I'm not using a home-made implementation, but this is library-agnostic.
I have a javascript frontend and a backend API, the userflow for uploading a file would be more or less:
Browser -> API (GET /sign):
Request:
{
filename: 'something.txt
size: 12345
type: image/jpeg
}
Response:
The backend using filename calculates a random key (so that collision is avoided), then using ACCESS_KEY, SECRET_KEY, BUCKET_NAME and some more info calculates a signature, sending back the required parameters to the frontend
Backend does not save any data in its db, as there's no actual data until the client uploads the file.
{
url: https://mybucket.s3.amazonaws.com
fields: {
acl: 'private',
key: 'mykey',
signature: 'mysignature',
policy: 'mybase64 encoded policy'
}
}
Browser -> s3 (POST mybucket.s3.amazonaws.com):
Request:
Browser sends the actual file and the signed parameters as a multipart/form-data
{
acl: 'private',
key: 'mykey',
signature: 'mysignature',
policy: 'mybase64 encoded policy'
}
Response (documentation aws RESTObjectPOST):
Among headers and body we have:
Bucket
Key
ETag (which "is an MD5 hash of the object")
Location
Many javascript libraries and guides (i.e. FineUploader, Heroku) now simply proxies the key from the POST response to the backend. Then the backend checks that the file exists and adds the new file key to its database.
Now, the files to be uploaded are private, meaning I'm generating signed GET URLs to download the files when the user wants them.
Let's say there's UserA and UserB, UserB uploaded FileB.
What's preventing UserA from uploading a file, but sending to the backend the FileB key (which can be guessed or just be random input until they get something existing) and thus the backend saving that userA has fileB?
What I tought of:
keys are random and a key/file can only belong to one user, thus when userA tells "I've uploaded fileB" the backend responds "fileB is of another user"/"Error". This seems to me the easiest way to solve the problem, but I think I'm missing something (concurrency, leftover files,...)
On GET /sign backend stores "UserA will upload FileA", thus on the callback the backend checks that UserA wanted to upload FileA, if mismatching "Error"
Checking the ETag returned by s3, so that they must have the file to calculate its md5 and the user can't get FileB.
Setting in the policy that the key is XXXX, and on the callback recalculate the signature and checking that it's the same.
Using a temporary upload bucket so that a malicious user would need not only to guess a random key but also that this key is of a file being uploaded now. When the callback is called the backend moves the file with the specified key to the final bucket.
Maybe I'm overlooking something, but I can't find any solution online apart from (I think) hacky ones (setting the key to start with the username, ...).
I can't post more than two links, so some aws documentation is left out.

Upload file using google cloud end points to google app engine application

I have an application on Google App Engine written in python, i am using this application as a server to my web application written in PHP which i am using as client to call my server side api on GAE using javascript end point.
I want to upload file from my web application to my GAE application using google cloud end points.I studied about blob store but it didn't help much.
Please suggest what should i do.

You have to use the Cloud Storage to store your files, instead of Blobstore.
Use GCS because:
Google is moving away from the blobstore
GCS offers more functionality like acl, folders, and more.
You can use filenames and a blobstore like serving url for images
You can create serving url's for non images
GCS is cheap and has a free default bucket
To using GCS, you have to use the https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/
Here is an exemple in Java to use this API:
GcsService gcsService = GcsServiceFactory.createGcsService();
GcsFilename filename = new GcsFilename(CONSTANTES.BUCKETNAME, unique+".jpg");
GcsFileOptions options = new GcsFileOptions.Builder()
.mimeType("image/jpg")
.acl("public-read")
.build();
GcsOutputChannel writeChannel = gcsService.createOrReplace(filename, options);

EDIT: This is written (my mistake) as if you were on java. Feel free to use the analogous pattern for PHP on App Engine.
What the other user wrote about GCS answers the storage portion of your question - you should definitely use GCS - but as far as your idea of POSTing the form data to an endpoints function, this is definitely not advised. API calls should be little tiny pieces of data. An API should be like a paper airplane, lightly flying to your server to request some real data.
The way to have users upload files to GCS is to serve them a page with a file upload form (enctype="multipart/form-data") where the form action parameter is an upload URL generated to template the page in your servlet's doGet(HttpServletRequest req, HttpServletResponse resp) method using the GCS client library function createUploadUrl(). You can use it like this:
String uploadUrl = blobstoreService
.createUploadUrl("/upload",
UploadOptions.Builder.withGoogleStorageBucketName("my_bucket"));
In this manner you can obtain an upload URL for GCS that the user's file POST will go to. The route /upload is a route on your app that will be redirected once the upload is received. Any extra form parameters you add will still be visible to the request handler (a doPost() method on a servlet) which you define for that route.
With all of this info, you're ready to begin serving file upload forms to your users, without worrying about how this interacts with Cloud Endpoints. Use Endpoints to define API calls needed by your javascript/Android/iOS client, not for handling file uploads.