Develop Reference

Count of JS S3 Object Access

In my app, my customers can create small widgets with text fields and text. They can customize the look and feel through JS and CSS. I upload the JS and CSS in my S3 bucket and use Cloudfront for CDN.
Once the widget is created, they can embed the widget on their website using embed code.
In the embed code, I have used a 1x1 pixel image which is used to send request to php so I can increase the visit counter.
public function track(Request $request) {
// increase stored number here
header('Content-Type: image/gif');
return base64_decode('R0lGODlhAQABAJAAAP8AAAAAACH5BAUQAAAALAAAAAABAAEAAAICBAEAOw==');
}
My server is getting overload because of the visit counter. I want to now track the number of visits to embedded widgets for each customer by using S3 and Cloudfront access count.
I searched and found - Getting the download count of a specific S3 object, but it is for download.
How can I get the count of times when S3 object is accessed?

Use cloudtrail to and parse the logs. List of actions trackable by cloudtrail on s3

You can use URL shortener service to redirect to your file (like goo.gl, bit.ly), they will track number of clicks,views etc, and it wont even overload your server.
Hope it helps.

Generate downloadable link to a file and automatically initiate download

Users generate files on my node js server by pressing a button on a web page.
The server then creates a .zip file.
I want to expose this zip file so that it can be downloaded automatically on to the users' client.
Once downloaded, I want the server to detect that the download is finished and delete the zip file.
1- How do I expose the file in node js? Should the system put it in public folder? That means it will be security risk and anyone can read it.How can I link to a file & make it downloadable which is not in public folder?
2- How do I detect that the download is finished? Should I run a cron job to delete the files without worrying about the download progress?

A few remarks that should help you:
If you are creating temporary files, a good practice is to create signed URLs. Those are URLS that contain specific token that is valid for a limited amount of time. Implementation is trivial - generate the file .zip and token, set timestamp preferably in the DB and construct signed link with token. If the file is not downloaded by client in a given amount of time, it is invalid.
Zip file should have unique name, preferably some random one (if it's not a problem, you could still use header called Content-Disposition to decide on its name during download). You store it in the TEMP dir inside your project.
After user clicks previously generated signed link with token that relates to that file, you start download (streaming). After streaming is complete (refer to NodeJS streams lib), you just delete the file.
And on the client side:
You create a button that leads to endpoint on server (triggers AJAX call or whatever). After clicking, you run mentioned above logic on server.
In return, user client gets generated link (leading to ANOTHER endpoint that handles those signed links only) that has to be followed to download the file
Using any kind of DOM manipulation, you create hidden <a/> tag with href leading to this link and then you trigger automatic click of this link in the JS code. Preferably, if you support new browsers, it's a good idea to add download attribute to it.
DO NOT:
put the file in the public folder. Create endpoint that will stream its contents to the client. Create just some temp dir for it.
run the CRON job for deleting the files. Or run it only if something fails, to clean old files. File should be deleted after it's downloaded (which you will know, as when your stream is closed, you will get a proper event).
IMPLEMENTATION SUGGESTIONS
Create two endpoints on the server (using Express or whatever framework for routing). One for requesting the file (that starts generation process) and another one for downloading it.
After the generation process is finished, store the .zip inside e.g. temp catalog and create token for it.
Store set of data like this in the DB for EVERY download:
zip file name
token (e.g. generated random hash)
timestamp of the generation
Pass the new link to the client (for the second endpoint that is used for download process). Client should initialise the download automatically without human interaction, as suggested above
When link is "clicked" by the code, your server receives another request on second endpoint and then:
checks if the token is still valid (e.g. for 30 seconds).
if not: 403 or 404
if yes: start streaming the data (create file stream and stream it to the client)
when streaming back, include proper headers with response, e.g. file name that client should see (this will obscure your internal file names from temp catalog), using Content-Disposition
After streaming is complete, delete the file
Create CRON job that will run e.g. once a day, ask the DB for ALL the files that have invalid tokens (expired ones) and will try to delete them, if they exist (but this should not be a common scenario, if you delete them properly when the streaming is finished).

How can I check s3 file uploaded or not using javascript

I'm implementing a direct pdf file upload from client machine to Amazon S3 via REST API using only Go langangue, All works fine but one thing is worrying me...
Here is the steps
User click on pdf button
New browser tab is open there is in html page(which says generating
your report)
On background pdf file is uploading(in process) on s3. And API return s3
url to client.
Problem
how can I check if the URL is active yet or not. If it's a 404 it doesn't redirect… waits another N seconds. Once it's a 200, then I redirect to s3 url.
How can I achieve this on javascript ?

AWS S3 ensures GET after PUT consistency for new objects. From https://aws.amazon.com/s3/faqs/
"
Q: What data consistency model does Amazon S3 employ?
Amazon S3 buckets in all Regions provide read-after-write consistency for PUTS of new objects and eventual consistency for overwrite PUTS and DELETES.
"
This ensures that once the upload is done, your object will be reachable. Now, with JS you can issue an Ajax request only if you're on same domain or you enable CORS on your S3 bucket. This is explained here: http://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html and it will allow you to check your object on S3 being there.
Otherwise, you would need a server-side component to check if the object is uploaded and call that resource from JS on the same domain.

Embed PDF w/POST Parameters

I need to pass a bunch of parameters to a web API method which returns a byte array that is needed to display a PDF report on a page. I tried doing this via a GET method but the query string is too long and it throws an exception. I have tried shortening the query string as much as physically possibly but it is still too long. All of the external PDF viewer libraries require a string path to the PDF. I need something that allows me to supply a string path and some parameters or just a string path but that uses a POST instead of a GET. I was thinking about POSTing the object values to the web API along with a GUID and storing it somehow on the server then with an tag calling another API method and retrieving the stored data by only passing the GUID as a parameter but I don't want to make a database table for only that to save and retrieve the data. I entertained the idea of saving the byte array to a PDF on the server and then simply pointing a PDF viewer to that after but I had no success with that either and that is not ideal.
File.WriteAllBytes(path, res);
The code above is the most common way I have found to save the file but when I open the new PDF file after it says it is corrupt and it won't allow me to open it. Ideally I would just find a method that allows me to POST data to my web API then embeds the result in a pdf.js/pdfObject/ control. Please help, this has stumped me for days. Thanks in advance!!

Filepicker.io - upload/store file after choosing export destination

In my web app, I'm generating some large image files and would like the user to be able to export them using filepicker.io
My problem is that it seems I have to call filepicker.store to upload the generated file BEFORE I call filepicker.exportFile, but I want to upload the file AFTER the user has chosen a destination. Is this possible?

While not the most elegant mechanism, you can use the fact that filepicker urls are read/write to store a "temporary" image using the exportFile call. You can then take the FPFile object that is returned and do a filepicker.write() call to store the file after the destination has been chosen.