I am working on a nodejs service where I need to read a PDF document from a file.
At a high level, here is the workflow. First time a user requests a PDF, I generate it (using pdfkitjs) and save it to the server. Then, when the user request the same document again, I need to read it from the server and send it back.
Is there a way that I can use PDFDocument from pdfkitjs to read the contents from the file and create a corresponding PDFDocument? Almost all searches come up with how to pipe the PDF to a stream, but now how to read from a stream.
If "the stream" is your server response then you shouldn't have to do much more than add the appropriate headers to the response first.
Related
Users generate files on my node js server by pressing a button on a web page.
The server then creates a .zip file.
I want to expose this zip file so that it can be downloaded automatically on to the users' client.
Once downloaded, I want the server to detect that the download is finished and delete the zip file.
1- How do I expose the file in node js? Should the system put it in public folder? That means it will be security risk and anyone can read it.How can I link to a file & make it downloadable which is not in public folder?
2- How do I detect that the download is finished? Should I run a cron job to delete the files without worrying about the download progress?
A few remarks that should help you:
If you are creating temporary files, a good practice is to create signed URLs. Those are URLS that contain specific token that is valid for a limited amount of time. Implementation is trivial - generate the file .zip and token, set timestamp preferably in the DB and construct signed link with token. If the file is not downloaded by client in a given amount of time, it is invalid.
Zip file should have unique name, preferably some random one (if it's not a problem, you could still use header called Content-Disposition to decide on its name during download). You store it in the TEMP dir inside your project.
After user clicks previously generated signed link with token that relates to that file, you start download (streaming). After streaming is complete (refer to NodeJS streams lib), you just delete the file.
And on the client side:
You create a button that leads to endpoint on server (triggers AJAX call or whatever). After clicking, you run mentioned above logic on server.
In return, user client gets generated link (leading to ANOTHER endpoint that handles those signed links only) that has to be followed to download the file
Using any kind of DOM manipulation, you create hidden <a/> tag with href leading to this link and then you trigger automatic click of this link in the JS code. Preferably, if you support new browsers, it's a good idea to add download attribute to it.
DO NOT:
put the file in the public folder. Create endpoint that will stream its contents to the client. Create just some temp dir for it.
run the CRON job for deleting the files. Or run it only if something fails, to clean old files. File should be deleted after it's downloaded (which you will know, as when your stream is closed, you will get a proper event).
IMPLEMENTATION SUGGESTIONS
Create two endpoints on the server (using Express or whatever framework for routing). One for requesting the file (that starts generation process) and another one for downloading it.
After the generation process is finished, store the .zip inside e.g. temp catalog and create token for it.
Store set of data like this in the DB for EVERY download:
zip file name
token (e.g. generated random hash)
timestamp of the generation
Pass the new link to the client (for the second endpoint that is used for download process). Client should initialise the download automatically without human interaction, as suggested above
When link is "clicked" by the code, your server receives another request on second endpoint and then:
checks if the token is still valid (e.g. for 30 seconds).
if not: 403 or 404
if yes: start streaming the data (create file stream and stream it to the client)
when streaming back, include proper headers with response, e.g. file name that client should see (this will obscure your internal file names from temp catalog), using Content-Disposition
After streaming is complete, delete the file
Create CRON job that will run e.g. once a day, ask the DB for ALL the files that have invalid tokens (expired ones) and will try to delete them, if they exist (but this should not be a common scenario, if you delete them properly when the streaming is finished).
I'm trying to do client-side processing of some data sent in a server's HTTP response.
Here's what I'm working with: I have a web application that sends commands to a backend simulation engine, which then sends back a bunch of data/results in the response body. I want to be able to access this response using JavaScript (note..not making a new response, but simply accessing the data already sent from the server).
Right now, I am able to do this via a kludgy hack of sorts:
var responseText = "{{response}}";
This is using Django's template system, where I have already pre-formatted the template context variable "response" to contain a pre-formatted string representation of a csv file (i.e., proper unicode separators, etc).
This results in a huge string being transmitted to the page. Right now, this supports my immediate goal of making this data available for download as a csv, but it doesn't really support more sophisticated tasks. Also, I'm not sure if it will scale well when my string is, say, 2 MB as opposed to less than 1 KB.
I'd like to have this response data stored more elegantly, perhaps as part of the DOM or maybe in a cache (?) [not familiar with this].
The ideal way to do this is to not load the csv on document load, either as a javascript variable or as part of the DOM. Why would you want to load a 2MB data every time to the user when his intention may not be to download the csv everytime?
I suggest creating a controller/action for downloading the csv and get it on click of the download button.
I have the below requirement.
I have a data grid, which has huge amount of data. I have a download button, when clicked data grid data has to be downloaded to an excel. This data grid data I want to send to server through a rest call using xhrpost. With this data i will construct an excel file on the fly on server side and i will return the response to the client. My client by seeing this response should open popup for downloading the response to save as the file.
I am seeing the following problems here :
the data is huge. so , I can't use window.open way for sending the data in GET format.
I don't have an URL on server which will access file, as i am generating file on the fly and sending response to client.
How will i achieve my use case to download file using xhrpost on the fly, not storing the file anywhere?
Please reply as I am breaking my head how to do this? It will be of great help.
Thanks,
Sreenivas
I'm posting json to my server and I want to return a csv response and have my browser automatically download the csv.
I know there have been some questions on SO exactly about this topic on the past. So far, I've gathered the following solutions:
Put the request into <form> element. This isn't possible because the JSON is nested and too large to be posted as a string.
Save the csv file (get request) onto the server and then have a separate post request to download it. This is not ideal because I don't want to keep a bunch of useless csv files on my server.
Is there a better solution? Can I make a Post request act like a form submission and automatically get the file to start downloading?
My web application uses file which has been updated by other process. My application reads the content of the file using ajax
xmlhttp.open("GET","/config/myfile",false);
xmlhttp.send();
Once response is received then app parses this response and shows that values on Web UI. The file contains 50 fields and whenever I want to read any single field I need to open whole file.
Is there any way to get the values of single field based on key instead of reading whole file.
As per my understanding we need to read and open file and then parse the response text. But would like to know is there any way to reduce the file calls with any other method.
I want to achieve this to reduce the file I/O operations. Since other processes are writing in to it and at same time my web app accessing to read the latest value.
Any other option would be appreciated.
Note :- I do not have to use any server side scripting lang option.
Regds