xhr2 file upload - accessing filenames with formData - javascript

I have an upload component to my application, which successfully uses the older form upload method in an iFrame. I have an Ajax poll that returns upload progress every 1000ms.
I am doing feature detection and allowing capable browsers to upload via xhr2. The theory is that I then have access to the progress on the client side; no need to poll, and the progress bar updates far more smoothly and elegantly! At the end of the batch upload, I need to redirect to a page showing a list of all the files uploaded.
You can upload files through xhr2 two ways: each file is its own upload, or you make a "formData" object containing all files. Each has benefits and drawbacks, and I think I'm trying to get the best of both worlds which may not be possible. The "best of both worlds" to me would meet these requirements:
Be able to display progress and filenames of individual files as they go
Be able to display a "total progress" bar as the package is sent
Capture the "package" of files on the server side and send back a redirect upon completion of the whole package.
Here's sample code for single-file transfer. I can meet criteria #1 easily, and could make code for criteria #2 with some effort. #3 is a bit more problematic as the client side simply sends a bunch of separate files as individual transfers.
function uploadFile(file) {
xhr = new XMLHttpRequest();
xhr.upload.addEventListener("progress", function(evt) {
if (evt.lengthComputable) {
//domNode.bar is a cached jQuery object
domNode.bar.css('width', (evt.loaded / evt.total) * 100 + "%");
console.log('my current filename is: ' + file.name;
}
else {
// No data to calculate on
}
}, false);
xhr.addEventListener("load", function() {
console.log("finished upload");
}, false);
xhr.open("post", remoteURL, true);
// Set appropriate headers
xhr.setRequestHeader("Content-Type", "application/octet-stream");
xhr.setRequestHeader("X-File-Name", file.name);
xhr.setRequestHeader("X-File-Size", file.size);
xhr.setRequestHeader("X-File-Type", file.type);
// Send the file
xhr.send(file);
}
}
Here's sample code for sending as formData. I cannot meet criteria #1 right now... at all. I can meet critera #2 and #3 fairly easily:
function uploadFile(form) {
xhr = new XMLHttpRequest();
xhr.upload.addEventListener("progress", function(evt) {
if (evt.lengthComputable) {
console.log(evt);
domNode.bar.css('width', (evt.loaded / evt.total) * 100 + "%");
// no longer have access to the file object
}
else {
// No data to calculate on
}
}, false);
xhr.addEventListener("load", function() {
console.log(xhr);
}, false);
xhr.open("post", remoteURL, true);
// Set appropriate headers
xhr.setRequestHeader("Content-Type", "multipart/form-data");
// Send the form
xhr.send(form);
}
}
// just assume that I've used formData.append to build a valid form object,
// which I have; and that this is triggered by a click event or something
uploadFile(form);
I'm not completely without options, but before I move forward with implementation I just wanted to sanity check that I'm not missing anything. Here are the options as I see them:
Continue to use only the Ajax calls for progress. Less smooth which is counter to our design goal, but the benefit is that I don't need to use an iFrame for the upload since I have the xhr2 API. I also get the single redirect when the stream closes.
Iterate over the files and do some heavy lifting on the client side for the progress; in addition to individual file progress bars, I can create an algorithm to track total progress. At the same time, I send a file count to the server, and use this count to hold off on a redirect URL until the last file arrives, and then I can send it.
Any thoughts?

If you want to send all files in one request, FormData is your only option of the two. Either option allows you to cover points 1 & 2.
Since you asked about other APIs, I feel it is justified in mentioning the cross-browser upload tool I maintain: Fine Uploader. My library provides callbacks that will allow you to easily achieve goals #1 & #2. As far as #3 is concerned, Fine Uploader sends each file in a separate request. However, it is not hard to make use of Fine Uploader's API to present a "total progress" bar. I've done just that in a project that uses Fine Uploader. The library provides a bunch of callbacks and exposes a comprehensive API that should be useful to you, given the points you have specified.
Note that upload progress is not currently available in IE9 and older in Fine Uploader, but you should be able to make use of your method to display progress via ajax calls. Your method is one possible approach, but there is another possible option that is scheduled for further review in the future: use of Apache/nginx's UploadProgress module. See feature request #506.

Related

How to get started adding fields in-browser that can be filled in on a PDF? (Like e-signature)

There are a million websites that offer e-signatures, usually by letting you draw a rectangle on a PDF (or file that is converted to a PDF) where the signature will be placed.
I assume that this is using the Canvas element and AJAX to send the location where the rectangle was drawn back to the server.
My nonprofit already uses a great open source document assembly tool called Docassemble. I'd like to leverage our existing form library to allow for direct integration with signatures. I actually already built a Docassemble app that does e-signing, but you need to manually place the fields in the file before uploading it. Placing the signature location in-browser would make it exponentially better. Having to pass it off to a third-party is probably more than we can pay for but also would be much less useful.
I'm not a novice programmer but I've never used the Canvas element. I really don't know where to start with this project. Any advice? Are there any libraries that would help? Docassemble is built on Python/Flask with Jquery but this seems mostly like a generic JavaScript question. I have seen so many cheap Docusign clones with this feature that I wonder if there's a library that I can't find with my Google-fu.
Have a look at https://github.com/szimek/signature_pad (live demo) which has no external deps.
The example in the docs/ directory implements a Save PNG button which downloads a PNG in the user's browser. I was able to modify these lines from the example to instead call my own upload function:
var dataURL = signaturePad.toDataURL();
// download(dataURL, "signature.png");
upload(dataURL)
This function makes the AJAX POST request:
function upload(data){
xhr = new XMLHttpRequest();
xhr.open('POST', '/submit');
xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
xhr.onload = function() {
if (xhr.status === 200) {
alert('Data submitted: ' + xhr.responseText);
}
else if (xhr.status !== 200) {
alert('Request failed. Returned status of ' + xhr.status);
}
};
xhr.send(encodeURI('data=' + data));
Now in my /submit route request.form['data'] is an inline image which can then be stored in the database, or rendered directly in the src attribute of an <img> tag.

How to execute javascript function after another function completes [duplicate]

I want to redirect the user to a different webpage after they click a hyperlink which allows them to download a file. However since they need to make a choice in the open/save file dialog, I don't want to redirect them until they accept the download.
How can I detect that they performed this action?
As i've found from years of maintaining download.js, there simply is no way to tell from JS (or likely in general, see below) what a user chooses to do with the download Open/Save dialog. It's a common feature request, and i've looked into it repeatedly over the years. I can say with confidence that it's impossible; I'll joyfully pay 10 times this bounty if someone can demo a mechanical way to determine the post-prompt user action on any file!
Further, it's not really just a matter of JS rules, the problem is complicated by the way browsers download and prompt such files. This means that even servers can't always tell what happened. There might be some specific work-arounds for a few specific cases, but they are not pretty or simple.
You could force your users to "re-upload" a downloaded file with an <input type=file> to validate it, but that's cumbersome at best, and the local file browse dialog could be alarming to some. It's the only sure-fire method to ensure a download, but for non-sensitive applications its very draconian, and it won't work on some "mobile" platforms that lack file support.
You might also try watching from the server side, pushing a message to the client that the file was hit on the server. The problem here is that downloads start downloading as soon as the Open/Save dialog appears, though invisibly in the background. That's why if you wait a few moments to "accept" a large file, it seems to download quickly at first. From the server's perspective, the activity is the same regardless of what the user does.
For a huge file, you could probably detect that the whole file was not transferred, which implies the user clicked "cancel", but it's a complicated syncing procedure pushing the status from backend to client. It would require a lot of custom programming with sockets, PHP message passing, EventSource, etc for little gain. It's also a race against time, and an uncertain amount of time at that; and slowing down the download is not recommended for user satisfaction.
If it's a small file, it will physically download before the user even sees the dialog, so the server will be useless. Also consider that some download manager extensions take over the job, and they are not guaranteed to behave the same as a vanilla browser. Forcing a wait can be treacherous to someone with a slow hard drive that takes "forever" to "finish" a download; we've all experienced this, and not being able to continue while the "spinny" winds down would lower user satisfaction, to put it mildly.
In short, there's no simple way, and really no way in general, except for huge files you know will take a long time to download. I've spent a lot of blood sweat and tears trying to provide my download.js users the ability, but there are simply no good options. Ryan dahl initially wrote node.js so he could provide his users an upload progress bar, maybe someone will make a server/client package to make it easy to do the same for downloads.
Here is a hacky solution.
My StreamSaver lib don't use blobs to download a file with a[download]. It uses service worker to stream something to the disc by emulating how the server handles download with content-disposition attachment header.
evt.respondWith(
new Response(
new ReadableStream({...})
)
)
Now you don't have any exact way of knowing what the user pressed in the dialog but you have some information about the stream. If the user press cancel in the dialog or abort the ongoing download then the stream gets aborted too.
The save button is trickier. But lets begin with what a stream bucket highWaterMark can tell us.
In my torrent example I log the writer.desiredSize. It's the correlation to how much data it is willing to receive. When you write something to the stream it will lower the desired size (whether it be a count or byte strategy). If it never increases then it means that user probably have paused the download. When it goes down below 0 then you are writing more data than what the user is asking for.
And every chunk write you do returns a promise
writer.getWriter().write(uint8).then(() => {
// Chunk have been sent to the destination bucket
// and desiredSize increase again
})
That promise will resolve when the bucket isn't full. But it dose not mean that the chunk have been written to the disc yet, it only means that the chunk has been passed from the one stream to another stream (from write -> readable -> respondWith) and will often do so in the beginning of the stream and when another earlier chunk have been written to the disc.
It's a possibility that the write stream can finishes even before the user makes a choice if the hole data can fit within the bucket (memory)
Tweaking the bucket size to be lower then the data can help
So you can make assumption on when the
download starts
finish
and pauses
but you won't know for sure since you don't get any events (apart from the abort that closes the stream)
Note that the torrent example don't show correct size if you don't have support for Transferable streams but you could get around this if you do everything inside a service worker. (instead of doing it in the main thread)
Detecting when the stream finish is as easy as
readableStream.pipeTo(fileStream).then(done)
And for future references WICG/native-file-system might give you access to write files to disc but it has to resolve a prompt dialog promise before you can continue and might be just what the user is asking for.
There are examples of saving a blob as a stream, and even more multiple blob's as a zip too if you are interested.
Given that user is, or should be aware that file should be downloaded before next step in process, user should expect some form of confirmation that file has been downloaded to occur.
You can create a unique idenfifier or timestamp to include within downloaded file name by utilizing <a> element with download attribute set to a the modified file name.
At click event of <button> element call .click() on <a> element with href set to a Blob URL of file. At a element click handler call .click() on an <input type="file"> element, where at attached change event user should select same file which was downloaded at the user action which started download of file.
Note the chaining of calls to .click() beginning with user action. See Trigger click on input=file on asynchronous ajax done().
If the file selected from user filesystem is equal to modified downloaded file name, call function, else notify user that file download has not been confirmed.
window.addEventListener("load", function() {
let id, filename, url, file;
let confirmed = false;
const a = document.querySelector("a");
const button = document.querySelector("button");
const confirm = document.querySelector("input[type=file]");
const label = document.querySelector("label");
function confirmDownload(filename) {
if (confirmed) {
filename = filename.replace(/(-\d+)/, "");
label.innerHTML = "download of " + filename + " confirmed";
} else {
confirmed = false;
label.innerHTML = "download not confirmed";
}
URL.revokeObjectURL(url);
id = url = filename = void 0;
if (!file.isClosed) {
file.close()
}
}
function handleAnchor(event) {
confirm.click();
label.innerHTML = "";
confirm.value = "";
window.addEventListener("focus", handleCancelledDownloadConfirmation);
}
function handleFile(event) {
if (confirm.files.length && confirm.files[0].name === filename) {
confirmed = true;
} else {
confirmed = false;
}
confirmDownload(filename);
}
function handleDownload(event) {
// file
file = new File(["abc"], "file.txt", {
type: "text/plain",
lastModified: new Date().getTime()
});
id = new Date().getTime();
filename = file.name.match(/[^.]+/g);
filename = filename.slice(0, filename.length - 1).join("")
.concat("-", id, ".", filename[filename.length - 1]);
file = new File([file], filename, {
type: file.type,
lastModified: id
});
a.download = filename;
url = URL.createObjectURL(file);
a.href = url;
alert("confirm download after saving file");
a.click();
}
function handleCancelledDownloadConfirmation(event) {
if (confirmed === false && !confirm.files.length) {
confirmDownload(filename);
}
window.removeEventListener("focus", handleCancelledDownloadConfirmation);
}
a.addEventListener("click", handleAnchor);
confirm.addEventListener("change", handleFile);
button.addEventListener("click", handleDownload);
});
<button>download file</button>
<a hidden>download file</a>
<input type="file" hidden/>
<label></label>
plnkr http://plnkr.co/edit/9NmyiiQu2xthIva7IA3v?p=preview
jquery.fileDownload allows you to do this:
$(document).on("click", "a.fileDownloadPromise", function () {
$.fileDownload($(this).prop('href'))
.done(function () { alert('File download a success!'); })
.fail(function () { alert('File download failed!'); });
return false;
});
Take a look at Github:
https://github.com/johnculviner/jquery.fileDownload
I had a project that I dabbled in recently that required me to specify whether a user could upload a particular kind of file, i.e. (a user can upload a png but not a pdf).
I may not used the most efficient method, but ultimately what I did was to code a small, built in "webapp" that functioned as a file browser, for upload or download.
I suppose the closest example without releasing my "secret project" would be https://encodable.com/filechucker/
Maybe you could write a simple integrated filebrowser such as that that cloud services use sometimes (i.e. dropbox) and have some functions that detect input with custom boxes and stuff.
Just a few thoughts.
window.showSaveFilePicker from the File System Access API does what you want, but unfortunately it's currently supported only by Chrome and Edge. It returns a promise -- if the user chooses to download the file the promise is resolved; if they cancel, an AbortError is raised.
try this:
<html>
<head>
<script type="text/javascript">
function Confirmation(pg) {
var res = confirm("Do you want to download?");
if(res){
window.open(pg,"_blank");
}
return res;
}
</script>
</head>
<body>
download
</body>
</html>

How can I get multiple files to upload to the server from a Javascript page without skipping?

I'm working on a research experiment which uses getUserMedia, implemented in recorder.js, to record .wav files from the user's microphone and XMLHttpRequest to upload them to the server. Each file is about 3 seconds long and there are 36 files in total. The files are recorded one after another and sent to the server as soon as they are recorded.
The problem I'm experiencing is that not all of the files end up on the server. Apparently the script or the php script are unable to catch up with all the requests in a row. How can I make sure that I get all the files? These are important research data, so I need every recording.
Here's the code that sends the files to the server. The audio data is a blob:
var filename = subjectID + item__number;
xhr.onload=function(e) {
if(this.readyState === 4) {
console.log("Server returned: ",e.target.responseText);
}
};
var fd=new FormData();
fd.append("audio_data",blob, filename);
xhr.open("POST","upload_wav.php",true);
xhr.send(fd);
And this is the php file on the server side:
print_r($_FILES);
$input = $_FILES['audio_data']['tmp_name'];
$output = "audio/".$_FILES['audio_data']['name'].".wav";
move_uploaded_file($input, $output)
This way of doing things is basically copied from this website:
Using Recorder.js to capture WAV audio in HTML5 and upload it to your server or download locally
I have already tried making the XMLHttpRequest wait by using
while (xhr.readyState != 4)
{
console.log("Waiting for server...")
}
It just caused the page to hang.
Would it be better to use ajax than XMLHttp Request? Is there something I can do to make sure that all the files get uploaded? I'm pretty new to Javascript so code examples are appreciated.
I have no idea what your architecture looks like, but here is a potential solution that will work to solve your problem.
The solution uses the Web Worker API to off load the file uploading to a sub-process. This is done with the Worker Interface of that API. This approach will work because there is no contention of the single thread of the main process - web workers work in their own processes.
Using this approach, we do three basic things:
create a new worker passing a script to execute
pass messages to the worker for the worker to deal with
pass messages back to the main process for status updates/replies/resolved data transformation/etc.
The code is heavily commented below to help you understand what is happening and where.
This is the main JavaScript file (script.js)
// Create a sub process to handle the file uploads
///// STEP 1: create a worker and execute the worker.js file immediately
let worker = new Worker('worker.js');
// Ficticious upload count for demonstration
let uploadCount = 12;
// repeatedly build and send files every 700ms
// This is repeated until uplaodCount == 0
let builder = setInterval(buildDetails, 700);
// Recieve message from the sub-process and pipe them to the view
///// STEP 2: listen for messages from the worker and do something with them
worker.onmessage = e => {
let p = document.createElement('pre');
// e.data represents the message data sent from the sub-process
p.innerText = e.data;
document.body.appendChild(p);
};
/**
* Sort of a mock to build up your BLOB (fake here of-course)
*
* Post the data needed for the FormData() to the worker to handle.
*/
function buildDetails() {
let filename = 'subject1234';
let blob = new Blob(['1234']);
///// STEP 3: Send a message to the worker with file details
worker.postMessage({
name: "audio_data",
blob: blob,
filename: filename
});
// Decrease the count
uploadCount--;
// if count is zero (== false) stop the fake process
if (!uploadCount) clearInterval(builder);
}
This is the sub-process JavaScript file (worker.js)
// IGNORE the 'fetch_mock.js' import that is only here to avoid having to stand up a server
// FormDataPolyFill.js is needed in browsers that don't yet support FormData() in workers
importScripts('FormDataPolyFill.js', 'fetch_mock.js');
// RXJS provides a full suite of asynchronous capabilities based around Reactive Programming (nothing to do with ReactJS);
// The need for your use case is that there are guarantees that the stream of inputs will all be processed
importScripts('https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.3.3/rxjs.umd.js');
// We create a "Subject" that acts as a vessel for our files to upload
let forms = new rxjs.Subject();
// This says "every time the forms Subject is updated, run the postfile function and send the next item from the stream"
forms.subscribe(postFile);
// Listen for messages from the main process and run doIt each time a message is recieved
onmessage = doIt;
/**
* Takes an event object containing the message
*
* The message is presumably the file details
*/
function doIt(e) {
var fd = new FormData();
// e.data represents our details object with three properties
fd.append(e.data.name, e.data.blob, e.data.filename);
// Now, place this FormData object into our stream of them so it can be processed
forms.next(fd);
}
// Instead of using XHR, this uses the newer fetch() API based upon Promises
// https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
function postFile(fd) {
// Post the file to the server (This is blocked in fetch_mock.js and doesn't go anywhere)
fetch('fake', {
method: 'post',
body: fd,
})
.then((fd) => {
// After the XHR request is complete, 'Then' post a message back to the main thread (If there is a need);
postMessage("sent: " + JSON.stringify(fd));
});
}
Since this will not run in stackoverflow, I've created a plunker so that you can run this example:
http://plnkr.co/edit/kFY6gcYq627PZOATXOnk
If all this seems complicated, you've presented a complicated problem to solve. :-)
Hope this helps.

Rails file download using send_data with follow-up action

I am working on a code base where I need to allow the user to download a PDF document that already resides on AWS S3. I have implemented a download concern that was used for a previous feature.
For this feature, I need to update the UI (A progress stepper) after the user has completed the file download. I was initially thinking that this would be as simple as:
User clicks download
API call is made where the file is downloaded using send_data. In this API call, I'd also update the Foo model to change state to indicate that the user has downloaded the file;
Execute a redirect_to request.referer to reload the data. The changed state in Foo will be responsible for showing the updated progress in the UI;
I was mistakenly thinking that this was going to be simple. The reasons for complexity:
send_data is already rendering data, so I can't refresh the page using redirect_to as this triggers a multiple render error;
send_data does not work with the remote: true option, so requesting data via an AJAX link and updating the ERB template is out;
I can write everything into a JS on click function, but this seems like a bit of a hack. I probably need to retrieve the file directly from AWS and skip my api? I'm suspecting that I might run into CORS issues as I don't have control over the server.
This is what my rails download method looks like currently:
def download
attachment = Attachment.find_by_id(params[:attachment_id])
content = send_data(
attachment.file.read,
filename: "#{attachment.title}.#{attachment.file.file.extension}",
type: attachment.content_type,
disposition: "attachment",
)
end
Th js code that basically worked looks like this where all the relevant paths & filenames are passed on to the JS via data-attributes:
$(document).on("click", "#download", function(e){
e.preventDefault();
const data = $('#temp-information').data();
var req = new XMLHttpRequest();
req.open("GET", data.path, true);
req.responseType = "blob";
const filename = data.title;
req.onload = function (event) {
var blob = req.response;
console.log(blob.size);
var link=document.createElement('a');
link.href=window.URL.createObjectURL(blob);
link.download= filename;
document.body.appendChild(link);
link.click();
};
if (typeof window.navigator.msSaveBlob !== 'undefined') {
// Fix to work in IE11
window.navigator.msSaveBlob(blob, filename);
} else {
req.send();
}
});
What is the most effective & rails'y way of handling a file download & updating the UI after the download has been completed?
It's not 100% clear what you're trying to accomplish. If you're trying to let the user see download progress, I'm not sure that you really need to do anything except send_data, and most browsers will then begin downloading the file, including showing a progress bar.
Since it seems you want to do something after the file download is complete, that's quite a bit trickier. There's nothing Rails-specific about the problem, and the approach you have used looks pretty reasonable to me.
On this SO thread you'll find a lengthy discussion of this problem and various ways that people have tried to solve it. In general the solutions follow the same basic structure, which is to simply poll the server.
In your Rails app you could implement that roughly as follows. Suppose you added a field status to your attachment model...
def download
attachment = Attachment.find_by_id(params[:attachment_id])
attachment.update(status: "downloading")
send_data(
attachment.file.read,
filename: "#{attachment.title}.#{attachment.file.file.extension}",
type: attachment.content_type,
disposition: "attachment",
)
attachment.update(status: "complete")
end
Then you can add an endpoint that returns the status of a file. Thus when the user starts to download the file you begin to poll that endpoint.
def attachment_status
attachment = Attachment.find_by_id(params[:attachment_id])
respond_to do |format|
format.json do
{status: attachment.status}
end
end
end
Then in Javascript, for example using HttpPromise:
var http = new HttpPromise;
function poll(doneFn) {
http.get("/status.json") // you will need to set your actual status endpoint path here
.success(function(data,xhr){
if (data.status == "complete") {
doneFn();
}
});
};
function downloadFinished(){
// ... do whatever you want on finish here ...
};
setInterval(function(){ poll(downloadFinished) }, 5000);
It's not the most beautiful thing in the world, but it should get the job done.
Good luck!

File drag and drop event in jquery

I'm trying to find a way of letting users drag and drop individual files into an area on my page that can then get submitted along with all my other form data.
In my research I've found multiple "drag and drop" upload scripts but they all do way, way too much. I want to handle the actual uploading myself and just provide a way for users to upload files without hitting the browse button.
Is there an event in jquery (or something similar) that I should be looking for?
Any help is much appreciated!
I came across this question while researching some AJAX file upload techniques.
I created a drag and drop upload script today (its still in proof of concept stage but heres the basic steps that I took.
$('drag-target-selector').on('drop', function(event) {
//stop the browser from opening the file
event.preventDefault();
//Now we need to get the files that were dropped
//The normal method would be to use event.dataTransfer.files
//but as jquery creates its own event object you ave to access
//the browser even through originalEvent. which looks like this
var files = event.originalEvent.dataTransfer.files;
//Use FormData to send the files
var formData = new FormData();
//append the files to the formData object
//if you are using multiple attribute you would loop through
//but for this example i will skip that
formData.append('files', files[0]);
}
now you can send formData to be processed by a php script or whatever else you want to use. I didn't use jquery in my script as there a lot of issues with it it seemed easier to use regular xhr. Here is that code
var xhr = new XMLHttpRequest();
xhr.open('POST', 'upload.php');
xhr.onload = function() {
console.log(xhr.responseText);
};
xhr.upload.onprogress = function(event) {
if (event.lengthComputable) {
var complete = (event.loaded / event.total * 100 | 0);
//updates a <progress> tag to show upload progress
$('progress').val(complete);
}
};
xhr.send(formData);

Categories

Resources