I want to upload a (single) file to a server and show the progress of the upload.
I know I can upload a file using HTTP POST. I'm not familiar with web-sockets, but as I understand, binary data can also be sent that way and because web sockets are bi-directional I could get the progress of the upload.
I'm not worried about older browsers so iframe's and flash solutions aren't very appealing unless there is a significant advantage in going that route.
I'm also curious as to the best server-side technology. Are their advantages to using a WSGI server like Django? Or maybe non-blocking I/O technology like Node.js? I'm not asking if web framework x is better than web framework y, or server x is better than server y. But simply what the ideal technology should have in order to facility uploads in the client.
Update: It seems like the server side does not have bearing on the technologies/API's available on the client to facilitate uploads.
Edit (2017-10-17): As of now, there is also the option to use Fetch API. It offers essentially the same capabilities as XMLHttpRequest behind a more modern promise-based API. There is a polyfill for browsers that don't support window.fetch() natively (which is mainly Internet Explorer and older Safari versions right now).
XMLHttpRequest vs. Web sockets vs. Something else
Clearly XMLHttpRequest. Its capabilities in modern browsers are enormous and cover almost all scenarios. It will produce a standard POST or PUT request, any web server and framework combination can deal with that.
While web sockets are nice for some scenarios, it's a different protocol that adds lots of complexity - they are only worth using if you need real-time responses from the server. And as you noted yourself, other approaches like Flash are merely ugly hacks.
Sending binary data
Normally, you won't have direct access to files. So you will have an <input type="file"> form field somewhere on your page and wait for the user to choose a file. The options then are:
Sending only the file contents: request.send(input.files[0]). The request body will be the file's contents and nothing else, no encoding will be performed and no metadata like file name will be transmitted. Browser compatibility: Chrome 7, Firefox 3.6, Opera 12, IE 10.
Sending the data of the entire form: request.send(new FormData(input.form)). Here the form contents will be encoded as multipart/form-data, meaning that you can send multiple form fields and metadata like field and file names will be transmitted as well. You can also modify the FormData object before sending it. Depending on the server-side framework, handling this request might be simpler than raw data, there are typically many helpers you can use. Browser compatibility: Chrome 6, Firefox 4, Opera 12, IE 10.
Sending a typed array: just in case you don't have a file but merely want to send some binary data you generate on the fly. No extra encoding is being performed here, so as far as the server side is concerned this works like sending file contents. Browser compatibility: Chrome 9, Firefox 9, Opera 11.60, IE 10.
Displaying upload progress
You can listen to progress events on XMLHttpRequest.upload. The progress events have loaded and total properties that allow determining how far you've got with your request. Browser compatibility: Chrome 7, Firefox 3.5, Opera 11.60, IE 10.
JavaScript libraries
There are of course existing libraries wrapping the functionality outlined here. These are mentioned in other answers, searching on the web will certainly turn up even more. I explicitly don't want to propose any libraries here - which of them if any you should use is purely a matter of preference.
My answer is quite late but here it goes:
Short answer:
XMLHttpRequest is the best way to upload files in a modern browser.
What is XMLHttpRequest?
XMLHttpRequest is a JavaScript object that was designed by Microsoft and adopted by Mozilla, Apple, and Google. It's now being standardized in the W3C. It provides an easy way to retrieve data from a URL without having to do a full page refresh. A Web page can update just a part of the page without disrupting what the user is doing. XMLHttpRequest is used heavily in AJAX programming.
Despite its name, XMLHttpRequest can be used to retrieve any type of data, not just XML, and it supports protocols other than HTTP (including file and ftp).
The XMLHttpRequest object has gotten a facelift in the Html5 specifications. Specifically the XMLHttpRequest Level 2.
Advantages:
Handling of byte streams such as File, Blob and FormData objects for uploading and downloading
Progress events during uploading and downloading
Cross-origin requests
Allow making anonymous request - that is not send HTTP Referer
The ability to set a Timeout for the Request
Uploading is happening in the background
The page the user is on remains intact
Does not require any change to the server side, so existing server side logic should remain unchanged, which makes adapting this technology that much easier.
The Html5 Progress Event:
As per the Html5 Progress Events spec, the Html5 progress event provides, among others, the following information :
total - Total bytes being transferred
loaded - Bytes uploaded thus far
lengthComputable - Specifies if the total size of the data/file being uploaded is known
Using the information above, it is fairly easy to provide the "Time remaining" information to the user.
Keep the user informed:
Information about the file that can be made available to the user:
File Name
File Size
Mime Type
A Progress bar with percent complete
The upload speed or upload bandwidth
The approximate time remaining
The bytes uploaded thus far
A response from the server side
File Upload using XMLHttpRequest Demo
Please check "Uploading files using Html5 with Progress indication demo" for an example. All of the JavaScript code required is in the page but no CSS is included. For security reasons, the file types are limited to jpg, png, gif and txt. Max file size is 2MB.
XMLHttpRequest Browser Compatibility:
Probably the Javascript's file API is the best way in modern browsers:
http://robertnyman.com/2010/12/16/utilizing-the-html5-file-api-to-choose-upload-preview-and-see-progress-for-multiple-files/
http://www.sitepoint.com/html5-javascript-file-upload-progress-bar/
Server side wise... I think any of the main frameworks has the HTTP file POST feature well covered.
Files can be uploaded via AJAX.
Use the jQuery form plugin. It does all the dirty work of binding the files to the form and serializing it. It is also capable of showing upload progress.
Server stack hasn't got much to do with it.
Demo
I personally like blueimp jQuery File Upload Plugin (https://blueimp.github.io/jQuery-File-Upload/)
File Upload widget with multiple file selection, drag&drop support,
progress bars, validation and preview images, audio and video for
jQuery. Supports cross-domain, chunked and resumable file uploads and
client-side image resizing. Works with any server-side platform (PHP,
Python, Ruby on Rails, Java, Node.js, Go etc.) that supports standard
HTML form file uploads.
Demos:
Basic
Basic Plus
Basic Plus UI
AngularJS
Download (GitHub):
https://github.com/blueimp/jQuery-File-Upload
Related
I need to get the binary data from input type file using JavaScript. In IE10+ there is no problem because of FileReader, but in IE9 I do not know how to figure out. I saw some post that recomends to use Flash or ActiveX but ActiveX will require extra changes in security settings in IE and Flash to install Shockwave. Is there no any other options? Really?
It is not exists any other options ? Really ?
Really (not client-side). That's why we have the File API (which is implemented in IE10+).
Well, okay, you could use a Java applet, but they have the same sorts of issues as ActiveX and Flash (and won't be supported in Chrome much longer [already aren't on *nix]).
To support IE9 you'll basically have to post the file to a server and handle processing it there.
If it's your server, you could put a base64 formatted file on it, and send it as text/plain. Then request that client-side, and easily convert it to binary.Here's a ready-made demo, from some IE8 issue of the past.It is xhr'ing "smiley.dataURL.txt".https://googledrive.com/host/0B8BLd2qPPV7XS0o2V0JQSkRFM2s/xhr-onload-test.html?dataURL
I have a program where the user does some actions (i.e. clicking on several buttons). I want to record their clicks and the buttons that they click to allow the user to then download a text file with a record of their clicks when they click a separate "download" button. I looked at the File-system APIs for HTML 5, but they seemed to not have cross-browser support. I would ideally like to have this entire file generation and download scheme be entirely client-side, but I am open to server-side ideas as well.
TL;DR: Essentially I'm looking for an equivalent to Java's FileWriter, FileReader, ObjectOutputStream, and ObjectInputStream within Vanilla JS or jQuery (would like to stay away from php, but I'll use it as a last option).
Also, why don't all browsers support the filesystem api? (I'm guessing that it would make MSWord and Pages go out of business with all the open source clientside text editors that could come out.)
Unfortunately the HTML5-File-system is no longer a part of the spec, long story short FF refused to implement because they claimed everything you could do in the File-System API was doable in the HTML5 Indexeddb (which was mostly true). Please see this blog post for more on why FF didn't implement. I do not know IE's story. (I may have exagerated why FireFox didn't implement, I'm still bummed because you cannot actually do everything in indexeddb that you can do in the noew "Chrome File-system API")
Typically if two of those three browsers implement a spec, it stays in the spec. Otherwise that spec gets orphaned. However, I'm fairly certain a large reason the file-system api didn't take off is because of the IndexedDB API (caniuse IndexedDB) really took off when both specs were introduced. If you want cross browser support, check this api out.
That all said if you are still set on the file-system api some developers wrote a nice wrapper around the IndexedDB, the File-system api wouldn't actually supply you with a stream anyway. You would have to keep appending events to a given file given a fileWriter object. you'd then have to read the entire file and send to the server via an ajax request and then downloaded from the server once successfully uploaded.
The better route would be to use the IndexedDB apiwhich as stated on developer.mozilla
Open a database.
Create an object store in upgrading database.
Start a transaction and make a request to do some database operation, like adding or retrieving data.
Wait for the operation to complete by listening to the right kind of DOM event.
Do something
with the results (which can be found on the request object).
Here are a couple tutorials on the IndexedDB.
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB
http://www.html5rocks.com/en/tutorials/indexeddb/todo/
As for giving the user that file, as mentioned briefly before you would have to upload the file to the server and download upon the "download" request. Unfortunately you have to trick the user into giving them the data already on their machine. Anyway, hope this all helps.
Simperium looks like an awesome way to sync data across a variety of platforms and to deal with on/offline access from mobile.
For a project I'm working on some of the data is in the form of generated image and video files. I can't find any information about whether it is possible to sync this kind of data through Simperium (I guess I could base64 encode the images but it seems like a hack).
Or would I need to sync the URLs and then manually download these resources and somehow store them locally?
Simperium has basic support for binary files on the iOS side, currently in testing. This isn't yet available in the JavaScript library, but it will be. The way it works is similar to what you described. Simperium can handle the syncing of both a URL and its associated binary content in cases where that makes sense.
On iOS, binary files are stored to the local file system (though small files can indeed be stored as base64 encoded strings if you prefer).
In JavaScript, if you're working on the client side, the situation is less clear given the storage limits imposed by browsers, but you always have the option of syncing and using standard links depending on what you're trying to do. On the server side, there are of course more options. If you have some uses cases to share, you should get in touch.
I have an application that displays a large set of data using Slickgrid. The dataset may be 30-50MB in size. I would like users to be able to download the current view of the data displayed. What is the best way to set this up?
I have considered the approach described here using data URIs, but the maximum size of a URI is much too small.
I have also considered the approach described here where the client posts arbitrary data to the server, which the server echos back as a download. I worry that the documents may exceed the maximum POST size.
Since you want to do this on client side and If HTML5 is an option why not HTML5 File System API
One option is to use the HTML5 FileWriter API. As of today it's only supported in Chrome (and the BlackBerry browser of all things).
I am trying to process an XML file using Javascript.
xhttp.open("GET","exportproject.xml",false);
What I want to do is, let the user specify the file (instead of hard-coding it to exportproject.xml) using file uploader and then process the same using Javascript instead of sending it to the server.
Is it possible?
You may want to take a look at the HTML5 FileReader API - http://www.html5rocks.com/en/tutorials/file/dndfiles/
If you don't mind a solution that requires a modern browser (basically, ie 9+) you can use the html5 file API with a basic <input type="file">.
Take a look at this link, there are a number of excellent examples to get you started.
Javascript cannot read a file from the client machine (where the browser runs). That would be a security violation. You will have to submit the file to server and process it.