In one of my views, I have a file upload control. It supports file uploading either via drag and drop, or via the standard file dialog opened after a button click.
How to do this in my e2e tests1?
1 Just one of the two options will be enough
You can upload files using Javascript blobs. This requires the FileApi, which isn't compatible with older browsers (http://caniuse.com/fileapi). But since you mentioned using drag and drop uploads, which uses the FileApi, it shouldn't matter too much.
There are two ways you can upload files using the blob API. One is very easy and the other is simply a continuation of the first.
Using Javascript, you can create a new blob with:
var blob = new Blob("content", contentType);
For example, this will create a blob object that contains the text "Hello World!".
var foo = new Blob("Hello World!", {type: "text/plain"});
You could also use the following method is better for non-plaintext files, such as pdf's. You have to convert the file to Base64 (you can use something like this) and create the blob using the Base64 data.
Use this function (a slightly modified version of this) to create the blob.
function b64toBlob(b64Data, contentType, sliceSize) {
b64Data = b64Data.replace(/\s/g, '');
contentType = contentType || '';
sliceSize = sliceSize || 1024;
function charCodeFromCharacter(c) {
return c.charCodeAt(0);
}
var byteCharacters = atob(b64Data);
var byteArrays = [];
for (var offset = 0; offset < byteCharacters.length; offset += sliceSize) {
var slice = byteCharacters.slice(offset, offset + sliceSize);
var byteNumbers = Array.prototype.map.call(slice, charCodeFromCharacter);
var byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
var blob = new Blob(byteArrays, {type: contentType});
return blob;
}
For example, this will create a PDF blob object.
var pdf = "JVBERi0xLjQKJcfsj6IKNSAwIG9...=="; //base64 encoded file as a String
var pdfBlob = b64toBlob(pdf, "application/pdf", 1024);
After you create the blob with one of the methods above, it can be treated as a file. For example, you could put the file into a FormData object (if you're doing uploads like this):
var fd = new FormData();
fd.append("uploadedFile", pdfBlob, "My PDF.pdf"*);
*Filename parameter only seems to work on Chrome as of now.
Related
I'm looking for a way to split up any text/data file on the front end in the browser before being uploaded as multiple files. My limit is 40KB per upload. So if a user uploads a 400KB file, it would split this file into 10 separate chunks or 10 separate files on the front end before uploading it to the server.
Currently, I'm doing it by converting this file into a base64 formatted string, then split this string by 40KB which comes out to 10 separate chunks. From there I upload each chunk as with a filename of chunk-1-of-10, chunk-2-of-10...
When pulling down these files, I just concat all these chunks back and deconvert it from base64 into its file format.
Is there a better way of doing this? Is there a library that handles all of this instead of writing it from scratch? I'm not sure if the base64 route is the best way to do this.
There is no need for reading the content into ram with the FileReader
using base64 will only increase the size of what you need to upload, base64 takes up ~33% more in size
Use Blob.slice to get chunks
blob slices (chunks) will not increase the memory, it will just create a new reference to the old blob with a changed offset and a new size to where it should start reading from.
when fetch sends the data it will be piped directly from the disk to the network without even touching the main thread.
// simulate a file from a input
const file = new File(['a'.repeat(1000000)], 'test.txt')
const chunkSize = 40000
const url = 'https://httpbin.org/post'
for (let start = 0; start < file.size; start += chunkSize) {
const chunk = file.slice(start, start + chunkSize + 1)
const fd = new FormData()
fd.set('data', chunk)
await fetch(url, { method: 'post', body: fd }).then(res => res.text())
}
You could avoid having to base64 encode by using a FileReader and then sending as binary:
const url = 'http://www.example.com/upload';
document.getElementById('file-uploader').addEventListener('change', function(e) {
const size = 40000;
var reader = new FileReader();
var buf;
var file = document.getElementById('file-uploader').files[0];
reader.onload = function(e) {
buf = new Uint8Array(e.target.result);
for (var i = 0; i < buf.length; i += size) {
var fd = new FormData();
fd.append('fname', [file.name, i+1, 'of', buf.length].join('-'));
fd.append('data', new Blob([buf.subarray(i, i + size)]));
var oReq = new XMLHttpRequest();
oReq.open("POST", url, true);
oReq.onload = function (oEvent) {
// Uploaded.
};
oReq.send(fd);
}
}
reader.readAsArrayBuffer(file);
});
<input type="file" id="file-uploader"/>
I'm looking for a way to split up any text/data file on the front end in the browser before being uploaded as multiple files. My limit is 40KB per upload. So if a user uploads a 400KB file, it would split this file into 10 separate chunks or 10 separate files on the front end before uploading it to the server.
Currently, I'm doing it by converting this file into a base64 formatted string, then split this string by 40KB which comes out to 10 separate chunks. From there I upload each chunk as with a filename of chunk-1-of-10, chunk-2-of-10...
When pulling down these files, I just concat all these chunks back and deconvert it from base64 into its file format.
Is there a better way of doing this? Is there a library that handles all of this instead of writing it from scratch? I'm not sure if the base64 route is the best way to do this.
There is no need for reading the content into ram with the FileReader
using base64 will only increase the size of what you need to upload, base64 takes up ~33% more in size
Use Blob.slice to get chunks
blob slices (chunks) will not increase the memory, it will just create a new reference to the old blob with a changed offset and a new size to where it should start reading from.
when fetch sends the data it will be piped directly from the disk to the network without even touching the main thread.
// simulate a file from a input
const file = new File(['a'.repeat(1000000)], 'test.txt')
const chunkSize = 40000
const url = 'https://httpbin.org/post'
for (let start = 0; start < file.size; start += chunkSize) {
const chunk = file.slice(start, start + chunkSize + 1)
const fd = new FormData()
fd.set('data', chunk)
await fetch(url, { method: 'post', body: fd }).then(res => res.text())
}
You could avoid having to base64 encode by using a FileReader and then sending as binary:
const url = 'http://www.example.com/upload';
document.getElementById('file-uploader').addEventListener('change', function(e) {
const size = 40000;
var reader = new FileReader();
var buf;
var file = document.getElementById('file-uploader').files[0];
reader.onload = function(e) {
buf = new Uint8Array(e.target.result);
for (var i = 0; i < buf.length; i += size) {
var fd = new FormData();
fd.append('fname', [file.name, i+1, 'of', buf.length].join('-'));
fd.append('data', new Blob([buf.subarray(i, i + size)]));
var oReq = new XMLHttpRequest();
oReq.open("POST", url, true);
oReq.onload = function (oEvent) {
// Uploaded.
};
oReq.send(fd);
}
}
reader.readAsArrayBuffer(file);
});
<input type="file" id="file-uploader"/>
I have an AJAX call which calls a secure PHP API call that generates PDF file data (on the fly, no actual file) and returns it. How can I use this PDF file data in javascript/jquery to create and download a PDF file to the user?
I've originally tried setting the ajax call's heads to that of a PDF, but obviously you can't download files via ajax.
To automatically download a file from ajax call you can do the following. In my example I returned an object that contained the file name, mime type, and Base64 data but maybe this can give you an idea on how to do what you are looking for.
PostDTO('/ReportService.asmx/DownloadReport', dto, function (result) {
var a = document.createElement('a');
if (window.URL && window.Blob && ('download' in a) && window.atob) {
// Do it the HTML5 compliant way
var blob = base64ToBlob(result.FileDataBase64, result.MimeType);
var url = window.URL.createObjectURL(blob);
a.href = url;
a.download = result.Filename;
a.click();
window.URL.revokeObjectURL(url);
}
});
function base64ToBlob(base64, mimetype, slicesize) {
if (!window.atob || !window.Uint8Array) {
// The current browser doesn't have the atob function. Cannot continue
return null;
}
mimetype = mimetype || '';
slicesize = slicesize || 512;
var bytechars = atob(base64);
var bytearrays = [];
for (var offset = 0; offset < bytechars.length; offset += slicesize) {
var slice = bytechars.slice(offset, offset + slicesize);
var bytenums = new Array(slice.length);
for (var i = 0; i < slice.length; i++) {
bytenums[i] = slice.charCodeAt(i);
}
var bytearray = new Uint8Array(bytenums);
bytearrays[bytearrays.length] = bytearray;
}
return new Blob(bytearrays, { type: mimetype });
};
You can download file from ajax.
I did it from an angular application.
The idea :
Get the file data and ask a response type "arrayBuffer"
Be sure the data response are not transformed by your framework (as it is with angular)
Create a blob : var blob = new Blob([data], {type: 'application/pdf');//check type
Create an url : var url = URL.createObjectURL(blob);
Open the url : window.open(url);
You can verify the blob generated via the console, the blob url is showed. That url must open your pdf.
I'm trying to create a web application that can be used via a file:// URI. This means that I can't use AJAX to load binary files (without turning off security features in the browser, which I don't want to do as a matter of principle).
The application uses a SQLite database. I want to provide the database to a sql.js constructor, which requires it in Uint8Array format.
Since I can't use AJAX to load the database file, I could instead load it with <input type="file"> and FileReader.prototype.readAsArrayBuffer and convert the ArrayBuffer to a Uint8Array. And that's working with the following code:
input.addEventListener('change', function (changeEvent) {
var file = changeEvent.currentTarget.files[0];
var reader = new FileReader();
reader.addEventListener('load', function (loadEvent) {
var buffer = loadEvent.target.result;
var uint8Array = new Uint8Array(buffer);
var db = new sql.Database(uint8Array);
});
reader.readAsArrayBuffer(file);
});
However, <input type="file"> requires user interaction, which is tedious.
I thought I might be able to work around the no-AJAX limitation by using a build tool to convert my database file to a JavaScript object / string and generate a ".js" file providing the file contents, and then convert the file contents to a Uint8Array, somehow.
Psuedocode:
// In Node.js:
var fs = require('fs');
var sqliteDb = fs.readFileSync('path/to/sqlite.db');
var string = convertBufferToJsStringSomehow(sqliteDb);
fs.writeFileSync('build/db.js', 'var dbString = "' + string + '";');
// In the browser (assume "build/db.js" has been loaded via a <script> tag):
var uint8Array = convertStringToUint8ArraySomehow(dbString);
var db = new sql.Database(uint8Array);
In Node.js, I've tried the following:
var TextEncoder = require('text-encoding').TextEncoder;
var TextDecoder = require('text-encoding').TextEncoder;
var sql = require('sql.js');
var string = new TextDecoder('utf-8').decode(fs.readFileSync('path/to/sqlite.db'));
// At this point, I would write `string` to a ".js" file, but for
// the sake of determining if converting back to a Uint8Array
// would work, I'll continue in Node.js...
var uint8array = new TextEncoder().encode(string);
var db = new sql.Database(uint8array);
db.exec('SELECT * FROM tablename');
But when I do that, I get the error "Error: database disk image is malformed".
What am I doing wrong? Is this even possible? The database disk image isn't "malformed" when I load the same file via FileReader.
Using the following code, I was able to transfer the database file's contents to the browser:
// In Node.js:
var fs = require('fs');
var base64 = fs.readFileSync('path/to/sqlite.db', 'base64');
fs.writeFileSync('build/db.js', 'var dbString = "' + base64 + '";');
// In the browser (assume "build/db.js" has been loaded via a <script> tag):
function base64ToUint8Array (string) {
var raw = atob(string);
var rawLength = raw.length;
var array = new Uint8Array(new ArrayBuffer(rawLength));
for (var i = 0; i < rawLength; i += 1) {
array[i] = raw.charCodeAt(i);
}
return array;
}
var db = new sql.Database(base64ToUint8Array(dbString));
console.log(db.exec('SELECT * FROM tablename'));
And that's working with the following code:
input.addEventListener('change', function (changeEvent) {
var file = changeEvent.currentTarget.files[0];
var reader = new FileReader();
reader.addEventListener('load', function (loadEvent) {
var buffer = loadEvent.target.result;
var uint8Array = new Uint8Array(buffer);
var db = new sql.Database(uint8Array);
});
reader.readAsArrayBuffer(file);
});
However, <input type="file"> requires user interaction, which is
tedious.
Using current working approach would be less tedious than attempting to create workarounds. If user intends to use application, user can select file from their filesystem to run application.
Before somebody says, "duplicate", I just want to make sure, that folks know, that I have already reviewed these questions:
1) Uses angular and php, not sure what is happening here (I don't know PHP): Download zip file and trigger "save file" dialog from angular method
2) Can't get this answer to do anything: how to download a zip file using angular
3) This person can already download, which is past the point I'm trying to figure out:
Download external zip file from angular triggered on a button action
4) No answer for this one:
download .zip file from server in nodejs
5) I don't know what language this even is:
https://stackoverflow.com/questions/35596764/zip-file-download-using-angularjs-directive
Given those questions, if this is still a duplicate, I apologize. Here is, yet, another version of this question.
My angular 1.5.X client gives me a list of titles, of which each have an associated file. My Node 4.X/Express 4.X server takes that list, gets the file locations, creates a zip file, using express-zip from npm, and then streams that file back in the response. I then want my client to initiate the browser's "download a file" option.
Here's my client code (Angular 1.5.X):
function bulkdownload(titles){
titles = titles || [];
if ( titles.length > 0 ) {
$http.get('/query/bulkdownload',{
params:{titles:titles},
responseType:'arraybuffer'
})
.then(successCb,errorCb)
.catch(exceptionCb);
}
function successCb(response){
// This is the part I believe I cannot get to work, my code snippet is below
};
function errorCb(error){
alert('Error: ' + JSON.stringify(error));
};
function exceptionCb(ex){
alert('Exception: ' + JSON.stringify(ex));
};
};
Node (4.X) code with express-zip, https://www.npmjs.com/package/express-zip:
router.get('/bulkdownload',function(req,resp){
var titles = req.query.titles || [];
if ( titles.length > 0 ){
utils.getFileLocations(titles).
then(function(files){
let filename = 'zipfile.zip';
// .zip sets Content-Type and Content-disposition
resp.zip(files,filename,console.log);
},
_errorCb)
}
});
Here's my successCb in my client code (Angular 1.5.X):
function successCb(response){
var URL = $window.URL || $window.webkitURL || $window.mozURL || $window.msURL;
if ( URL ) {
var blob = new Blob([response.data],{type:'application/zip'});
var url = URL.createObjectURL(blob);
$window.open(url);
}
};
The "blob" part seems to work fine. Checking it in IE's debugger, it does look like a file stream of octet information. Now, I believe I need to get that blob into the some HTML5 directive, to initiate the "Save File As" from the browser. Maybe? Maybe not?
Since 90%+ of our users are using IE11, I test all of my angular in PhantomJS (Karma) and IE. When I run the code, I get the old "Access is denied" error in an alert window:
Exception: {"description":"Access is denied...<stack trace>}
Suggestions, clarifications, answers, etc. are welcome!
Use this one:
var url="YOUR ZIP URL HERE";
window.open(url, '_blank');
var zip_file_path = "" //put inside "" your path with file.zip
var zip_file_name = "" //put inside "" file name or something
var a = document.createElement("a");
document.body.appendChild(a);
a.style = "display: none";
a.href = zip_file_path;
a.download = zip_file_name;
a.click();
document.body.removeChild(a);
As indicated in this answer, I have used the below Javascript function and now I am able to download the byte[] array content successfully.
Function to convert byte array stream (type of string) to blob object:
var b64toBlob = function(b64Data, contentType, sliceSize) {
contentType = contentType || '';
sliceSize = sliceSize || 512;
var byteCharacters = atob(b64Data);
var byteArrays = [];
for (var offset = 0; offset < byteCharacters.length; offset += sliceSize) {
var slice = byteCharacters.slice(offset, offset + sliceSize);
var byteNumbers = new Array(slice.length);
for (var i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
var byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
var blob = new Blob(byteArrays, {type: contentType});
return blob;
};
An this is how I call this function and save the blob object with FileSaver.js (getting data via Angular.js $http.get):
$http.get("your/api/uri").success(function(data, status, headers, config) {
//Here, data is type of string
var blob = b64toBlob(data, 'application/zip');
var fileName = "download.zip";
saveAs(blob, fileName);
});
Note: I am sending the byte[] array (Java-Server-Side) like this:
byte[] myByteArray = /*generate your zip file and convert into byte array*/ new byte[]();
return new ResponseEntity<byte[]>(myByteArray , headers, HttpStatus.OK);
I updated my bulkdownload method to use $window.open(...) instead of $http.get(...):
function bulkdownload(titles){
titles = titles || [];
if ( titles.length > 0 ) {
var url = '/query/bulkdownload?';
var len = titles.length;
for ( var ii = 0; ii < len; ii++ ) {
url = url + 'titles=' + titles[ii];
if ( ii < len-1 ) {
url = url + '&';
}
}
$window.open(url);
}
};
I have only tested this in IE11.