Develop Reference

Javascript Read Excel file on server with SheetJS

I am using a library called SheetJS and I want to read an excel sheet that resides on the server without using nodejs, only pure javascript. Is this possible?
There is a message in the documentation that says "readFile is only available in server environments. Browsers have no API for reading arbitrary files given a path, so another strategy must be used"
With the above message, I assume the author is referring to a situation where the file is residing on the client side.
This is what I have done so far
var wb = XLSX.readFile("myFile.xlsx"); //my file is in same directory on server
I get error "xlsx.full.min.js:22 Uncaught TypeError: Cannot read property 'readFileSync' of undefined"

This worked for me
/* set up async GET request */
var req = new XMLHttpRequest();
req.open("GET", url, true);
req.responseType = "arraybuffer";
req.onload = function(e) {
var data = new Uint8Array(req.response);
var workbook = XLSX.read(data, {type:"array"});
/* DO SOMETHING WITH workbook HERE */
}
req.send();

I had many issues with reading the file server-side, with a number of errors including type error, charCodeAt. So this provides a client and server-side solution using a reader. The excel file comes from a file upload button, and uses node.js.
Client-side:
let excelInput = document.getElementById("fileToUpload");
let excelFile = excelInput.files[0];
let reader = new FileReader();
So you get the file using files[0] from that element and create a fileReader.
You can see Aymkdn's solution on Github. https://github.com/SheetJS/sheetjs/issues/532. It uses the Uint8Array to work.
reader.readAsArrayBuffer(excelFile);
reader.onload = function() {
excelArray = new Uint8Array(reader.result); //returns Uint8Array using the result of reader
let binary = "";
var length = excelArray.byteLength;
for (var i = 0; i < length; i++) {
binary += String.fromCharCode(excelArray[i]);
//uses a for loop to alter excelArray to binary
}
let formData = new FormData(); //create form data
formData.append("excel", binary); //append binary to it
fetch('/excel', {method: "POST", body: formData}) //post as normal
.then((data) => {
console.log('Success:', data);
})
.catch((error) => {
console.error('Error:', error);
});
}
Server-side:
app.post('/excel', function(req, res) {
let data = req.body;
var workbook = sheetJS.read(data, {type: 'buffer'});
console.log("workbook is", workbook);
res.send();
}

No file was sent to SAP Leonardo Image Feature Extraction API using FormData

I'm testing SAP Leonardo Image Feature Extraction API (https://sandbox.api.sap.com/ml/featureextraction/inference_sync). I have the base64 string of the image and I want to transform it to a file object and zip it, then to send the zipped image file to this API using XMLHttpRequest. But the response text is "Service requires a list of (zipped) images".
I attach my HTTP request header and parameters in below screenshots.
Although we see a messy code in parameters, the zipped file Download here is created successfully.
If you cannot download the zipped file, please refer to the screenshot below.
Everything seems to be fine. However, the response text is as below with status 400.
My javascript code is shown below. What is wrong? It drives me crazy...
dataURItoBlob: function(dataURI, fileName) {
//convert base64/URLEncoded data component to raw binary data held in a string
var byteString;
if (dataURI.split(',')[0].indexOf('base64') >= 0)
byteString = atob(dataURI.split(',')[1]);
else
byteString = unescape(dataURI.split(',')[1]);
//separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0];
//write the bytes of the string to a typed array
var ia = new Uint8Array(byteString.length);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
var blob = new Blob([ia], {encoding:"UTF-8",type:mimeString});
//A Blob() is almost a File() - it's just missing the two properties below which we will add
blob.lastModifiedDate = new Date();
blob.name = fileName + '.' + mimeString.split('/')[1];
return blob;
},
onSubmit: function(oEvent) {
var oImage = this.getView().byId('myImage');
//oImage.getSrc() : 'data:image/png;base64,iVBORw0KGgo...'
var imageFile = this.dataURItoBlob(oImage.getSrc(), 'myImage');
var zip = new JSZip();
zip.file(imageFile.name, imageFile);
zip.generateAsync({
type:"blob",
compression: 'DEFLATE', // force a compression for this file
compressionOptions: {
level: 6,
},
}).then(function(content) {
//saveAs(content, "hello.zip");
// start the busy indicator
var oBusyIndicator = new sap.m.BusyDialog();
oBusyIndicator.open();
var formData = new FormData();
formData.append('files', content, 'myImage.zip');
var xhr = new XMLHttpRequest();
xhr.withCredentials = false;
xhr.addEventListener("readystatechange", function () {
if (this.readyState === this.DONE) {
oBusyIndicator.close();
//navigator.notification.alert(this.responseText);
console.log(this.responseText);
}
});
//setting request method
//API endpoint for API sandbox
//Destionation '/SANDBOX_API' in HCP is configured as 'https://sandbox.api.sap.com'
var api = "/SANDBOX_API/ml/featureextraction/inference_sync";
xhr.open("POST", api);
//adding request headers
xhr.setRequestHeader("Content-Type", "multipart/form-data");
xhr.setRequestHeader("Accept", "application/json");
//API Key for API Sandbox
xhr.setRequestHeader("APIKey", "yQd5Oy785NkAIob6g1eNwctBg4m1LGQS");
//sending request
xhr.send(formData);
});
},

I fix this issue by myself. I put my solution just for others' information. It's very easy and only below code needs to be removed before sending request. I have no idea why. Please suggest if you know the reason. Thanks in advance!
xhr.setRequestHeader("Content-Type", "multipart/form-data");
Best Regards,
Shelwin Wei

Is there a way to convert a file to byte in angular js?

I am trying to upload a file (bytes) to server. is there a way to convert this from ui side, i am using angular js?
Api i am using is done with ASP.new and the my sql table has fields:
FileName (string) and File (Collection of Byte).
Here is what i have tried so far
$scope.upload = function(){
var file = $scope.myFile;
var fileName=file.name;
Upload.uploadFile((file), uploadUrl,fileName);
};
App.service('fileUpload', ['$http', function ($http) {
this.uploadFile = function(File, uploadUrl,fileName){
var fd = new FormData();
console.log(File); //returns {}
fd.append('File', new Uint8Array(File));
var data ={
FileName : fileName,
};
fd.append("FileName", JSON.stringify(data.FileName));
$http.post(uploadUrl, fd, {
transformRequest: angular.identity,
headers: {'Content-Type': 'application/json'}
})
}
}]);

Not sure if this will suit your needs, but I'm assuming you just need a file uploaded using Angular. I ran into problems doing this myself until I found a little directive called ng-file-upload. Pretty simple to use and include and even provides fallbacks for older browsers.
Hope that helps :)

I agree, ng-file-upload is great. I'm also similarly posting my file to a database with base64 string as my "file_content" of my json object. With the resulting FileList* object from ng-file-upload (or a normal <input type="file"/>), I use the FileReader* object to read the file as a DataUrl (FileReader has a few other readAs methods too).
Note: I also use LoDash* (_.last and _.split).
uploadFile() {
var reader = new FileReader();
reader.onload = (event) => {
var data = {
file_id: 1,
//strip off the first part ("data:[mime/type];base64,") and just save base64 string
file_content: _.last(_.split(event.target.result, ','));,
file_name: this.file.name,
mime_type: this.file.type
}
/*... POST ...*/
};
//this.file is my model for ng-file-upload
reader.readAsDataURL(this.file);
}
http://jsfiddle.net/eliseosoto/JHQnk/
Then to download the file later, I use FileSaver.js* saveAs:
downloadFile(fileFromDB) {
var convertDataURIToBinary = () => {
var raw = window.atob(fileFromDB.file_content);
var rawLength = raw.length;
var array = new Uint8Array(new ArrayBuffer(rawLength));
for (var i = 0; i < rawLength; i++) {
array[i] = raw.charCodeAt(i);
}
return array;
}
var bin_array = convertDataURIToBinary();
saveAs(new Blob([bin_array], { type: fileFromDB.mime_type }), fileFromDB.file_name);
}
*sorry, my reputation isn't high enough to post links to all these, but you could google them and it should be the first result

JS - How to compute MD5 on binary data

EDIT: changed title from "JS File API - write and read UTF-8 data is inconsistent" to reflect the actual question.
I have some binary content i need to calculate the MD5 of. The content is a WARC file, that means that it holds text as well as encoded images. To avoid errors in the file saving, I convert and store all the data in arrayBuffers. All the data is put in UInt8Arrays to convert it to UTF-8.
My first attempt, for testing, is to use the saveAs library to save files from Chrome extensions. This means I was using a blob object to be passed on to the method and create the file.
var b = new Blob(arrayBuffers, {type: "text/plain;charset=utf-8"});
saveAs(b,'name.warc');
I haven't found a tool to compute the MD5 from a Blob object so what I was doing was using a FileReader to read the blob file as binary data and then use an MD5 tool (I used cryptoJS as well as a tool from faultylabs) to compute the result.
f = new FileReader();
f.readAsBinaryString(b);
f.onloadend = function(a){
console.log( 'Original file checksum: ', faultylabs.MD5(this.result) );
}
The resources (images) are downloaded directly in arraybuffer format so I have no need to convert them.
The result was wrong, meaning that checking the MD5 from the code and checking it from the file I saved on my local machine gave 2 different results. Reading as text, obviously shoots out an error.
The workaround I found, consists in writing the blob object on the disk using the filesystem API and then read it back as binary data, compute the MD5 and then save that retrieved file as WARC file (not directly the blob object but this "refreshed" version of the file).
In this case the computed MD5 is fine ( I calculate it on the "refreshed" version of the warc file) but when I launch the WARC replay instance with the "refreshed" warc archive, it throws me errors - while with the original file I don't have any problem (but the MD5 is not correct).
var fd = new FormData();
// To compute the md5 hash and to have it correct on the server side, we need to write the file to the system, read it back and then calculate the md5 value.
// We need to send this version of the warc file to the server as well.
window.requestFileSystem = window.requestFileSystem || window.webkitRequestFileSystem;
function computeWARC_MD5(callback,formData) {
window.requestFileSystem(window.TEMPORARY, b.size, onInitFs);
function onInitFs(fs) {
fs.root.getFile('warc.warc', {create: true}, function(fileEntry) {
fileEntry.createWriter(function(fileWriter) {
fileWriter.onwriteend = function(e) {
readAndMD5();
};
fileWriter.onerror = function(e) {
console.error('Write failed: ' + e.toString());
};
fileWriter.write(b);
});
});
function readAndMD5() {
fs.root.getFile('warc.warc', {}, function(fileEntry) {
fileEntry.file( function(file) {
var reader = new FileReader();
reader.onloadend = function(e) {
var warcMD5 = faultylabs.MD5( this.result );
console.log(warcMD5);
var g = new Blob([this.result],{type: "text/plain;charset=utf-8"});
saveAs(g, o_request.file);
formData.append('warc_file', g)
formData.append('warc_checksum_md5', warcMD5.toLowerCase());
callback(formData);
};
reader.readAsBinaryString(file);
});
});
}
}
}
function uploadData(formData) {
// upload
$.ajax({
type: 'POST',
url: server_URL_upload,
data: fd,
processData: false,
contentType: false,
// [SPECS] fire a progress event named progress at the XMLHttpRequestUpload object about every 50ms or for every byte transmitted, whichever is least frequent
xhrFields: {
onprogress: function (e) {
if (e.lengthComputable) {
console.log(e.loaded / e.total * 100 + '%');
}
}
}
}).done(function(data) {
console.log('done uploading!');
//displayMessage(port_to_page, 'Upload finished!', 'normal')
//port_to_page.postMessage( { method:"doneUpload" } );
});
}
computeWARC_MD5(uploadData, fd);
saveAs(b, 'warc.warc');
Could anybody explain me why there is this discrepancy? What am I missing in treating all the objects I am dealing with as binary data (store, read)?

Basically I tried another route and converted the blob file back to arraybuffer and computed the MD5 on that. At that point, the file's MD5 and the arraybuffer's are the same.
var b = new Blob(arrayBuffers, {type: "text/plain;charset=utf-8"});
var blobHtml = new Blob( [str2ab(o_request.main_page_html)], {type: "text/plain;charset=utf-8"} );
f = new FileReader();
f.readAsArrayBuffer(b);
f.onloadend = function(a){
var warcMD5 = faultylabs.MD5(this.result);
var fd = new FormData();
fd.append('warc_file', b)
fd.append('warc_checksum_md5', warcMD5.toLowerCase());
uploadData(fd);
}
I guess the result from a binary string and from a buffer array is different, that's why also the MD5 is inconsistent.

Ember - HandleBar does not display blob image

I am getting image from the server in Base64 encoding.
My Base64 response is correct.
On the client side I am using following :
var blob = base64ToBlob(content, {type: 'image/jpeg'});
URL = window.URL || window.webkitURL;
var blobUrl = URL.createObjectURL(blob);
//Displaying image in div works.
var elImage = $("#dbimage");
elImage.append("<img src='data:image/jpeg;base64," + content+ " '/>");
///This doesn;t work.
return blobUrl;
var base64ToBlob = function(base64) {
var binary = atob(base64);
var len = binary.length;
var buffer = new ArrayBuffer(len);
var view = new Uint8Array(buffer);
for (var i = 0; i < len; i++) {
view[i] = binary.charCodeAt(i);
}
return new Blob([view], {type: 'image/jpeg'});
};
I am returning the blobURL to HandleBar template. It does not seem to recognize the blob.
This conversion is encapsulated in a promise. In the HandleBar template I am writing something like this :
<img {{bind-attr src=image.content}} />
This promise is present in the image attribute. The image displayed in the div works fine, but the one in the blob does not display.
Edit :
I am returning the promise, like this :
image : function() {
var promise = jQuery.ajax({
url: '/getattachments/?id=' + this.id + "&name=" + docname,
contentType: 'image/jpeg',
type: 'GET',
processData : false,
success: function(content) {
var blob = base64ToBlob(content, { type: 'image/jpeg'} );
URL = window.URL || window.webkitURL;
var blobUrl = URL.createObjectURL(blob);
return blobUrl;
},
error : function(content)
{
Em.Logger.info('Model:', this.id, 'has no image', err);
return '';
}
return DS.PromiseObject.create({ promise: promise });
}.property('_attachments'),
});
Pl help.
Thanks.

In the particular answer you provided there are a couple of little issues.
The property needs to be a computed property, not just a function
foo: function(){
}.property() // <-------this here
Additionally you were using part of Ember Data (the DS namespace) which isn't part of Ember, it is a separate product. This is fine, but you'll need to include the file if you're going to use it.
And since the promise is returning straight text, and not an object, you'll need to hook up to the content property of the promise object.
<img {{bind-attr src=img.content}}/>
http://emberjs.jsbin.com/qiyodojo/8/edit