Develop Reference

How to read and write from a file in Javascript

There are many similar questions to this, however, when I used the code provided, it didn't work. My code is as follows:
function write(fs) {
fs.root.getFile('archive.txt', {create: true}, function(fileEntry) {
// Create a FileWriter object for our FileEntry (log.txt).
fileEntry.createWriter(function(fileWriter) {
fileWriter.onwriteend = function(e) {
console.log('Write completed.');
};
fileWriter.onerror = function(e) {
console.log('Write failed: ' + e.toString());
};
var blob = new Blob([prompt("MESSAGE: ")], {type: 'text/plain'});
fileWrite.write(blob);
}, errorHandler);
}, errorHandler);
}
function onInitFs(fs) {
fs.root.getFile('archive.txt', {}, function(fileEntry) {
// Get a File object representing the file,
// then use FileReader to read its contents.
fileEntry.file(function(file) {
var reader = new FileReader();
reader.onloadend = function(e) {
var txtArea = document.createElement('textarea');
txtArea.value = this.result;
document.body.appendChild(txtArea);
};
reader.readAsText(file);
}, errorHandler);
}, errorHandler);
}
window.requestFileSystem(window.TEMPORARY, 5*1024*1024 /*5MB*/, onInitFs, errorHandler);
The file archive.txt does exist but when I call the function, it doesn't work. So instead I used window.requestFileSystem() which I found on a website. However, when I compile this code through Github, it doesn't work.
Also, if someone could tell me a way to read and write to a file without using php as this is all in html file using Github without git. I have another file in Github in the same directory as this. Would I need to include the full directory rather than archive.txt?

It's not possible without server side code.
I've asked a similar question a while ago.

What's DataTransferItemList and what is DataTransferItemList.add doing?

So I'm trying to understand paste and copy API in Google Chrome. I don't understand either.
As of copy, you'll probably want to use javascript to add something in clipboard. I'm working with images (strings work well actually1):
//Get DataTransferItemList
var files = items.items;
if(files) {
console.log(files);
//Create blob from canvas
var blob = Blob.fromDataURL(_this.editor.selection.getSelectedImage().toDataURL("image/png"));
var file;
try {
//Try to create file from blob, which may fail
file = new File([blob], "image.png", {type:"image/png"});
}
catch(e) {
return false;
}
if(file) {
//I think this should clear previous data from clipboard
files.clear();
//Add a file as image/png
files.add(file, "image/png");
}
//console.log(files.add(file));
}
The problem is, that I don't really know how does that add method work. I found this "documentation" for DataTransferItemList which says:
add(any data, optional DOMString type)
What's any data? (how can anybody even write this in documentation?) While something is added to clipboard, I don't know what it is. I can't paste it anywhere - except Chrome. If I inspect paste event with my copied file, this is in DataTransferItemList:
It can be converted to File, but if I try to turn it back to <img>:
ImageEditorKeyboard.prototype.processFile = function(file) {
//File reader converts files to something else
var reader = new FileReader();
//Refference to this class
var _this = this;
//happens when the file is loaded
reader.onload = function(event) {
var img = new Image;
img.onload = function() {
_this.processImage(this);
};
img.src = event.target.result;
}; // data url!
//Read file
reader.readAsDataURL(file);
}
I get the error2:
If I log the value of event.target.result it turns out that the data was empty:
console.error("String '", event.target.result, "' ain't a valid URL!");
Q: What is the exact specification of DataTransferItemList, it's methods and properties, especially the .add method?
1: To add string to clipboard (during copy event of course!), call this: event.clipboardData.setData(data, "text/plain");. The second argument doesn't seem to have any functionality - using image/png will not do anything.
2: Funny thing is that this error can't be caught.

JS - How to compute MD5 on binary data

EDIT: changed title from "JS File API - write and read UTF-8 data is inconsistent" to reflect the actual question.
I have some binary content i need to calculate the MD5 of. The content is a WARC file, that means that it holds text as well as encoded images. To avoid errors in the file saving, I convert and store all the data in arrayBuffers. All the data is put in UInt8Arrays to convert it to UTF-8.
My first attempt, for testing, is to use the saveAs library to save files from Chrome extensions. This means I was using a blob object to be passed on to the method and create the file.
var b = new Blob(arrayBuffers, {type: "text/plain;charset=utf-8"});
saveAs(b,'name.warc');
I haven't found a tool to compute the MD5 from a Blob object so what I was doing was using a FileReader to read the blob file as binary data and then use an MD5 tool (I used cryptoJS as well as a tool from faultylabs) to compute the result.
f = new FileReader();
f.readAsBinaryString(b);
f.onloadend = function(a){
console.log( 'Original file checksum: ', faultylabs.MD5(this.result) );
}
The resources (images) are downloaded directly in arraybuffer format so I have no need to convert them.
The result was wrong, meaning that checking the MD5 from the code and checking it from the file I saved on my local machine gave 2 different results. Reading as text, obviously shoots out an error.
The workaround I found, consists in writing the blob object on the disk using the filesystem API and then read it back as binary data, compute the MD5 and then save that retrieved file as WARC file (not directly the blob object but this "refreshed" version of the file).
In this case the computed MD5 is fine ( I calculate it on the "refreshed" version of the warc file) but when I launch the WARC replay instance with the "refreshed" warc archive, it throws me errors - while with the original file I don't have any problem (but the MD5 is not correct).
var fd = new FormData();
// To compute the md5 hash and to have it correct on the server side, we need to write the file to the system, read it back and then calculate the md5 value.
// We need to send this version of the warc file to the server as well.
window.requestFileSystem = window.requestFileSystem || window.webkitRequestFileSystem;
function computeWARC_MD5(callback,formData) {
window.requestFileSystem(window.TEMPORARY, b.size, onInitFs);
function onInitFs(fs) {
fs.root.getFile('warc.warc', {create: true}, function(fileEntry) {
fileEntry.createWriter(function(fileWriter) {
fileWriter.onwriteend = function(e) {
readAndMD5();
};
fileWriter.onerror = function(e) {
console.error('Write failed: ' + e.toString());
};
fileWriter.write(b);
});
});
function readAndMD5() {
fs.root.getFile('warc.warc', {}, function(fileEntry) {
fileEntry.file( function(file) {
var reader = new FileReader();
reader.onloadend = function(e) {
var warcMD5 = faultylabs.MD5( this.result );
console.log(warcMD5);
var g = new Blob([this.result],{type: "text/plain;charset=utf-8"});
saveAs(g, o_request.file);
formData.append('warc_file', g)
formData.append('warc_checksum_md5', warcMD5.toLowerCase());
callback(formData);
};
reader.readAsBinaryString(file);
});
});
}
}
}
function uploadData(formData) {
// upload
$.ajax({
type: 'POST',
url: server_URL_upload,
data: fd,
processData: false,
contentType: false,
// [SPECS] fire a progress event named progress at the XMLHttpRequestUpload object about every 50ms or for every byte transmitted, whichever is least frequent
xhrFields: {
onprogress: function (e) {
if (e.lengthComputable) {
console.log(e.loaded / e.total * 100 + '%');
}
}
}
}).done(function(data) {
console.log('done uploading!');
//displayMessage(port_to_page, 'Upload finished!', 'normal')
//port_to_page.postMessage( { method:"doneUpload" } );
});
}
computeWARC_MD5(uploadData, fd);
saveAs(b, 'warc.warc');
Could anybody explain me why there is this discrepancy? What am I missing in treating all the objects I am dealing with as binary data (store, read)?

Basically I tried another route and converted the blob file back to arraybuffer and computed the MD5 on that. At that point, the file's MD5 and the arraybuffer's are the same.
var b = new Blob(arrayBuffers, {type: "text/plain;charset=utf-8"});
var blobHtml = new Blob( [str2ab(o_request.main_page_html)], {type: "text/plain;charset=utf-8"} );
f = new FileReader();
f.readAsArrayBuffer(b);
f.onloadend = function(a){
var warcMD5 = faultylabs.MD5(this.result);
var fd = new FormData();
fd.append('warc_file', b)
fd.append('warc_checksum_md5', warcMD5.toLowerCase());
uploadData(fd);
}
I guess the result from a binary string and from a buffer array is different, that's why also the MD5 is inconsistent.

Create a download link from a Blob URL

In a Google chrome extension I am working on, a file is downloaded from a server with an XMLHttpRequest. This file contains some binary data which are stored in an ArrayBuffer object. In order to provide the possibility to download this file I am using the createObjectURL API.
function publish(data) {
if (!window.BlobBuilder && window.WebKitBlobBuilder) {
window.BlobBuilder = window.WebKitBlobBuilder;
}
var builder = new BlobBuilder();
builder.append(data);
var blob = builder.getBlob();
var url = window.webkitURL.createObjectURL(blob);
$("#output").append($("<a/>").attr({href: url}).append("Download"));
}
It is working fine; except that the filename is an opaque UUID like 9a8f6a0f-dd0c-4715-85dc-7379db9ce142. Is there any way to force this filename to something more user-friendly?

you can force an arbitrary filename by setting the "download" attribute of your anchor
see: http://updates.html5rocks.com/2011/08/Downloading-resources-in-HTML5-a-download

I have never tried it before, but it should be possible to create a new File object (which allows you to specify a file name) and write your blob to it. Something along the lines of:
function publish(data, filename) {
if (!window.BlobBuilder && window.WebKitBlobBuilder) {
window.BlobBuilder = window.WebKitBlobBuilder;
}
fs.root.getFile(filename, {
create: true
}, function (fileEntry) {
// Create a FileWriter object for our FileEntry (log.txt).
fileEntry.createWriter(function (fileWriter) {
fileWriter.onwriteend = function (e) {
console.log('Write completed.');
};
fileWriter.onerror = function (e) {
console.log('Write failed: ' + e.toString());
};
var builder = new BlobBuilder();
builder.append(data);
var blob = builder.getBlob();
fileWriter.write(blob);
}, errorHandler);
}, errorHandler);
}
I think this could work for you.

If there is no way to read a binary file on client-side Javascript, how to parse a text file?

I'm going to put a text file on my ISP's server, e.g. http://home.ISP.net/~foobar/text.txt.
How can I read that with Javascript in the browser e.g. from http://home.ISP.net/~foobar/textreader.html?
I already know that I can't read a binary file that's on a web server from inside the browser.
Thanks.

Since you're gonna read it from the same domain, just use Ajax to load the file.
I think you could read a file in the client-side by just using a file input tag, and loading it with a FileReader object, though I don't know if it's supported on a browser other than Firefox.

You can read whatever file you want in the web server from inside the browser as long as it's in a "protected" path like "web-inf"...
Probably the easiest way to read the file is using the "load" from jQuery
http://api.jquery.com/load/
or the "ajax" one:
http://api.jquery.com/jQuery.ajax/

In my cross-platform app I had the need to get documents from a cloud-sharepoint and store it on the local filesystem via cordova. Here the relevant parts of code that is working for me;
1) Ajax call to get and download the file
$.ajax({
url: fileurl, // full qualified URL to document
dataType: 'text',
mimeType: 'text/plain; charset=x-user-defined',
async: false,
cache: false
}).done(function(data) {
var data = str2ab(data);
writeData(newfile, data);
});
2) Prepare the fetched binary data
function str2ab(str) {
var buf = new ArrayBuffer(str.length); // 2 bytes for each char
var bufView = new Uint8Array(buf);
for (var i=0, strLen=str.length; i<strLen; i++) {
bufView[i] = str.charCodeAt(i);
}
return buf;
}
3) Write data to filesystem using cordova plugin. Hint: it is assumed the filesystem "fs" has been initialized before.
function writeData(path, data)
{
var content, newContent = "";
var newfile = path ;
//
fs.root.getFile(newfile, {create: true, exclusive: false},
function(fileEntry)
{
fileEntry.createWriter(function(fileWriter) {
fileWriter.onwriteend = function(e) {
console.log('Write completed: ' + fileEntry.fullPath);
};
fileWriter.onerror = function(e) {
console.log('Write failed: ' + e.toString());
};
fileWriter.write(data);
}),
function() {
alert("ERROR WRITE FILE");
},
function() {
alert("ERROR");
}
}
);
};

Your HTML document can use an XMLHttpRequest to fetch the text document.
Note that the file could be a binary file (e.g. an image) although parsing it would require some JavaScript trickery (e.g. by treating it as a string of bytes and processing it with string functions like substr).