How to read a file with JavaScript to WebAssembly? - javascript

How can I pass a File to be read within the WebAssembly memory context?
Reading a file in the browser with JavaScript is easy:
<input class="file-selector" type="file" id="files" name="files[]" />
I was able to bootstrap WebAssembly code written in Rust with the crate stdweb, add an event listener to the DOM element and fire up a FileReader:
let reader = FileReader::new();
let file_input_element: InputElement = document().query_selector(".file-selector").unwrap().unwrap().try_into().unwrap();
file_input_element.add_event_listener(enclose!( (reader, file_input_element) move |event: InputEvent| {
// mystery part
}));
In JavaScript, I would get the file from the element and pass it to the reader, however, the API of stdweb needs the following signature:
pub fn read_as_array_buffer<T: IBlob>(&self, blob: &T) -> Result<(), TODO>
I have no idea how to implement IBlob and I am sure that I am missing something obvious either with the stdweb API or in my understanding of WebAssembly/Rust. I was hoping that there is something less verbose than converting stuff to UTF-8.

It works when the FileReader itself is passed from JavaScript to WebAssembly. It also seems like a clean approach because the data has to be read by the JavaScript API anyway - no need to call JS from WASM.
index.html
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Read to wasm</title>
</head>
<body>
<input type="file" id="file-input"/>
<script src="reader.js"></script>
<script>
var fileReader = new FileReader();
fileReader.onloadend = e => Rust.reader
.then(reader=> {
window.alert(reader.print_result(fileReader));
});
var fileInputElement = document.getElementById("file-input");
fileInputElement.addEventListener("change", e => fileReader.readAsText(fileInputElement.files[0]));
</script>
</body>
</html>
main.rs
#![feature(proc_macro)]
#[macro_use]
extern crate stdweb;
use stdweb::js_export;
use stdweb::web::FileReader;
use stdweb::web::FileReaderResult;
#[js_export]
fn print_result(file_reader: FileReader) -> String {
match file_reader.result() {
Some(value) => match value {
FileReaderResult::String(value) => value,
_ => String::from("not a text"),
}
None => String::from("empty")
}
}
fn main() {
stdweb::initialize();
stdweb::event_loop();
}

The following code is what I use to interact with another javascript library to read a sql file all without using javascript. This is based on the wasm-bindgen library, and I believe may be helpful to newer folks stumbling onto this answer.
[wasm_bindgen]
pub fn load_accounts_from_file_with_balances(file_input : web_sys::HtmlInputElement) {
//Check the file list from the input
let filelist = file_input.files().expect("Failed to get filelist from File Input!");
//Do not allow blank inputs
if filelist.length() < 1 {
alert("Please select at least one file.");
return;
}
if filelist.get(0) == None {
alert("Please select a valid file");
return;
}
let file = filelist.get(0).expect("Failed to get File from filelist!");
let file_reader : web_sys::FileReader = match web_sys::FileReader::new() {
Ok(f) => f,
Err(e) => {
alert("There was an error creating a file reader");
log(&JsValue::as_string(&e).expect("error converting jsvalue to string."));
web_sys::FileReader::new().expect("")
}
};
let fr_c = file_reader.clone();
// create onLoadEnd callback
let onloadend_cb = Closure::wrap(Box::new(move |_e: web_sys::ProgressEvent| {
let array = js_sys::Uint8Array::new(&fr_c.result().unwrap());
let len = array.byte_length() as usize;
log(&format!("Blob received {}bytes: {:?}", len, array.to_vec()));
// here you can for example use the received image/png data
let db : Database = Database::new(array);
//Prepare a statement
let stmt : Statement = db.prepare(&sql_helper_utility::sql_load_accounts_with_balances());
stmt.getAsObject();
// Bind new values
stmt.bind();
while stmt.step() { //
let row = stmt.getAsObject();
log(&("Here is a row: ".to_owned() + &stringify(row).to_owned()));
}
}) as Box<dyn Fn(web_sys::ProgressEvent)>);
file_reader.set_onloadend(Some(onloadend_cb.as_ref().unchecked_ref()));
file_reader.read_as_array_buffer(&file).expect("blob not readable");
onloadend_cb.forget();
}

I managed to access the file object and pass it to the FileReaderin the following way:
let reader = FileReader::new();
let file_input_element: InputElement = document()
.query_selector(".file-selector")
.unwrap()
.unwrap()
.try_into()
.unwrap();
file_input_element.add_event_listener(
enclose!( (reader, file_input_element) move |event: InputEvent| {
let file = js!{return #{&file_input_element}.files[0]};
let real_file: stdweb::web::Blob = file.try_into().unwrap();
reader.read_as_text(&real_file);
}
This code compiles. However, the data never gets available via reader.result().

Related

Reading numbers from array in javascript and plotting these using plotly

I'm trying to make an interface that lets a user upload CSV files and plots these using plotly, only using javascript and obviously the plotly library. I'm close, but my suspicion is that there's an issue with the asynchronous reading of the csv files.
As you can probably see, I'm relatively new to javascript, so any feedback is welcome. I cannot however use any other libraries or packages plotly.
The problem is that the resulting figure only shows the initialized values (1).
EDIT: The heatmap function works on test data, or if I modify specific elements of the data_y object, just not when I update the information from the file.
There's a button that allows uploading of the csv files. On event this code triggers:
<script>
let picker = document.getElementById('picker');
picker.addEventListener('change', event => {0
file_list = event.target.files;
var fig_y = [];
for (let i = 0 ; i< file_list.length ; i++){
if(file_list[i].name == (".DS_Store")){continue}
else {
var ready = read_data(file_list[i]);
fig_y.push(ready);
}
}
console.log(fig_y);
plot_heatmap(fig_y);
}
);
</script>
The data is read using this code.
<script>
function read_data(input){
var xs = 1212; // length of the data
file_contents = [];
var data_y = Array(xs).fill(1);
let file = input;
let reader = new FileReader();
reader.readAsText(file);
reader.onload = function(){
file_contents = reader.result.split('\n');
// open the data file. First two lines contain a description of the data.
for (let j = 2 ; j<file_contents.length-1 ; j++) {
// the relevant data is the third number in the column
var nr = file_contents[j].split(",").map(Number)[2];
data_y[j-2] = nr;
}
}
return data_y;
}
</script>
the code that makes the plotly heatmap.
<script>
function plot_heatmap(data_z){
var data = [
{
z: data_z,
type: 'heatmap'
}
];
Plotly.newPlot('raw_data', data);
};
</script>
OK, so I figured out the answer. It comes from the asynchronous reading of the text files. Putting the plot_heatmap function in the following Timeout function solved the issue (well, maybe it's more a workaround).
setTimeout(() => { plot_heatmap(fig_y); }, 100);
Actually, by changing the length of the timeout, I could catch JS in its act and see half the heatmap filled in with the real values and the other half still with the initialized value!

How to use FileReader.readAsText() synchronously?

I am trying to read a CSV file using FileReader.readAsText() in JavaScript. But I am not able to get the value synchronously. I tried multiple approaches. But none is working. Below is the code I have so far:
1st Approach:
<input
id = "inputfile"
type = "file"
name = "inputfile"
onChange = {uploadFile} >
const uploadFile = (event: React.ChangeEvent < HTMLInputElement > ) => {
let resultSyncOutput = '';
connst files = event?.target.files;
if (files && files.length > 0) {
readAsTextCust(files[0]).then(resultStr => {
resultSyncOutput = resultStr;
});
}
// At this line I am not able to get the value of resultSyncOutput with the content of file sychronously
//Do something with the result of reading file.
someMethod(resultSyncOutput);
}
async function readAsTextCust(file) {
let resultStr = await new Promise((resolve) => {
let fileReader = new FileReader();
fileReader.onload = (e) => resolve(fileReader.result);
fileReader.readAsText(file);
});
console.log(resultStr);
return resultStr;
}
This is the first approach I tried ie, using async/await. I also tried to do it without aysnc/await and still was not able to succeed. It is critical for this operation to be synchronous.
NB: I checked a lot of answers in Stack Overflow and none provides a solution to this problem. So please do not mark this as duplicate.
Answer to this particualr question is not provided anywhere

How to read file using FileReader.readAsText synchronously in JavaScript?

I am trying to read a CSV file using FileReader.readAsText() in JavaScript. But I am not able to get the value synchronously. I tried multiple approaches. But none is working. Below is the code I have so far:
1st Approach:
<input
id = "inputfile"
type = "file"
name = "inputfile"
onChange = {uploadFile} >
const uploadFile = (event: React.ChangeEvent < HTMLInputElement > ) => {
let resultSyncOutput = '';
connst files = event?.target.files;
if (files && files.length > 0) {
readAsTextCust(files[0]).then(resultStr => {
resultSyncOutput = resultStr;
});
}
// At this line I am not able to get the value of resultSyncOutput with the content of file sychronously
//Do something with the result of reading file.
someMethod(resultSyncOutput);
}
async function readAsTextCust(file) {
let resultStr = await new Promise((resolve) => {
let fileReader = new FileReader();
fileReader.onload = (e) => resolve(fileReader.result);
fileReader.readAsText(file);
});
console.log(resultStr);
return resultStr;
}
This is the first approach I tried ie, using async/await. I also tried to do it without aysnc/await and still was not able to succeed. It is critical for this operation to be synchronous. Also use of Ajax is not allowed in project.
NB: I checked a lot of answers in Stack Overflow and none provides a solution to this problem. So please do not mark this as duplicate. Answer to this particular question is not provided anywhere.
Please help me even if this looks simple to you
I found the solution resultSyncOutput = await readAsTextCust(files[0]); and declaring the calling function as async worked.
You need to set a callback function for the onload event callback to get the results. Also notice that you need to call readAsText on the uploaded CSV file.
You can use FileReader API inside input's onChange callback this way:
<input
id = "inputfile"
type = "file"
name = "inputfile"
onChange = function (event: React.ChangeEvent<HTMLInputElement>) {
const reader = new FileReader();
reader.onload = (e) => {
console.log(e.target?.result) // this is the result string.
};
reader.readAsText(event.target.files?.[0] as File);
};
No need for async/await as well.

Obtain FileReader result in JavaScript for dynamic file inputs [duplicate]

This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Closed 3 years ago.
I'm new to JS and I've to do a feature at work that I don't know very well how deal with it.
I have a form, where the user can add items dynamically pressing buttons. One of the buttons allows to add an input to upload a file. For symplicity I will resume the HTML to something like this:
The original div in the form it's:
<div id="userContent">
</div>
When the user press the button it adds elements, in this case file inputs:
<div id="userContent">
<div id="inputFile-1" class="customTabContent">
<input id="file-1" type="file" required="required">
</div>
<div id="inputFile-2" class="customTabContent">
<input id="file-2" type="file" required="required">
</div>
<div id="inputFile-3" class="customTabContent">
<input id="file-3" type="file" required="required">
</div>
</div>
When the user finish adding elements and press the button to submit the data, I read the DOM and to see what has created the user, first I get the parent with:
var parent = document.getElementById('userContent');
Then, I iterate over the childs and for each child I get the input:
var input = document.getElementById('file-1');
var file = input.files[0];
And then I use this function to read the file and return the value in base64, using closures to obtain the value:
function getBase64(file) {
var base64data = null;
var fileReader = new FileReader();
fileReader.onloadend = (function () {
return function () {
var rawData = fileReader.result;
/* Remove metadata */
var cleanedData = rawData.replace(/^data:(.*;base64,)?/, '');
/* Ensure padding if the input length is not divisible by 3 */
if ((cleanedData.length % 4) > 0) {
cleanedData += '='.repeat(4 - (cleanedData.length % 4));
}
base64data = cleanedData;
}
})();
fileReader.readAsDataURL(file);
return base64data;
}
My problem it's that on the website, I get an error saying that base64data it's null but if I put a breakpoint on return base64data; at the end of getBase64(file) the variable has the value on base64, and if I release the debugger I can send the value to the server.
Reading the documentation, FileReader it's asynchronous so I think that the problem seems be related to it, but how can I achieve what I want to do? I don't know why on the debugger I can get the value but outside the debugger no. I'm very stuck....
Regards.
P.D: If I don't answer it's because I'm not at work, so sorry for the delay on the response.
-------- EDIT 1
Thanks to Sjoerd de Wit I can get the result but I can't assign to a variable:
var info = {'type': '', 'data': ''};
........
} else if (it_is_a_file) {
var file = input.files[0];
var base64data = null;
getBase64(file).then((result) => {
base64data = result;
info['type'] = 'file';
info['data'] = base64data;
});
} else if
.......
return info;
But on info I get {'type': '', 'data': ''}. I'm using wrong the promise? Thanks.
-------- EDIT 2
This problem was because as a noob on JavaScript, I didn't know that using FLASK, you have to use forms and get the data in a different way.
So, the answer to this question was search how get data from a FORM with FLASK.
But I'm going to mark the answer as correct beacuse you can get the value, as I was asquing for.
You could turn the function to return a promise and then resolve the base64data when it's loaded.
function getBase64(file) {
return new Promise((resolve, reject) => {
var fileReader = new FileReader();
fileReader.onloadend = (function () {
return function () {
var rawData = fileReader.result;
/* Remove metadata */
var cleanedData = rawData.replace(/^data:(.*;base64,)?/, '');
/* Ensure padding if the input length is not divisible by 3 */
if ((cleanedData.length % 4) > 0) {
cleanedData += '='.repeat(4 - (cleanedData.length % 4));
}
resolve(cleanedData);
}
})();
fileReader.readAsDataURL(file);
});
}
Then where you want to read it you can do:
getBase64(file).then((base64data) => {
console.log(base64data);
})

How to using javascript to read files into an array in order

I have a bunch of text files on server side with file names 0.txt, 1.txt, 2.txt, 3.txt and so forth. I want to read the content of all files and store them in an array A, such that A[0] has 0.txt's content, A[1] has 1.txt's, ...
How can I do it in Javascript / jquery?
Originally, I used $.ajax({}) in jQuery to load those text files. But it didn't work, because of the asynchronous nature of ajax. I tried to set $.ajax({...async=false...}), but it was very slow -- I have ~1000 10KB files to read in total.
from your question, you want to load txt file from server to local:
var done = 0, resultArr = [], numberOfFiles = 1000;
function getHandler(idx) {
return function(data) {
resultArr[idx] = data;
done++;
if (done === numberOfFiles) {
// tell your other part all files are loaded
}
}
}
for (var i = 0; i < numberOfFiles; i++) {
$.ajax(i + ".txt").done(getHandler(i));
}
jsFiddle: http://jsfiddle.net/LtQYF/1/
What you're looking for is File API introduced in HTML5 (working draft).
The examples in this article will point you in the right direction. Remember that the end user will have to initiate the action and manually select the files - otherwise it would have been a terrible idea privacy- and security-wise.
Update:
I found (yet again) the mozilla docos to be more readable! Quick html mockup:
<input type="file" id="files" name="files[]" onchange="loadTextFile();" multiple/>
<button id="test"onclick="test();">What have we read?</button>
...and the JavaScript:
var testArray = []; //your array
function loadTextFile() {
//this would be tidier with jQuery, but whatever
var _filesContainer = document.getElementById("files");
//check how many files have been selected and iterate over them
var _filesCount = _filesContainer.files.length;
for (var i = 0; i < _filesCount; i++) {
//create new FileReader instance; I have to read more into it
//but I was unable to just recycle one
var oFReader = new FileReader();
//when the file has been "read" by the FileReader locally
//log its contents and push them into an array
oFReader.onload = function(oFREvent) {
console.log(oFREvent.target.result);
testArray.push(oFREvent.target.result);
};
//actually initiate the read
oFReader.readAsText(_filesContainer.files[i]);
}
}
//sanity check
function test() {
for (var i = 0; i < testArray.length; i++) {
console.warn(testArray[i]);
}
}​
Fiddled
You don't give much information to give a specific answer. However, it is my opinion that "it doesn't work because of the asynchronous nature of ajax" is not correct. You should be able to allocate an array of the correct size and use a callback for each file. You might try other options such as bundling the files on the server and unbundling them on the client, etc. The designs, that address the problem well, depend on specifics that you have not provided.

Categories

Resources