Is there a more efficient way to check number of columns? - javascript

So I am trying to do some client side validation on an app I am working on, and I want to be able to check the number of columns in a CSV that the user uploads. The CSVs could potentially be very large (up to 30 thousand rows) so I'm worried that my current method, which reads the entire file, is not the best way to do it.
This is what I have right now.
// Function to validate the File Upload
$(document).ready(function() {
$("#csv").change(function () {
console.log("Read CSV Function Triggered");
let theFile = document.getElementById("csv").files[0];
let reader = new FileReader();
reader.addEventListener('load', readFile);
reader.readAsText(theFile);
function readFile(event) {
let csvText = event.target.result;
let firstLine = csvText.split('\n').shift(); // First Line
$("#result").html(firstLine);
}
});
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<input type="file" id="csv" />
<br>
<span id="result">0</span>
In the example I just make the first line display on the page, but in the app I have a parser library that I couldn't include in this example which converts the first line to an array and then I get the length of the array. So like I said before, this works, but I am worried that when large CSVs are uploaded it may affect browser performance.
Any suggestions?

Related

Retreiving contents of previously obtained text

This is a follow-up to a previous post (solved). In the previous post we loaded a delimited text file then took action on the contents of the file. But what we did was in one smooth progression of events. In my actual implementation, events will occur separately. First, a tab delimited text file will be loaded, using the FileReader API, and displayed on the page. Much later a function will need to retrieve the loaded content and do stuff with it. I thought I would have no difficulty separating the tasks, but I was wrong. I don't know what I'm doing wrong or if I'm going about it the wrong way. I'd appreciate some input.
Note: I don't necessarily have to use the information directly from the pre-tag "fileDisplayArea". I just thought that would work and be easy. I could use the string stored in memory, if that is better. The string should never be more than 6 or 7 Mb.
What do I need to do to, so at a later time I can use the string loaded and placed in "fileDisplayArea"?
Thanks in advance,
Andrew
var fileInput = document.getElementById('fileInput');
fileInput.addEventListener('change', function (e) {
var file = fileInput.files[0];
var textType = /text.*/;
if (file.type.match(textType)) {
var reader = new FileReader();
reader.onload = function (e) {
// Entire file
fileDisplayArea.innerText = reader.result;
}
reader.readAsText(file);
} else {
fileDisplayArea.innerText = "File not supported!"
}
});
// doIt functionality
document.getElementById("doIt").addEventListener("click", function () {
var myTemp = document.getElementById("fileDisplayArea");
console.log(myTemp);
// trying to retreive original reader.result contents so I can do stuff with it
// i thought it would be as simple as using getElementByID, but i'm obviously wrong
}, false);
<br />
Step 1:<br />
Select a text file:
<input type="file" id="fileInput">
<hr />
Step 2:<br />
Retreive the contents previously loaded. (Normally 'doIt' would be called from another function, not a button.)
<button id="doIt">doIt</button>
<hr />
<pre id="fileDisplayArea"></pre>

How is a file split / chunked? How do I access the chunks sequentially?

For example, if I have an image and I download it -- how does the computer know that all the bytes are supposed to be in that sequential order?
When I upload an image, is it possible to upload it in "chunks" so I can access/use whatever I have uploaded thus far (i.e. I have only uploaded the top half of the image) -- how would I access it?
The same would go for video or PDF, etc.
You can read file into chunks and upload file in chunks.
See the example below.
<input type="file" name="filebrowsefileid">
var filePicker = document.getElementById('filebrowsefileid');
var file = filePicker.files[0];
var chunck = file.slice(a,b);
var reader = new FileReader();
reader.readAsBinaryString(chunck );
....
reader["onloadend"] = function () {
reader.result; // This is what you want
}

Parse a local file to an array using JavaScript

I am trying to read a local file and have each line as an index in an array using JavaScript. I have been searching for the past 20 minutes and either I'm stupid or there really isn't an answer that pertains to my problem (...but it's probably the former :P). I am really new to JavaScript so if you have an answer could you please comment the code just to I know what's going on?
Also, from the searching I've done on the internet some people said JavaScript can't read local file for security reasons so if that is correct is there another language I can use? I'm a bit familiar with PHP if that is an option, which I doubt it is.
EDIT
As per thg435's question, I'll explain what I am trying to accomplish.
My project is to analyze a BUNCH of water quality data that has been collected by the Ontario gov't (which I've done) and display it in some way. I have chosen to display it on a webpage using the Google Maps API. I currently have a file of chemicals that were found. Each line is a different chemical. I would like to read the file in an array then create an option menu displaying the chemicals in the array.
Also, the local file I would like to read will the be the same name and location all the time. I have seen people have boxes where the user clicks and chooses their file or to drag and drop but that's not what I'm looking for.
I don't think I explained this properly. I have a file in the same directory as my HTML and JavaScript files that contains words. Example:
Line 1: "Iron"
Line 2: "Aluminum"
Line 3: "Steel"
etc...
I would like to read the file and parse each line into a different index in an array. I don't want the user to be able to choose which file to read using the <input ... /> thing.
You're going to want to take a look at the FileReader API. This should allow you to read the text of a local file via readAsText(). This won't work in every browser but should work in all modern browser. You can see which browsers support it here.
Example:
<input id="file" type="file" />
var filesInput = document.getElementById("file");
filesInput.addEventListener("change", function (event) {
var files = event.target.files;
var file = files[0];
var reader = new FileReader();
reader.addEventListener("load", function (event) {
var textFile = event.target;
alert(textFile.result);
});
reader.readAsText(file);
});
It's not possible to invoke the FileReader API without user interaction. Consequently, your user would have to select whatever file to load in order for it to be read in pure JS. Since I'm assuming this will be up on a server, why not just put the list of chemicals also up on the server and GET the JSON encoded array of the results. Then you can decode them with Javascript.
You can access local files in 2 ways that I know of. The first way is making the user drag-and-drop the files onto the page, and using an <input type="file"> tag.
For the former, you would need to do the following:
addEventListener('dragover', function(e){e.preventDefault();});
addEventListener('drop', function(e) {
eventHandler.call(e.dataTransfer||e.clipboardData);
e.preventDefault();
});
For the latter, you'd need to add an event listener for the change event on the input:
document.getElementById('upload').addEventListener('change', eventHandler)
And for both, you'd need to have this as a basic callback function:
function eventHandler() {
var file = this.files[0]; //get the files
var reader = new FileReader(); //initiate reader
reader.onloadend = callbackFn; //set event handler
reader.readAsText(file); //initiate reading of files
if (this.id) { //only run if this is the input
var id = this.id;
this.outerHTML = this.outerHTML; //this resets the input
document.getElementById(id).addEventListener('change', eventHandler); //reattach event handler
}
function callbackFn(e) {
document.getElementById('output').value = e.target.result; //output it to a textarea
}
}
Here is a demo where the text contents (that what you see when opening it in notepad) of any file you drop in it, or any file you select from the input, is put in the textarea.
For more information, see https://developer.mozilla.org/en-US/docs/Using_files_from_web_applications.

How to remove one specific selected file from input file control

How to remove one specific selected file from input file control?
I have an input file control with the option to select multiple files; however, I want to validate a file and if it has an wrong extension then I should remove that file from the file control itself, is it possible?
I tried as below
<input type="file" name="fileToUpload" id="fileToUpload" multiple/>
<script> $("#fileToUpload")[0].files[0] </script>
Below is the screenshot of the object but I am not able to modify it
As other people pointed out, FileList is read only. You can get around this by pushing those files into a separate Array though. You can then do whatever you want with that curated list of files. If uploading them to a server is the goal, you can use the FileReader API.
Below is a round about way of completely avoiding needing to modify the FileList.
Steps:
Add normal file input change event listener
Loop through each file from change event, filter for desired validation
Push valid files into separate array
Use FileReader API to read files locally
Submit valid, processed files to server
Event handler and basic file loop code:
var validatedFiles = [];
$("#fileToUpload").on("change", function (event) {
var files = event.originalEvent.target.files;
files.forEach(function (file) {
if (file.name.matches(/something.txt/)) {
validatedFiles.push(file); // Simplest case
} else {
/* do something else */
}
});
});
Below is a more complicated version of the file loop that shows how you can use the FileReader API to load the file into the browser and optionally submit it to a server as a Base64 encoded blob.
files.forEach(function (file) {
if (file.name.matches(/something.txt/)) { // You could also do more complicated validation after processing the file client side
var reader = new FileReader();
// Setup listener
reader.onload = (function (processedFile) {
return function (e) {
var fileData = { name : processedFile.name, fileData : e.target.result };
// Submit individual file to server
$.post("/your/url/here", fileData);
// or add to list to submit as group later
validatedFiles.push(fileData);
};
})(file);
// Process file
reader.readAsDataURL(file);
} else {
/* still do something else */
}
});
A note of caution about using FileReader API. Base64 encoding a file will increase its size by around 30%. If that isn't acceptable you will need to try something else.
I know this is an old post but I have spent ages trying to work around this one so I'll post my solution. There is a way to update the fileList of an fileField element with another fileList - which can be done with DataTransfer:
let updateFileList = function (fileField, index) {
let fileBuffer = Array.from(fileField.files);
fileBuffer.splice(index, 1);
/** Code from: https://stackoverflow.com/a/47172409/8145428 */
const dT = new ClipboardEvent('').clipboardData || // Firefox < 62 workaround exploiting https://bugzilla.mozilla.org/show_bug.cgi?id=1422655
new DataTransfer(); // specs compliant (as of March 2018 only Chrome)
for (let file of fileBuffer) { dT.items.add(file); }
fileField.files = dT.files;
}
The above function takes the fileField as a DOM Object and the index of the file in the fileField's fileList to be removed and updates the passed fileField accordingly.
Hope this saves somebody else some time!
How to set File objects and length property at FileList object where the files are also reflected at FormData object?
I thought that I should add my comment here as well here (I've answered here: JavaScript delete File from FileList to be uploaded)
I found a workaround. This will not require AJAX for the request at all and the form can be sent to the server. Basically you could create an hidden or text input and set it's value attribute to the base64 string created after processing the file selected.
<input type=hidden value=${base64string} />
You will probably consider the idea to create multiple input file instead of input text or hidden. This will not work as we can't assign a value to it.
This method will include the input file in the data sent to the database and to ignore the input file you could:
in the back-end don't consider the field;
you can set the disabled attribute to the input file before serialising the form;
remove the DOM element before sending data.
When you want to delete a file just get the index of the element and remove the input element (text or hidden) from the DOM.
Requirements:
You need to write the logic to convert files in base64 and store all files inside an array whenever the input file trigger the change event.
Pros:
This will basically give you a lot of control and you can filter, comparing files, check for file size, MIME type, and so on..
Based on #liamthorne4 answer, here is a working solution for upload, list and delete element from list, tested on Firefox and Chrome:
HTML:
<button onclick="uploadFile(event)">Upload files</button>
<div id="listfiles" class="view_list"></div>
<input class="hidden" type="file" id="input_file_id" onchange="fileList(event)" name="files[]" multiple>
JS:
function uploadFile(evt) {
evt.preventDefault();
$('#input_file_id').click();
}
function fileList(e) {
var files = $('#input_file_id').prop("files");
var names = $.map(files, function(val) { return val.name; });
for (n in names) {
$("#listfiles").append("<div id=preload_"+n+" title='"+names[n]+"'><p>"+names[n]+"</p><a onclick=deleteFile("+n+")>Delete</a></div>");
}
}
function deleteFile(index) {
filelistall = $('#input_file_id').prop("files");
var fileBuffer=[];
Array.prototype.push.apply( fileBuffer, filelistall );
fileBuffer.splice(index, 1);
const dT = new ClipboardEvent('').clipboardData || new DataTransfer();
for (let file of fileBuffer) { dT.items.add(file); }
filelistall = $('#input_file_id').prop("files",dT.files);
$("#preload_"+index).remove()
}
html
<input id="fileInput" name="fileInput" type="file" />
<input onclick="clearFileInput()" type="button" value="Clear" />
javascript
function clearFileInput(){
var oldInput = document.getElementById("fileInput");
var newInput = document.createElement("input");
newInput.type = "file";
newInput.id = oldInput.id;
newInput.name = oldInput.name;
newInput.className = oldInput.className;
newInput.style.cssText = oldInput.style.cssText;
// copy any other relevant attributes
oldInput.parentNode.replaceChild(newInput, oldInput);
}

Reading client side text file using Javascript

I want to read a file (on the client side) and get the content in an array. It will be just one file. I have the following and it doesn't work. 'query_list' is a textarea where I want to display the content of the file.
<input type="file" id="file" name="file" enctype="multipart/form-data"/>
<script>
document.getElementById('file').addEventListener('change', readFile, false);
function readFile (evt) {
var files = evt.target.files;
var file = files[0];
var fh = fopen(file, 0);
var str = "";
document.getElementById('query_list').textContent = str;
if(fh!=-1) {
length = flength(fh);
str = fread(fh, length);
fclose(fh);
}
document.getElementById('query_list').textContent = str;
}
</script>
How should I go about it? Eventually I want to loop over the array and run some SQL queries.
If you want to read files on the client using HTML5's FileReader, you must use Firefox, Chrome or IE 10+. If that is true, the following example reads a text file on the client.
your example attempts to use fopen that I have never heard of (on the client)
http://jsfiddle.net/k3j48zmt/
document.getElementById('file').addEventListener('change', readFile, false);
function readFile (evt) {
var files = evt.target.files;
var file = files[0];
var reader = new FileReader();
reader.onload = function(event) {
console.log(event.target.result);
}
reader.readAsText(file)
}
For IE<10 support you need to look into using an ActiveX Object like ADO.Stream Scripting.FileSystemObject http://msdn.microsoft.com/en-us/library/2z9ffy99(v=vs.85).aspx but you'll run into a security problem. If you run IE allowing all ActiveX objects (for your website), it should work.
There is such thing as HTML5 File API to access local files picked by user, without uploading them anywhere.
It is quite new feature, but supported by most of modern browsers.
I strongly recommend to check out this great article to see, how you can use it.
There is one problem with this, you can't read big files (~400 MB and larger) because straight forward FileAPI functions attempting to load entire file into memory.
If you need to read big files, or search something there, or navigate by line index check my LineNavigator, which allows you to read, navigate and search in files of any size. Try it in jsFiddle! It is super easy to use:
var navigator = new FileNavigator(file);
navigator.readSomeLines(0, function linesReadHandler(err, index, lines, eof, progress) {
// Some error
if (err) return;
// Process this line bucket
for (var i = 0; i < lines.length; i++) {
var line = lines[i];
// Do something with it
}
// End of file
if (eof) return;
// Continue reading
navigator.readSomeLines(index + lines.length, linesReadHandler);
});
Well I got beat to the answer but its different:
<input type="file" id="fi" />
<button onclick="handleFile(); return false;">click</button>
function handleFile() {
var preview = document.getElementById('prv')
var file = document.getElementById('fi').files[0];
var div = document.body.appendChild(document.createElement("div"));
div.innerHTML = file.getAsText("utf-8");
}
This will work in FF 3.5 - 3.6, and that's it. FF 4 and WebKit you need to use the FileReader as mentioned by Juan Mendes.
For IE you may find a Flash solution.
I work there, but still wanted to contribute because it works well: You can use the filepicker.io read api to do exactly this. You can pass in an dom element and get the contents back, for text or binary data, even in IE8+

Categories

Resources