CSV is too large to be processed via AJAX - javascript

I'm having a problem when sending large amounts of data through an AJAX request. I'm pulling in an XLS file from a website and attempting to pass it through an API by parsing the data. I'm doing this in VBScript/Classic ASP so there is no native function to parse XLS so I'm first attempting to convert it to a CSV file through Javascript.
I'm using something called SheetJS (http://oss.sheetjs.com/js-xls/) which is a great tool and it works just as I need it to. I can run an Excel file through it and it outputs the correct CSV data. I then try to send that data via AJAX to the ASP page with my code and I get a 500 error that I've isolated to being an issue with the file being too large. I was able to isolate to about 1652 lines of my Excel file and anything past that generates a CSV file too large to send.
All I am getting is a 500 error so I'm not really sure what else to do from this point. Is there a data limit on AJAX functions? Or is it a time limit type issue? I don't know how to find out which it is. Any suggestions on how to get a more detailed error message AND any fixes for this issue?

While you can't work with .xls files natively in VBScript, there is ADO that makes it easy to do that via COM. This way scales well wrt memory. Start your research here.

Related

Displaying a generated JSON file on a simple website

I have a Java tool, which schedules a task every 24 hours and writes the result in a result.json file. Now I want to display this result.json file on a simple website, but I know that it's not natively possible with JavaScript to access local files. But what other, simple ways exist for this problem? I try to avoid a webservice to keep the scheduling-program and the website on the same server.
Thanks!
You can use the object FileReader for read the file stream from your system and gets the JSON file in a string, later use JSON.parse() to get the JSON in JS object and iterate over it for pretty representing in document DOM, or you can print the string in the HTML without parse to JS object.
Here there are a very completely example of FileReader.

Javascript D3-fetch write to csv

i have some code that reads a csv using d3-fetch. I load my csv in the way:
D3.csv(./mydata.csv).then(function (info) {
Console.log(info);
});
this works fine and gives me my data. But how would i write to this data?
Keep in mind D3 is supposed to be run in a web browser. Web browsers can read files through HTTP but they cannot write files.
A server could write files on a file system it controls. So you would need to write a server endpoint like /editCSV and then make a POST request to it from the browser. Your server could then write the changes to the CSV file.
In order to do this, you need to have control of the server (for example, a Node.js server) that is serving the CSV and the related file.

Excel to PDF conversion in PHP

It is asked to me to do a small excel to pdf conversion.
Let me describe what is exactly to be done.
in excel file there are list of companies which looks like this...
and to retrieve a company name turn into this in PDF format, like this...
I am thinking to follow a path...
1-using php and js I will get excel file with "input type=file"
2-using pdf reader php script I will retrieve the data required.
After this I am confusing...
3-I can retrieve data by searching and parsing a company name( for example getting an input: "company:A") till next keyword of "company" for all columns
or should I put all data from excel into CSV file and do the parsing part there???
4- after that, I am thinking to use, one of the tcpdf example methodologies however here comes another confusing point for me... because I don't know which type of output should get from excel or csv and load into pdf converter method.
An enlightening path would be appreciated since I am confused
Regards
Build your Data on HTML format first then if you got the correct display on html then you can now convert it to a PDF file.

How to access a local JSON file with Javascript?

I have a Python program that generates an html page for reporting results. The html page is saved in an output directory on disk alongside a javascript file that helps with dynamic table handling. I also save a JSON file to this output directory that I would like to read in with my javascript file. This JSON file has data from the Python run (saved dictionary) that I would like to be able to access. So in an output directory on disk I have:
C:/somedirectory/output/report.html
C:/somedirectory/output/tables.js
C:/somedirectory/output/data.json
All files have been created from my program.
My html page has a table with checkboxes and if those checkboxes are selected I would like to update a second table based on data saved in the JSON file. Thus I would like to open my html report in any browser and read in the JSON file as a javascript object.
I have been trying to use ajax and .getJSON but am getting the
No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access.
I have searched and seen many similar problems but have not come across anything that quite fits what I need. Thoughts and a work around would be greatly appreciated. Thanks.
Update
Since everything is run locally on the client side I have decided to embed the JSON data (python dictionary) and javascript code directly into the html report output. This way the data is internally accessible and the html file can be passed around without dependency issues. The user with the answer I selected below has a link that eludes to this solution.
JavaScript runs on the client machine, hence it can only access files on the client machine using a special setup.
If you want it to read JSON on your server, you should use the path:
http://example.com/output/data.json
Better way would be to read/write JSON file from Python and then send the table data to JavaScript as in this answer: Send data from Python to Javascript (JSON)

How do you check whether a CSV file is corrupted in PHP and/or Javascript?

I'm making an HTML5/jQuery/PHP app which involves uploading CSV files (via a drag and drop), and processing them to a MySQL database. So far, I can upload the files, and I know how to read them into a database.
My question: I am wondering if it is possible to detect whether a CSV file is in a corrupted format by PHP or Javascript/jQuery? For example, I can rename somefile.png (an image) to somefile.csv, and it still gets uploaded. If I open up the renamed file in Notepad++, all I see is garbage, which is expected.
I would like to do this on the clientside, so I can alert the user (via JQuery) whether the file is in a corrupted format. I'd also like to check on the serverside (via PHP) before I start iterating over each CSV file for db processing.
My first thoughts would be to use regular expressions, but I am unsure how to make ones for this particular problem. I know the basics of regular expressions, but haven't really used them in advanced settings before.
First of all you should check content-type of picked file, it should be "text/csv". At the server-side you can check file via fgetscsv PHP function (http://php.net/manual/function.fgetcsv.php) (catch null or false on error)
You don't want to be validating it if you're going to be reading it right after. Just read it in and catch any errors as you read. That way you come to know whether file is valid or corrupt.

Categories

Resources