I'm coding a webpage that needs to read some data from different csv on a path depending on the country of the user.
the path is something like this:
./csv/m2-2022-10-25_13_45_55_es.csv
m2-2022-10-25_13_45_56_fr.csv
m2-2022-10-25_13_46_04_it.csv
etc
And those files will be replaced regularly, the only that we'll always have is the country code (es, fr, it, etc).
So, what I need is to list all the files on the path to an array, and loop through the array to find if the last characters of the filename are $countryCode + ".csv", and there run some code.
But I can't find how, all the solutions I find are using Node.js, but are there a solution using only Javascript (or jQuery)?
Regards!
You cannot use pure Javascript to do that, because if you wanted to search files in your computer only using javascript, it would be a huge security breach.
You must use node.js to open files but you can make an API to your nodejs file from your javascript and you can send as a response the content of your file.
Here some links that might help you :
FS : https://nodejs.org/api/fs.html
NodeJS api : https://medium.com/swlh/how-to-create-a-simple-restful-api-in-node-js-ae4bfddea158
You can check a similar question here:
Get list of filenames in folder with Javascript
You can't access to filesystem from the frontend, this it would be a huge security breach, because anyone could access to your filesystem tree.
You have to do a function in backend to build the array you want and send it to frontend.
If you create a function in backend file that returns the array of files in the folder, you can call it from the frontend via XMLHttpRequest or Fetch to get the array in frontend and be able to use in your js file.
Related
This question already has answers here:
Javascript read file without using input
(3 answers)
Closed 1 year ago.
I am trying to create a members badge draw for a club.
I have 1000+ records in "members.csv" containing "memNumber, lastName, firstName" and "expiryDate".
I can manually load the "members.csv" file using the <input> method in "index.html" then with "script.js" turn the data into an array and retrieve a random members details.
This will only be run locally on one PC using a "Task Scheduler" and won't be published online.
My "index.html", "script.js" and "members.csv" files are all in the same folder.
How can I load "members.csv" automatically on page-load rather than using the <input> method ?
Any help would be appreciated. Cheers, Vince.
there is some ways you can do to solve this problem. Main problem is to have the data from members.csv on page load. All of the solutions below already mentioned on the comment section, but i just want to clear it out a little bit.
To have access to local file (including your members.csv), browsers have some strict rules related to security (unless the one that needs user interaction such as the <input>)
First Solution: using AJAX and install a local server
By using AJAX, the javascript can request the csv file using HTTP Protocol (rather than File Protocol) on localhost on page load.
After that, you can process the file as you've previously processed.
Second Solution: convert it to JavaScript Object
If the file is static (rarely-changed) and you don't want to install any local web server, then you can convert the csv file into javascript object using some free online services like CSV to JSON.
After that, you copy the converted result from the service and assign it to a variable. Then, you can process it.
Hope this answer helps you :)
I am trying to make a compiler in which users make code at abc.com/newProject and their output is in an iframe, that iframe need to be served files that are made at the abc.com/newProject. So I will be doing a POST of JSON obj at abc.com/compile-project that will create files and those will be used by the iframe, after being used those should get deleted. Files are basically JS files that iframe will fetch using script in header.
So a pseudo-code will look something like this:-
app.post('/compile-project', function(req, res){
//Directory created using node tmp
//files created in the directory
//These files are accessible using <script src="/js/file1.js"></script>
//when the current connection requests the files they get deleted
});
Any help will be appreciated thanks.....
I am trying to make a compiler [...] Any help will be appreciated thanks.....
I would strongly discourage you from doing that if you don't know what you're doing (and considering the fact that you're asking how to save a file then apparently you don't).
The requirements that you described are extremely simple but you need to have much deeper understanding of everything that's going on to avoid serious security problems that you will encounter with no doubt along the way.
What you describes can be done without even using a file system, since all your files are served only once so it doesn't make much sense to store them in actual files. But even if you insist on the file system then all you need is to use fs.mkdtemp to create a temporary directory, use something like the uuid module for unique IDs to use in the filenames, then use fs.writeFile to write a file. This is all you need for the file upload endpoint. Now in the download endpoint all you need is to use fs.readFile to read the file and fs.unlink to remove it. That's it.
Now, it will surely get you into trouble of failures on browser reloads, back button not working, and finally security issues of people being able to serve any random code from your servers leading to vulnerabilities too numerous to even list here.
Take a look at the source code of repl.it and JS Bin on GitHub:
https://github.com/replit/repl.it
https://github.com/jsbin/jsbin
to appreciate the scope of the project that you are willing to undertake.
Is it possible to list the files in a directory using only javascript? To clarify, I mean list the files on the server, not the files on the clients computer. For instance:
www.domain.com/files/
contains 4 images (.jpg)
What you asked for:
fs.readdirSync("./")
For example, in Express:
router.get("/someFiles", function(){
return "<ul>"
+ fs.readdirSync("/path/to/someFiles")
.map(function(fName){
return "<li>"+fName+"</li>";
})
+"</ul>"
;
});
What I understand you are searching for:
Express Serve-Index
No, it isn't possible to get the local directories of the client for obvious security reasons. If you want to receive a file, then use a file input field.
Neither is possible to get the server files natively, because JavaScript is a client side language.
I'm trying to figure out the best way to accomplish the following:
Download a large XML (1GB) file on daily basis from a third-party website
Convert that XML file to relational database on my server
Add functionality to search the database
For the first part, is this something that would need to be done manually, or could it be accomplished with a cron?
Most of the questions and answers related to XML and relational databases refer to Python or PHP. Could this be done with javascript/nodejs as well?
If this question is better suited for a different StackExchange forum, please let me know and I will move it there instead.
Below is a sample of the xml code:
<case-file>
<serial-number>123456789</serial-number>
<transaction-date>20150101</transaction-date>
<case-file-header>
<filing-date>20140101</filing-date>
</case-file-header>
<case-file-statements>
<case-file-statement>
<code>AQ123</code>
<text>Case file statement text</text>
</case-file-statement>
<case-file-statement>
<code>BC345</code>
<text>Case file statement text</text>
</case-file-statement>
</case-file-statements>
<classifications>
<classification>
<international-code-total-no>1</international-code-total-no>
<primary-code>025</primary-code>
</classification>
</classifications>
</case-file>
Here's some more information about how these files will be used:
All XML files will be in the same format. There are probably a few dozen elements within each record. The files are updated by a third party on a daily basis (and are available as zipped files on the third-party website). Each day's file represents new case files as well as updated case files.
The goal is to allow a user to search for information and organize those search results on the page (or in a generated pdf/excel file). For example, a user might want to see all case files that include a particular word within the <text> element. Or a user might want to see all case files that include primary code 025 (<primary-code> element) and that were filed after a particular date (<filing-date> element).
The only data entered into the database will be from the XML files--users won't be adding any of their own information to the database.
All steps could certainly be accomplished using node.js. There are modules available that will help you with each of these tasks:
node-cron: lets you easily set up cron tasks in your node program. Another option would be to set up a cron task on your operating system (lots of resources available for your favourite OS).
download: module to easily download files from a URL.
xml-stream: allows you to stream a file and register events that fire when the parser encounters certain XML elements. I have successfully used this module to parse KML files (granted they were significantly smaller than your files).
node-postgres: node client for PostgreSQL (I am sure there are clients for many other common RDBMS, PG is the only one I have used so far).
Most of these modules have pretty great examples that will get you started. Here's how you would probably set up the XML streaming part:
var XmlStream = require('xml-stream');
var xml = fs.createReadStream('path/to/file/on/disk'); // or stream directly from your online source
var xmlStream = new XmlStream(xml);
xmlStream.on('endElement case-file', function(element) {
// create and execute SQL query/queries here for this element
});
xmlStream.on('end', function() {
// done reading elements
// do further processing / query database, etc.
});
Are you sure you need to put the data in a relational database, or do you just want to search it in general?
There don't seem to be any actual relations in the data, so it might be simpler to put it in a document search index such as ElasticSearch.
Any automatic XML to JSON converter would probably produce suitable output. The large file size is an issue. This library, despite its summary saying "not streaming", is actually streaming if you inspect the source code, so it would work for you.
I had task with xml files as you wrote. This are principals I used:
All incoming files I stored as is in DB (XMLTYPE), because I need a source file info;
All incoming files parsed with XSL transformation. For example, I see that it is three entity here: fileInfo, fileCases, fileClassification. You can write XSL transformation to compile source file info in 3 entity types (in tags FileInfo, FileCases, FileClassification);
When you have output transformed XML you can make 3 procedures, that inserts data into DB (each entity in DB area).
I have a file server located at say http://myshare.com. This server is used to just host all my files. My file server has a directory named 'myfiles'. So my URI looks something like this: 'http://myshare.com/myfiles'. This location has say 10 files.
First question: How do I get the names of all the files located on this remote file server using Node.js?
Second question: If 'myfiles' directory has sub-directories, how do I traverse all the sub-directories and list all the files in them using Node.js?
Any help would be appreciated. Thanks
I suppose that the files are on the same machine, since you can always mount them with cifs or similar and then it would be the same than local files, making live easier since is OS who manages them instead of your code.
First, you have to familiarize with node's file object anf the function "readdir" either on its async or sync version.
http://nodejs.org/api/fs.html#fs_fs_readdir_path_callback
Then you parse the array of files returned .
For subdirectories you have to code a recursive function that calls itself on every directory. For every file you have to check the fsstats and check the property isDirectory() over the returned object.
getting fsStat object http://nodejs.org/api/fs.html#fs_fs_stat_path_callback
fsStat object http://nodejs.org/api/fs.html#fs_class_fs_stats
As a sample, you can check this answer https://stackoverflow.com/a/5827895/1680125
Hope it helps