Develop Reference

Search through files in path using only javascript

I'm coding a webpage that needs to read some data from different csv on a path depending on the country of the user.
the path is something like this:
./csv/m2-2022-10-25_13_45_55_es.csv
m2-2022-10-25_13_45_56_fr.csv
m2-2022-10-25_13_46_04_it.csv
etc
And those files will be replaced regularly, the only that we'll always have is the country code (es, fr, it, etc).
So, what I need is to list all the files on the path to an array, and loop through the array to find if the last characters of the filename are $countryCode + ".csv", and there run some code.
But I can't find how, all the solutions I find are using Node.js, but are there a solution using only Javascript (or jQuery)?
Regards!

You cannot use pure Javascript to do that, because if you wanted to search files in your computer only using javascript, it would be a huge security breach.
You must use node.js to open files but you can make an API to your nodejs file from your javascript and you can send as a response the content of your file.
Here some links that might help you :
FS : https://nodejs.org/api/fs.html
NodeJS api : https://medium.com/swlh/how-to-create-a-simple-restful-api-in-node-js-ae4bfddea158

You can check a similar question here:
Get list of filenames in folder with Javascript
You can't access to filesystem from the frontend, this it would be a huge security breach, because anyone could access to your filesystem tree.
You have to do a function in backend to build the array you want and send it to frontend.
If you create a function in backend file that returns the array of files in the folder, you can call it from the frontend via XMLHttpRequest or Fetch to get the array in frontend and be able to use in your js file.

Preview generated docx before downloading [duplicate]

I have successfully done code to display a PDF file in the browser instead of the "Open/Save" dialog. Now, I'm stuck trying to display a Word document in the browser. I want to display a Word document in Firefox, IE7+, Chrome etc.
Can any one help? I am always getting the "Open/Save" dialog while displaying the Word doc in browser. I want to implement this functionality using JavaScript.

No browsers currently have the code necessary to render Word Documents, and as far as I know, there are no client-side libraries that currently exist for rendering them either.
However, if you only need to display the Word Document, but don't need to edit it, you can use Google Documents' Viewer via an <iframe> to display a remotely hosted .doc/.docx.
<iframe src="https://docs.google.com/gview?url=http://remote.url.tld/path/to/document.doc&embedded=true"></iframe>
Solution adapted from "How to display a word document using fancybox".
Example:
JSFiddle
However, if you'd rather have native support, in most, if not all browsers, I'd recommend resaving the .doc/.docx as a PDF file Those can also be independently rendered using PDF.js by Mozilla.
Edit:
Huge thanks to cubeguerrero for posting the Microsoft Office 365 viewer in the comments.
<iframe src='https://view.officeapps.live.com/op/embed.aspx?src=http://remote.url.tld/path/to/document.doc' width='1366px' height='623px' frameborder='0'>This is an embedded <a target='_blank' href='http://office.com'>Microsoft Office</a> document, powered by <a target='_blank' href='http://office.com/webapps'>Office Online</a>.</iframe>
One more important caveat to keep in mind, as pointed out by lightswitch05, is that this will upload your document to a third-party server. If this is unacceptable, then this method of display isn't the proper course of action.
Live Examples:
Google Docs Viewer
Microsoft Office Viewer

The answers by Brandon and fatbotdesigns are both correct, but having implemented the Google docs preview, we found multiple .docx files that couldn't be handled by Google. Switched to the MS Office Online preview and works likes a charm.
My recommendation would be to use the MS Office Preview URL over Google's.
https://view.officeapps.live.com/op/embed.aspx?src=http://remote.url.tld/path/to/document.doc'

There are a few JS libraries that can handle .docx (not .doc) to HTML conversion client-side (in no particular order):
https://github.com/VolodymyrBaydalka/docxjs — works in the browser, docx only, complex layouts might be off, live demo
https://github.com/lalalic/docx2html (npm) — docx to html, most elements are supported, works in browser or nodejs
https://github.com/mwilliamson/mammoth.js — supports headings, lists, tables, endnotes, footnotes, images and text boxes
https://github.com/artburkart/docx2html — apparently, works in the browser
Note: If you are looking for the best way to convert a doc/docx file on the client side, then probably the answer is don't do it. If you really need to do it then do it server-side, i.e. with libreoffice in headless mode, apache-poi (java), pandoc etc.

A great solution if your data is confidential
Since the documents are confidential, they should not be processed on third-party resources.
This solution is open-source:
On the server-side: use Gotenberg to convert word & excel files to PDF.Note: Gotenberg works like a charm, it is based on the LibreOffice engine.
On the frontend: It's very easy to render the PDF file with javascript. (You can use libraries like: pdf.js, react-pdf, etc.)

ViewerJS is helpful to view/embed openoffice format like odt,odp,ods and also pdf.
For embed openoffice/pdf document
<iframe src = "/ViewerJS/#../demo/ohm2013.odp" width='700' height='550' allowfullscreen webkitallowfullscreen></iframe>
/ViewerJS/ is the path of ViewerJS
#../demo/ohm2013 is the path of your file want to embed

I think I have an idea.
This has been doing my nut in too and I'm still having trouble getting it to display in Chrome.
Save document(name.docx) in word as single file webpage (name.mht)
In your html use
<iframe src= "name.mht" width="100%" height="800"> </iframe>
Alter the heights and widths as you see fit.

If you wanted to pre-process your DOCX files, rather than waiting until runtime you could convert them into HTML first by using a file conversion API such as Zamzar. You could use the API to programatically convert from DOCX to HMTL, save the output to your server and then serve that HTML up to your end users.
Conversion is pretty easy:
curl https://api.zamzar.com/v1/jobs \
-u API_KEY: \
-X POST \
-F "source_file=#my.docx" \
-F "target_format=html5"
This would remove any runtime dependencies on Google & Microsoft's services (for example if they were down, or you were rate limited by them).
It also has the benefit that you could extend to other filetypes if you wanted (PPTX, XLS, DOC etc)

Native Documents (in which I have an interest) makes a viewer (and editor) specifically for Word documents (both legacy binary .doc and modern docx formats). It does so without lossy conversion to HTML. Here's how to get started https://github.com/NativeDocuments/nd-WordFileEditor/blob/master/README.md

PDFTron WebViewer supports rendering of Word (and other Office formats) directly in any browser and without any server side dependencies.
To test, try https://www.pdftron.com/webviewer/demo

You can also use some existing API's like GroupDocs.Viewer which can convert your document into image or html and then you will be able to display it in your own application.

Use Libre Office API
Here is an example
libreoffice --headless --convert-to html docx-file-path --outdir html-dir-path

Converting large XML file to relational database

I'm trying to figure out the best way to accomplish the following:
Download a large XML (1GB) file on daily basis from a third-party website
Convert that XML file to relational database on my server
Add functionality to search the database
For the first part, is this something that would need to be done manually, or could it be accomplished with a cron?
Most of the questions and answers related to XML and relational databases refer to Python or PHP. Could this be done with javascript/nodejs as well?
If this question is better suited for a different StackExchange forum, please let me know and I will move it there instead.
Below is a sample of the xml code:
<case-file>
<serial-number>123456789</serial-number>
<transaction-date>20150101</transaction-date>
<case-file-header>
<filing-date>20140101</filing-date>
</case-file-header>
<case-file-statements>
<case-file-statement>
<code>AQ123</code>
<text>Case file statement text</text>
</case-file-statement>
<case-file-statement>
<code>BC345</code>
<text>Case file statement text</text>
</case-file-statement>
</case-file-statements>
<classifications>
<classification>
<international-code-total-no>1</international-code-total-no>
<primary-code>025</primary-code>
</classification>
</classifications>
</case-file>
Here's some more information about how these files will be used:
All XML files will be in the same format. There are probably a few dozen elements within each record. The files are updated by a third party on a daily basis (and are available as zipped files on the third-party website). Each day's file represents new case files as well as updated case files.
The goal is to allow a user to search for information and organize those search results on the page (or in a generated pdf/excel file). For example, a user might want to see all case files that include a particular word within the <text> element. Or a user might want to see all case files that include primary code 025 (<primary-code> element) and that were filed after a particular date (<filing-date> element).
The only data entered into the database will be from the XML files--users won't be adding any of their own information to the database.

All steps could certainly be accomplished using node.js. There are modules available that will help you with each of these tasks:
node-cron: lets you easily set up cron tasks in your node program. Another option would be to set up a cron task on your operating system (lots of resources available for your favourite OS).
download: module to easily download files from a URL.
xml-stream: allows you to stream a file and register events that fire when the parser encounters certain XML elements. I have successfully used this module to parse KML files (granted they were significantly smaller than your files).
node-postgres: node client for PostgreSQL (I am sure there are clients for many other common RDBMS, PG is the only one I have used so far).
Most of these modules have pretty great examples that will get you started. Here's how you would probably set up the XML streaming part:
var XmlStream = require('xml-stream');
var xml = fs.createReadStream('path/to/file/on/disk'); // or stream directly from your online source
var xmlStream = new XmlStream(xml);
xmlStream.on('endElement case-file', function(element) {
// create and execute SQL query/queries here for this element
});
xmlStream.on('end', function() {
// done reading elements
// do further processing / query database, etc.
});

Are you sure you need to put the data in a relational database, or do you just want to search it in general?
There don't seem to be any actual relations in the data, so it might be simpler to put it in a document search index such as ElasticSearch.
Any automatic XML to JSON converter would probably produce suitable output. The large file size is an issue. This library, despite its summary saying "not streaming", is actually streaming if you inspect the source code, so it would work for you.

I had task with xml files as you wrote. This are principals I used:
All incoming files I stored as is in DB (XMLTYPE), because I need a source file info;
All incoming files parsed with XSL transformation. For example, I see that it is three entity here: fileInfo, fileCases, fileClassification. You can write XSL transformation to compile source file info in 3 entity types (in tags FileInfo, FileCases, FileClassification);
When you have output transformed XML you can make 3 procedures, that inserts data into DB (each entity in DB area).

Passing Value from C++ to Javascript

i have a c++ file which reads values from a sensor and I want to display those values on a website dynamically. So Im looking for a way to pass these values(integers) from my cpp file to an javascript which displays them on the site.
My first, simple try was to write the values into a js file as variables every second from my cpp script. The Js then uses this file as a source and displays its variables on the site:
cpp:
fprintf(file, "var mx=%d, my=%d, mz=%d, ax=%d, ay=%d, az=%d, gx=%d, gy=%d, gz=%d;\n",
imu.raw_m[0], imu.raw_m[1], imu.raw_m[2], // M = Magnetometer
imu.raw_a[0], imu.raw_a[1], imu.raw_a[2], // A = Accelerometer
imu.raw_g[0], imu.raw_g[1], imu.raw_g[2] // G = Gyroscope
);
html/js:
<script src="./imu.js" type="text/javascript"></script>
The Problem now is of course, that I need to refresh the page all the time, because the imu.js file is cached by the website.
I'd rather have a way to directly pass to integers from the cpp file to the js script. I read something about json or Googles V8 script. But I'd like to hear your suggestions first.
By the way, Im running this on a raspi, if this is important.
Thanks for your help
EDIT:
I'm goning to try it with a mysql database, in which my cpp file writes the data from the sensor with Connector/c++ from http://dev.mysql.com/doc/connector-cpp/en/ and my website reads them.

You could compile your C++ code into a Node.js plugin, you can then register a JavaScript function with your plugin which the C++ calls when it updates the value. That way you can pass values directly from C++ into Javascript in a managed and controlled way.
Node.js has the added benefit of being able to host your webpage and do all the Websocket and HTTP stuff that can be a pain in C++.

You do not have to refresh if your script is smart about how to access the data file! In case you do have a webserver at hand: Take care that your data file is accessible by your webserver and then let your script request the file via ajax (link to w3schools)

I'm doing something similar on a BeagleBone Black. With websocketd you can turn pretty much any program into a websocket endpoint and then send data via stdin and stdout commands. This would be a particularly good solution for you since websockets are designed to handle information that's constantly changing.

How to create a file using javascript in Mozilla Firefox

I want to write a function in javascript which creates a file and write some content to it, iam working with firefox, can anybody help me in this case.
Thanks...

You can write files in JavaScript in Firefox, but you have to use an XPCOM object (internal browser API). This is not allowed for JavaScript loaded from a web page, and it is intended to be used by JavaScript running inside a Firefox add-on (with high level of privileges).
There is a way for unprivileged (web page) JavaScript to request more privileges and if the user grants it (there will be a pop up dialog asking for permission), the web page code would be able to write to a file.
But before you read further, a warning:
This is not standard JavaScript and I would not recommend this approach unless you are developing a very specific application, that will be used in a very specific way (like for example, http://www.tiddlywiki.com/ a client-side JavaScript-HTML only wiki).
Requesting XPCOM privileges on a website is a bad practice! It's basicly equivalent to running an .exe you just downloaded from a site. You are asking a user to grant full access to their computer (read, write, execute) with the identity of the user running Firefox.
Request permission to use XPCOM (this will prompt the user for confirmation, no way to avoid it):
netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");
Then, write to a file using an XPCOM object (example code from Mozilla Developer Network):
1. // file is nsIFile, data is a string
2. var foStream = Components.classes["#mozilla.org/network/file-output-stream;1"].
3. createInstance(Components.interfaces.nsIFileOutputStream);
4.
5. // use 0x02 | 0x10 to open file for appending.
6. foStream.init(file, 0x02 | 0x08 | 0x20, 0666, 0);
7. // write, create, truncate
8. // In a c file operation, we have no need to set file mode with or operation,
9. // directly using "r" or "w" usually.
10.
11. // if you are sure there will never ever be any non-ascii text in data you can
12. // also call foStream.writeData directly
13. var converter = Components.classes["#mozilla.org/intl/converter-output-stream;1"].
14. createInstance(Components.interfaces.nsIConverterOutputStream);
15. converter.init(foStream, "UTF-8", 0, 0);
16. converter.writeString(data);
17. converter.close(); // this closes foStream
You can find more information about I/O in Firefox using XPCOM here: https://developer.mozilla.org/en-US/docs/Code_snippets/File_I_O

Javascript from websites cannot access the local file system.
If you like to store data either store it on the server or in a cookie.

writing to the file system directly from a browser is prohibited for security reasons. With html5 however it'll be possible to have offline storage support. Take a look here.
Grz, Kris.

Javascript executes in a client-side context.
http://www.tek-tips.com/viewthread.cfm?qid=1171273&page=1

There will be an API for this.. File Writer API. The early specification is here:
http://www.w3.org/TR/file-writer-api/
It is not implemented in any browser yet.
Update: It seems there already exists an implementation. Check out http://caniuse.com/filesystem and http://www.html5rocks.com/en/tutorials/file/filesystem/

While everyone who's responded that javascript does not have the ability to write files on a remote server are correct, and this is true for security reasons, what you want to accomplish may still be possible.
For example, if you wanted to make it possible to create a file on your website with the use of javascript, you can do so with some server side scripting language and and AJAX call.
Example:
You have a file on your server called update_last_access.php which will create a file which stores the last time the file was accessed in some arbitrary file.
If you then had your javascript function make an AJAX call out to that script, for instance, in jquery
$.get("update_last_access.php")
Then this would execute the server side script and write to the file.
Before any more help can be provided for you, you're going to have to clarify what you're trying to do.

You can read files from the filesystem in JavaScript with Firefox 3.6 - see my EPUB reader proof of concept, for example.
You can't write files directly from JavaScript, though. You have to go via a server.

Mozilla is planning to include FileSaver to Gecko 9:
https://bugzilla.mozilla.org/show_bug.cgi?id=557540