Develop Reference

Filenames containing diacritics are not displayed properly using Javascript in NodeJs

So, after having not programmed for 23 years, I decided to start learning Javascript.
I am trying to write a program to read through my music files and create a HTML page based on the files found in a specific directory.
It goes well until I hit filenames containing diacritics in it (like é, ü, ø etc).
For Example: André Hazes turns into : André Hazes
For Example: Andrea Bocelli & Sarah Brightman - Time to Say Goodbye [Con Te Partirò] (single) turn into Andrea Bocelli & Sarah Brightman - Time to Say Goodbye [Con Te PartiroÌ€] (single)
The link I have created doesn't work anymore
The command I use to create the HTML statement is:
<td>${item.vFilename}</td>
This is the code I use to read the files from the filesystem. I work on a Mac, OS Catalina, so basically an Unix variant.
// List all files in a directory in Node.js recursively in a synchronous fashion
var ReadDirFiles = function(pdir, pfilelist) {
files = vFileSystem.readdirSync(pdir,"utf-8");
filelist = pfilelist;
files.forEach(function(file) {
if (vFileSystem.statSync(pdir + '/' + file).isDirectory()) {
filelist = ReadDirFiles(pdir + '/' + file, filelist);
}
else {
vstats = vFileSystem.statSync(pdir + '/' + file);
// debug info
// console.log(vstats);
filelist.push({vFilename: file, vDir: pdir, vBirthtime: formatDate(vstats.birthtime), vSize: vstats.size});
}
});
return filelist;
};
This is the statement I use to write the output to disk and it turns out the problem is in the write statement:
fs.writeFileSync(buildPathHtml.buildPathHtml(), html);
When the output is written back to disk, the conversion of the diacritics happens.
Anyone knows the trick how to work diacritics?

Try using encoding and decoding functions in your script. There are many functions doing this in Javascript, just use what you need. For a complete encoding/decoding you can use (or only copy & paste) into your script the code suggested in https://www.strictly-software.com/htmlencode/. There is a little encoding Javascript library doing the job (encoder.js).

Maybe your file system isn't utf8 based. I read somewhere it's Western for Mac.

Not able to render pdf by passing Uint8Array in the viewer.html file parameter while using pdfjs

I am trying to create an app with electron and react in which I am trying to render a pdf using pdfjs from the local filesystem. I will get the pdf data from electron which will then be passed to react app.
After going through the code for viewer.js file I can see that I can send the file parameter value as Uint8Array and in turn, it will call the PDFJS's getDocument with data, so I tried a little example but somehow I am not able to get it working.
Pasting the code below
base64ToUint8Array = () => {
// taken from pdfjs github example
let raw = atob(
'JVBERi0xLjcKCjEgMCBvYmogICUgZW50cnkgcG9pbnQKPDwKICAvVHlwZSAvQ2F0YWxvZwog' +
'IC9QYWdlcyAyIDAgUgo+PgplbmRvYmoKCjIgMCBvYmoKPDwKICAvVHlwZSAvUGFnZXMKICAv' +
'TWVkaWFCb3ggWyAwIDAgMjAwIDIwMCBdCiAgL0NvdW50IDEKICAvS2lkcyBbIDMgMCBSIF0K' +
'Pj4KZW5kb2JqCgozIDAgb2JqCjw8CiAgL1R5cGUgL1BhZ2UKICAvUGFyZW50IDIgMCBSCiAg' +
'L1Jlc291cmNlcyA8PAogICAgL0ZvbnQgPDwKICAgICAgL0YxIDQgMCBSIAogICAgPj4KICA+' +
'PgogIC9Db250ZW50cyA1IDAgUgo+PgplbmRvYmoKCjQgMCBvYmoKPDwKICAvVHlwZSAvRm9u' +
'dAogIC9TdWJ0eXBlIC9UeXBlMQogIC9CYXNlRm9udCAvVGltZXMtUm9tYW4KPj4KZW5kb2Jq' +
'Cgo1IDAgb2JqICAlIHBhZ2UgY29udGVudAo8PAogIC9MZW5ndGggNDQKPj4Kc3RyZWFtCkJU' +
'CjcwIDUwIFRECi9GMSAxMiBUZgooSGVsbG8sIHdvcmxkISkgVGoKRVQKZW5kc3RyZWFtCmVu' +
'ZG9iagoKeHJlZgowIDYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDEwIDAwMDAwIG4g' +
'CjAwMDAwMDAwNzkgMDAwMDAgbiAKMDAwMDAwMDE3MyAwMDAwMCBuIAowMDAwMDAwMzAxIDAw' +
'MDAwIG4gCjAwMDAwMDAzODAgMDAwMDAgbiAKdHJhaWxlcgo8PAogIC9TaXplIDYKICAvUm9v' +
'dCAxIDAgUgo+PgpzdGFydHhyZWYKNDkyCiUlRU9G');
let uint8Array = new Uint8Array(new ArrayBuffer(raw.length));
for (var i = 0; i < raw.length; i++) {
uint8Array[i] = raw.charCodeAt(i);
}
return uint8Array;
render(){
let pdfUint8Array = this.base64ToUint8Array();
return(
<iframe
src={`/web/viewer.html?file=${pdfUint8Array}`}
title="frame"
width="100%"
height="700px"
/>
);
}
From the error on the browser console I can see that the file param value is still taken as url hence pdf not getting rendered.
Error
fetch_stream.js:101 GET http://localhost:3001/web/37,80,68,70,45,49,46,55,10,10,49,32,48,32,111,98,106,32,32,37,32,101,110,116,114,121,32,112,111,105,110,116,10,60,60,10,32,32,47,84,121,112,101,32,47,67,97,116,97,108,111,103,10,32,32,47,80,97,103,101,115,32,50,32,48,32,82,10,62,62,10,101,110,100,111,98,106,10,10,50,32,48,32,111,98,106,10,60,60,10,32,32,47,84,121,112,101,32,47,80,97,103,101,115,10,32,32,47,77,101,100,105,97,66,111,120,32,91,32,48,32,48,32,50,48,48,32,50,48,48,32,93,10,32,32,47,67,111,117,110,116,32,49,10,32,32,47,75,105,100,115,32,91,32,51,32,48,32,82,32,93,10,62,62,10,101,110,100,111,98,106,10,10,51,32,48,32,111,98,106,10,60,60,10,32,32,47,84,121,112,101,32,47,80,97,103,101,10,32,32,47,80,97,114,101,110,116,32,50,32,48,32,82,10,32,32,47,82,101,115,111,117,114,99,101,115,32,60,60,10,32,32,32,32,47,70,111,110,116,32,60,60,10,32,32,32,32,32,32,47,70,49,32,52,32,48,32,82,32,10,32,32,32,32,62,62,10,32,32,62,62,10,32,32,47,67,111,110,116,101,110,116,115,32,53,32,48,32,82,10,62,62,10,101,110,100,111,98,106,10,10,52,32,48,32,111,98,106,10,60,60,10,32,32,47,84,121,112,101,32,47,70,111,110,116,10,32,32,47,83,117,98,116,121,112,101,32,47,84,121,112,101,49,10,32,32,47,66,97,115,101,70,111,110,116,32,47,84,105,109,101,115,45,82,111,109,97,110,10,62,62,10,101,110,100,111,98,106,10,10,53,32,48,32,111,98,106,32,32,37,32,112,97,103,101,32,99,111,110,116,101,110,116,10,60,60,10,32,32,47,76,101,110,103,116,104,32,52,52,10,62,62,10,115,116,114,101,97,109,10,66,84,10,55,48,32,53,48,32,84,68,10,47,70,49,32,49,50,32,84,102,10,40,72,101,108,108,111,44,32,119,111,114,108,100,33,41,32,84,106,10,69,84,10,101,110,100,115,116,114,101,97,109,10,101,110,100,111,98,106,10,10,120,114,101,102,10,48,32,54,10,48,48,48,48,48,48,48,48,48,48,32,54,53,53,51,53,32,102,32,10,48,48,48,48,48,48,48,48,49,48,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,48,55,57,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,49,55,51,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,51,48,49,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,51,56,48,32,48,48,48,48,48,32,110,32,10,116,114,97,105,108,101,114,10,60,60,10,32,32,47,83,105,122,101,32,54,10,32,32,47,82,111,111,116,32,49,32,48,32,82,10,62,62,10,115,116,97,114,116,120,114,101,102,10,52,57,50,10,37,37,69,79,70 404 (Not Found)
I can also see while console logging the file param as file [object ArrayBuffer] if I pass pdfUint8Array.buffer in the file param but the pdf is still isn't rendered as it is taken as a string as pdfjs try to display it with url.

CodeSAndbox Demo (open the result of the codesandbox in a new tab for it
to work)
Hey Hiten,
Apparently in mozilla official documents for PDF.js, They say that no need to convert to Byte array (uint8Array), you can parse the base64 code into pdf directly....
Just render the iframe with data:application/pdf;base64 before the raw base64 data like this and it will appear :)
<iframe
title="frame"
width="300px"
height="700px"
src={`data:application/pdf;base64,${this.base64PDF()}`} />
A good reference in another Stackoverflow Thread discussing rendering a pdf file using a base64 instead of url
I hope I have helped you ✌

My main aim here was to render the pdf from my local filesystem in the electron app that I am creating.
My earlier approach was to pass on the raw base64 data of the pdf in an iframe with src as web/viewer.html of the pdfjs but due to some limitation in electron, this wasn't working.
So, I figured out an easy solution that while starting electron I ran a node server on the directory from where I wanted to serve the pdf files and used the URL in the form http://localhost/{port_number}/{path_to_my_pdf_files} to access the pdf in pdfjs
<iframe
src={`/web/viewer.html?file=http://localhost:${process.env.REACT_APP_PDF_SERVER_PORT}/${path}`}
title="iframe"
width="100%"
height="700px"
/>

JXA get file's Size & checksum?

I am using JXA (javascript for automation) on my Mac to try to automate iTunes Connect screenshot uploads. Because of this, I want to automate grabbing each image and uploading it, but for iTunes Connect to allow this (using iTMSTransporter), I need to edit a XML file and add in each image's size in bits, as well as get the checksum (type="md5").
I know I could do this manually, but I want to automate it as this will save me a lot of time in the long run, with tons of different screenshots for each localization.
I've used JXA to grab the images, and get their dimensions, but can't figure out getting the size & checksum. Maybe someone can help? Or if not using JXA, maybe there is some sort of other script that JXA can run (like a shell script, which I have no experience with what so ever as of now), or maybe some script I could just run ahead of time and export the XML to a file manually. From there I could use JXA to parse that file.
Here is what I have so far for what it takes to get the image file:
desktopPath = finder.desktop.url();
desktopPath = desktopPath.substring(7, desktopPath.length);
var imagePath = Application('System Events').folders.byName(desktopPath + '/myImage.png');
imageEvents = Application("Image Events");
imageEvents.launch();
imageEvents.name();
img = imageEvents.open(Path(imagePath));
// now I don't know what to do with the image as the documentation is quite difficult for me to understand

I figured it out. I had to use shell scripts to access this info. Idk if there is another way, but this way worked...
// to get the size (newString is the path (as a string) to the file I am getting the size for
var theSize = app.doShellScript("stat -f%z " + newString.replace(" ", "\\ "));
// to get the MD5 checksum (newString is again the path (as a string) to the file I am getting the checksum for
var md5String = newString;
md5String = md5String.replace(" ", "\\ ");
var checksum = app.doShellScript("md5 " + md5String);
checksum = checksum.replace(/\/$/, "").split(' ').pop();
// I popped because I had to format the returned string so it's just the MD5 and not the file path as well. Maybe there is an easier way in shell script, but I'm a newbie to shell scripting

How can I force the browser/javascript to clear/ignore a cached file?

I have a Javascript program that extracts data from a text file (just a plain *.txt file) and then displays it in a table. It works as intended except for one issue: If I update the text file where the data lives in the update does not shows up on the table. The reason is that the file is being cached by the browser.
Is there a way to force Javascript to read from the latest version of the file and not from the cached version? Again, this is not a problem with the javascript file as doing Ctrl-5 and or Shift+Ctrl+R does not works and also updating the javascript file itself behaves as expected. it is only the file where the data is in that is the problem.
To read the text file I use the Webix toolkit which uses AJAX to read the file. The reading and parsing of the text file is done via Javascript and the Webix toolkit. There is very little html in my program.
//the name of the file is dependant of an user specified input
var theURL = rossObj.store_number.store_number + "macaddress.txt ";
webix.ajax(theURL,{
//response
error:function(text, xml, XmlHttpRequest){
webix.alert({ ok:"OK", type:"alert-warning", text:"could not open the following URL --> "+theURL, width:"400px"});
return "mac_address_error";
},
success:function(text, xml, XmlHttpRequest){
//parse the file;
}
});

Try adding a random number to the end of the url as a query string, the server will ignore but the browser will treat it as a new file
var theURL = rossObj.store_number.store_number +
"macaddress.txt?" +
Math.floor((Math.random() * 10000) + 1);

How to fetch file content (basically read) a local file in javascript for UIAutomation iOS

Is there a possible way to read a local file in JavaScript.
MyFolder:
db.csv
Parse.js
Trying to fetch the contents of file db.csv in Parse.js, But in vain.
Can you share some links where I can get enough knowledge how to read a file.
Running Instruments in Xcode5, with test scripts in .js file where I have to feed in some values from a .csv file.

iOS UIAutomation, apple provides an api for running a task on the target's host.
performTaskWithPathArgumentsTimeout
Using this, we can have a bash script to printout the contents of a file that we wanted to fetch in the first case.
Bash script can be as simple as this for this requirement.
#! /bin/bash
FILE_NAME="$1"
cat $FILE_NAME
Save it as for example FileReader.sh file.
And in your automation script,
var target = UIATarget.localTarget();
var host = target.host();
var result = host.performTaskWithPathArgumentsTimeout(executablePath,[filePath,fileName], 15);
UIALogger.logDebug("exitCode: " + result.exitCode);
UIALogger.logDebug("stdout: " + result.stdout);
UIALogger.logDebug("stderr: " + result.stderr);
where in,
executablePath is where the command need to be executed.
var executablePath = "/bin/sh";
filePath is the location of the created FileReader.sh file. When executed, outputs the content to standard output (in our requirement).
[give full absolute path of the file]
fileName is the actual file to fetch contents from.
[give full absolute path of the file] In my case I had a Contents.csv file, which I had to read.
and the last parameter is the timeout in seconds.
Hope this helps others, trying to fetch contents (reading files) for performing iOS UIAutomation.
References:
https://stackoverflow.com/a/19016573/344798
https://developer.apple.com/library/iOS/documentation/UIAutomation/Reference/UIAHostClassReference/UIAHost/UIAHost.html

If the file is on the same domain as the site you're in, you'd load it with Ajax. If you're using Ajax, it's be something like
$.get('db.csv', function(csvContent){
//process here
});
Just note that the path to the csv file will be relative to the web page you're in, not the JavaScript file.
If you're not using jQuery, you'd have to manually work with an XmlHttpRequest object to do your Ajax call.
And though your question doesn't (seem to) deal with it, if the file is located on a different domain, then you'd have to use either jsonP or CORS.
And, just in case this is your goal, no, you can't, in client side JavaScript open up some sort of Stream and read in a file. That would be a monstrous security vulnerability.

This is a fairly simple function in Illuminator's host functions library:
function readFromFile(path) {
var result = target.host().performTaskWithPathArgumentsTimeout("/bin/cat", [path], 10);
// be verbose if something didn't go well
if (0 != result.exitCode) {
throw new Error("readFromFile failed: " + result.stderr);
}
return result.stdout;
}
If you are using Illuminator, this is host().readFromFile(path).