Develop Reference

List Files on a server via front-end javascript

A have a folder filled with files accessible to the end user, and am working on a javascript file to parse through them and deliver them as needed. However, rather than manually updating the list, I'd like the javascript to scan the folder and then list iterate through an array of the files in that folder. Is there a decent way in front-end JS to do this? All solutions I've looked into have turned out to be purely for Node.
For example, say I have a folder structure like so...
/ (Web Root)
|__ /Build_a_card
|__ /Cool pictures
|__ /Summer Pictures
summer_dog.gif
smiling_sun.svg
|__ /Winter Pictures
snowman.png
cat.jpg
And then in the javascript I'd run something like
var image_list = get_list("/Cool Pictures");
build_select_list(image_list);
function get_list(folder_to_look_in){
var the_list = ???
return the_list;
}
...
And then, for example, the JS is run, and after some parsing, the user would see...
<select>
<option value="summer_pictures/summer_dog.gif">summer_dog.gif</option>
<option value="summer_pictures/smiling_sun.svg">smiling_sun.svg</option>
<option value="winter_pictures/snowman.png">snowman.png</option>
<option value="cat.jpg">cat.jpg</option>
</select>
In an insane world, since the individual files in the folder are accessible to javascript, hypothetically I could brute-force every single possible file name in the folder and return success on each one:
function get_list(folder){
var list_of_files = {};
var starting_character = 0;
list_of_files = every_single_option({starting_character}, 0, 40, folder)
}
}
function every_single_option(existing_characters, current_depth, max_depth, folder){
this_string = String.fromCharCode(existing_characters);
if (request_url(this_string, folder)){
there_array[this_string] = this_string;
}
var there_array = {}
var i;
if (current_depth < max_depth){
while (i < 127){
let temp_array = there_array;
temp_array[i] = i;
mix_source(there_array, every_single_option(existing_characters, current_depth + 1, max_depth, folder))
}
}
return there_array;
}
function request_url(url, folder){
var oReq = new XMLHttpRequest();
oReq.addEventListener("load", reqListener);
oReq.open("GET", "/" + folder + "/" + url);
oReq.send();
}
function mix(source, target) {
for(var key in source) {
if (source.hasOwnProperty(key)) {
target[key] = source[key];
}
}
}
but as mentioned, doing it that way would be insane (both ridiculously slow and very bad code design, resorting to brute-forcing your own website is just dumb.)
but it does hypothetically prove that there's no reason javascript shouldn't be able to just get a directory listing assuming public permissions. Alternatively, I could make some API with the backend that allows fetching a JSON that lists it, but that's requiring backend code for something that's a frontend process. I'm trying to pull this off with something sane and simple, but the question is... how?
(If you insist on posting a jquery way to do this, please also post a non-jquery way as well as there is no jquery available in my environment.)

So, refusing to admit it's impossible, I engineered a solution that works, and requires no API.
That said, the server has to not be actively blocking the javascript from viewing the directory. In other words, the server hasn't turned indexing off, and the directory doesn't have an index.html or equivalent to rewrite any attempt to index, and the server isn't doing some url-rewriting. In other words, this should work in any server environment that doesn't rewrite or block indexes.
Here's a rough draft (still buggy, needs finished):
var request = new XMLHttpRequest();
request.open('GET', '/my/directory/', true);
request.onload = function() {
if (request.status >= 200 && request.status < 400) {
// Success!
var resp = request.responseText;
}
};
request.send();
var directory_listing = resp;
var regexp = /\b((?:[a-z][\w-]+:(?:\/{1,3}|[a-z0-9%])|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}\/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'".,<>?«»“”‘’]))/i;
var match, files = [];
while ((match = regexp.exec(resp)) != null) {
files.push(match.index);
}
console.log(files);

Building off lilHar's answer, we can use DOMParser to create a shadow-DOM for the directory page we're accessing, and then use that to find any links we need:
// relative path to the desired directory
const directory = "/DIRECTORY-NAME/";
// selector for the relevant links in the directory's index page
const selector = "LINK SELECTOR";
const request = new XMLHttpRequest();
request.open("GET", directory, true);
request.onload = () => {
// succesful response
if(request.status >= 200 && request.status < 400)
{
// create DOM from response HTML
const doc = new DOMParser().parseFromString(request.responseText, "text/html");
// get all links
const links = doc.querySelectorAll(selector);
console.log("Links:", links);
links.forEach(link => {
// do stuff with the links
});
}
};
request.send();

Is there a decent way in front-end JS to do this?
No. Nor is that a way that isn't decent.
The front end can communicate with the server via HTTP or WebSockets.
Neither of those provides any built-in mechanism for exploring a filesystem.
You need the server to provide an API (e.g. a web service) which provides the information you want.

By-pass virus scan for Google Drive links and get the confirm ID

With some help from this thread I came up with the code below. How can I fetch the Google Drive file ID, open the direct link to the file and snatch the virus scan confirm ID that is required to stream files over 100 MB and then puzzle back the link? I'm kind of stuck at the xhr part.
function fixGoogleDriveURL(url) {
if (url.indexOf('drive.google.com') !== -1) {
var DocIDfull = url;
var DocIDstart = DocIDfull.indexOf('open?id=');
if (DocIDstart == -1) {
// invalid
return url;
}
var DocID = DocIDfull.slice(DocIDstart+8);
url = 'https://drive.google.com/uc?export=download&id=' + DocID;
var xhr = new XMLHttpRequest();
xhr.onload = function () {
if (xhr.readyState === xhr.DONE) {
if (xhr.status === 200) {
var token = xhr.responseText.match("/confirm=([0-9A-Za-z]+)&/");
window.location.replace(url + '&confirm=' + token[1]);
// should I add url += '&confirm=' + token[1] here instead of window.location?
}
}
};
xhr.open("GET", url);
xhr.send();
}
return url;
}
console.log(fixGoogleDriveURL('https://drive.google.com/open?id=1C25uoL6nIqqNhex3wm8VwODsO2q2pXBt') + "\n<-- should output:\nhttps://drive.google.com/uc?export=download&id=1C25uoL6nIqqNhex3wm8VwODsO2q2pXBt&confirm=XXXXX");

Scraping GDrive using Client-Side JavaScript isn't explicitly allowed by Google and therefore your Ajax call/XHR fails.
The only way to get around that restriction is by using a proxy in the middle that will forward Google's Website code but add appropriate Access-Control Allow-Origin Headers.
You can either use your own server for that (some minimal server-side script code will do) or you can use a service like http://multiverso.me/AllOrigins/ or https://corsproxy.github.io/ to proxy the request for you.
The AllOrigins site has some example code for use with jQuery, but basically they work by URI encoding the URL you want to access and appending that string to the site's proxy URL.
Here's an article by freecodecamp.org that outlines how to use these services (skip to the Don’t Let CORS Stop You! section.
Note: A security advice: These services are working fine right now, but they could go out of business tomorrow and start serving malicious data instead or redirect your file requests to completely different files or completely different websites altogether. It's up to you to decide if you want to trust these strangers or not.

How to refresh a page whenever my json data file changes

So I've got this local file named 'data.json' containing various data. I want to refresh my page only when some data in the json file changes. Appreciate your help if you can explain me with bit of code. I searched all over internet, I couldnt find appropriate answer.

Create a timer, fetch the json file every X milliseconds. If the json contents has changed since the last fetch, reload the page. The sample code below uses JQuery to fetch the json file, and checks every 2000 milliseconds. Be sure the json file contains valid json.
<html>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
<script>
var previous = null;
var current = null;
setInterval(function() {
$.getJSON("data.json", function(json) {
current = JSON.stringify(json);
if (previous && current && previous !== current) {
console.log('refresh');
location.reload();
}
previous = current;
});
}, 2000);
</script>
</html>

Detecting a file change
Well, first, you have to trigger an event or something when the file changes. Some information about that can be found here: Check if file has changed using HTML5 File API
(Copied from the link) Something like that should do the job:
(function() {
var input;
var lastMod;
document.getElementById('btnStart').onclick = function() {
startWatching();
};
function startWatching() {
var file;
if (typeof window.FileReader !== 'function') {
display("The file API isn't supported on this browser yet.");
return;
}
input = document.getElementById('filename');
if (!input) {
display("Um, couldn't find the filename element.");
}
else if (!input.files) {
display("This browser doesn't seem to support the `files` property of file inputs.");
}
else if (!input.files[0]) {
display("Please select a file before clicking 'Show Size'");
}
else {
file = input.files[0];
lastMod = file.lastModifiedDate;
display("Last modified date: " + lastMod);
display("Change the file");
setInterval(tick, 250);
}
}
function tick() {
var file = input.files && input.files[0];
if (file && lastMod && file.lastModifiedDate.getTime() !== lastMod.getTime()) {
lastMod = file.lastModifiedDate;
alert("File changed: " + lastMod);
}
}
})();
Refreshing the page
In this case, the your problem is with the refresh. Usually a page can be refreshed using location.reload(), but in your case, refreshing the page will lose the connection to the file (the user will have to re-select it in the file input)
If you want to update some data using the new file, just retrigger it, but I strongly recommend to not refresh the page.
However, if you do want to refresh the page entirely, you can make a kind of a "helper-app" (A background application that will read the file continously and via websocket notify the Javascript when the file has changed).
You can do something like that using Websockets or $ajax (for jQuery) or XMLHttpRequest (non jQuery).
The helper app can be written in Java, C# or Python (C# for windows only) or any other language that HTTP server or Websocket server can be implemented in.

Check this stackOverflow question and answer
Is it possible to retrieve the last modified date of a file using Javascript?
If it's on the same server as your calling function you can use
XMLHttpRequest-
This example is not asynchronous, but you can make it so if you wish.
function fetchHeader(url, wch) {
try {
var req=new XMLHttpRequest();
req.open("HEAD", url, false);
req.send(null);
if(req.status== 200){
return req.getResponseHeader(wch);
}
else return false;
} catch(er) {
return er.message;
}
}
alert(fetchHeader(location.href,'Last-Modified'));
Refresh a page using javascript or html
Ways to refresh Page
Here are the first 20:
location = location
location = location.href
location = window.location
location = self.location
location = window.location.href
location = self.location.href
location = location['href']
location = window['location']
location = window['location'].href
location = window['location']['href']
location = window.location['href']
location = self['location']
location = self['location'].href
location = self['location']['href']
location = self.location['href']
location.assign(location)
location.replace(location)
window.location.assign(location)
window.location.replace(location)
self.location.assign(location)
and the last 10:
self['location']['replace'](self.location['href'])
location.reload()
location['reload']()
window.location.reload()
window['location'].reload()
window.location['reload']()
window['location']['reload']()
self.location.reload()
self['location'].reload()
self.location['reload']()
self['location']['reload']()
So simply Combine two and two together you get what you want
If you want to periodically check that
setInterval(function(){
//the function here
and compare and update last_mod_date var if there changes else keep it like that
}, 3000);
Reference date comparison Example Mozilla
var
nLastVisit = parseFloat(document.cookie.replace(/(?:(?:^|.*;)\s*last_modif\s*\=\s*([^;]*).*$)|^.*$/, "$1")),
nLastModif = Date.parse(document.lastModified);
if (isNaN(nLastVisit) || nLastModif > nLastVisit) {
document.cookie = "last_modif=" + Date.now() + "; expires=Fri, 31 Dec 9999 23:59:59 GMT; path=" + location.pathname;
if (isFinite(nLastVisit)) {
alert("This page has been changed!");
}
}

If you're using Node, it has a built-in fs.watch() function that basically checks to see if/when a file has changed. Otherwise, you'd likely want a setInterval to periodically get the JSON file via an AJAX call and update your variables/DOM. You could compare the old JSON object to the new one and if they're different, update the DOM/variables with the new data.

You want to examine your json file in a very thorough way, in order to understand if it has changed.
So what you should do is:
use jQuery getJSON() to load the initial data from your json file to a localStorage object.
then use jQuery getJSON() in a timed loop to get new data from your json file, compare them in-deep and very strict way with a little help from this awsome function posted as an answer in a similar question here. If your localStorage objects, initial JSON and new JSON match Object.deepEquals(initialJSON, newJSON) then no change was made, if not then refresh the page.

Simple server-side html/JS router for three apps

I'm not a javascript programmer but have a need to do very simple routing in my website's main index.htm file. Situation: We have, let's say, three main ISAPI applications, app1.dll, app2.dll and app3.dll, all hosted by the same in-house server.
These also represent three external domains, e.g., app1.com, app2.com and app3.com, all of which hit the same server and index.htm page. What I want to do in my index.htm page is to simply redirect to the appropriate server application without further user action (ie., using the same browser window).
Now, poking around I've learned that I can obtain and display the hostname with the following script:
<script>
document.getElementById("hostname").innerHTML =
"Hostname is: " + window.location.hostname + "<br/>";
</script>
Now what I want to do is something along the following (forgive any incorrect syntax):
<script>
if { hostname = "www.app1.com" {
var url = "http://www.app1.com/isapi/app1/app1.dll/";
} else {
if { hostname = "www.app2.com" {
var url = "http://www.app1.com/isapi/app1/app2.dll/";
} else {
if { hostname = "www.app3.com" {
var url = "http://www.app3.com/isapi/app1/app3.dll/";
} else {
}
}
}
"GO TO URL";
</script>
How do I encode "GO TO URL" without buttons, etc., and further user action? I believe I need a GET command, something like this:
var xhr = new XMLHttpRequest();
xhr.open('GET', url, false);
xhr.send();
return;
but I can get this to execute.
Any suggestions most welcome.

Do you just want to redirect the user to a certain url?
You can do that through the location api.
window.location = "http://stackoverflow.com"
This will redirect the user to stackoverflow.com
So in your case just do: window.location = url
And since the the window object is the global scope you can just do location = url
EDIT:
And I would also recommend you using a switch statement instead of the nested if statements. It will be alot more readable:
var url = "";
switch(hostname){
case "www.app1.com":
url = "url..."; break;
case "otherdomain":
url = "url..."; break;
default:
url = "url..."; //if none of the above ones
}
window.location = url;

How can I set the http RequestHeader for a file load in the $script.js library?

The $script.js library I am using
https://github.com/ded/script.js/
has the following Javascript. What I need to do is set the request header in this code block but I am not sure how to do this:
var el = doc.createElement("script"),
loaded = false;
el.onload = el.onreadystatechange = function () {
if ((el.readyState && el.readyState !== "complete" && el.readyState !== "loaded") || loaded) {
return false;
}
el.onload = el.onreadystatechange = null;
loaded = true;
// done!
};
el.async = true;
el.src = path;
document.getElementsByTagName('head')[0].insertBefore(el, head.firstChild);
Here's an example of something similar (not part of $script.js) that does set the request header:
var xmlhttp = new XMLHttpRequest();
xmlhttp.open("GET", "/Scripts/Pages/Home.js", false);
xmlhttp.setRequestHeader("X-Custom-Header", "My Values");
xmlhttp.send();
var m = document.createElement('script');
m.appendChild(document.createTextNode(xmlhttp.responseText));
document.getElementsByTagName('head')[0].appendChild(m);
The first code block (the one I need to use but can modify slightly) is not very clear to me. Does anyone know how I can change this first code blocks (used in $script.js) to set the request header?

If I'm interpreting the library correctly, XMLHttpRequest isn't used directly by $script.js. It appears to be creating <script></script> tags dynamically and letting the browser handle downloading the script files. Using this library, it does not appear you can specify custom headers.
Altering the request headers the browser sends via Javascript is not possible according to some of the articles I've read including the answer to this question on SO: Is it possible to alter http request's header using javascript?
If you want to set custom headers, you'll need to use a dynamic loading library which uses XHR to load the scripts. Here is another article I've found with patterns for this type of functionality: On-Demand Javascript.