capture file into javascript variable instead of downloading it - javascript

I am writing a app that visits a web site that can give me a link to a file, like this: http://www.thrustcurve.org/download.jsp?id=2199
If I visit this link, a small text file is downloaded. What I would like to do instead is to capture this text into a javascript variable so I can search around in it and extract the data I need.
Is this even possible?
Further details: although I am old and have lots of programming experience, I am a total noob in the javascript/web/server/modern space (think FORTRAN 77).
I now teach high school physics and am trying to build a web-based rocket simulator for my students to use on their chromebooks. The creator of thrustcurve.org has generously made data about rocket motors available on the web, but I need some bits that can only be found inside these little text files. Maybe it would be possible to work with the downloaded files on the chrome books, but I really have no idea how to begin there. If you are patient enough to have read this far, you can see the kind of javascript I have been able to accomplish at noragulfa.com

You can use XMLHttpRequest to perform HTTP requests, but due to security restrictions the browser blocks requests to “external domains” (thus, you can download files only from your domain). For more info, read about Cross-Origin Resource Sharing (CORS).
To solve your task, you have several options:
1) Download required files from thrustcurve.org and store them on your server. This is the best option since you will not be dependent on an external server (besides, hotlinking may upset the thrustcurve.org owner). In this case XMLHttpRequest will be able to access files using relative URLs:
var url = '/thrustcurve-downloads/Estes_A8.eng';
2) Contact the thrustcurve.org owner and ask him to enable Access-Control-Allow-Origin from anywhere. In this case XMLHttpRequest will be able to access files using full URLs:
var url = 'http://www.thrustcurve.org/download.jsp?id=2199';
3) Create a proxy that passes HTTP requests to thrustcurve.org. For example, since you are using nginx, you can simple add the following to your configuration file:
location /thrustcurve {
proxy_pass http://www.thrustcurve.org/;
}
In this case XMLHttpRequest will be able to access files using relative URLs:
var url = '/thrustcurve/download.jsp?id=2199';
4) Use third-party proxies (not a very reliable solution, but great for tests). As an example, I will use this option.
var url = 'http://cors-anywhere.herokuapp.com/http://www.thrustcurve.org/download.jsp?id=2199';
var xhr = new XMLHttpRequest();
xhr.onload = function () {
console.log(xhr.response);
};
xhr.open('GET', url);
xhr.responseType = 'text';
xhr.send();
UPD: A full example how to download files using a XMLHttpRequest and PHP.
1) Create the file thrustcurve.php on your root server with the following contents:
<?php
// Change this to FALSE if don't want to store files locally
$store_files_locally = true;
$id = (int) filter_input(INPUT_GET, 'id');
if ($id > 0) {
if ($store_files_locally) {
// Specify the directory where you want to store engine files
// It will create the directory if it doesn't exist
$dir = __DIR__ . '/thrustcurve-downloads';
if (!is_dir($dir) && !mkdir($dir, true, 0777)) {
http_response_code(500);
die('Cannot create the downloads directory');
}
// If file exists, load the engine from the local file
$file = "{$dir}/{$id}.eng";
if (is_file($file)) {
$engine = file_get_contents($file);
die($engine);
}
}
// Download the engine file from the remote server
$url = "http://www.thrustcurve.org/download.jsp?id={$id}";
$engine = trim(#file_get_contents($url));
// The downloaded file is considered valid engine only if it starts with semicolon
if (strpos($engine, ';') === 0) {
if ($store_files_locally) {
file_put_contents($file, $engine);
}
die($engine);
}
}
http_response_code(404);
echo "File #{$id} not found";
2) To download files using JavaScript, use the following:
var xhr = new XMLHttpRequest();
xhr.onload = function () {
if (xhr.status === 200) {
console.log(xhr.response);
} else {
console.error(xhr.response);
}
};
xhr.open('GET', '/thrustcurve.php?id=2198');
xhr.responseType = 'text';
xhr.send();

Related

How to read text file from filesystem in JavaScript

I have tried different approaches to read a text file from the local file system in JavaScript and display the content of the file in alert() but all to no avail.
Approach 1
function readTextFile(file) {
var rawFile = new XMLHttpRequest();
rawFile.open("GET", file , false);
rawFile.onreadystatechange = function () {
if (rawFile.readyState === 4) {
if (rawFile.status === 200 || rawFile.status == 0) {
var allText = rawFile.response;
document.getElementById("content").innerText = allText;
alert(allText);
}
} else {
alert("Can't read the file");
}
}
rawFile.send(null);
}
readTextFile("FormulaQuestion.txt");
The FormulaQuestion.txt file is in the same directory with the html file, this approach shows an empty alert window on the browser
Approach 2 using fetch method
fetch('FormulaQuestion.txt')
.then(response => response.text())
.then((data) => {
alert(data);
})
This doesn't show anything
Approach 3 using JQuery
$.get('FormulaQuestion.txt', function (data) {
alert(data)
}, 'text');
This doesn't work either.
I am building a desktop application that uses a web browser control to load html file which is embedded into the application. The application reads the string from sqlite database and save it in the FormulaQuestion.txt file, then refreshes the WebControl component which reloads the html file.
Now when the html file is reloaded, the JavaScript should read the text file and display it on alert() which once the alert is able to display the file content, i will then set it to a paragraph and remove the alert().
Please someone should help me out.
Browsers by design do not allow access to the file system for JavaScript, as allowing such access would be a serious security concern.
To provide the FormulaQuestion.txt file to your script you will need to host the file on a server and request it via a HTTP request (like with your fetch). The key thing here is that a server is needed to actually transmit the file over the HTTP protocol to your script.
If working locally, there are many options for running a local server.
The npm serve module,
Wamp
Apache
You may also want to try out some free tier services like Vercel or Netlify. Both I believe allow you to just drag/drop a file and it will host it for you.

List Files on a server via front-end javascript

A have a folder filled with files accessible to the end user, and am working on a javascript file to parse through them and deliver them as needed. However, rather than manually updating the list, I'd like the javascript to scan the folder and then list iterate through an array of the files in that folder. Is there a decent way in front-end JS to do this? All solutions I've looked into have turned out to be purely for Node.
For example, say I have a folder structure like so...
/ (Web Root)
|__ /Build_a_card
|__ /Cool pictures
|__ /Summer Pictures
summer_dog.gif
smiling_sun.svg
|__ /Winter Pictures
snowman.png
cat.jpg
And then in the javascript I'd run something like
var image_list = get_list("/Cool Pictures");
build_select_list(image_list);
function get_list(folder_to_look_in){
var the_list = ???
return the_list;
}
...
And then, for example, the JS is run, and after some parsing, the user would see...
<select>
<option value="summer_pictures/summer_dog.gif">summer_dog.gif</option>
<option value="summer_pictures/smiling_sun.svg">smiling_sun.svg</option>
<option value="winter_pictures/snowman.png">snowman.png</option>
<option value="cat.jpg">cat.jpg</option>
</select>
In an insane world, since the individual files in the folder are accessible to javascript, hypothetically I could brute-force every single possible file name in the folder and return success on each one:
function get_list(folder){
var list_of_files = {};
var starting_character = 0;
list_of_files = every_single_option({starting_character}, 0, 40, folder)
}
}
function every_single_option(existing_characters, current_depth, max_depth, folder){
this_string = String.fromCharCode(existing_characters);
if (request_url(this_string, folder)){
there_array[this_string] = this_string;
}
var there_array = {}
var i;
if (current_depth < max_depth){
while (i < 127){
let temp_array = there_array;
temp_array[i] = i;
mix_source(there_array, every_single_option(existing_characters, current_depth + 1, max_depth, folder))
}
}
return there_array;
}
function request_url(url, folder){
var oReq = new XMLHttpRequest();
oReq.addEventListener("load", reqListener);
oReq.open("GET", "/" + folder + "/" + url);
oReq.send();
}
function mix(source, target) {
for(var key in source) {
if (source.hasOwnProperty(key)) {
target[key] = source[key];
}
}
}
but as mentioned, doing it that way would be insane (both ridiculously slow and very bad code design, resorting to brute-forcing your own website is just dumb.)
but it does hypothetically prove that there's no reason javascript shouldn't be able to just get a directory listing assuming public permissions. Alternatively, I could make some API with the backend that allows fetching a JSON that lists it, but that's requiring backend code for something that's a frontend process. I'm trying to pull this off with something sane and simple, but the question is... how?
(If you insist on posting a jquery way to do this, please also post a non-jquery way as well as there is no jquery available in my environment.)
So, refusing to admit it's impossible, I engineered a solution that works, and requires no API.
That said, the server has to not be actively blocking the javascript from viewing the directory. In other words, the server hasn't turned indexing off, and the directory doesn't have an index.html or equivalent to rewrite any attempt to index, and the server isn't doing some url-rewriting. In other words, this should work in any server environment that doesn't rewrite or block indexes.
Here's a rough draft (still buggy, needs finished):
var request = new XMLHttpRequest();
request.open('GET', '/my/directory/', true);
request.onload = function() {
if (request.status >= 200 && request.status < 400) {
// Success!
var resp = request.responseText;
}
};
request.send();
var directory_listing = resp;
var regexp = /\b((?:[a-z][\w-]+:(?:\/{1,3}|[a-z0-9%])|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}\/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'".,<>?«»“”‘’]))/i;
var match, files = [];
while ((match = regexp.exec(resp)) != null) {
files.push(match.index);
}
console.log(files);
Building off lilHar's answer, we can use DOMParser to create a shadow-DOM for the directory page we're accessing, and then use that to find any links we need:
// relative path to the desired directory
const directory = "/DIRECTORY-NAME/";
// selector for the relevant links in the directory's index page
const selector = "LINK SELECTOR";
const request = new XMLHttpRequest();
request.open("GET", directory, true);
request.onload = () => {
// succesful response
if(request.status >= 200 && request.status < 400)
{
// create DOM from response HTML
const doc = new DOMParser().parseFromString(request.responseText, "text/html");
// get all links
const links = doc.querySelectorAll(selector);
console.log("Links:", links);
links.forEach(link => {
// do stuff with the links
});
}
};
request.send();
Is there a decent way in front-end JS to do this?
No. Nor is that a way that isn't decent.
The front end can communicate with the server via HTTP or WebSockets.
Neither of those provides any built-in mechanism for exploring a filesystem.
You need the server to provide an API (e.g. a web service) which provides the information you want.

By-pass virus scan for Google Drive links and get the confirm ID

With some help from this thread I came up with the code below. How can I fetch the Google Drive file ID, open the direct link to the file and snatch the virus scan confirm ID that is required to stream files over 100 MB and then puzzle back the link? I'm kind of stuck at the xhr part.
function fixGoogleDriveURL(url) {
if (url.indexOf('drive.google.com') !== -1) {
var DocIDfull = url;
var DocIDstart = DocIDfull.indexOf('open?id=');
if (DocIDstart == -1) {
// invalid
return url;
}
var DocID = DocIDfull.slice(DocIDstart+8);
url = 'https://drive.google.com/uc?export=download&id=' + DocID;
var xhr = new XMLHttpRequest();
xhr.onload = function () {
if (xhr.readyState === xhr.DONE) {
if (xhr.status === 200) {
var token = xhr.responseText.match("/confirm=([0-9A-Za-z]+)&/");
window.location.replace(url + '&confirm=' + token[1]);
// should I add url += '&confirm=' + token[1] here instead of window.location?
}
}
};
xhr.open("GET", url);
xhr.send();
}
return url;
}
console.log(fixGoogleDriveURL('https://drive.google.com/open?id=1C25uoL6nIqqNhex3wm8VwODsO2q2pXBt') + "\n<-- should output:\nhttps://drive.google.com/uc?export=download&id=1C25uoL6nIqqNhex3wm8VwODsO2q2pXBt&confirm=XXXXX");
Scraping GDrive using Client-Side JavaScript isn't explicitly allowed by Google and therefore your Ajax call/XHR fails.
The only way to get around that restriction is by using a proxy in the middle that will forward Google's Website code but add appropriate Access-Control Allow-Origin Headers.
You can either use your own server for that (some minimal server-side script code will do) or you can use a service like http://multiverso.me/AllOrigins/ or https://corsproxy.github.io/ to proxy the request for you.
The AllOrigins site has some example code for use with jQuery, but basically they work by URI encoding the URL you want to access and appending that string to the site's proxy URL.
Here's an article by freecodecamp.org that outlines how to use these services (skip to the Don’t Let CORS Stop You! section.
Note: A security advice: These services are working fine right now, but they could go out of business tomorrow and start serving malicious data instead or redirect your file requests to completely different files or completely different websites altogether. It's up to you to decide if you want to trust these strangers or not.

Is it possible to serve a string via a URL request from one Javascript to another?

I am not even sure that what I ask is possible at all, but maybe somebody would be able to give me some ideas...
Here is the situation. I have a certain third-party Javascript module which is being run in browser. The module exposes an API call which loads some configuration (XML file) from a provided URL. I would like to pass it instead some XML which I generate in my own script. I cannot change the module; so, what I am looking for is some way to specify some URL which will serve my local data. Is there any way to do something like this?
Testing with the "javascript:" pseudo protocol didn't work - making it look like XMLHttpRequest does not accept "pseudo" protocols.
Testing with data URIs did work:
var xmlSource =
`<?xml version="1.0" encoding="UTF-8"?>
<text>
<para>hello world</para>
</text>
`;
var xmlDataURL = "data:text/xml," + xmlSource;
loadConfig( xmlDataURL);
was succesfully sent using a dummy version of loadConfig:
// Dummy loadConfig:
function loadConfig( url)
{ var req = new XMLHttpRequest();
req.open("GET", url);
req.onreadystatechange = function ()
{ if( req.readyState == 4)
console.log(req.responseText);
};
req.send();
}

Javascript: can't load JSON file from localhost

I'm currently working through the book "Head first HTML5 programming". I want to load the content of a file named sales.json from a web server on my own machine. I used wampserver for this.
In the folder wamp/www/gumball/ I put all relevant .html, .js and .css files, and also the sales.json file.
My JavaScript code is very simple:
window.onload = function() {
var url = "http://localhost/gumball/sales.json";
var request = new XMLHttpRequest();
request.open("GET", url);
request.onload = function() {
if (request.status == 200) {
updateSales(request.responseText);
}
};
request.send(null);
}
function updateSales(responseText) {
var salesDiv = document.getElementById("sales");
salesDiv.innerHTML = responseText;
}
This doesn't do anything! Typing the link: http://localhost/gumball/sales.json in my browser opens the right file, so the link should be correct. Even when using the .js files that come with the book (with a finished version of the application I'm trying to make), nothing loads.
Testing with alert statements tells me the request.onload event never happens. I'm clueless as to why this is the case.
A fact I don't quite understand yet: when I type: http://localhost/gumball/sales.json: in my browser (I added a colon at the end of the link), I get a 403 Forbidden error! Why does this happen? Does this have something to do with my problem?
I open html document with firefox
Your HTML document must be open with a URL in http://, not file://, if you want it to be able to open in javascript another document, unless the second document is served with relevant CORS headers.
This is due to same origin policy.
As you have a local WAMP server, there is no problem : simply open your file using a http:// URL like you do for your JSON file.

Categories

Resources