ESP8266 serving HTML+js - javascript

I try to host an HTML file on an esp8266 access point. I can properly show an .html file. Unfortunately, when accessing the html page, my browser cannot display javascript content. Strangely, when I work locally on my machine - it works perfectly fine. When I access the page on the esp8266 I receive the error
"Not found: dygraph.min.js."
Obviously, the browser does not find the javascript source. I wonder why. I have tried out several ways of naming and referencing, but I was not lucky until now.
I upload the files with the ESP8266 Sketch Data Upload tool to the SPIFFS. In the html file I reference the js as <script type="text/javascript"
src="dygraph.min.js"></script>.
Did anybody experience anything like this before? The whole code can be found here:
https://github.com/JohnnyMoonlight/esp8266-AccessPoint-Logger-OfflineVisualisation
I am looking forward for your input!
Thanks and best!

Take a read through your code, and imagine the requests that will be made of your web server.
Your code is written to handle requests for two URLs: / and /temp.csv - that's it.
When /temp.csv is accessed, you serve the contents of index.html. When the browser interprets that file it will try to load /dygraph.min.js from your ESP. You don't have a handler for that file. So the load fails.
You need to add a handler for it and then serve the file. So you'll need to add a line like:
server.on("/dygraph.min.js", handleJS);
and define function void handleJS() that does what handleFile() does.
You'll need to do the same thing for the /dygraph.css; you don't have a handler for it either.
I would do it this way:
void handleHTML() {
handleFile("index.html");
}
void handleJS() {
handleFile("dygraph.min.js");
}
void handleCSS() {
handleFile("dygraph.css");
}
void handleFile(char *filename) {
File f = SPIFFS.open(filename, "r");
// the rest of your handleFile() code here
}
and in your setup():
server.on("/", handleRoot);
server.on("/temp.csv", handleHTML);
server.on("/dygraph.css", handleCSS);
server.on("/dygraph.min.js", handleJS);
Separately:
Your URL to file mappings are messed up. The code I shared above is consistent with what you have now, but normally you'd want / to serve index.html; you have it serving a fragment of HTML.
Normally /temp.csv would serve a comma-separated value file. I see you have one, in the repo and you have code to add data to it; you're just not serving it. Right now you have that serving index.html. Once you start successfully loading the Javascript you'll have problems with that.
You'll need to sort those out to get this working right.
Also, in loop() you should move server.handleClient(); to be the first thing in the loop. The way you have it written you're only checking to see if there's a web request if it's time to take another temperature reading. You should always check to see if there's a web request, otherwise you're unnecessarily slowing down web service.
One last thing, completely separate from the web server code, and I wouldn't worry about this till you get the rest of your code working: your code is writing to SPIFFS roughly every 5 seconds. SPIFFS is stored in flash memory on the ESP8266. ESP8266 boards use cheap flash memory that doesn't last a long time - it wears out after maybe 10,000 to 100,000 write cycles (this is a little complicated; it's broken into "pages" and the individual cells in the pages wear out, but you have to write the entire page at the same time).
It's hard to say for sure what its lifetime will be; it depends on the specific ESP8266 boards and flash chips involved. 10,000 write cycles means the flash memory on your board might start failing after 50,000 seconds - 100,0000 write cycles would give you about 500,000 writes -- if you keep writing to the same spot. It depends on how often the same place in flash is getting written to. If that's a problem for you, you might want to increase the delay between writes or do something else with your data.
You might not run into this because you're appending to a file - you'll still rewrite the same blocks of flash memory many times, but not 10,000 times - unless you often remove the CSV file and start over. So this might be a problem for you long term or might not.
You can read more about these problems at https://design.goeszen.com/mitigating-flash-wear-on-the-esp8266-or-any-other-microcontroller-with-flash.html
Good luck!

Related

How do I display my project correctly in a server?

I sent my project to my server but no one cant see changes what i did in local mode(i have index.html and other js and php). I had the same problem with another project with index.php but soved adding this <?php time();?> at the end of scrip. Is there any similar solution for javascript?
This is what i did
<script src="assets/js/funciones.js?<?php time();?>"></script>
The problem is that you're changing static files, but not their filenames.
By default apache/nginx/etc serve static content with headers that say "cache this for a very long time" because it's static content, why would you not?
Tacking on random trash to the URL like you're doing with you JS is a kludge that permanently breaks all caching and ensures that users will repeatedly download the exact same static file every time they request a page. You can make the trash less random to break the cache less frequently, but it's still an inefficient kludge. [Albeit a popular one, to my immense annoyance.]
Ideally for resource bundles like JS and CSS, you make a new resource bundle file every time you change it, eg: somefile-v1234.js or somefile-20211007.js and update the reference in your HTML source. This has the side-benefit of ensuring that the versions of your resource bundles always match.
The same goes for any other static file: images, CSV, etc.
The trouble you're having now is that you've updated some HTML pages and the only way to break the cache is to have the user perform an action, like hitting CTRL+F5 to force a refresh.
There are a couple ways around this:
Change the Apache/Nginx/etc settings to set shorter expiries for static file cache headers. You may be able to target specific files like index.html, but YMMV.
Serve the content with PHP. Anything served via a PHP script will not have any cache headers set by default, as the assumption is that the output of a script is dynamic. You can also issue the caching headers yourself in PHP to control what gets cached for how long.
Lastly, you cannot solve this problem retroactively. If a user has a cached version of the HTML page that has not yet reached its expiration, the user MUST take action to break that cache. There's nothing that can be done server side because the valid cache tells the client that it doesn't have to ask the server.
Once you get to the point of your application being popular enough to warrant putting a CDN in front of it this problem gets much worse as now there's a cache in the middle that the user doesn't have control of, and it's potentially an expensive problem because some CDN providers charge a fee for forcing CDN cache invalidations.

How to include a .js file in PL/SQL on Oracle10gXE

again. I'm making a PL/SQL generated HTML5 web page. It's running a Oracle 10g XE server. Okay, now when the setup is clear, my problem - I need to include a Java Script file in the page. Simply
HTP.P('<script type="text/javascript" src="js/ScriptFileName.js"></script>');
Doesn't work of course. So i created a folder object and granted read,write to PUBLIC. Then changed the string to match the newly created object, instead of path. Still doesn't work. I know, i can write
HTP.P(<script type="text/javascript"> MY JAVA SCRIPT HERE</script>);
And i've done so with other scripts(Even had to write CSS this way). But this time this will not work. Reason being - the JavaScript i'm trying to run was normalized(or rather unnormalized), so it's written all in one line. And there is a lot of it too. I tried to reverse it to normal, but faild many a time.
So, I went online and searched for a solution. Found one. It seem's that this include should go not to the page, but to server config. Makes sense, since PL/SQL is server sided. But when i went looking for the usual httpd.conf, it's nowhere to be found in Database directory.So i went online again, result - NOT A WORD OF WHERE THE HELL ARE HTTP SERVER CONFIGS IN 10gXE IN ANY ORACLE MANUALS. Searched some forums - exactly 1 person asked where httpd.conf in XE is, and didn't get an answer. Please, help. I'm desperate.
P.S. I don't use APEX. I don't get that mumbo-jumbo. So i write in Notepad and run the scripts in SQL line.
Firstly, XE has its own built in HTTP server called the 'Embedded PL/SQL Gateway' or EPG. But you don't HAVE to use that. You can use an Oracle HTTP Server with the mod_plsql plugin. Or you can use the Apex listener.
The question is on what server is "ScriptFileName.js" ?
Is it a flat file on the database server ? If so, you'll need to use the Oracle HTTP Server (or Apache or similar) to serve it. The database is pretty much unaware of files on its server and the EPG can't deliver them. [At least not in any practical sense, you could do weird things with chicken entrails and UTL_FILE, but you don't want to go there.]
Is it a file stored in the database ? That sounds exotic, but it is pretty much how all the CSS, images etc are served up through the EPG. The best explanation on how to get files in and out of there is by Dietmar
Is it a file stored on a separate machine ? Often the best answer. The "src=" directive will be read by the end users browser. That will do an HTTP get to the URL. It doesn't have to be a URL on the same domain/host as the rest of the page.

How do I protect JavaScript files?

I know it's impossible to hide source code but, for example, if I have to link a JavaScript file from my CDN to a web page and I don't want the people to know the location and/or content of this script, is this possible?
For example, to link a script from a website, we use:
<script type="text/javascript" src="http://somedomain.example/scriptxyz.js">
</script>
Now, is possible to hide from the user where the script comes from, or hide the script content and still use it on a web page?
For example, by saving it in my private CDN that needs password to access files, would that work? If not, what would work to get what I want?
Good question with a simple answer: you can't!
JavaScript is a client-side programming language, therefore it works on the client's machine, so you can't actually hide anything from the client.
Obfuscating your code is a good solution, but it's not enough, because, although it is hard, someone could decipher your code and "steal" your script.
There are a few ways of making your code hard to be stolen, but as I said nothing is bullet-proof.
Off the top of my head, one idea is to restrict access to your external js files from outside the page you embed your code in. In that case, if you have
<script type="text/javascript" src="myJs.js"></script>
and someone tries to access the myJs.js file in browser, he shouldn't be granted any access to the script source.
For example, if your page is written in PHP, you can include the script via the include function and let the script decide if it's safe" to return it's source.
In this example, you'll need the external "js" (written in PHP) file myJs.php:
<?php
$URL = $_SERVER['SERVER_NAME'].$_SERVER['REQUEST_URI'];
if ($URL != "my-domain.example/my-page.php")
die("/\*sry, no acces rights\*/");
?>
// your obfuscated script goes here
that would be included in your main page my-page.php:
<script type="text/javascript">
<?php include "myJs.php"; ?>;
</script>
This way, only the browser could see the js file contents.
Another interesting idea is that at the end of your script, you delete the contents of your dom script element, so that after the browser evaluates your code, the code disappears:
<script id="erasable" type="text/javascript">
//your code goes here
document.getElementById('erasable').innerHTML = "";
</script>
These are all just simple hacks that cannot, and I can't stress this enough: cannot, fully protect your js code, but they can sure piss off someone who is trying to "steal" your code.
Update:
I recently came across a very interesting article written by Patrick Weid on how to hide your js code, and he reveals a different approach: you can encode your source code into an image! Sure, that's not bullet proof either, but it's another fence that you could build around your code.
The idea behind this approach is that most browsers can use the canvas element to do pixel manipulation on images. And since the canvas pixel is represented by 4 values (rgba), each pixel can have a value in the range of 0-255. That means that you can store a character (actual it's ascii code) in every pixel. The rest of the encoding/decoding is trivial.
The only thing you can do is obfuscate your code to make it more difficult to read. No matter what you do, if you want the javascript to execute in their browser they'll have to have the code.
Just off the top of my head, you could do something like this (if you can create server-side scripts, which it sounds like you can):
Instead of loading the script like normal, send an AJAX request to a PHP page (it could be anything; I just use it myself). Have the PHP locate the file (maybe on a non-public part of the server), open it with file_get_contents, and return (read: echo) the contents as a string.
When this string returns to the JavaScript, have it create a new script tag, populate its innerHTML with the code you just received, and attach the tag to the page. (You might have trouble with this; innerHTML may not be what you need, but you can experiment.)
If you do this a lot, you might even want to set up a PHP page that accepts a GET variable with the script's name, so that you can dynamically grab different scripts using the same PHP. (Maybe you could use POST instead, to make it just a little harder for other people to see what you're doing. I don't know.)
EDIT: I thought you were only trying to hide the location of the script. This obviously wouldn't help much if you're trying to hide the script itself.
Google Closure Compiler, YUI compressor, Minify, /Packer/... etc, are options for compressing/obfuscating your JS codes. But none of them can help you from hiding your code from the users.
Anyone with decent knowledge can easily decode/de-obfuscate your code using tools like JS Beautifier. You name it.
So the answer is, you can always make your code harder to read/decode, but for sure there is no way to hide.
Forget it, this is not doable.
No matter what you try it will not work. All a user needs to do to discover your code and it's location is to look in the net tab in firebug or use fiddler to see what requests are being made.
From my knowledge, this is not possible.
Your browser has to have access to JS files to be able to execute them. If the browser has access, then browser's user also has access.
If you password protect your JS files, then the browser won't be able to access them, defeating the purpose of having JS in the first place.
I think the only way is to put required data on the server and allow only logged-in user to access the data as required (you can also make some calculations server side). This wont protect your javascript code but make it unoperatable without the server side code
I agree with everyone else here: With JS on the client, the cat is out of the bag and there is nothing completely foolproof that can be done.
Having said that; in some cases I do this to put some hurdles in the way of those who want to take a look at the code. This is how the algorithm works (roughly)
The server creates 3 hashed and salted values. One for the current timestamp, and the other two for each of the next 2 seconds. These values are sent over to the client via Ajax to the client as a comma delimited string; from my PHP module. In some cases, I think you can hard-bake these values into a script section of HTML when the page is formed, and delete that script tag once the use of the hashes is over The server is CORS protected and does all the usual SERVER_NAME etc check (which is not much of a protection but at least provides some modicum of resistance to script kiddies).
Also it would be nice, if the the server checks if there was indeed an authenticated user's client doing this
The client then sends the same 3 hashed values back to the server thru an ajax call to fetch the actual JS that I need. The server checks the hashes against the current time stamp there... The three values ensure that the data is being sent within the 3 second window to account for latency between the browser and the server
The server needs to be convinced that one of the hashes is
matched correctly; and if so it would send over the crucial JS back
to the client. This is a simple, crude "One time use Password"
without the need for any database at the back end.
This means, that any hacker has only the 3 second window period since the generation of the first set of hashes to get to the actual JS code.
The entire client code can be inside an IIFE function so some of the variables inside the client are even more harder to read from the Inspector console
This is not any deep solution: A determined hacker can register, get an account and then ask the server to generate the first three hashes; by doing tricks to go around Ajax and CORS; and then make the client perform the second call to get to the actual code -- but it is a reasonable amount of work.
Moreover, if the Salt used by the server is based on the login credentials; the server may be able to detect who is that user who tried to retreive the sensitive JS (The server needs to do some more additional work regarding the behaviour of the user AFTER the sensitive JS was retreived, and block the person if the person, say for example, did not do some other activity which was expected)
An old, crude version of this was done for a hackathon here: http://planwithin.com/demo/tadr.html That wil not work in case the server detects too much latency, and it goes beyond the 3 second window period
As I said in the comment I left on gion_13 answer before (please read), you really can't. Not with javascript.
If you don't want the code to be available client-side (= stealable without great efforts),
my suggestion would be to make use of PHP (ASP,Python,Perl,Ruby,JSP + Java-Servlets) that is processed server-side and only the results of the computation/code execution are served to the user. Or, if you prefer, even Flash or a Java-Applet that let client-side computation/code execution but are compiled and thus harder to reverse-engine (not impossible thus).
Just my 2 cents.
You can also set up a mime type for application/JavaScript to run as PHP, .NET, Java, or whatever language you're using. I've done this for dynamic CSS files in the past.
I know that this is the wrong time to be answering this question but i just thought of something
i know it might be stressful but atleast it might still work
Now the trick is to create a lot of server side encoding scripts, they have to be decodable(for example a script that replaces all vowels with numbers and add the letter 'a' to every consonant so that the word 'bat' becomes ba1ta) then create a script that will randomize between the encoding scripts and create a cookie with the name of the encoding script being used (quick tip: try not to use the actual name of the encoding script for the cookie for example if our cookie is name 'encoding_script_being_used' and the randomizing script chooses an encoding script named MD10 try not to use MD10 as the value of the cookie but 'encoding_script4567656' just to prevent guessing) then after the cookie has been created another script will check for the cookie named 'encoding_script_being_used' and get the value, then it will determine what encoding script is being used.
Now the reason for randomizing between the encoding scripts was that the server side language will randomize which script to use to decode your javascript.js and then create a session or cookie to know which encoding scripts was used
then the server side language will also encode your javascript .js and put it as a cookie
so now let me summarize with an example
PHP randomizes between a list of encoding scripts and encrypts javascript.js then it create a cookie telling the client side language which encoding script was used then client side language decodes the javascript.js cookie(which is obviously encoded)
so people can't steal your code
but i would not advise this because
it is a long process
It is too stressful
use nwjs i think helpful it can compile to bin then you can use it to make win,mac and linux application
This method partially works if you do not want to expose the most sensible part of your algorithm.
Create WebAssembly modules (.wasm), import them, and expose only your JS, etc... workflow. In this way the algorithm is protected since it is extremely difficult to revert assembly code into a more human readable format.
After having produced the wasm module and imported correclty, you can use your code as you normallt do:
<body id="wasm-example">
<script type="module">
import init from "./pkg/glue_code.js";
init().then(() => {
console.log("WASM Loaded");
});
</script>
</body>

What is a "?" for in the src attribute of a html script tag?

If this has been asked before, I apologize but this is kinda of a hard question to search for. This is the first time I have come across this in all my years of web development, so I'm pretty curious.
I am editing some HTML files for a website, and I have noticed that in the src attribute of the script tags that the previous author appended a question mark followed by data.
Ex: <script src="./js/somefile.js?version=3.2"></script>
I know that this is used in some languages for value passing in GET request, such as PHP, but as I far as I ever knew, this wasn't done in javascript - at least in calling a javascript file. Does anyone know what this does, if anything?
EDIT: Wow, a lot of responses. Thanks one and all. And since a lot of people are saying similar things, I will post an global update instead of commenting everyone.
In this case the javascript files are static, hence my curiosity. I have also opened them up and did not see anything attempt to access variables on file load. I've never thought about caching or plain version control, both which seam more likely in this circumstance.
I believe what the author was doing was ensuring that if he creates version 3.3 of his script he can change the version= in the url of the script to ensure that users download the new file instead of running off of the old script cached in their browser.
So in this case it is part of the caching strategy.
My guess is it's so if he publishes a new version of the JavaScript file, he can bump the version in the HTML documents. This will not do anything server-side when requested, but it causes the browser to treat it as a different file, effectively forcing the browser to re-fetch the script and bypass the local cache of the file.
This way, you can set a really high cache time (like a week or a month!) but not sacrifice the ability to update scripts frequently if necessary.
What you have to remember is that this ./js/somefile.js?version=3.2 doesn't have to be a physical file. It can be a page which creates the file on the fly. So you could have it where the request says, "Hey give me version 3 of this js file," and the server side code creates it and writes it to the output stream.
The other option is to force the browser to not cache the file and pull down the new one when it makes the request. Since the URI changed, it will think the file is completely new.
A (well-configured) web server will send static files like JavaScript source code once and tell the web browser to cache that file locally for a certain period of time (could be a day, a week, a month, or longer). When the browser sees another request for that same file, it will just use that version instead of getting new code from the server.
If the URL changes -- for example by adding a query string -- then the browser suspects that its cached version is no good and gets a new one. As such, the ? helps developers say "Oops, I changed this file, make sure the browser gets a new copy."
In this case it's probably being used to ensure the source file isn't cached between versions.
Of course, it could also be used server side to generate the javascript file, without knowing what you have on the other end of the request, it's difficult to be definitive.
BTW, the ?... portion of the url is called the query string.
this is used to guarantee that the browser downloads a new version of the script when available. The version number in the url is incremented each time a new version is deployed so that the browser see it as a different file.
Just because the file extension is .js doesn't mean that the target is an actual .js file. They could set up their web server to pass the requested URL to a script (or literally have a script named somefile.js) and have that interpret the filename and version.
The query string has nothing to do with the javascript. Some server side code is hosting up a different version depending on that querystring it appears.
You should never assume anything about paths in a URL. The extension on a path in a URL doesn't really tell you anything. URLs can be completely dynamic and served by some server side code or can rewritten in web servers dynamically.
Now it is common to add a querystring to urls when loading javascript files to prevent client side caching. If the page updates and references a new version of the script then the page can bust through and cause the client to refresh it's script.

are there any negative implications of sourcing a javascript file that does not actually exist?

If you do script src="/path/to/nonexistent/file.js" in an HTML file and call that in a browser, and there are no dependencies or resources anywhere else in the HTML file that expect the file or code therein to actually exist, is there anything inherently bad-practice about doing this?
Yes, it is an odd question. The rationale is the developer is dealing with a CMS that allows custom (self-contained) javascript files to be provided in certain circumstances. The problem is the CMS is not very flexible when it comes to creating conditional includes for javascript. Therefore it is easier to just make references to the self-contained js files regardless of whether they are actually at the specified path.
Since no errors are displayed to the user, should this practice be considered a viable option?
Well the major drawback is performance since the browser will try (hard) to download the file and your server will look for it. Finally the browser may download the 404 page instead - thus slowing down the page load.
If you have the script referred to in the <head> tag, ( not recommended for starters ), it will slow down the initial page-render time somewhat too.
If instead of quickly returning a 404, your site just accepts the connection and then never responds, this can cause the page to take an indefinite amount of time to load, and in some cases, lock up the entire user interface.
( At least that was the case with one revision of FireFox, I hope they've fixed it since I saw that happen ~2 years ago.* )
You should at least put the tags as low in the page order as you can afford to to remedy this problem.
Your best bet by far is to have one consistent no-op url that is used as a fill-in for all "doesn't exist" JavaScript files, that returns a 0-byte response with HTTP headers telling the UA to cache it till the cows come home, that should negate most your server<->client load penalties beyond the first hit ( and that should hardly hurt people even on ye-olde dialup )
*Lesson learned: don't put script-src references in head, especially for 3rd-party scripts hosted outside your machine, because then you can have the joy of having clients be able to access your website, but run the risk of the page being inoperable because of a bit of advertising JS that was inaccessible due to some internet weirdness. Even if they're a reputable-ish 3rd-party.
If your web server is configured to do work on a 404 error ("you might be looking for this", etc) then you're also causing unnecessary load on the server.
you should ask yourself why you were too lazy to test this yourself :)
i tested 1000 randomized javascript filenames and it took several nanoseconds to load, so no, it doesn't make a difference. example:
script src="/7701992spolsky.js"
This was on my local machine however, so it should take N * roundTripTime for the browser to figure out for remote servers, where N is the number of bad scripts.
If however, you have random domain names that don't exist, like
script src="http://www.randomsite7701992.com/spolsky.js"
then it will take a long f-in time.
If you choose to implement it this way, you could tune the web server that if the referenced JS file is not found, instead of 404, it could return a redirect (301) to an empty/default JS file.
If you are using asp.net you can look into using custom handlers (ASHX files).
Here's an example:
public class JavascriptHandler : IHttpHandler {
public void ProcessRequest (HttpContext context)
{
context.Response.ContentType = "text/plain";
//Some code to check if javascript code exists
string js = "";
if(JavascriptExists())
{
js = GetJavascript();
}
context.Response.write(js);
}
}
Then in your html header you could declare a file pointing to the custom handler:
src="/js/javascripthandler.ashx"

Categories

Resources