Final Update
For most practical purposes, this question is obsolete as both firefox and chrome have native support for avif through the standard picture html tag with a source marked as type="image/avif". See https://reachlightspeed.com/blog/using-the-new-high-performance-avif-image-format-on-the-web-today/ . Fire fox still likes to hang and often forces a control F5 to bypass caches and requires sending the correct content type from the server. Hopefully will be fixed soon.
Here is the commit where I got avif support working: https://github.com/quackack/quackack-comics/commit/f1a98ed1f40b6a22584d61bc338bd91df3232fa5#diff-e25b0950ce48f4e928f98e0a6fbb694c . Note that it contains many unrelated changes and in fact avif is barely mentioned, only as a content type and a file extension.
Original Question
I am trying to change a website where I host web comics from using jpegs to the newer avif image format. It is much smaller and seems to be the new image format with the most widespread support. Unfortunately, web browsers don't properly support the new format yet. So I was planning to use this package: https://github.com/Kagami/avif.js to allow my comics to be rendered. Some basic tests showed that AVIF would give the same quality as jpeg for less than half the space.
Unfortunately, after more than 5 hours of time spent on this, I am unable to get this to work with my react framework. You can see my website at the time of writing at 'https://github.com/quackack/quackack-comics/tree/cda4c3893d8477192c4ff3aa78d00096b7621ff7'.
I tried using npm install to install avif.js and then added
require("avif.js").register("/avif-sw.js");
to index.js . But I get error
Failed to register/update a ServiceWorker for scope ‘http://localhost:3000/’: Bad Content-Type of ‘text/html’ received for script ‘http://localhost:3000/avif-sw.js’. Must be a JavaScript MIME type
And avif files are still not able to load. I think the requests are getting rerouted to index.html instead of the javascript package. It seems like the appropriate thing to do is something like
import * as avif from 'avif.js';
avif.register('avif.js/avif-sw.js');
But this fails too with the same error, as do many other similar variations.
At this point I am inclined to wait for proper browser support for avif, as I don't get enough traffic to worry about data costs anyway. If this could easily be fixed, then I would love to have the improvements from avif. I just want smaller file sizes and widespread browser support.
Update
Okay, I found that I could get this to work if I changed from the default react bundler (which I believe to be webpack) to Parcel. Then it does work exactly as you expect... until I try to deploy the project.
There is an issue where I cannot load the service worker when i try to deploy my single page webapp to AWS as a single page web app. There it makes a request to my url with avif-sw.js where there is not actually a js file. I believe the issue is closely related to https://github.com/parcel-bundler/parcel/issues/670
So the first key is to use Parcel to build your web app. But Parcel still does something wrong with deployment it seems. I will continue to investigate this in a few days.
Here is the almost working version using parcel: https://github.com/quackack/quackack-comics/tree/parcel
Update 2
My earlier update was incorrect. I only thought it was working because of a cached service worker. My final solution is in the answer below.
It doesn't seem to be a problem now. Simply convert your images to avif with tools like https://avif.io/ and use them as the image source or background source via typical CSS. As Chrome and Firefox now support it (even though users still have to enable it on Firefox), everything goes well. Even works on mobile now! :)
Okay, I finally got it working. Unfortunately, mobile devices don't seem to be able to handle the large file sizes so I had to keep using jpeg anyway. It worked on my laptop though.
Here is the commit that got everything to work: https://github.com/quackack/quackack-comics/commit/75e75307e688f0e515b4bbc9eb22eef290d2c209
What I had to do:
Switch to Parcel.
Copy the contents of the avif.js library into source before building. I used the command:
copyfiles -f node_modules/avif.js/*.js .
Put this specifically into reg.js:
require("avif.js").register("./avif-sw.js");
navigator.serviceWorker.register("./avif-sw.js", undefined);
What the last two steps do is trick parcel into actually keeping a copy of "avif-sw.js" around that can actually be loaded as a service worker. Probably with a bit more tinkering you can get this to work without using parcel at all, just by copying local and then registering. No requires required. But I stopped investigating after I found this solution can't work on mobile.
This was exceptionally hard to debug because service workers are cached by the browser and I had to clear broswer data after every edit. It was also hard to debug because the source files are cached to so I had to delete my projects cache and build frequently too.
You might also want to use npm module "http-server-spa", or similar, to test how your built SPA will act when deployed.
Related
I currently have a node.js script that automatically creates a group of files and then zips them ready for being uploaded on a site. I'm trying to add one extra piece of functionality to the script that will log into the site and upload the file itself.
I've done some reading around and found a lot about headless browsers but not sure if that's the right path to go down as they seem to rely on other applications like chromium and they're focused on testing sites.
Does anyone know where I should start looking?
In my current project I am using the following library from Google, puppeteer. I personally found it to be very easy to use, and it even provides access to the dev protocol that Google Chrome has.
I've done some reading around and found a lot about headless browsers but not sure if that's the right path to go down as they seem to rely on other applications like chromium and they're focused on testing sites.
Yes, they are often used for testing, to assure that the correct things are rendered on screen etc. However, in many scenarios, like yours, the use of a headless browser to interact with a website is totally legit in a non-testing scenario.
I have latests CefGlue version (3.2272.2035)
I need to save current page as PDF. I'd like to interact with my CEF somehow (JS/C#) and make it create PDF for me.
I've tried to use javascript.window.print() for that purpose with no success, because, all i've got is such window.
I've found OnPrintJob method in CefPrintHandler but I don't know what is the right way to call it + comments say: "Implement this interface to handle printing on Linux"
and I need to handle printing to pdf on Windows environments (both x64/x86)
Any code example would be appreciated. Thanks in advance for any help.
You'd need to merge this patch and recompile cef from source; Besides, you might have to add some glue logic to CefGlue. I tested the patch myself (on C++ API) and it worked fine.
EDITED: the patch will be merged in trunk.
If you only need to print some pages (specially if these pages are under your control), you will do just fine. However if you need (as I did at the time) to print almost anything, you'll probably face the fact that some web pages don't render anything useful to the print view. Also, even those well-behaved diverge significantly from what you see on the screen - and that's not a CEF behavior, as even google chrome showed the same issues. In my application this was a no-go so I dropped printing and started capturing the screen (and implemented saving that to a pdf using a pdf library in C++), but perhaps your application isn't as demanding as mine was. Cheers!
How is it possible to programmatically save a web page snapshot with all its elements (css, js, images, ...) into one file?
I need to archive some web pages regularly. However, just saving their HTML code is useless - not only because of images missing but esp. because the absence of CSS on today's pages can turn a web page into unrecognizable mess.
I remember the .mht format that worked like this, but that required manual saving, and it was just a feature of IE. I believe there is an open-source solution that can achieve this programmatically, but despite hours of searching I cannot find it on the web.
HTTrack, -%M
Use wget in terminal
wget -p -k http://www.example.com/
It'll make a clone of site frontend html, css, js, svg etc. But not in one file as asked. Rather, it'll recreate the whole folder structure
E.g. if folder structure of www.example.com is as
/css/*
/js/*
/index.html
then it'll create the same structure locally.
Docs: https://www.gnu.org/software/wget/manual/wget.html
I think #reisio (+1) has you covered...
...But if only to plug a great free tool, I would point out the Firefox extension Save Complete, which does an admirable job of grabbing "complete" pages on an ad hoc basis. The output will be a single HTML file with an accompanying directory stuffed with all the resources - you can easily zip them up for archiving.
It's not without fault - I've had issues with corrupted .png files lately on OSX, but I use it frequently for building mockups off of live pages and it's a huge time-saver. (Also of note, it hasn't been updated for FF 4 yet, and is the sole reason I rolled back to 3.6)
If you are using Google Chrome just use the save page as menu entry (CTRL + s), and select complete website from the options at the bottom of the file dialog. This save the HTML and all required resources (in a separate folder).
Apple's Safari has a pretty good solution. It saves all HTML and CSS (sadly no JS) but in a format called webarchive. It's one file, but it requires Safari to save and open, and Safari requires a Mac. Even though Safari for Windows does exist, it's too old to work with webpages, and it doesn't even support saving as webarchive, or opening them. If you have a Mac, open any website in Safari and press ⌘S and then make sure that Web Archive appears in the drop down.
There is also a Chrome extension that can open these types of files, but not save them.
Apologies for replying to such an old thread, just wanted to spread this info!
I have a web page which includes insane amount of minified JS files. The web page works perfectly fine on my local network but throws some JS error on staging. There is an issue in JS and I wan't to debug it. When I load the JS in Firebug's script tag it appears in one long horizontal line. Is there a way out in Firebug that expands or beautifies the script for debugging? I know I can use jsbeautifier but they wont help me. I can not upload an expanded file to CDN, defeats the purpose of using CDN.
Points to be noted,
a) I can not control the box which serves JS, its on CDN, I mentioned it.
b) I can use beautifiers etc but would that help me in debugging the script in run time? IMHO, no
c) Being bound by NDA and other legal things I can not share the script but its a generic problem, you can encounter it with a minified jQuery
Beautify your script
Add the CDN host in /etc/hosts or your local DNS to resolve it to your own web server
Host the beautified version and everything that you need on said web server
Both Firefox and Chrome (versions as of this edit) support beautifying script with the {} button available in the inspector.
Just load the minified file and press the {} button at the bottom and it instantly beautifies, making breakpoints and other debugging possible. (True for Chrome too)
This is a common problem and the Chrome dev team have recently come up with an elegant solution, which they've called Source Maps - see http://www.html5rocks.com/en/tutorials/developertools/sourcemaps/ for more info, I think you'll find it's exactly what you (and the rest of us) have been crying out for! :)
This is more a workaround, but it can help. The idea is that we will replace files coming from the server by files on your machine.
This will work with any browser.
It takes a bit of setup the first time (15 minutes maybe), but then it can be very convenient.
It can also helps testing your bug-fixes in a live/prod environment.
Get Fiddler (it's a web debugging proxy), install it, run it.
http://www.fiddler2.com/fiddler2/
(Restart browser after install to get the Fiddler extension)
If you debug an HTTPS website, check this first:
http://www.fiddler2.com/Fiddler/help/httpsdecryption.asp
From now on, you should see in Fiddler ("Web Sessions" pane on the left) all downloads made by your browser, including JS files.
If not, check this : Fiddler not displaying sessions
Find the file you want to debug in the list (Ctrl+F works)
Click on the file. Then either:
get the file content from the inspectors pane (textView tab), beautify it, save to a file on your local computer
or have access to a file which contains the source code (ex: from your source control)
Go to AutoResponder tab (top left pane).
Select "Enable automatic responses" checkbox.
Select "Unmatched requests passthrough" checkbox.
Drag your file from left pane to right pane (prefills rule editor at the bottom)
Set the other field with the path of your local file
Click the Save button
Reload the page and enjoy your debugging session.
Fiddler can do many more things, but this use-case answers the initial question.
Consider a Change!
Firefox w/ Firebug was my favorite JavaScript debugging method for almost a year, but I've recently moved to Google-Chrome's Developer-Tools which is far more superior.
Chrome supports an On-The-Fly (built-in feature) beautification of JavaScript resources
Once beautified, you are free to debug the JavaScript resource file, as it was "natively" downloaded beautified from the web-server. Breaking-points are set by clicking the line number.
One of the most extremely powerful feature,
Is once You've Stopped In A Breaking-point, You Are Free To Execute Commands (using console) In The Same Scope You ARE In The Breaking-point. In Firefox you can't do that.
Its so easy to debug (even anonymous functions), You'll never be back to Firefox.
Try It!
Pretty-print your JavaScript. Google this and you'll find multiple on-line JS beautifiers.
I happen to use http://jsbeautifier.org/ myself and it works fine, but search for others and use one that suits your needs.
Caveat: You still won't be able to get meaningful local variable names (which are usually renamed by a minifier). If the code was compiled by the Closure Compiler, then you absoutely won't get any useful information back at all, even when beutified, because then all variables and functions and properties are mangled (not only local ones).
Now, if your problem is with debugging code that comes from outside (e.g. a CDN), obviously that code would be minified, and you can't save your beautified version back there. In this case, you can replace the tags that load code from a CDN with a url pointing to your local version, then you can beautify the code (downloaded from the CDN) into your own server and you can then debug with FireBug.
Now, if you don't even control the HTML that contains those tags (e.g. they reside on a outside server), then unfortunately there is no way for you to do what you want without physically downloading the entire site to your own server. Even if you downloaded the entire site (with all the files), it may not work since the site may be driven by a back-end processing language or accesses a back-end database. In such case you'll also need to simulate all those data. It can be done, however -- you just have to go through a lot of pain. My recommendation is to save a version of the web page and run it on your own server, serving beautified code from your own server to debug.
Placing breakpoints on JavaScript makes debugging much easier, but if your code has already made it to production then it's probably been minified. How can you debug minified code? Helpfully, some of the browsers have an option to un-minify your JavaScript.
In Chrome and Safari, simply select the 'Scripts' tab, find the relevant file and then press the "{ }" (pretty print) icon located in the bottom panel.
In Internet Explorer, click the tool icon by the script selection drop down to find the option to format the JavaScript.
Opera will automatically prettify minified JavaScript. Source
I'm pretty new to workign with Javascript.
In most languages you can run the code quickly locally on your machine. From what I've seen, in JS you generally only use it via the browser, and so I've been uploading my code an viewing its effects in the browser. This has proven very tiresome. Also, if I mak one error, it seems like my JS/JQuery will just do NOTHING, instead of giving me a useful error, message, which is making it painfully slow to code in.
IS there some way to run JS locally to see that it is working as I go? And then only upload it to the web when I'm mostly done? What ways are there for me to do this? What ways aer there for me to unit test the Javascript locally? Say I have some JAML that should render as <p>HI</p>, how do I run this locally in a unit test?
Thanks for the help,
Alex
EDIT:
Thanks for all the great suggestions. I'll have to take a bit of time and go through them to see which ones best help me in my situation.
Since you're using jQuery, I assume that you actually want to manipulate the various elements on your page. So depending on your specific development enviroment, uploading it each time is probably the way to go anyway. If you can set up a dev enviroment on your local machine (not always possible) then go with that.
As an actual answer to your question, I suggest using Chrome's developer tools, it doesn't just have the console, but an element inspector, and a resource tracker (resource tracker is invaluable when working with JSON and AJAX, since invalid json will fail silently)
As far as I know, the firebug plugin for firefox (dont use it myself) has a similar feature set, so if you're more comfortable with that go with it.
Just remember, as a developer, your development (and debuggin) enviroment is just as important as the code that you are writing.
EDIT: Noticed that you mentioned unit testing. There are several unit testing frameworks out there for JS including one that integrates with firebug called FireUnit. Do a quick google search to find more if you want.
You don't need to upload the JS file to a server to test it. Just write an html and declare the js binding
<script
src="js/yourJSFile.js"
type="text/javascript"></script>
Edit the JS file in your favorite editor and then refresh the page to test it.
For unit testing the best option is Selenium. It allows you to record an interaction with the browser and then play it back.
You can use Firebug with Firefox to debug JS, and Google Chrome has a debugger built-in (use the Tools -> Developer Tools menu).
You can run Javascript from the local file on your machine in your browser, so you can skip the uploading step.
Also, I'd recommend using Firefox/Firebug combo for developing Javascript as it will be very handy, especially for the part you mentioned about not seeing what's going wrong with your code.
Even if you upload your javascript it gets downloaded back to you as soon as you visit the webpage that invoques it. Its run client side always. So stick to local and use firebug as the others have said. Google`s developer tool is quite nice too.
In the browser if you open the developer tools, follow the following steps:
1) Navigate to sources
2) Under sources, click snippet and open run.js
3) You can use run.js to write as much code as you want and run it locally only to see if your code is working or not (it will give you output on the console)
4) Also you can get used to some keyboard shortcuts to make it faster for you.
5) For small javascript codes, you can navigate to console and run your code there
If you want to do unit testing with Javascript there are extension of Firebug that can help you with that. I haven't try any of them, so I can't really tell you which one are worth considering, but you can easily find them if you search for the keyword "Firebug unit testing" on Google.
What seems to be comming on top is FireUnit. You can find some information about how it works here.
Consider Spider Monkey, which is a javascript engine separate from a browser. If what you are developing does not involve rendering to a webpage or can be separated from the rendering code (good practice!), then this could be useful.
I prefer Chrome to Firefox and I just found Web Server for Chrome.
It's just a Google App that quickly sets up a web server for you and will be set up anywhere you are logged into Chrome. It only allows file access to your current devices, or if you specify, other devices only on the current LAN.
You just point it to the directory with your index.html file and type http://127.0.0.1:8887 in your browser.
Additionally to the answers given you can use Jasmine for automated testing.
A tutorial that seems to help get started with automated testing on Jasmine is provided by Evan Hahn.
I used it and for me it works like a charm. Especially if test driven development is what you are going for!