I am trying to analyze some JavaScript code for which I make use of function rewriting so that calls to a JavaScript library go through my JavaScript code. My JavaScript code is part of a Chrome Extension. From a Chrome extension content script, I install/inject the code into the target page's DOM.
This works fine for functions that are induced after the load of page. The library calls go through my function. But, there's JavaScript code that runs while the page is actually loading (probably while the DOM is being rendered). This happens before my custom script is injected. This way, the function calls before the custom script is injected are lost to me, or those JavaScript calls do not go through my function.
I make use of Content Script to actually inject other JavaScript by appending to the DOM as mentioned in the following Stack Exchange question:
Insert code into the page context using a content script
I know I can cause the loading time of Content Script to be at the start/end of the DOM but this is another script file that I append to the DOM of the target page. I do not seem to understand how to control it.
The problem explained in Is it possible to run a script in context of a webpage, before any of the webpage's scripts run, using a chrome extension?
is exactly the same, but the solution does not seem to work. My intention is to make the injected script execute before any JavaScript code executes from the webpage. By specifying document_start in manifest.json, content script execution can be made to run before the webpage, but not the script that I inject through the content script (injecting script as explained in first link). This injected script is not running in any specific manner with respect to the webpage
Manifest.json:
Manifest file has the content script content.js added at document_start, so content.js is run before the target webpage (underlying page) runs.
"content_scripts":[
{
"matches":["<all_urls>"],
"js":["content.js"],
"run_at":"document_start",
"all_frames":false
}
],
content.js:
content.js has the below code with which I add the main.js to the DOM, so that I am actually able to interact with the JavaScript that is in the target page's environment. I do this from a different file and attach it to the DOM because I cannot interact with the target page's JavaScript through the Content Scripts, since they both do not interfere with each other.
To explain further, main.js has some JavaScript that intercepts JavaScript calls during the execution of JavaScript in target page. JavaScript in target page makes calls to a library and I intend just to write a wrapper on those library functions.
var u = document.createElement('script');
u.src = chrome.extension.getURL('main.js');
(document.head||document.documentElement).appendChild(u);
u.onload = function() {
u.parentNode.removeChild(u);
};
I expect that main.js is available in the target page's domain and any of the scripts in the target page, since I inject it through the content script that is run at document_start.
Assume I have a call to some JavaScript function like this in my target page HTML, someJSCall() is defined by the target page's domain.
<html onLoad="someJSCall( )">
In this scenario, main.js (code injected through my Chrome extension) is already available. So calls to the JavaScript library from someJSCall() function go through main.js wrapper functions.
This works fine.
The problem is when there are IIFE (immediately invoked function expressions) defined in the target page's JavaScript. If these IIFE calls make library calls, this does not go through my main.js interceptions. If I look at the files loaded in the browser through Chrome Dev Tools, I see that main.js is still not loaded while IIFE calls are executing.
I hope I have explained the problem in detail.
Based on the additional information you added to the question about 2.5 weeks after I answered, you are adding code to the page context by including a "main.js", which is a separate file in your extension, using a <script> that looks something like:
<script src="URL_to_file_in_extension/main.js"/>
However, when you do that you introduce an asynchronous delay between when the <script> is inserted into the page and when the "main.js" is fetched from the extension and executed in the page context. You will not be able to control how long this delay is and it may, or may not, result in your code running prior to any particular code in the page. It will probably run prior to code that has to be fetched from external URLs, but may not.
In order to guarantee that your code runs synchronously, you must insert it in a <script> tag as actual code, not using the src attribute to pull in another file. That means the code which you want to execute in the page must exist within the content script file you are loading into the page.
Needing to execute code in the page context is a fairly common requirement. I've needed to do so in browser extensions (e.g. Chrome, Firefox, Edge, etc.) and in userscripts. I've also wanted to be able to pass data to such code, so I wrote a function called executeInPage(), which will take a function defined in the current context, convert it to text, insert it into the page context and execute it while passing any arguments you have for it (of most types). If interested, you can find executeInPage() in my answer to Calling webpage JavaScript methods from browser extension and my answer to How to use cloneInto in a Firefox web extension?
The following is my original answer based on the original version of the question, which did not show when the content script was being executed, or explain that the code being added to the page was in a separate file, not in the actual content script.
You state in your question that you "can handle the loading time of Content Script to be at the start/end of the DOM", but you don't make clear why you are unable to resolve your issue by executing your content script at document_start.
You can have your script injected prior to the page you are injecting into being built by specifying document_start for the run_at property in your manifest.json content_scripts entry, or for the runAt option passed to chrome.tabs.executeScript(). If you do this, then your script will start running when document.head and document.body are both null. You can then control what gets added to the page.
For chrome.tabs.executeScript() exactly when your script runs depends on when you execute chrome.tabs.executeScript() in relation to the process of loading the page. Due to the asynchronous nature of the processing (your background script is usually running in a different process), it is difficult to get your script consistently injected when document.head and document.body are both null. The best I've accomplished is to have the script injected sometimes when that is the case, and sometimes after the page is populated, but prior to any other resources being fetched. This timing will work for most things, but if you really need to have your script run prior to the page existing, then you should use a manifest.json content_scripts entry.
With your content script running prior to the existence of the head and body, you can control what gets inserted first. Thus, you can insert your <script> prior to anything else on the page. This should make your script execute prior to any other script in the page context.
Here's the scenario, not sure what I'm missing.
Page A.htm makes an ajax request for page B.htm, and inserts the response into the page.
Page B.htm contains links to several other JS files, many of which contain a document.ready() function to initialize them.
This works fine when A.htm and B.htm are on the same server but not when they are on different servers.
What I think I'm seeing here, is that when page A and B are on different servers (cross domain ajax), the external resources are being returned asynchronously, or at least out of order, so scripts are executing expecting JQuery.UI to be loaded already, when it is not.
Appreciate any pointers or advice. Apologies for the poor explanation.
You are injecting HTML + script tags via jQuery. In this case *:
HTML content except scripts are injected in the document
Then all scripts are executed one by one
If a script is external then it is downloaded and executed asynchronously
Therefore an external or inline script that depends on jQuery UI might execute before jQuery UI.
One possible solution is to change the way your pages work:
Get rid of external scripts in pageb.html but keep inline scripts
Load the required scripts in pagea.html
Load pageb.html
Another solution is to roll your own jQuery function that will:
Strip all <script src> elements from HTML
Download and execute those scripts in order
Inject the remaining HTML
* The exact behavior is not documented. I had to look into the source code to infer the details.
you are correct in your impression that the issue is a difference in how the requests are handled cross-domain.
Here is a link to get you on the right track : How to make synchronous JSONP crossdomain call
However, you will have to actually re-achitect your solution somewhat to check if the resource has been loaded before moving on. There are many solutions (see the link)
You can set a timer interval and check for something in the dom, or another reasonable solution (despite it's lack of efficiency) is to create a "proxy" serverside (eg php) file on your server and have that file do the cross-domain request, then spit out the result.
Note that since jquery UI is a rather large file, it's conceivable that the cross-domain request finishes first, and executes immediately, even though jqueryUI is not loaded yet. In any case, you're going to have to start thinking about having your app react rather than follow a sequence.
I have a few scripts that are common among all html pages for my application. Call this file commonfunctions.js. Each html page will load it as you move around the application along with appending the last modification date for this js file (that's gotten from the server). Firebug is adding the file every time to the list of loaded scripts as well as an eval/seq/# (where # is the number of times this file has been loaded starting at 7 for some reason). For example, if I have 3 pages called one.html, two.html, and three.html each with this line of code:
<script type="text/javascript" src="commonfunctions.js?mod=11/33/2012"></script>
If I were to go from one.html->two.html->one.html->three.html, Firebug would list the scripts loaded as:
commonfunctions.js?mod=11/33/2012
commonfunctions.js?mod=11/33/2012/eval/seq/7
commonfunctions.js?mod=11/33/2012/eval/seq/8
commonfunctions.js?mod=11/33/2012/eval/seq/9
and so on as I visit the three pages more.
Why is this happening and is there a way to stop it? I read that it could be that firebug will make its own url if it doesn't know the url due to an eval() or event attribute; however, these scripts are being loaded via regular tags.
I'm concerned because I'm not sure if this means the browser has now compiled and is executing or storing multiple copies of the same script--very wasteful in both conditions.
The script may have been loaded via script tag, but somewhere within commonfunctions.js a call to eval()has been made. Or three, obviously.
I wanted to know what is the most recommended way in terms of optimization, caching and page load speed. Should I dynamically insert the JS files in the BODY or HEAD of my website document when using Defer loading?
If you are using DEFER, it does not matter whether you put the script tag in the HEAD or BODY as far as when the script will run, or cache. Either way, it will run after the page is finished parsing, and will cache according to the server headers sent with the javascript file.
You may still want to put the script tags at the end of the body to optimize for the case where an older browser ignores the DEFER attribute.
Alternatively, you may want to put the script tags in the HEAD to get the javascript files earlier in the fetching order (before images, audio, or video files specified in the body, for example).
I have a couple of questions about the attributes async & defer for the <script> tag which to my understanding only work in HTML5 browsers.
One of my sites has two external JavaScript files that currently sit just above the </body> tag; the first is jquery sourced from google and the second is a local external script.
With respects to site load speed
Is there any advantage in adding async to the two scripts I have at the bottom of the page?
Would there be any advantage in adding the async option to the two scripts and putting them at the top of the page in the <head>?
Would this mean they download as the page loads?
I assume this would cause delays for HTML4 browsers, but would it speed up page load for HTML5 browsers?
Using <script defer src=...
Would loading the two scripts inside <head> with the attribute defer the same affect as having the scripts before </body>?
Once again I assume this would slow up HTML4 browsers.
Using <script async src=...
If I have two scripts with async enabled
Would they download at the same time?
Or one at a time with the rest of the page?
Does the order of scripts then become a problem? For example one script depends on the other so if one downloads faster, the second one might not execute correctly etc.
Finally am I best to leave things as they are until HTML5 is more commonly used?
This image explains normal script tag, async and defer
Async scripts are executed as soon as the script is loaded, so it
doesn't guarantee the order of execution (a script you included at
the end may execute before the first script file )
Defer scripts guarantees the order of execution in which they appear
in the page.
Ref this link : http://www.growingwiththeweb.com/2014/02/async-vs-defer-attributes.html
Keep your scripts right before </body>. Async can be used with scripts located there in a few circumstances (see discussion below). Defer won't make much of a difference for scripts located there because the DOM parsing work has pretty much already been done anyway.
Here's an article that explains the difference between async and defer: http://peter.sh/experiments/asynchronous-and-deferred-javascript-execution-explained/.
Your HTML will display quicker in older browsers if you keep the scripts at the end of the body right before </body>. So, to preserve the load speed in older browsers, you don't want to put them anywhere else.
If your second script depends upon the first script (e.g. your second script uses the jQuery loaded in the first script), then you can't make them async without additional code to control execution order, but you can make them defer because defer scripts will still be executed in order, just not until after the document has been parsed. If you have that code and you don't need the scripts to run right away, you can make them async or defer.
You could put the scripts in the <head> tag and set them to defer and the loading of the scripts will be deferred until the DOM has been parsed and that will get fast page display in new browsers that support defer, but it won't help you at all in older browsers and it isn't really any faster than just putting the scripts right before </body> which works in all browsers. So, you can see why it's just best to put them right before </body>.
Async is more useful when you really don't care when the script loads and nothing else that is user dependent depends upon that script loading. The most often cited example for using async is an analytics script like Google Analytics that you don't want anything to wait for and it's not urgent to run soon and it stands alone so nothing else depends upon it.
Usually the jQuery library is not a good candidate for async because other scripts depend upon it and you want to install event handlers so your page can start responding to user events and you may need to run some jQuery-based initialization code to establish the initial state of the page. It can be used async, but other scripts will have to be coded to not execute until jQuery is loaded.
HTML5: async, defer
In HTML5, you can tell browser when to run your JavaScript code. There are 3 possibilities:
<script src="myscript.js"></script>
<script async src="myscript.js"></script>
<script defer src="myscript.js"></script>
Without async or defer, browser will run your script immediately, before rendering the elements that's below your script tag.
With async (asynchronous), browser will continue to load the HTML page and render it while the browser load and execute the script at the same time.
With defer, browser will run your script when the page finished parsing. (not necessary finishing downloading all image files. This is good.)
Both async and defer scripts begin to download immediately without pausing the parser and both support an optional onload handler to address the common need to perform initialization which depends on the script.
The difference between async and defer centers around when the script is executed. Each async script executes at the first opportunity after it is finished downloading and before the window’s load event. This means it’s possible (and likely) that async scripts are not executed in the order in which they occur in the page. Whereas the defer scripts, on the other hand, are guaranteed to be executed in the order they occur in the page. That execution starts after parsing is completely finished, but before the document’s DOMContentLoaded event.
Source & further details: here.
Faced same kind of problem and now clearly understood how both will works.Hope this reference link will be helpful...
Async
When you add the async attribute to your script tag, the following will happen.
<script src="myfile1.js" async></script>
<script src="myfile2.js" async></script>
Make parallel requests to fetch the files.
Continue parsing the document as if it was never interrupted.
Execute the individual scripts the moment the files are downloaded.
Defer
Defer is very similar to async with one major differerence. Here’s what happens when a browser encounters a script with the defer attribute.
<script src="myfile1.js" defer></script>
<script src="myfile2.js" defer></script>
Make parallel requests to fetch the individual files.
Continue parsing the document as if it was never interrupted.
Finish parsing the document even if the script files have downloaded.
Execute each script in the order they were encountered in the document.
Reference :Difference between Async and Defer
async and defer will download the file during HTML parsing. Both will not interrupt the parser.
The script with async attribute will be executed once it is downloaded. While the script with defer attribute will be executed after completing the DOM parsing.
The scripts loaded with async doesn't guarantee any order. While the scripts loaded with defer attribute maintains the order in which they appear on the DOM.
Use <script async> when the script does not rely on anything.
when the script depends use <script defer>.
Best solution would be add the <script> at the bottom of the body. There will be no issue with blocking or rendering.
Good practice is to keep all the files in your source folder to load source files fast. You need to download all the script, style, icon, and image-related files and put these files into your project folder.
Create these folders in your project to keep different source files and then load the required files into the pages from this folder.
JS: to keep script-related files.
CSS: to keep style-related files.
images: to keep image/icon-related files
fonts: to keep font-related files
When to use defer and async attributes on the <script> tag
defer attribute: First the defer attribute will download the script file and then wait for HTML parsing. After the end of the HTML parsing, the script will execute. In other words, it will guarantee all the scripts will execute after the HTML parsing.
The defer attribute is useful when the script is used for DOM manipulations.
async attribute: The async attribute will download the script file and execute without waiting for the end of HTML parsing. In other words, it does not guarantee that all the scripts will execute after the HTML parsing.
The async attribute is useful when the script is not used for DOM manipulation. Sometimes you need a script only for server-side operations or for handling cache or cookies, but not for DOM manipulations.
Useful link when to use defer and async:
https://stackoverflow.com/a/68929270/7186739
I think Jake Archibald presented us some insights back in 2013 that might add even more positiveness to the topic:
https://www.html5rocks.com/en/tutorials/speed/script-loading/
The holy grail is having a set of scripts download immediately without blocking rendering and execute as soon as possible in the order they were added. Unfortunately HTML hates you and won’t let you do that.
(...)
The answer is actually in the HTML5 spec, although it’s hidden away at the bottom of the script-loading section.
"The async IDL attribute controls whether the element will execute asynchronously or not. If the element's "force-async" flag is set, then, on getting, the async IDL attribute must return true, and on setting, the "force-async" flag must first be unset…".
(...)
Scripts that are dynamically created and added to the document are async by default, they don’t block rendering and execute as soon as they download, meaning they could come out in the wrong order. However, we can explicitly mark them as not async:
[
'//other-domain.com/1.js',
'2.js'
].forEach(function(src) {
var script = document.createElement('script');
script.src = src;
script.async = false;
document.head.appendChild(script);
});
This gives our scripts a mix of behaviour that can’t be achieved with plain HTML. By being explicitly not async, scripts are added to an execution queue, the same queue they’re added to in our first plain-HTML example. However, by being dynamically created, they’re executed outside of document parsing, so rendering isn’t blocked while they’re downloaded (don’t confuse not-async script loading with sync XHR, which is never a good thing).
The script above should be included inline in the head of pages, queueing script downloads as soon as possible without disrupting progressive rendering, and executes as soon as possible in the order you specified. “2.js” is free to download before “1.js”, but it won’t be executed until “1.js” has either successfully downloaded and executed, or fails to do either. Hurrah! async-download but ordered-execution!
Still, this might not be the fastest way to load scripts:
(...) With the example above the browser has to parse and execute script to discover which scripts to download. This hides your scripts from preload scanners. Browsers use these scanners to discover resources on pages you’re likely to visit next, or discover page resources while the parser is blocked by another resource.
We can add discoverability back in by putting this in the head of the document:
<link rel="subresource" href="//other-domain.com/1.js">
<link rel="subresource" href="2.js">
This tells the browser the page needs 1.js and 2.js. link[rel=subresource] is similar to link[rel=prefetch], but with different semantics. Unfortunately it’s currently only supported in Chrome, and you have to declare which scripts to load twice, once via link elements, and again in your script.
Correction: I originally stated these were picked up by the preload scanner, they're not, they're picked up by the regular parser. However, preload scanner could pick these up, it just doesn't yet, whereas scripts included by executable code can never be preloaded. Thanks to Yoav Weiss who corrected me in the comments.
Rendering engine goes several steps till it paints anything on the screen.
it looks like this:
Converting HTML bytes to characters depending on encoding we set to the document;
Tokens are created according to characters. Tokens mean analyze characters and specify opening tangs and nested tags;
From tokens separated nodes are created. they are objects and according to information delivered from tokenization process, engine creates objects which includes all necessary information about each node;
after that DOM is created. DOM is tree data structure and represents whole hierarchy and information about relationship and specification of tags;
The same process goes to CSS. for CSS rendering engine creates different/separated data structure for CSS but it's called CSSOM (CSS Object Model)
Browser works only with Object models so it needs to know all information about DOM and CSSDOM.
The next step is combining somehow DOM and CSSOM. because without CSSOM browser do not know how to style each element during rendering process.
All information above means that, anything you provide in your html (javascript, css ) browser will pause DOM construction process. If you are familiar with event loop, there is simple rule how event loop executes tasks:
Execute macro tasks;
execute micro tasks;
Rendering;
So when you provide Javascript file, browser do not know what JS code is going to do and stops all DOM construction process and Javascript interptreter starts parsing and executing Javascript code.
Even you provide Javascript in the end of body tag, Browser will proceed all above steps to HTML and CSS but except rendering. it will find out Script tag and will stop until JS is done.
But HTML provided two additional options for script tag: async and defer.
Async - means execute code when it is downloaded and do not block DOM construction during downloading process.
Defer - means execute code after it's downloaded and browser finished DOM construction and rendering process.
It seems the behavior of defer and async is browser dependent, at least on the execution phase. NOTE, defer only applies to external scripts. I'm assuming async follows same pattern.
In IE 11 and below, the order seems to be like this:
async (could partially execute while page loading)
none (could execute while page loading)
defer (executes after page loaded, all defer in order of placement in file)
In Edge, Webkit, etc, the async attribute seems to be either ignored or placed at the end:
data-pagespeed-no-defer (executes before any other scripts, while page is loading)
none (could execute while page is loading)
defer (waits until DOM loaded, all defer in order of placement in file)
async (seems to wait until DOM loaded)
In newer browsers, the data-pagespeed-no-defer attribute runs before any other external scripts. This is for scripts that don't depend on the DOM.
NOTE: Use defer when you need an explicit order of execution of your external scripts. This tells the browser to execute all deferred scripts in order of placement in the file.
ASIDE: The size of the external javascripts did matter when loading...but had no effect on the order of execution.
If you're worried about the performance of your scripts, you may want to consider minification or simply loading them dynamically with an XMLHttpRequest.
Default - By default, as soon as the browser sees a script tag it downloads the file and then executes the script file. The script files are executed in the order of their occurrence.
async - The browser will download the script file and continue parsing HTML parallelly until the file is downloaded. The file is executed as soon as it is downloaded.
defer - The browser will download the script and do HTML parsing at the same time. After parsing is done, the script files are executed in the order of their occurrence.
Note:
In defer, the js files are executed in the order of their occurrence in the HTML file while in the case of the async attribute the script files are executed in the order of download time.
Both the async and defer attributes are used to load external JavaScript files asynchronously, which means that the HTML parsing and rendering process is not blocked while the external file is being downloaded and executed.
When you download a web page, 2 major things happen in your browser:
HTML Parsing
Loading of the scripts
2.1 Fetching scripts from the network
2.2 Executing the scripts line by line
However, there are some differences between the two attributes:
1.async attribute:
The async attribute allows the browser to download the script file asynchronously while continuing to parse the HTML document. Once the script file is downloaded, it will be executed immediately, regardless of whether or not the HTML document has finished parsing. This means that the script may execute before the rest of the page has loaded.
Example:
<script async src="script.js"></script>
2.defer attribute:
The defer attribute also downloads the script file asynchronously, but it will not be executed until the HTML document has finished parsing. This means that the script will be executed in the order in which it appears in the HTML document, and after the page has finished loading.
Example:
<script defer src="script.js"></script>
In general, the async attribute is used when the script file does not depend on other scripts or on the HTML document being fully loaded, while the defer attribute is used when the script file depends on other scripts or on the HTML document being fully loaded.
Summary:
Async is suitable if your script doesn’t contains DOM manipulation and other scripts doesn’t depend upon on this.
Eg: bootstrap cdn,jquery
Defer is suitable if your script contains DOM manipulation and other scripts depend upon on this.
Eg: <script src=”createfirst.js”> //let this will create element <script src=”showfirst.js”> //after createfirst create element it will show that.
Thus make it:
Eg: <script defer src=”createfirst.js”> //let this will create element <script defer src=”showfirst.js”> //after createfirst create element it will
This will execute scripts in order.
But if i made:
Eg: <script async src=”createfirst.js”> //let this will create element <script defer src=”showfirst.js”> //after createfirst create element it will
Then, this code might result unexpected results.
Coz: if html parser access createfirst script.It won’t stop DOM creation and starts downloading code from src .Once src got resolved/code got downloaded, it will execute immediately parallel with DOM.
What if showfirst.js execute first than createfirst.js.This might be possible if createfirst takes long time (Assume after DOM parsing finished).Then, showfirst will execute immediately.