Develop Reference

How can I speedup adding large amounts of complex HTML to the DOM

I have a plain webpage that loads the majority of it's content through an AJAX call that returns a large amount of complex HTML. When I place the retrieved HTML in the DOM, it cripples the browser for quite a while (between 5 seconds on Chrome and 35 seconds on Edge).
Example how I append the HTML to the DOM:
$.ajax("example.php").done(function (response) {
const contentElement = document.getElementById('results');
contentElement.innerHTML = response;
});
I want to avoid having to return JSON and converting it to HTML at all cost because of the complexity of the application.
The odd thing is that the browser gets crippled a little while after the inserted HTML is already visible. See the timeline below, where I can see the HTML on my screen (with proper styling) before the ~5 sec long Parse HTML events happen.
How do I speed up the parsing and appending of the HTML to the DOM?
Edit: I have tried multiple browsers and multiple methods of injecting the HTML (documentFragments, innerHTML, jquery .html(), append()). All methods are roughly as slow.
Edit2: The exact HTML injected can be seen in this gist: https://gist.github.com/Rhinni/3032e74bab0de8f40e08a3392c0243b1

Part 1 - It's not the way the code is loaded, it's just the code is invalid and wouldn't work even if hardcoded on the page.
"The odd thing is that the browser gets crippled a little while after the inserted HTML is already visible. See the timeline below, where I can see the HTML on my screen (with proper styling) before the ~5 sec long Parse HTML events happen."
There are some things that should be addressed concerning the practicality of the HTML (its preposterousness speaks for itself), its validity (which it isn't) and functionality, (which it isn't and probably never had any).
You should validate your HTML because it is very invalid, but before we get into that, when you decide to validate that mess, you'll need to separate it into about 16 parts because most online services will fall or cut the validation process early if given that much to process at one time.
The following is a list of issues that are not isolated problems due to a typo. These issues are repeated multiple times. What concerns me the most is that the values and the majority of the variables appear to be customized by hand. Hopefully I am mistaken and you didn't spend hours on customizing values that will hinder rather than be of any real use.
1. #IDs must be unique -- under no circumstances should there ever be a duplicated #ID on the same page.
14 #accordion- fixed, 14 #headingOne - fixed, 7 #model, 7 #type, 7#brand,...
There's more duped #IDs, I changed the #accordion to #acordion1 to 14 because it was necessary for each #accordion to function rather than just the first one. All related attributes that has a direct relationship with #accordion needs to be changed as well, I managed to change toggle-parent="#accodion for the sake of functionality once again. So there's 15 functioning accordions, I added a Home tab with a properly designed accordion which you can use as a template if you decide to redesign the other 14 accordions.
2. In order to use Bootstrap components, you make them according to the document.
The OP code wasn't even close to having any tabs, if you referred to the Bootstrap documents or even W3School's short tutorials, you'd know that you are required to have an <a> for each tab, so your code was short of 16 <a> to toggle 16 tabs. This is why your page is only showing the first tab of 16 tabs, it isn't because the browser just fails midway.
3. Another invalid thing I noticed was that the attribute readonly (and required to a lesser extent) was applied to almost every form control.
Why would you need the readonly attribute on a <select> tag? When assigning attributes to elements, don't start adding a ton of attributes to everything. The clutter makes readability, maintenance, and debugging impossible.
4. There are 2 Plunks:
Plunk 1 is the solution to the OP (Original Post) question which is explained in detail in Part 2 of this answer. The HTML has been partially fixed, I don't have enough time to fix everything.
It has 16 tabs and 15 accordions that work.
Load time has been reduced from 34 sec to 2 sec. with Edge. It appears that Edge heroically tries to make sense of the HTML that was parsed and then fails. The real browsers like Firefox and Chrome justs dumps it and leaves it there.
Plunk 2 is the HTML from the OP code and my solution loading it.
The results are the same, OP code is failing due to the code itself, not because of a loading problem.
Part 2 - A stable way to parse a huge string into HTML. Not needed if OP code actually worked.
OP experiencing heavy latency when attempting to add a huge amount of markup to DOM by innerHTML. Up to 34 seconds to render it completely using Edge while other browsers OP reported at 3 seconds.
I got the load time down to 2 to 3 seconds on Edge and instantly on the real browsers (Chrome and Firefox).
Although OP had tried using createDocumentFragment() already, I believe it is key to a quick load and parse of said HTML. The other key components that the OP probably didn't use are:
insertAdjacentHTML() and Immediately Invoked Function Expression
Using insertAdjacentHTML() method instead of innerHTML property. insertAdjacentHTML() is a powerful and versatile version of innerHTML.
Similarities:
Both will take a given string and parse as HTML.
Both are quick.
Differences:
insertAdjacentHTML() inserts HTML into the DOM, it doesn't overwrite any existing HTML in an element or anywhere in the DOM. innerHTML overwrites the inside of an element.
innerHTML is directed by reference to an element to which it will take a string and overwrite all content of said element with the given string. If innerHTML is just directed to an element without a string, then it will return with the HTML content of said element. innerHTML ability to GET is the only thing insertAdjacentHTML() it can't do. In contrast, insertAdjacentHTML() ability to SET is powerful as explained: insertAdjacentHTML() is directed not only by reference to an element it is told exactly where to go in relation to the referenced element by its first parameter which is one of the 4 DOMStrings that correlate to a position:
"beforebegin" places the string right before the beginning of the element.
`$elector.before(str)`★
"afterend" places the string right after the end of the element.
`$elector.after(str)`★
"afterbegin" places the string within the element right after the brginning. In other words the string is inserted before the element's contents.
`$elector.prepend(str)`★
"beforeend" places the string within the element right before the end. Basically the string is placed after the element's content. This position is the most optimized for speed since the are no proceeding siblings to slow things down.
`$elector.append(str)`★
insertAdjacentHTML() second parameter is the string that will be parsed into HTML. Using Template Literals instead of literal strings allows us to a whole extra level of easy string manipulation.
`element.insertAdjacentHTML("beforeend", <input id="${ID+i}" type="${typeArr[i]}" value="${Math.floor(Math.random() * i}">)`
Immediately Invoked Function Expression is a function with a special pattern.
It is usually two anonymous functions:
The outer function is wrapped in parenthesis.
The inner function usually forms a closure.
Anonymous/expression functions are created when evaluated and then they are immediately invoked due to the extra parenthesis wrapped around them.
They have no name and the variables that the inner function uses can only be accessed by the outer function because they are local scope.
These conditions make an IIFE a one time thing. The signature of an IIFE have slight variances in their signature but the gist of one goes like this:
`(function() { var x = function() {...} x})();`
DOM manipulation is processor intensive and the more we avoid it the better. DocumentFragment was made in order for us to do all of the menial yet numerous tasks involving the DOM -- off of the DOM. We can add as many elements, text, attributes, set event handlers, etc. to the DocumentFragment and its descendants without touching the DOM. Once everything is complete, only one DOM operation needs to be done:
`document.body.appendChild(frag);`
Demo - if you'd like to test an actual working Demo, review this Plunk
<!DOCTYPE html>
<html>
<head>
<meta charset='utf-8'>
<title>Monstrosity</title>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.0.0/css/bootstrap.min.css" />
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css" />
</head>
<body>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.0.0/js/bootstrap.min.js"></script>
<script>
(function() {
const str = `Huge Disaster of HTML`
const frag = document.createDocumentFragment();
const node = document.createElement('div');
let build = function(node, str, frag) {
node.insertAdjacentHTML('beforeend', str);
frag.appendChild(node);
document.body.appendChild(frag);
}
build(node, str, frag);
}());
</script>
</body>
</html>

Simply appending or inserting the HTML you have provided into the browser does not seem to have any adverse affects. At least for me on my browser/computer. (chrome)
Run this example and see if you experience any delays or pauses..
See Example
document.getElementById("test").innerHTML = toAppend;
Obviously this is not a complete test because I am missing your CSS and I have modified your html slightly by removing line breaks so that I could assign the text to a variable in the text editor.
If the example works for you then we need to investigate further the data coming from the server, and try adding css to the equation etc..
If the example is causing delays then the problem is likely hardware related, maybe you don't have enough available memory and or cpu which is crippling the browser.

I fixed the issue with the help of the answers of Trevor and Zer00ne, but it was something completely different.
The issue was caused by the Laravel Debugbar which tracks AJAX requests by default, and parses the response for debugging purposes. Disabling AJAX request tracking in the Debugbar config solved the issue.

Dynamically displaying large amounts of text by partially displaying data in textarea

The Goal:
Optimize a web page that needs to display large amount of text data (25mb+) in a textarea or content editable div, without losing too much performance.
Currently, loading a 10mb file takes ~3 seconds on Chrome. I would like this to be 1 second max.
Ideas and Attempts:
I am loading local text files from a users computer using <input type="file"> and have no problem loading large files directly into memory. However as soon as I try and display this text data in a textarea, I naturally run into performance issues.
I have spellcheck, auto capitalization, and auto complete all disabled, and this certainly helped, however I would like to minimize the amount of lag when attempting to render large files (files greater than 10mb, max of 100mb would be ideal).
<textarea autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false">
One idea I had was to only render say 100 lines before and 100 lines after the current line, and when the users scrolls the textarea, I would then just switch out what data is being displayed. I can swap a few hundred lines without any noticeable lag, but hundreds of thousands locks up the entire page.
I was also looking at projects such as CodeMirror which is used in some javascript based text editors, and chrome dev tools. However a quick test showed similar performance issues when initially loading large amounts of text.
Another idea was to use highlight.js to render the text dom elements, but I was also noticing large amounts of large when dealing with thousands of DOM elemetns.
This site seems to tackle a similar example by dynamically creating and displaying the dom elements, instead of attempting to render all at once.
I have found that if the number of records in the grid becomes more then just a few thousands the grid gets very slow because the rendering speed is directly related to the number of nodes in the DOM. If the number of nodes in the DOM is more then 40-50k (depending on your computer configuration and amount of memory), your browser will crash or will become unresponsive.
So, I decided to set out on a quest to do two things: (1) dynamically create records as user scrolls (2) optimize grid to handle large data sets. After a few weeks of work, the grid was optimized and ready for testing.
I think this is similar to my first idea, but I have not tried this approach yet.
I'm hoping someone who's had experience with something similar might be able to offer some advice on which path to take, or offer some additional ideas.
Before anyone asks, I cannot not show this data, it needs to be user editable, it does not need to highlight code, or show line numbers. Lastly, the entire text file is being loaded with a FileReader into a variable.
I would like to avoid uploading the file to my webserver if possible for end user privacy and NDA concerns.
Server config: Ubuntu 16.04 LAPP Stack with Laravel 5.4, open to NodeJS solutions though. Use of jQuery is allowed.
Proposed Solution(s):
Lazy Loading - only displaying say 300 line "chunks" at a time as the user scrolls. Would need to make the scrollbar the appropriate height ahead of time in this case though. - Also, should unload these "chunks" as the users scrolls to reduce total the DOM rendering load.
Pseduo code:
c_chunk = scrollpos / scrollheight * totalChunks;
chunk_block = chunk[c_chunk--] + chunk[c_chunk] + chunk[c_chunk++];
my_textarea.val(chunk_block);
Any thoughts on this method?
Thanks all.

As promised here's my solution:
So after fumbling around with various ideas and trying different methods, I think I finally settled on using Ace for my project. I chose Ace for a few reasons.
Well documented API that was easy to understand in addition to a fairly large and active community.
Performance, Ace hands down gets far superior performance compared to my test with Code Mirror using large files. (500,000 lines, github says Ace has been tested with up to 4 million lines.
Features, out of the box Ace has more features to get you up and running with little user setup or configuration. Getting started took less than 10 lines total!
It doesn't support as many browsers as Code Mirror, but my project is running with WebGL so I was not very concerned.
So you understand my reasoning, and you want to get Ace up and running yourself? Easy, just hop on over to the build repo and grab one whichever flavor suites your needs. I grabbed the src-min release as I will just add it to my build script.
Then just include the javascript file in whatever way you do:
<script src="/ace/ace.js" type="text/javascript" charset="utf-8"></script>
And an element to your page with id="editor". In my case I'll attach it directly to a div:
<div id="editor">Any text you want to have auto displayed</div>
and in a javascript file:
var editor = ace.edit("editor");
editor.setTheme("ace/theme/chrome");
editor.session.setMode("ace/mode/javascript");
editor.$blockScrolling = Infinity;
If you just want to look around to see what languages/themes/options/etc. are available, just head over to Ace's kitchen sink page and play around with the editor.
Now I also needed a few functions, such as being able to load a file into the text area - and you can do that with the following:
Load local file:
// Add event listener to the file input
// Note: This will not fire twice if the user opens File A, then re-opens file A
// To detect re-opening a file you will need to clear the input
// after the file is read so that it "changes" upon re-opening
document.getElementById('open-file').addEventListener('change', handleFileOpen, false);
// First we create the function handler that fires when our input is changed
function handleFileOpen(e) {
var file = e.target.files[0]; // Get first file selected
// Load file using file reader function
loadFile(file);
}
function loadFile(file) {
// Create file reader and relevant Ace code
var reader = new FileReader();
reader.onload = function(e) {
// Get text contents of file
var data = e.target.result;
// Update Ace Editor with loaded data
editor.setValue(data, -1);
};
// Now that we defined the function we want to run, read the file
reader.readAsText(file);
}
The -1 in the setValue() function means place cursor at the beginning of the file (top) - 1 would be the bottom.
There are tons of events and properties you can hook into and mess in order to tweak this editor to your exact liking. They are all extremely straight forward and easy to use so if you're undecided, it's worth the time to try it out.
I wasted 2 days trying to get clusterize + highlightjs to work with my setup and gave up finally and replaced that entire setup with Ace in less than a day. Really impressed with editor so far!
Here's a link to their "How-to Guide": https://ace.c9.io/#nav=howto
and their API reference (which is extremely useful): https://ace.c9.io/#nav=api

Effeciency, hidden HTML or JavaScript DOM appending?

I am working on a simple Cordova app with about 4 page types and I am trying to think through which is the better way to handle the inner HTML templates.
Hidden HTML hard coded into the HTML files that is hidden and populated/revealed by my JS.
Using a JS template system and appending and removing from the DOM.
I feel that appending all that to the DOM for a page is inefficient when I could just update the sections that change. But perhaps an append is lightweight enough where I shouldn't worry about it.

There are a number of ways you can do it. In terms of load on the browser. That is hard to say. From your question it is hard to know what is in these pages, what are you displaying, is it live data, static html etc.
When you first plot out an app, if you are from the old class of building multiple page websites, it can be a little concerning as to how well your app/page will run with all those pages crammed in to one, and all that data plus code.
The answer is, amazingly well. If done properly in modern browsers, and for example Ipads the app will run to near native performance.
The options you have are
Map all the pages into one HTML document. Hide each page content using css display:none, flip them into view using css animation, fading or just display:block.
Use a javascript routing library to map urls to blocks of code that deal with each page, this makes mapping out your app much easier, and means that buttons can just link to your pages, like a real website. see http://projects.jga.me/routie/
Building all the page templates into one page can make it hard to code, as the page becomes enormous, consider breaking the inner content of each page into separate files, you can then give each page holder a url and use a small xhr request to load the page on-the fly, once loaded you can cache it into memory or even local-storage, depending on whether you remove it when it is closed or keep it hidden.
In my experience you can put an enormous number or nodes into one page and have very little speed drop, bear in mind if you use something like jquery and do a lot of $(".page > .page1 > .items li") your going to have a slow app.
Tips
Use element ID's everywhere document.getElementById(..) is 100's of times faster in a loop that $(...)
cache elements when you find them, if you need them later store them in a memory cache.
keep for loop inner code to a minimum.
use a decent click touch libary like http://hammerjs.github.io/ and delegate all the events of the body tag or at least on each page.
If you need to touch the server, load data, think dom first, device second server later. A good app is a responsive app, that responds to the user instantly.

I know this has been posted a while ago, but for the sake of the users I am going to add my answer.
I completely agree with MartinWebb but my answer will shed some light on the results of his options. I am currently working on a similar project. Please note that this answer pertains to cordova (previously called phonegap) specifically. My app has about 5 pages with +-20 different components (input's, div's, h1's, p's, etc.). This is what i tried and the result of each:
jQuery was my first option, mainly because it is easy to use and reduces the amount of code required to accomplish a said goal. Result: First time I tried this approach I though I would spice it up with animations and transformations. The result of this was a very unresponsive app. I removed the animation and transformation, however due to the nature of my application I required multiple dynamically added components and jQuery just wasn't up for the task.
Css display:none and visible:hidden was my next option. I used javascript's dom to display certain div's. Result: This works if your not planning on switching many div shortly after one another eg. a simple menu. It quickly became apparent that this wasn't going to work. Also this does not eliminate my need for the dom. Remember document.getElementById('menu').style.display = "none"; is still part of the dom. This as a solution, for me, is poor. There is a reason that var menu= document.createElement('div'); is part of the language. Which brings me to my last option.
Building a page 90% on javascript's dom was my last option. Logically I could not see how 600 lines of code cold trump one .innerHTML or .style.display = "block"; but it did. Result: It was by far the most responsive of all the solutions.
I'm not saying that all webpages should be coded with dom appending, but as I stated previously, for a cordova app of a few pages (<6), with a few components a javascript dom appending approach would be best. It takes longer to code, but you will be rewarded with control and efficiency. I would suggest coding the backbone of your app in html and populating and controlling with javascript's dom.
Best of luck.

The first option, <div>s with display:none; would be more efficient by a small margin, but you can get the best of both worlds by compiling your JavaScript and templates together into a single file using something like browserify or require.js.

How to use progressmeter in an XUL application?

This might be a simple question for geeks, but not for me, at least. I was developing a simple XUL program from scratch. I used wizard tag to simplify the GUI that appears to the user. On a wizard page I have inserted a progressmeter to show the progress of a function called myFunction() which belongs to the JavaScript which is available in the XUL file. How can I properly update the progressmeter based on the real progress of the function?

Yes, the reverse thread work round does work, but as per the doc (here: https://developer.mozilla.org/en/nsIThread) processNextEvent may wait for a very long time if there are no pending events! Err.. since xulrunner is single thread there appear to be few, resulting in a huge slow down. Waggling the mouse helps though.
On further investigation use processNextEvent(false) works without delay.

Have you tried setting the progression of the progressmeter using its "value" property?
myProgressmeter.value = 50;
You just need to increment this value depending on the progression of your function.
Note that the value should be set between 0 and myProgressmeter.max

Ways to increase performance when set big value to innerHTML

I'm trying to set a huge (200K) response to innerHTML of a node. The goal is to get better time than 2.7 sec in Internet Explorer 6.
Any ideas?

This won't make it happen any faster but it will stop the browser from locking up on the user; they can continue to use the page while this happens in the background:
function asyncInnerHTML(HTML, callback) {
var temp = document.createElement('div'),
frag = document.createDocumentFragment();
temp.innerHTML = HTML;
(function(){
if(temp.firstChild){
frag.appendChild(temp.firstChild);
setTimeout(arguments.callee, 0);
} else {
callback(frag);
}
})();
}
Using it:
var allTheHTML = '<div><a href="#">.............</div>';
asyncInnerHTML(allTheHTML, function(fragment){
myTarget.appendChild(fragment); // myTarget should be an element node.
});
This technique will take longer than plain innerHTML but the user will be able to carry on using your site without noticing a delay.

First off, I would make sure your HTML is as simple as possible. As minimal structure markup as possible. Use CSS everywhere, including its cascading feature so you're not setting class attribute everywhere, etc.
Then try some further research in the best methods per browser. For example, some do better by building DOM objects, some setting all of it to the innerHTML of a single non-tree DOM node then adding it to the tree, etc. I suggest you read Quirksmode.org's Benchmark - W3C DOM vs. innerHTML.
You'll also want to probably use a script like JimmyP suggested to add in chunks, so the browser UI doesn't appear to hang. Although, you don't want to split it up too much, or you'll be unnecessarily triggering too many page refreshes when you don't need them.

why not bring the response in json format and process it on the client side, for example here a big amount of data is driven each time you change any navigation option, and all rows are rendered by the client.

Make the HTML as simple as possible, or use XML with the shortest possible element / attribute names.
Style with CSS (Don't use XML with XSLT, that could end up being even slower to parse / build).
The handling of XML in different browsers is not compatible unfortunately.

With a large HTML payload you run the risk of the "Operation Aborted" error in IE 6. Take some time and review the following StackOverflow questions, as the error arises from updating the DOM using innerHTML and to date MS does not have a complete fix in place:
Why Does Asp.net cause the operation aborted error in ie7?
How To Overcome IE Bug When Dynamically Attaching DIV To Body From Script?

Make sure your JSON/HTML/Plain Text is as simple as possible. The easiest solution would be to do less work, but if you really do need to stream 200k, make sure the transport to the browser is compressed, which should be configurable in your app and/or web server. Gzip'ping content (assuming you're using an AJAX-friendly service) really does help out, especially with simple text.
Look at other things like any loops that can be simplified, etc., but it sounds like making sure the data is ready and making sure that data can be sent across the wire efficiently will be the most help.
If IE is the outlier and all other browsers work well, you might need to move on. There's only so much you can do, but I suspect it's not the main problem.

I've heard that innerText is about two times faster than innerHTML in IE. Not sure if it's true, though, but it might be worth a try.
if (window.ActiveXObject) { // we're using IE
document.getElementById('myElement').innerText = 'Bla bla bla bla';
// or create a textnode:
var txt = document.createTextNode('Lorem ipsum');
document.getElementById('myElement').appendChild(txt);
} else {
// other code here
}
Update: Note that if you want to modify the HTML of myElement, don't use innerText – the HTML will be visible in plain text, like if you were using < and >.