what's the best way for d3 to manipulate html? - javascript

I'm using mbostock's awesome d3 for simple html rendering, the simplest way I could think of, selecting an empty div and filling it with HTML.
d3.select($('#thediv')[0])
.selectAll('p')
.data(l).enter()
.append('p')
.html(function(d){return 'd3 ' + d;});
In Win7 Chrome, I noticed that, for large datasets, this seems to be very slow. (3.5 s for 20000 items)
This example renders a very simple (though long) list: http://jsfiddle.net/truher/NQaVM/
I tried two solutions, visibility toggling:
$('#thediv').toggle()
and writing to a detached element:
var p = $('<div>')
d3.select(p[0])...
$('#theparent').append(p)
The results were this:
Firefox: uniformly fast (600-700 ms)
Chrome: either toggling or detaching is fast (800 ms), otherwise very slow (3.5 s)
Internet Explorer: detaching is faster but still slow (1.8 s), otherwise very slow (3.2 s)
My question is this: Is there a better solution?

Generally the two things that end up being slow when doing DOM manipulation are modifications to the DOM tree and browser redraws.
Using visibility toggling allows you to avoid the browser redraw in any modern, optimized DOM engine. You will still pay the price of modifying the DOM tree in line though.
Editing a detached element will give you both the no redraw perk as well as not having to pay for the updates to the DOM.
In all cases the second option, out of flow DOM manipulation, will be faster or at worst as fast. The relative difference in speed between browsers is more than probably due to differences in their DOM and Javascript engines. There are some benchmarks that might give you more insight into this.
Besides the speed of the DOM manipulations itself you could start looking into loop unrolling and other optimizations (by Jeff Greenberg) to minimize the work your actual script is doing. Other than that you are doing it right already.
If you are interested in knowing more details about the browser internal you should defiantly have a look at the long article at html5rock about browser internals. There is also a nice article over at Google developers about how to speed up JavaScript in webpages.

Related

Speed difference between inserting html and changing display style property

Assuming you have a relatively small piece of HTML (let's say under 100 tags and <4KB in size) and you want to occasionally display and hide it from your user (think menu, modal... etc).
Is the fastest approach to hide and show it using css, such as:
//Hide:
document.getElementById('my_element').style.display = 'none';
//Show:
document.getElementById('my_element').style.display = 'block';
Or to insert and remove it:
//Hide
document.getElementById('my_element_container').innerHTML = '';
//Show:
const my_element_html = {contents of the element};
document.getElementById('my_element_container').innerHTML = my_element_html;
// Note: insertAdjacentHTML is obviously faster when the container has other elements, but I'm showcasing this using innerHTML to get my point across, not necessarily because it's always the most efficient of the two.
Obviously, this can be benchmarked on a case by case basis, but, with some many browser versions and devices out there, any benchmarks that I'd be able to run in a reasonable amount of time aren't that meaningful.
I've been unable to find any benchmarks related to this subject.
Are there any up to date benchmarks comparing the two approaches? Is there any consensus from browsers developers as to which should, generally speaking, be preferred when it comes to speed.
In principle, DOM manipulation is slower than toggling display property of existing nodes. And I could stop my answer here, as this is technically correct.
However, repaint and reflow of the page is typically way slower and both your methods trigger it so you could be looking at:
display toggle: 1 unit
DOM nodes toggle: 2 units
repaint + reflow of page: 100 units
Which leaves you comparing 101 units with 102 units, instead of comparing 3 with 4 (or 6 with 7). I'm not saying that's the order of magnitude, it really depends on the actual DOM tree of your real page, but chances are it's close to the figures above.
If you use methods like: visibility:hidden or opacity:0, it will be way faster, not to mention opacity is animatable, which, in modern UIs, is preferred.
A few resources:
Taming huge collections of DOMs
Render-tree Construction, Layout, and Paint
How Browsers Work: Behind the scenes of modern web browsers
An introduction to Web Performance and the Critical Rendering Path
Understanding the critical rendering path, rendering pages in 1 second
Web performance, much like web development, is not a "press this button" process. You need to try, fail, learn, try again, fail again...
If your elements are always the same, you might find out (upon testing) caching them inside a variable is much faster than recreating them when your show method is called.
Testing is quite simple:
place each of the methods inside a separate function;
log the starting time (using performance.now());
use each method n times, where n is: 100, 1e3, 1e4,... 1e7
log finishing time for each test (or difference from its starting time)
Compare. You will notice conclusions drawn from 100 test are quite different than the ones from 1e7 test.
To go even deeper, you can test differences for different methods when showing and for different methods when hiding. You could test rendering elements hidden and toggle their display afterwards. Get creative. Try anything you can think of, even if it seems silly or doesn't make much sense.
That's how you learn.

Slow javascript execution in Iframe only in IE

The Problem:
I've developed a web application. It is embedded in a site with the help of an iFrame.
If I run the application as a stand alone (IE9) on say: www.example.com/webapp it loads in about ten seconds flat (it's a rather large application). Chrome and FF are much faster.
If It's embedded in an iFrame however, IE completely loses it with javascript execution times up to 40-60 seconds until the app is done loading. Once the application is loaded however there are no issues and it runs flawlessly.
Recap: Stand alone: OK, in iFrame: Not OK.
In the web application a few xml's are loaded, specifically a very large one which is about 8mb. The xml's are parsed and content is created using KnockoutJS. However this is not very relevant as I've narrowed it down to the XML parsing which is done with jQuery.
Stand alone the parsing takes about 10 seconds in IE9. Embedded it's around 40-60. I've consoled out the status logs and timestamps and I can physically see the javascript is running incredibly slow embedded. Every trace-out takes 4-6 times as long which corresponds with the increased overall load time.
FireFox and Chrome are immune and show no slowdown or so little slowdown that it's unnoticeable.
I've tried iFrame and Object embedding. Same results.
The question
Do you know why simple javascript execution (XML Parsing when the xml IS loaded and in memory), would take 4-6 times longer when embedded in an iframe than in stand alone?
Bonus info
I'm not talking about page load here. Everything loads fine. Even the host page. This is not yet another page is hanging until iframe is ready problem. the problem is the execution inside the iframe being slow. I've tried embedding on same domain, foreign domain, internal, external. Same problem everywhere. As soon as I iframe the damn thing, load performance goes to hell. Once it's loaded, everything is fine and everything runs very well.
PS: I hope the bolding of what i find is keywords is OK. It's supposed to be a help, not be annoying. I personally have problems focusing on large amounts of text.
**
Performance Monitor while it's loading:
IE9**
http://imgur.com/iYdMuPe
I found that setting element size with jQuery .height(n) and .width(n) can be extremly slow, you may use .css("width",x) and .css("height",x) instead.
First, hit F-12 and confirm the document mode is the same in both instances. If not, change the document mode of the outer frame to match..
If they are already the same, try instead to load the iFrame script dynamically after the outer page is complete. Older versions of IE handle resource allocation oddly and could be part of the problem.
Granted, not the answer to your question but bringing 8 MB of XML to the client is quite inefficient. Can any of this be stripped out or entirely processed server side?
Lastly, IE is slow to move and add DOM elements (compared to Chrome). Your best bet is to add them all at once. So if you are updating the UI as you parse the XML (instead of all at once after parsing), that will slow you down considerably.
Similar to what #ern0 said, if you are manipulating height and width in your script and are experiencing slowness then changing from using jQuery's .height() and .width() methods to vanilla JS could realize a significant performance improvement.
Getters
Here is a performance test for reading the element's current height. It shows that the vanilla JS property offsetHeight is significantly faster than the .height(), .css("height") and .style.height techniques.
The difference is so significant that it is not even a competition.
Setters
Here is a performance test for setting the element's current height. It shows that the vanilla JS property .style.height is significantly faster* than the .height(), and .css("height") methods.
Again, the difference is so significant that it is not even a competition.
Summary
The .style.height property excels in both getting and setting by an incredible margin, as compared to the jQuery methods. The read-only offsetHeight property is significantly faster than the style.height property for getting, but (as it is read-only) it cannot be used for setting the height. As such, it may be easier to just change the code to use .style.height, if it still achieves the desired effect.
The height and width properties and methods should be pretty much the same. If you want to add performance benchmarks for them too, that is fine, but you should get the same outcome, with the width properties and methods finishing in the same place as their corresponding height counterparts.
Apparently IE had a serious problem with getting attributes of an xml node through jQuery in a deeply nested loop. Changing this to pure JS reduced load time to about 15 seconds. Still not great, but much, much better!

Reflow/Layout performance for large application

I am using GWT to build a HTML application where the performance is correct in general.
Sometimes, it can load many objects in the DOM and the application becomes slow. I used Chrome Developer Tools Profiler to see where that time was spent (under Chrome once the app is compiled ie no GWT overhead) and it is clear that the methods getAbsoluteLeft()/getBoundingClientRect() consume the major part of this time.
Here is the implementation used under Chrome (com.google.gwt.dom.client.DOMImplStandardBase) :
private static native ClientRect getBoundingClientRect(Element element) /*-{
return element.getBoundingClientRect && element.getBoundingClientRect();
}-*/;
#Override
public int getAbsoluteLeft(Element elem) {
ClientRect rect = getBoundingClientRect(elem);
return rect != null ? rect.getLeft()
+ elem.getOwnerDocument().getBody().getScrollLeft()
: getAbsoluteLeftUsingOffsets(elem);
}
This makes sense to me, as the more elements in the DOM, the more time it may take to calculate absolute positions. But it is frustrating because sometimes you know just a subpart of your application has changed whereas those methods will still take time to calculate absolute positioning, probably because it unnecessarily recheck a whole bunch of DOM elements. My question is not necessarily GWT oriented as it is a browser/javascript related problem :
Is there any known solution to improve GWT getAbsoluteLeft/javascript getBoundingClientRect problem for large DOM elements application ?
I did not find any clues on the internet, but I thought about solution like :
(reducing number of calls for those methods :-) ...
isolate part of the DOM through iframe, in order to reduce the number of elements the browser has to evaluate to get an absolute position (although it would make difficult components to communicate ...)
in the same idea, there might be some css property (overflow, position ?) or some html element (like iframe) which tell the browser to skip a whole part of the dom or simply help the browser to get absolute position faster
EDIT :
Using Chrome TimeLine debugger, and doing a specific action while there are a lot of elements in the DOM, I have the average performance :
Recalculate style : nearly zero
Paint : nearly 1 ms
Layout : nearly 900ms
Layout takes 900ms through the getBoundingClientRect method. This page list all the methods triggering layout in WebKit, including getBoundingClientRect ...
As I have many elements in the dom that are not impacted by my action, I assume layout is doing recalculation in the whole DOM whereas paint is able through css property/DOM tree to narrow its scope (I can see it through MozAfterPaintEvent in firebug for example).
Except grouping and calling less the methods that trigger layout, any clues on how to reduce the time for layout ?
Some related articles :
Minimizing browser reflow
I finally solve my problem : getBoundingClientRect was triggering a whole layout event in the application, which was taking many times through heavy CSS rules.
In fact, layout time is not directly proportional to the number of elements in the DOM. You could draw hundred thousands of them with light style and layout will take only 2ms.
In my case, I had two CSS selectors and a background image which were matching hundred thousands of DOM elements, and that was consuming a huge amount of time during layout. By simply removing those CSS rules, I reduce the layout time from 900ms to 2ms.
The most basic answer to your question is to use lazy evaluation, also called delayed evaluation. The principle is that you only evaluate a new position when something it depends upon has changed. It generally requires a fair amount of code to set up but is much cleaner to use once that's done. You'd make one assignment to something (such as a window size) and then all the new values propagate automatically, and only the values that need to propagate.

Should I use multiple canvases (HTML 5) or use divs to display HUD data?

I am in process of making a game where the health bar (animated) and some other info represented visually like some icons showing the number of bombs the player has etc. Now, this can be done both in canvas (by making another canvas for info that sits over the main canvas, or it can be done using many divs and spans with absolute positioning. This is my first time in making a browser based game so if any experienced people view this, tell me what you recommend. I would like to know that which method would be faster.
The game will also be running on mobile devices. Thanks!
There is no straighforward answer and I suggest you do FPS testing with different browser how it plays out for your use case. If you do not wish to go such in-depth I suggest you simply draw the elements inside canvas and if you need to hide them then leave out drawHUD() call from your rendering loop.
For HTML HUD overlay on <canvas> the following factors should be considered
Can the web browser compositor do hardware accelerated <canvas> properly if there are DOM elements upon the canvas
HTML / DOM manipulation will be always slower than <canvas> operations due to inherited complexity dealing with DOM elements
<canvas> pixel space stays inside <canvas> and it might be difficult to have pixel-perfect aligment if you try to draw elements on <canvas> outside the canvas itself
HTML offers much more formatting options for text than canvas drawString() - is HTML formatting necessary
Use the canvas. Use two canvases if you want, one overlaid over the other, but use the canvas.
Touching the DOM at all is slow. Making the document redo its layout because the size of DOM elements moved is very slow. Dealing with the canceling (or not) of even more events because there are DOM items physically on top of the canvas can be a pain and why bother dealing with that?
If your HUD does not update very often then the fastest thing to do would be drawing it to an in-memory canvas when it changes, and then always drawing that canvas to the main canvas when you update the frame. In that way your drawHud method will look exactly like this:
function drawHUD() {
// This is what gets called every frame
// one call to drawImage = simple and fast
ctx.drawImage(inMemoryCanvas, 0, 0);
}
and of course updating the HUD information would be like:
function updateHUD() {
// This is only called if information in the HUD changes
inMemCtx.clearRect(0, 0, width, height);
inMemCtx.fillRect(blah);
inMemCtx.drawImage(SomeHudImage, x, y);
var textToDraw = "Actually text is really slow and if there's" +
"often repeated lines of text in your game you should be" +
"caching them to images instead";
inMemCtx.fillText(textToDraw, x, y);
}
Since HUDs often contain text I really do urge caching it if you're using any. More on text performance here.
As others have said, there is no universally best approach, as it depends on the specifics of what you need to render, how often, and possibly what messaging needs to happen between graphical components.
While it is true the DOM reflows are expensive, this blanket warning is not always applicable. For instance, using position:fixed; elements avoids triggering reflows for the page (not necessarily within the element if there are non-fixed children). Repaint is (correct me if this is wrong) expensive because it is pixel pushing, and so is not intrinsically slower than pushing the same number of pixels to a canvas. It can be faster for some things. What's more, each has certain operations that have performance advantages over the other.
Here are some points to consider:
It's increasingly possible to use WebGL-accelerated canvas elements on many A-grade browsers. This works fine for 2D, with the advantage that drawing operations are sent to the GPU, which is MUCH faster than the 2D context. However this may not be available on some target platforms (e.g., at the time of this writing, it is available in iOS Safari but not in the iOS UIWebView used if you target hybrid mobile applications.) Using a library to wrap canvas can abstract this and use WebGL if its available. Take a look at pixi.js.
Conversely, the DOM has CSS3 animations/transitions which are typically hardware-accelerated by the GPU automatically (with no reliance on WebGL). Depending on the type of animation, you can often get much faster results this way than with canvas, and often with simpler code.
Ultimately, as a rule in software performance, understanding the algorithms used is critical. That is, regardless of which approach used, how are you scheduling animation frames? Have you looked in a profiler to see what things take the most time? This practice is excellent for understanding what is impacting performance.
I've been working on an app with multiple animations, and have implemented each component both as DOM and canvas. I was initially surprised that the DOM version was higher performant than the canvas (wrapped with KineticJS) version, though I know see that this was because all the animated elements were position:fixed and using CSS (under the hood via jQuery UI), thereby getting GPU performance. However the code to manage these elements felt clunky (in my case, ymmv). Using a canvas approach allows more pixel-perfect rendering, but then it loses the ability to style with CSS (which technically allows pixel-perfect rendering as well but may be more or less complex to achieve).
I achieved a big speed up by throttling the most complex animation to a lower framerate, which for my case is indistinguishable from the 60fps version but runs smooth as butter on an older iPad 2. Throttling required using requestAnimationFrame and clamping calls to be no more often than the desired framerate. This would be hard to do with CSS animations on the DOM (though again, these are intrinsically faster for many things). The next thing I'm looking at is syncing multiple canvas-based components to the same requestAnimationFrame loop (possibly independently throttled, or a round-robin approach where each component gets a set fraction of the framerate, which may work okay for 2-3 elements. (Incidentally, I have some GUI controls like sliders that are not locked to any framerate as they are should be as close to 60fps as possible and are small/simple enough that I haven't seen performance issues with them).
I also achieved a huge speed boost by profiling and seeing that one class in my code that had nothing to do with the GUI was having a specific method called very often to calculate a property. The class in question was immutable, so I changed the method to memoize the value and saw the CPU usage drop in half. Thanks Chrome DevTools and the flame chart! Always profile.
Most of the time, the number of pixels being updated will tend to be the biggest bottleneck, though if you can do it on the GPU you have effectively regained all the CPU for your code. DOM reflows should be avoided, but this does not mean avoid the DOM. Some elements are far simpler to render using the DOM (e.g. text!) and may be optimized by the browser's (or OS's) native code more than canvas. Finally, if you can get acceptable performance for a given component using either approach (DOM or canvas), use the one that makes the code simplest for managing that type of component.
Best advice is to try isolated portions in the different approaches, run with a profiler, use techniques to over-draw or otherwise push the limits to see which approach can run fastest, and do NOT optimize before you have to. The caveat to this rule is the question you are asking: how do I know in advance which technical approach is going to allow the best performance? If you pick one based on assuming the answer, you are basically prematurely optimizing and will live with the arbitrary pain this causes. If instead you are picking by rapid prototyping or (even better) controlled experiments that focus on the needs of your application, you are doing R&D :)
Browserquest displays their HUD using HTML elements, which has the benefit that you don't have to worry about redrawing etc. (and the performance will be pretty good, given that the entire browser engine is optimized to render the DOM pretty fast.
They (browserquest) also use several layered canvas elements for different game elements. I don't know the exact structure, but I guess that on which canvas an element is displayed depends on how often it needs to be redrawn.

Javascript performance problems with too many dom nodes?

I'm currently debugging a ajax chat that just endlessly fills the page with DOM-elements. If you have a chat going for like 3 hours you will end up with god nows how many thousands of DOM-nodes.
What are the problems related to extreme DOM Usage?
Is it possible that the UI becomes totally unresponsive (especially in Internet Explorer)?
(And related to this question is off course the solution, If there are any other solutions other than manual garbage collection and removal of dom nodes.)
Most modern browser should be able to deal pretty well with huge DOM trees. And "most" usually doesn't include IE.
So yes, your browser can become unresponsive (because it needs too much RAM -> swapping) or because it's renderer is just overwhelmed.
The standard solution is to drop elements, say after the page has 10'000 lines worth of chat. Even 100'000 lines shouldn't be a big problem. But I'd start to feel uneasy for numbers much larger than that (say millions of lines).
[EDIT] Another problem is memory leaks. Even though JS uses garbage collection, if you make a mistake in your code and keep references to deleted DOM elements in global variables (or objects references from a global variable), you can run out of memory even though the page itself contains only a few thousand elements.
Just having lots of DOM nodes shouldn't be much of an issue (unless the client is short on RAM); however, manipulating lots of DOM nodes will be pretty slow. For example, looping through a group of elements and changing the background color of each is fine if you're doing this to 100 elements, but may take a while if you're doing it on 100,000. Also, some old browsers have problems when working with a huge DOM tree--for example, scrolling through a table with hundreds of thousands of rows may be unacceptably slow.
A good solution to this is to buffer the view. Basically, you only show the elements that are visible on the screen at any given moment, and when the user scrolls, you remove the elements that get hidden, and show the ones that get revealed. This way, the number of DOM nodes in the tree is relatively constant, but you don't really lose anything.
Another similar solution to this is to implement a cap on the number of messages that are shown at any given time. This way, any messages past, say, 100 get removed, and to see them you need to click a button or link that shows more. This is sort of what Facebook does with their profiles, if you need a reference.
Problems with extreme DOM usage can boil down to performance. DOM scripting is very expensive, so constantly accessing and manipulating the DOM can result in a poor performance (and user experience), particularly when the number of elements becomes very large.
Consider HTML collections such as document.getElementsByTagName('div'), for example. This is a query against the document and it will be reexecuted every time up-to-date information is required, such as the collection's length. This could lead to inefficiencies. The worst cases will occur when accessing and manipulating collections inside loops.
There are many considerations and examples, but like anything it depends on the application.

Categories

Resources