I know this question sounds very trivial, but I just want to know how 'browser processes DOM and what makes it become slow with large number of DOM elements? Is this just about the size? What if DOM elements are not high in number but javascript objects are? Would it still respond slow?
I guess, if there are events attached to javascript objects and we don't dispose them, it responds slow because it has to execute all the event handlers (in a sequential manner), but other than that what are the other reason where 'memory leak' slows down the browser? (Assuming browser has consumed lots of memory but enough memory is still usable in system).
Update:
Surprisingly, CPU and memory usage is always under control while browser responds slow.
If a page is loaded with all its elements and it doesn't change, then there is no reason why it should be slow no matter the amount of DOM elements. However, if you have a dynamic page, there are loads of operations that cause the entire layout to redraw itself. This is called layout thrashing and can have dramatic effects on performance.
The most obvious is that your browser consumps all your memory and when it comes time to render something during scrolling, for example it has no more memory.
If there are no memory issues, then JohanP is write - there is no reasons.
Why do browsers slow down when loading a lot of data? Because they have to load a lot of data. Large images are obviously the worst culprit in terms of load speed, but page load is directly correlated to the number of kilobytes being transmitted. If you have a lot of code, it's going to have a large filesize.
As for JavaScript, there are four main causes of leaks:
Accidental Global Variables -- Variables not explicitly defined will assume a global scope:
function foo(arg) {
bar = "This is a global variable";
}
Forgotten Timers Or Callbacks -- When declaring a variable in a timed function like setInterval(function() {}), the variable still exists at the end of the interval.
Out Of DOM References -- When assigning reference to an element, which later is removed, the reference still exists:
var button: document.getElementById('button');
document.body.removeChild(document.getElementById('button'));
Closures -- Loops often don't get closed 'correctly', losing variable scope, and leaking in the process. See JavaScript closures.
See 4 Types of Memory Leaks in JavaScript and How to Get Rid Of Them for further information on JavaScript memory leaks.
Hope this helps!
Related
I'm making a rather extensive game using Javascript. It is a kind of online game maker that allows players to upload media files and use them to create worlds. Unfortunately, it is rather prone to crashing the browser tab at unpredictable moments. So far, I have found no pattern to this - sometimes it happens within a few minutes, other times it can run for hours without a problem.
I have tried enabling logging in Chrome, but the crashes don't seem to generate an error report in the chrome_debug file.
I thought it might be that the program was using too much memory (given the game's open-ended nature, some worlds can involve downloading rather large data files - though this seems unrelated to when the crash actually happens - while large worlds do seem to be more crash-prone, they do not always crash when the world's data is loaded).
I tried using Electron to turn it into an executable app, but the app still crashes. That shouldn't happen if it's a memory issue, right?
Is there any way of finding out what is causing the code to crash?
Most unpredictable crashes in Javascript are caused by memory leaks - objects that are still stored in memory and not being picked up by the garbage collector. Every object in Javascript is stored in a variable somewhere within the global scope, or is associated with another object that is itself connected to the global scope. When a "branch" of the "tree" is removed and can no longer be accessed by the global scope, the garbage collector destroys it.
However, if an object is not being removed from the global scope when it should be, it remains in memory. This usually happens when objects are added to an array but are not removed from that array when they are no longer in use. Over time, these objects build up until the process crashes due to memory overload.
To find memory leaks in Chrome, press F12 and open the Performance tab. By recording the page over time, you can view the amount of memory being used. The green line (nodes) is the most important here - it refers to the number of objects in memory. If nodes are constantly increasing over time (there will always be increases and decreases, but if the overall level is constantly rising) this generally means there's a memory leak.
To find which specific objects are causing the problem, open the Memory tab to take snapshots or timeline profiles of the memory heap. This gives you a count of the specific objects that are in memory at any given time. If there are more of some kind of object than there should be, that's where the leak is.
I am working with quite large volume of data.
Mechanism:
JavaScript is reading WebSQL database, then assembles data into Object that has tree structure.
Then applies to tree object knockout.js (makes elements observable) then data-binds
and then applies Jquery Mobile UI at the end.
Whole process takes unacceptable amount of time.
I have already optimized algorithm that makes tree object out of data,
also optimised conversion to observables mechanism by pushing items directly into ko.observable arrays and calling hasMutated only once.
I am applying knockout.js IF bindings to not process invisible tree nodes in UI until parent is opened.
Performance here is key.
After inspecting page load in timeline in Chrome developer tools I have noticed that Garbage Collector is doing cleans on every concurrent call when I am building tree object.
Question: Is there a way to temporarily disable Chrome GC and then enable it again after I am done with page processing?
P.S I know I could add reference to part that gets collected, basically introduce object that dominates and prevents GC collection, but this would require substantial changes through the code, and I am not sure I could keep it long enough, and it is likely to introduce memory leak. Surely there must be better way
No, there is no way to disable the garbage collector. There cannot be, because what is Chrome supposed to do when more memory is requested but none is available?
(Also, the garbage collector is very fine-grained and complicated; your screenshot is a bit too small to be readable, but in all likelihood what you're seeing are small steps of incremental work to keep up with allocations, and/or "minor GC" cycles that only operate on the relatively small area of the heap where new allocations happen.)
If you want to reduce time spent in GC, then the primary way how to achieve that is to allocate fewer and/or smaller objects. Yes, that can mean changing your application's design so that objects are reused instead of being short-lived, or similar changes in strategy.
If you allocate a lot, you will see a lot of GC activity, there is just no way around that. This is true even in languages/runtimes that are not considered "garbage collected", e.g. in C/C++ using new/delete a lot also has a performance cost.
I am currently developing a website that is pure javascript and relies heavily on the jQuery & jQuery UI libraries (this site is not intended for use by a general public, hence progressive enhancement is not a strict requirement for this project). I am encountering a significant memory leak on executing the following code:
oDialogBox = $("<div>...</div>");
/* Add useful things to the dialog box here */
oDialogBox.appendTo("body");
oDialogBox.dialog({
/* Other dialog box settings here */
close: function(event, ui) {
oDialogBox.dialog("destroy");
oDialogBox.remove();
oDialogBox = null;
}
});
At any given time in this dialog box, I am creating, removing and modifying a large number of instances of jQuery UI buttons, multiselects (per the Multiselect widget created by Eric Hynds) and on click event handlers. According to jQuery UI documentation, calling .remove() on oDialogBox should result in all child widgets being unbound and deleted. Yet my detached DOM tree shows a significant number of garbage elements that the GC isn't collecting.
It is highly likely I have missed a large set of closures that need to be finished off safely. How do I do the following:
1) How do I identify which closures are keeping a given detached DOM object alive (either in Firefox or Chrome)?
2) Assuming the complete set of closures is identified, does anything beyond nulling the variable need to be done to assure marking the DOM element for garbage collection?
3) I have also noticed my list of arrays stored by the page is giant and contains references to DOM elements not being gathered by the GC. Is there a documented best practice for cleaning arrays from javascript and allowing all elements to be marked for deletion? (Note: this is a current prime suspect for the source of the memory leak)
I'm afraid that I don't have a great answer for #1. I haven't found any really good tools for this myself, even given how good the development tools have become over the last few years. The best advice I can give is to always keep things in the smallest scope you possibly can. If things don't escape, it's generally easier to simply figure out where the references must be.
As to #2, there can be further concerns. If the object referenced by variable v1 closes over the free variables of some function, removing v1 will not be enough to make them eligible for garbage collection if another variable v2 closes over v1 in some other function. So I guess if you really mean the "complete set of closures", then you should be all set. But this might get hairy. Again, if most object have references only in narrow scopes, these problems are much less severe.
For #3, what sorts of arrays are you discussing? If it's jQuery collections, then perhaps you simply have too many of them around. The only reason I know for them to stay around for a long time is to bind event handlers to them, and that is almost always better handled by event delegation on parent elements. If it's you're own custom arrays, do you really have a good reason to store references to them in arrays that last for any substantial length of time? I've rarely found one.
I'm suspecting that if i set div.innerHTML="" instead of using while(div.firstChild)div.removeChild(div.firstChild) the memory will be hogged until the page refreshes or the browser closes.
My question is how do we actully go about testing if my hypothesis is true?
Firstly, whether memory is returned to the system or not depends on the Javascript garbage collector (gc) and therefore results will vary from browser to browser.
It's difficult to measure memory usage by looking at the process as there are several layers of memory management in place. To see how this can have an impact, consider that a huge javascript object might have been erased forever, but that memory might still not have yet been released back to the operating system because the web browser might keep hold of it in case you need to create more big objects. Another example is that most gc routines only run periodically, so it's possible that object is still in memory but will be reclaimed later.
However, it's pretty easy to determine if a particular operation is leaking memory as all you have to do is repeat it in an endless loop. e.g.
remove references to existing html elements
construct new html elements
add them to the page
Try this code:
var div = document.getElementById("test")
while(true) {
// remove operation, change me
while(div.firstChild) {
div.removeChild(div.firstChild);
}
// create some new content
for(var i=0; i<1000; i++) {
var p = document.createElement('p');
p.appendChild(document.createTextNode('text'));
div.appendChild(p);
}
}
My results in Chrome, using the Task Manager (shift+esc):
Leaving the loop running endlessly without deleting anything eventually results in the "Aw! Snap" screen, which indicates memory has been exhausted
Using the removeChild technique leads to memory usage stabilizing at around 750MB
Using the innerHTML technique leads to memory usage stabilizing at around 750MB
If memory isn't leaking you'll notice a pattern similar to this: Increases to 300, drops to 150, increases to 400, drops to 250, eventually stabilising. This is the memory management system running out of memory, triggering the gc to reclaim memory that has been deallocated and increasing the available memory footprint each time until reaching a soft limit that has been set to avoid the process impacting others. This is a typical memory management scheme and more can be found by reading this wikipedia article: http://en.wikipedia.org/wiki/Garbage_collection_(computer_science)
Since both results stabilize, I'd conclude that (for Chrome at least) both techniques work the same though you might get different results from other browsers.
As there doesn't seem to be a difference, removeChild should be preferred as innerHTML is not included in the W3C standard and is frowned upon by some developers. See more here: Alternative for innerHTML?
The reason innerHTML and removeChild have no differences is because underneath, they both have to de-reference elements to stop them from being visible on the screen. Memory leaks are most likely to occur when you have circular references (A points to B, B points to A, nobody else points to either) but this is only a problem in older browsers. This link has some good guidelines about how to avoid js memory leaks: Javascript memory management pitfalls?
I have been reading up to try to make sense of memory leaks in browsers, esp. IE. I understand that the leaks are caused by a mismatch in garbage collection algorithms between the Javascript engine and the DOM object tree, and will persist past. What I don't understand is why (according to some statements in the articles I'm reading) the memory is not reclaimed after the page is unloaded by the browser. Navigating away from a webpage should put all the DOM and javascript objects out of scope at that point, shouldn't it?
Here's the problem. IE has a separate garbage collector for the DOM and for javascript. They can't detect circular references between the two.
What we used to was to clean up all event handlers from all nodes at page unload. This could, however, halt the browser while unloading. This only addressed the case where the circular reference was caused by event handlers. It could also be caused by adding direct references from DOM nodes to js objects which had a reference to the DOM node itself.
Another good thing to remember is that if you are deleting nodes, it's a good idea to remove the handlers yourself first. Ext-js has a Ext.destroy method that does just that (if you've set the handlers using ext).
Example
// Leaky code to wrap HTML elements that allows you to find the custom js object by adding
//a reference as an "expando" property
function El(node) {
this.dom = node;
node.el = this;
}
Then Microsoft hacked IE so it removed all event handlers and expando properties when unloading internally, therefore it's much faster than doing it with js. This fix seemed to fix our memory problems, but not all problems as there are people still having the problem.
MS's description of the problem
MS releases patch that "fixes" memory leaks:
Blog about fixed memory leaks
IE still has some problems
At our company, we use ext-js. By always setting event handlers using ext-js, which has a an internal clean up routine, we have not experienced memory leaks. In reality, memory usage grows but stops at about 250Mb for a machine with 4Gb of RAM. We don't think that's too bad since we load about 2Mb(uncompressed) of js files and all the elements on the page are dynamic.
There's a lot to be said about this and we've researched this extensively where I work. Feel free to ask a more specific question. I may be able to help you.
The best thing I ever read about Javascript memory leaks was written by Doulgas Crockford.
To answer your question, yes, the browser absolutely should unload all the objects (and most importantly, event handlers) at the appropriate time. If it did, it wouldnt have leaks :)
You don't have to make sense of them -- they are bugs in broswers and being fixed from versions to versions.