I am in the process of profiling a javascript library I wrote, looking for memory leaks. The library provides an API and service to a back-end. It does not do any html or dom manipulation. It does not load any resources (images, etc). The only thing it does is make xhr requests (using jquery), including one long poll, and it passes and receives data to and from the UI via events (using a Backbone event bus).
I have tested this library running it overnight for 16 hours. The page that loads it does nothing but load the library and sends a login request to start the service. There were no html, css, or other dom changes over the course of the test.
All that happened over the course of the test was the library sent a heartbeat (xhr request) to the server every 15 seconds, and received a heartbeat via the long poll every 30 seconds.
I ran the test with the chrome task manager open, and with the chrome debugger open, in order to force GCs from the timeline.
At the start of the tests, after I forced an initial GC - these were the stats from the chrome task manager:
Memory - 11.7mb
Javascript memory - 6.9 mb / 2.6mb live
Shared memory - 21.4 mb
Private memory 19mb
After 16hrs I forced a GC - these were the new stats:
Memory - 53.8mb
Javascript memory - 6.9 mb / 2.8mb live
Shared memory - 21.7 mb
Private memory 60.9mb
As you can see the JS heap grew by only 200k.
Private memory grew by 42mb!
Can anyone provide possible explanations of what caused the private memory growth? Does having the chrome debugger open cause or affect the memory growth?
One other thought I had is that forcing the GC from the timeline debugger only reclaimed memory from the JS heap - so other memory was not reclaimed. Therefore this may not be a 'leak' per se, as it may eventually be collected - though I'm not sure how to confirm this. Especially without knowing what this memory represents.
Lastly, I did read that xhr results are also stored in private memory. Does anyone know if this is true? If so, the app did perform approx 5700 xhr requests over this timeframe. If most of the 42mb was due to this, that would mean approx 7k was allocated - which seems high, considering the payloads were very small. If this is the case; when would this memory be released, is there anything I can do to cause it to be released, and would having the chrome debugger open impact this?
I was unable to find information on precisely when and what goes to private memory versus javascript memory. However I can answer this: "would having the chrome debugger open impact this" ... YES.
Having the developer tools opened causes the browser to gather/retain/display a lot more information about each and every XHR that is being made. This data is not gathered/retained when the developer tools are closed (as most know since it is sometimes bloody annoying when you open the dev tools to late and it missed that one request you cared about).
You can open the dev tools, trigger the GC, then close the dev tools, opening them only again to trigger the GC when you want to take the metric. Also you can use this sucker chrome://memory-redirect/ to keep track of the growth without opening the dev tools.
Related
I have web application running in chrome browser. When I ran for long time I can see increase in memory footprint and java script memory.
My application is switching screen from one page to another page and back to initial page every 15 seconds.
How do start analysis?
When you talk about memory leaks in JS, it may be due to an unintentional global variables, persistant closures or detached DOM nodes which are still referenced.
There are few tools in DevTools which you can use to detect the above cases.
You can use a Heap Snapshot and see the allocations to which you have direct references from the code.
You can use the Allocation Timeline to link the heap snapshot information to a timeline and let you see where the memory is allocated.
I'm working with a team on a single page application with quite a few timers, and about 10 REST API calls per second. We've used the Chrome memory profiling tools to locate all of our memory leaks--at this point the heap size stays about the same (i.e. in between garbage collections).
However, we're noticing that the longer the application runs, the more memory the Chrome tab itself uses. After a few hours, we've seen the tab memory grow to about 350MB, while the heap size remains at 6MB. If run overnight, the tab consumes over 500MB and becomes unusable by morning.
We need the application to run for long periods of time (we're hoping to restart it only once a week), so this is a problem.
Here's a screenshot of the heap timeline over the course of an hour. At the end, the tab's private memory size is 350MB, while the heap's size remained at around 5.4MB.
We've seen this in Chrome 40 and Chrome 38, on Windows 7. We have caching disabled, and are running the application in incognito mode.
Any idea why the tab memory size would grow to the point where Chrome is unusable, while the JS heap usage remains so small?
EDIT: We ran the app for a couple days and the tab memory size is up to 800MB, heap size still about the same.
EDIT (6/9/15): I'm not sure if this is documented anywhere, but it seems like Chrome changes the amount of memory it will allocate to a tab, based on the amount of free memory on the system. So, if you have a lot of free memory, tabs will use a lot of memory, and if you don't have much, it won't use as much. This doesn't seem to be proportional to heap size, it's probably just stuff that Chrome keeps around to make itself faster. This is only a theory, based on monitoring Chrome memory usage :)
This is probably too old and not relevant anymore, but for everyone else who experienced this: it's probably because of the opened dev tools. I had an application that would accumulate memory over time with the heap size staying the same, but when I closed the dev tools the memory went way down and didn't get back up like it did when dev tools was open. When profiling long term memory footprint, use chrome task manager first.
I can guess that in your case it happens because devtools has to store all those REST API calls (to show them in Network tab) and I'm not sure if chrome saves them to disk temporarily to save memory.
I am troubleshooting what appears to be a memory leak in our configuration page. The page is used to change the configuration of our service and also displays health diagnostics. This means that we are querying the service periodically for configuration and instrumentation information (typically we use a query interval of 30sec, but to troubleshoot I am querying at 100ms intervals). We rely on knockoutjs, datajs, jquery and spinjs.
I've found that if I leave the page open overnight at the 100ms query interval that the private bytes for the chrome browser tab grows from about 50MB to 335MB. I have four pages with the issue, but am focused on one during my troubleshooting effort. Using chrome://memory-redirect/ I can see the page (process id 26148) memory.
However, the JavaScript heap memory appears to be flat over the same period at 3.6MB. Using the heap profiling tools in Chrome it shows that all of my object allocations are garbage collected.
In the above picture the gray allocations indicate that the objects have been cleaned up by the GC.
The memory timeline also is constant.
I also forced two GCs and confirmed that the number of documents, nodes and listeners was constant between the two GCs.
My questions are:
Where is the process memory being used that is not part of the JavaScript heap?
Given our JavaScript heap memory is flat, could that extra memory be a memory leak caused by our JavaScript code?
Thanks for all the help!
You are comparing apples and oranges - Garbage collected memory in a sub-heap and whole application memory.
You've used Chrome to inspect the JavaScript heap and found evidence that indicates the JavaScript part of your application is running OK.
You're also using a tool to monitor the global memory usage of Chrome itself. That is all the memory that Chrome is using for any task, including tasks not directly related to your application, but to the functioning of the browser itself.
Perhaps you've found a use case that triggers a memory leak in the Chrome internals?
Or perhaps it isn't a memory leak, but memory fragmentation in the non-garbage collected internal heaps used by Chrome?
According to this web page Chrome is written in a mixture of C, C++, Java, JavaScript and Python. This means we have deterministic memory allocators for C and C++ and three different types of garbage collected heaps for Java, JavaScript and Python. Bad news: Python's handling of integers isn't so kind on memory use when it comes to garbage collection (last time I checked, which was a few years ago, maybe they've improved it).
But I've had Chrome sessions run for weeks without issue. So I do wonder what is happening.
You don't say which OS you are using but if you are using Microsoft Windows then you could use C++ Memory Validator to inspect where each allocation was made (full callstack, how many bytes, etc) while Chrome is running (Launch Chrome from C++ Memory Validator, load your applicaton, let it do it's thing then go to the Memory tab and click Refresh - it will display all the live allocations that can be tracked - any statically linked heaps won't be trackable as you won't have the symbols to allow them to be hooked). OK, you don't have symbols to make teh callstacks readable but you can still identify allocations happening at the same place. That may give you a clue as to the cause of the leak/fragmentation so that can report this to the Chrome devs for a closer look.
Do you get the same behaviour in Firefox? If you then you could do what I suggest with C++ Memory Validator but do it on a build of Firefox that you've built yourself - you'll have symbols and source and know exactly where the problem is.
Disclaimer. I am the designer of C++ Memory Validator.
My site is pretty standard ecom site, it isn't a JS backed standalone app or anything, it's just a site which uses JS for standard stuff, as well as some jquery plugins to do a few things.
I'm trying to do some JS memory evaluation on my site. I've done this by looking at the Chrome Task Manager and through Heap Snapshots.
Initailly my site on first load sits between 35MB (i.e 35,000K) and 40MB on the task manager. This is the largest of any tab, if I have several tabs of other websites open at the same time.
If I refesh the page it jumps up to 55-60, another refresh sees it jump to 65-70MB.
On a normal page in a workflow, it fluctuates between 45-65 (sometimes 75 depending on what you're doing). Clicking around and doing the workflow from page to page sees the memory jump up to 85-100, and increases as you continue through the site.
I've tried to do a few things like check for:
detacted nodes
heap snapshots & looking at the deltas
amix's MemoryLeakChecker checking size of objects
I'd need a deeper dive to look for circular references or closure problems.
Heap snapshots don't reveal much, most of the top lists are (array), (string), (system). The snapshots sit between 4.8MB, 5.1MB, 5.8MB, 6.8MB and increase.
I've got a few questions as result:
How do I understand the different metrics between snapshot memory and task manager memory
Are there any good tutorials (apart from the ones on the Google Developers site)?
How much memory is considered acceptable? Given in the task manager my site is always the highest?
Do I have a memory leak? Apart from the steps I've described above (which I haven't found anything concrete from) is there any other ways I can find leaks?
Can you suggest any tools apart from the Chrome Dev Tools (a lot of the tools mentioned on Google for Firefox are not compatible with the latest version, eg: Leak Monitor for FF)
As a side note, most of my functions are low key operations, and don't exceed 200ms (based on a CPU profile). What is a good benchmark I should be aiming for? Is 200ms high?
What you are describing is not a memory leak, it's a garbage that Chrome knows of and that will be removed whenever Chrome decides it's time to do it. To explain this, lets have a closer look at the scenario you have described.
Making memory to 'leak'
First lets open up a new incognito window (just to be sure that browser extensions are not affecting our results) and navigate to google.com.
Then, lets open the Task Manager and enable "JavaScript Memory" column (by right-clicking on the Task Manager window). We need this column to be sure that the memory we will be 'leaking' is being, in fact, allocated by JavaScript. We end up with something like this:
Now, as you suggested, we should reload the page couple of times and observe the memory of our tab going up:
So far, so good - everything works exactly as you described it.
Wait a second...
However, lave your cursor inactive for half a minute, or go to another tab and you will observe a huge memory usage drop on our 'Tab:google'. Why is that? What happened there? Who cleaned up our 'leaked' memory for us?
The Memory Usage Drop
To investigate that, lets repeat what we have done so far, so that 'Tab:google' uses a lot of memory again. Then, lets open Chrome Developer Tools and start recording on the 'Timeline' tab. After that, lets change a tab for couple of seconds and when memory drops stop 'recording' on the 'Timeline'. You should end up with this:
In the last couple of seconds of our recording mysterious 'GC Events' appeared. Exactly in the same time when the memory was released. Coincidence? Nope.
GC Events
GC stands for the Garbage Collector. It's a mechanism that "attempts to reclaim garbage, or memory occupied by objects that are no longer in use by the program". So it turns out that memory of our tab was polluted by garbage and GC was capable of getting rid of these garbage for the whole time (you can even force garbage collection using button at the bottom of the 'Timeline' tab). So why it decided not to? Why it waited for us to stop interacting with the page or change the tab?
Lazy Garbage Collector
The short answer is that garbage collection has to 'freeze' the execution of all scripts before any work can be done. Also, it can take significant amount of CPU time to execute. This can result in lag, choppy animations, unresponsive controls etc. That's why Chrome waits for the right moment to call the garbage collection. And the best moment to do it is when user is not looking.
In addition, please note that 'GC Events' come in series, there are always couple of them with short breaks in between. These breaks are meant for 'normal' JavaScript to execute making the garbage collection less noticeable.
Live Objects
Take a look at "JavaScript Memory" tab at the top two screenshots in this post again. You will notice that this column contains two numbers. First one is memory "reserved for JavaScript VM
heap", the other one is "how much memory live (reachable) objects
comprise" (source). When benchmarking your applications you should worry only about the second value, all the rest will be handled by GC.
An example of a leak
A real JavaScript leak can happen ie. in a web chat application. If, over time, it will use more and more 'live' memory while always displaying only last 10 messages then we can talk about a leak. Such leak, will eventually crash a tab (or a browser).
Conclusion
For scripts running on the page, reloading the page (or going to another location) is equal to restarting your computer while your ANSI C app is running. After that, you should think about all the memory allocated by your scripts as wiped out. The only reason why, in practice, this may not happen immediately after reloading the page is that browser is waiting for the right moment to clean up. And you, as a web developer, should not be concerned about it.
If you still think that your page are leaking you can use the answer from this question to track down the leaked objects.
I have a touch screen kiosk application that I'm developing that will be deployed on the latest version of Chrome.
The application will need to make AJAX calls to an web service, every 10 minutes or so to pull thru any updated content.
As it's a kiosk application, the page is unlikely to be reloaded very often and theoretically, unless the kiosk is turned off, the application could run for days at a time.
I guess my concern is memory usage and whether or not a very long running setTimeout loop would chew through a large amount of memory is given sufficient time.
I'm currently considering the use of Web Workers and I'm also going to look into Web Sockets but I was wondering if anyone had any experience with this type of thing?
Cheers,
Terry
The browser has a garbage collector so no problems on that. as long as you don't introduce memory leaks through bad code. here's an article and another article about memory leak patterns. that's should get you started on how to program efficiently, and shoot those leaky code.
also, you have to consider the DOM. a person in SO once said that "things that are not on screen should be removed and not just hidden" - this not only removes the entity in a viewing perspective, but actually removes it from the DOM, remove it's handlers, and the memory it used will be freed.
As for the setTimeout, lengthen the interval between calls. Too fast, you will chew up memory fast (and render the page quite... laggy). I just tested code for a timer-based "hashchange" detection, and even on chrome, it does make the page rather slow.
research on the bugs of chrome and keep updated as well.