I would like to find a way, in any desktop browser, to determine which JavaScript-allocated objects are being freed by garbage collection. Please note, I am not trying to track "leaks." Leaks are objects that did not get freed.
In Firefox, the web app I'm looking at occasionally stutters (stops responding to keystrokes for >1 second); profiling reveals that during that time, the browser is performing a lengthy GC. Over the previous several seconds, a significant amount of memory was added (in small chunks, not all at once). When the GC happens, almost all of that memory gets freed.
What I'm trying to figure out is: What are all these short-lived objects, anyway? I would love to know their values, their types, or where in the JS they were allocated.
The web app exhibits these same allocation patterns in all browsers (although only Firefox seems to suffer from a lengthy GC); so I would happily use any browser (Mac or Windows) to debug this.
I've figured out how to use Chrome's dev tools to look at leaked objects, but as I mention above, leaks are not my concern. I can't figure out a way to see what objects got allocated and then freed.
GC is just the end of the lifecycle. At the beginning there are allocations.
So instead of trying to figure out what gets collected you can use the devtools' allocation recording.
Related
I have some JavaScript Functions with source code coming from larger JavaScript-files. They are created this way:
const function = Function("foo", "...<large JS source code>...");
When looking at the memory snapshots in Chrome, the source code of these functions is retained and create a large memory overhead. Is it possible to "release"/"drop" source code of functions in JavaScript?
EDIT
I am really keeping the function itself around and wonder whether it’s possible to tell the function to drop the source code from memory.
Internally Chrome uses the V8 JavaScript engine. This engine is a just-in-time JavaScript engine which means that it takes your JavaScript and it compiles it to machine code when it needs it. This is expensive; so V8 will cache the result of the compilation and (in most cases) will re-use the previously compiled code if you call the function again.
I believe function memory reclamation is handled by the V8 garbage collector. When a variable (or function) falls out of scope (there aren't any references left to it anywhere, including closures) then the garbage collector is free to reclaim that memory including the source code and cached machine code. The garbage collector runs periodically and will clean up any memory from anything still in scope. Generally speaking you shouldn't try to force garbage collection on your own, it should happen automatically, but with Chrome there is a way to force garbage collection using the developer tools.
If you remove any references to your function (remember this includes closures) and force garbage collection you should see the memory reclaimed. Use the Chrome Developer memory tools to see if your function has been reclaimed or not (look at "heap snapshots").
There's one other caveat to this: even if the memory is reclaimed it won't necessarily be released back to the operating system or even cleared. Many applications handling large amounts of small memory allocations will try to improve performance by trying to re-use previously allocated memory before asking the operating system for more. So if you're using a low-level memory inspector you may still see your code hanging around in memory even if it's been garbage collected and there aren't any useful references to it. Without diving pretty deep into the V8 internals it probably isn't possible to determine from a memory dump if your code is still in memory because of a memory leak or because Chrome allocated the memory and simply hasn't released it back to the operating system after internally cleaning up references to that memory.
I've been trying to optimize an angular site, and I'm getting a huge amount of delay in the responsiveness of my page when switching between certain routes. Each page displayed is not massive, but it has a fair number of elements in, and a reasonable number of bindings. I've already done what I can with bindonce, so I went and looked in the debugger with Chrome and I see most of my time appears to be spent doing GC.
What's strange is there seems to be huge gaps between each GC, and I'm trying to figure out what exactly those are.
I'm guessing it's when it's actually removing the items and the little bars are when it's doing the mark and sweep, but I'm not as familiar with this level of depth of analyzing JS. Most of my work has been in C++/C#/Java.
In half a second more than 20MB of garbage was collected. GC is pretty busy. This also means that your software is pretty busy as well, producing at least the same memory usage through certain objects.
In order to better understand where the garbage came from, profiling heap allocations might prove useful at this point.
Under Profiles, you might take snapshots of heap allocations and see what type of objects were created, which objects consumed the most memory etc.
I am troubleshooting what appears to be a memory leak in our configuration page. The page is used to change the configuration of our service and also displays health diagnostics. This means that we are querying the service periodically for configuration and instrumentation information (typically we use a query interval of 30sec, but to troubleshoot I am querying at 100ms intervals). We rely on knockoutjs, datajs, jquery and spinjs.
I've found that if I leave the page open overnight at the 100ms query interval that the private bytes for the chrome browser tab grows from about 50MB to 335MB. I have four pages with the issue, but am focused on one during my troubleshooting effort. Using chrome://memory-redirect/ I can see the page (process id 26148) memory.
However, the JavaScript heap memory appears to be flat over the same period at 3.6MB. Using the heap profiling tools in Chrome it shows that all of my object allocations are garbage collected.
In the above picture the gray allocations indicate that the objects have been cleaned up by the GC.
The memory timeline also is constant.
I also forced two GCs and confirmed that the number of documents, nodes and listeners was constant between the two GCs.
My questions are:
Where is the process memory being used that is not part of the JavaScript heap?
Given our JavaScript heap memory is flat, could that extra memory be a memory leak caused by our JavaScript code?
Thanks for all the help!
You are comparing apples and oranges - Garbage collected memory in a sub-heap and whole application memory.
You've used Chrome to inspect the JavaScript heap and found evidence that indicates the JavaScript part of your application is running OK.
You're also using a tool to monitor the global memory usage of Chrome itself. That is all the memory that Chrome is using for any task, including tasks not directly related to your application, but to the functioning of the browser itself.
Perhaps you've found a use case that triggers a memory leak in the Chrome internals?
Or perhaps it isn't a memory leak, but memory fragmentation in the non-garbage collected internal heaps used by Chrome?
According to this web page Chrome is written in a mixture of C, C++, Java, JavaScript and Python. This means we have deterministic memory allocators for C and C++ and three different types of garbage collected heaps for Java, JavaScript and Python. Bad news: Python's handling of integers isn't so kind on memory use when it comes to garbage collection (last time I checked, which was a few years ago, maybe they've improved it).
But I've had Chrome sessions run for weeks without issue. So I do wonder what is happening.
You don't say which OS you are using but if you are using Microsoft Windows then you could use C++ Memory Validator to inspect where each allocation was made (full callstack, how many bytes, etc) while Chrome is running (Launch Chrome from C++ Memory Validator, load your applicaton, let it do it's thing then go to the Memory tab and click Refresh - it will display all the live allocations that can be tracked - any statically linked heaps won't be trackable as you won't have the symbols to allow them to be hooked). OK, you don't have symbols to make teh callstacks readable but you can still identify allocations happening at the same place. That may give you a clue as to the cause of the leak/fragmentation so that can report this to the Chrome devs for a closer look.
Do you get the same behaviour in Firefox? If you then you could do what I suggest with C++ Memory Validator but do it on a build of Firefox that you've built yourself - you'll have symbols and source and know exactly where the problem is.
Disclaimer. I am the designer of C++ Memory Validator.
My site is pretty standard ecom site, it isn't a JS backed standalone app or anything, it's just a site which uses JS for standard stuff, as well as some jquery plugins to do a few things.
I'm trying to do some JS memory evaluation on my site. I've done this by looking at the Chrome Task Manager and through Heap Snapshots.
Initailly my site on first load sits between 35MB (i.e 35,000K) and 40MB on the task manager. This is the largest of any tab, if I have several tabs of other websites open at the same time.
If I refesh the page it jumps up to 55-60, another refresh sees it jump to 65-70MB.
On a normal page in a workflow, it fluctuates between 45-65 (sometimes 75 depending on what you're doing). Clicking around and doing the workflow from page to page sees the memory jump up to 85-100, and increases as you continue through the site.
I've tried to do a few things like check for:
detacted nodes
heap snapshots & looking at the deltas
amix's MemoryLeakChecker checking size of objects
I'd need a deeper dive to look for circular references or closure problems.
Heap snapshots don't reveal much, most of the top lists are (array), (string), (system). The snapshots sit between 4.8MB, 5.1MB, 5.8MB, 6.8MB and increase.
I've got a few questions as result:
How do I understand the different metrics between snapshot memory and task manager memory
Are there any good tutorials (apart from the ones on the Google Developers site)?
How much memory is considered acceptable? Given in the task manager my site is always the highest?
Do I have a memory leak? Apart from the steps I've described above (which I haven't found anything concrete from) is there any other ways I can find leaks?
Can you suggest any tools apart from the Chrome Dev Tools (a lot of the tools mentioned on Google for Firefox are not compatible with the latest version, eg: Leak Monitor for FF)
As a side note, most of my functions are low key operations, and don't exceed 200ms (based on a CPU profile). What is a good benchmark I should be aiming for? Is 200ms high?
What you are describing is not a memory leak, it's a garbage that Chrome knows of and that will be removed whenever Chrome decides it's time to do it. To explain this, lets have a closer look at the scenario you have described.
Making memory to 'leak'
First lets open up a new incognito window (just to be sure that browser extensions are not affecting our results) and navigate to google.com.
Then, lets open the Task Manager and enable "JavaScript Memory" column (by right-clicking on the Task Manager window). We need this column to be sure that the memory we will be 'leaking' is being, in fact, allocated by JavaScript. We end up with something like this:
Now, as you suggested, we should reload the page couple of times and observe the memory of our tab going up:
So far, so good - everything works exactly as you described it.
Wait a second...
However, lave your cursor inactive for half a minute, or go to another tab and you will observe a huge memory usage drop on our 'Tab:google'. Why is that? What happened there? Who cleaned up our 'leaked' memory for us?
The Memory Usage Drop
To investigate that, lets repeat what we have done so far, so that 'Tab:google' uses a lot of memory again. Then, lets open Chrome Developer Tools and start recording on the 'Timeline' tab. After that, lets change a tab for couple of seconds and when memory drops stop 'recording' on the 'Timeline'. You should end up with this:
In the last couple of seconds of our recording mysterious 'GC Events' appeared. Exactly in the same time when the memory was released. Coincidence? Nope.
GC Events
GC stands for the Garbage Collector. It's a mechanism that "attempts to reclaim garbage, or memory occupied by objects that are no longer in use by the program". So it turns out that memory of our tab was polluted by garbage and GC was capable of getting rid of these garbage for the whole time (you can even force garbage collection using button at the bottom of the 'Timeline' tab). So why it decided not to? Why it waited for us to stop interacting with the page or change the tab?
Lazy Garbage Collector
The short answer is that garbage collection has to 'freeze' the execution of all scripts before any work can be done. Also, it can take significant amount of CPU time to execute. This can result in lag, choppy animations, unresponsive controls etc. That's why Chrome waits for the right moment to call the garbage collection. And the best moment to do it is when user is not looking.
In addition, please note that 'GC Events' come in series, there are always couple of them with short breaks in between. These breaks are meant for 'normal' JavaScript to execute making the garbage collection less noticeable.
Live Objects
Take a look at "JavaScript Memory" tab at the top two screenshots in this post again. You will notice that this column contains two numbers. First one is memory "reserved for JavaScript VM
heap", the other one is "how much memory live (reachable) objects
comprise" (source). When benchmarking your applications you should worry only about the second value, all the rest will be handled by GC.
An example of a leak
A real JavaScript leak can happen ie. in a web chat application. If, over time, it will use more and more 'live' memory while always displaying only last 10 messages then we can talk about a leak. Such leak, will eventually crash a tab (or a browser).
Conclusion
For scripts running on the page, reloading the page (or going to another location) is equal to restarting your computer while your ANSI C app is running. After that, you should think about all the memory allocated by your scripts as wiped out. The only reason why, in practice, this may not happen immediately after reloading the page is that browser is waiting for the right moment to clean up. And you, as a web developer, should not be concerned about it.
If you still think that your page are leaking you can use the answer from this question to track down the leaked objects.
I'm loading some fairly big data sets in to firefox (500 k or so) and drawing tables with them. Firefox is using up to 400 megs of memory. How can I tell if Firefox is leaking memory, or is just using a lot of memory just because it can?
Is there another browser which will use less memory if it doesn't need it? I get the feeling that firefox grabs a bunch of memory for itself and doesn't release it unless it has to. It may be apparent that my understanding of memory management in general is very superficial.
There is a setting you can set in firefox that forces it to free as much memory as it can on minimise:
On the website url toolbar type
about:config
A page full of words
will come up. right click anywhere
and choose New -> Boolean
For the
name input type
"config.trim_on_minimize" Select
True
Restart FireFox.
If it is leaking memory, then the mem usage (number of K) in your Windows task manager will continue to grow. If this number is fairly consistent, then Firefox is behaving normally. It is my opinion that Firefox requires far too much memory to behave normally.
How can I tell if Firefox is leaking memory, or is just using a lot of memory just because it can?
Open another page in a different tab. Then close your 'big' tab and see if the memory is freed.
Memory Leak - memory which is not released when it should be
If the memory Firefox is allocating to hold your data is released when you navigate away from your page, there is no memory leak. You can than argue if Firefox if using too much memory or not, but that is something beyond your scope.
You do have a leak on your hands if that memory is not released. In that case you should check if it is something you can handle in your JS code, or a firefox bug. If it's the latter, go ahead to the mozilla bugzilla, and open a ticket.
You can turn on Windows Performance Monitor and see if the firefox.exe process is growing memory over time to confirm if there is a memory leak.
I get the feeling that firefox grabs a bunch of memory for itself and doesn't release it unless it has to. It may be apparent that my understanding of memory management in general is very superficial.
well, at 1 point Firefox requires a lot of memory, say 400Mb.
maybe after that, it doesn't require so many memory, BUT
since your system do not need this memory, it does not reclaim it to Firefox, which keeps it.
If ever you launch other processes that need a lot of memory, then your OS will claim back memory to other processes that are running with a high priority.
To summarize my ideas : it's probably not a memory leak.
Is there another browser which will use less memory if it doesn't need it?
why don't you try them?