Something wrong is happening with my pixi.js game. It allocated 1MB a second and 3 seconds later GS releases it. And so on, infinitely.
Of course I read this, but it seems like Chrome Tools are unable to detect a problem - when I record the Allocation Timeline - it shows some rare spikes, which, when selected - show some functions, but also there are constant tiny spikes of memory allocation, which don't show anything. I select them, and in a list of functions I see nothing!
In my frame by frame code I optimized everything - when I turn off pixi - the memory doesn't move. Only when I do the pixi render the scene on every frame - then this constant allocation/release starts and never ends. On PC it's ok, but on mobile every 10 seconds it freezes for 5 seconds - impossible to play.
Did anybody encounter frequent allocations/GC in their code? If yes - how did you debug it, how did you fix it?
In my experience Pixi.js has a GC spikes even with empty scene, you can test it yourself. Feel free to open github issue in their repo. I believe they had some discussions about leaks already. But I don't think pixi itself should impact that much, unless you have thousands and thousands of objects.
Are you sure you did everything? You should abuse Object pool pattern and pre-allocations in you code. This is especially true when you need to constantly create/delete objects (Something like bullets).
General information
https://www.html5rocks.com/en/tutorials/speed/static-mem-pools/
Upd:
For debugging chrome tools is pretty much okay.
https://developers.google.com/web/tools/chrome-devtools/memory-problems/
Related
I have somewhat expensive task that needs to be executed a few times by my users. The task uses the ImageTracer library. This task starts off taking about 1 second, but usually after executing a few times (2-10) the task suddenly takes 20+ seconds, even though none of the input arguments are different. Sometimes I never hit this increase, and sometimes it takes much longer to hit.
Other libraries I am using:
Vue.js
Fabric.js
My first guess was that this is some sort of memory issue, so I went to chrome dev tools and captured the heap size. You can download the profile here and here is a screenshot:
You can clearly see that the last 2 task executions took significantly longer than the first couple. I am not very experienced with js and the chrome dev tools, but that doesn't seem like a memory leak to me. The heap size is not significantly larger at the end than at the beginning (both around 8mb). Please correct me if I am wrong and this does indicate a memory leak.
However those memory spikes seem like they might be an issue (or maybe a symptom of some other issue). Zooming in you can see an immediate jump from 15mb to 80mb and then back down to 15mb only 20ms later. Strangly the jump seems to coincide with a Major GC.
I am not sure if this is a problem and if it is how to further debug it.
I tried whittling the code down to a small self contained reproducible example, but as I got the code smaller and smaller the error became more and more rare to trigger, eventually disappearing. To me this also indicated I am dealing with some sort of memory issue.
I have made a somewhat simplified version that does show my issue however.
For me this problem is only reproducible in Chrome (also tested in Safari and Firefox). I am using a 2018 MacBook Pro with 16GB of memory and running Catalina.
My Questions:
Can anyone see an obvious issue in the profile posted above?
Is this most likely a memory bug and if so what are the next steps to continue debugging?
Are there any workarounds that would enable me to essentially reload or reset the offending js library (since the first couple of task executions always seem to go smoothly.
SHORT QUESTION
I have a bug that only disappears when I restart my device. I would like to know what is released when restarting an iOS device in order to have an idea about what my bug is.
Releasing my cache and my RAM don't help to fix my bug, so I wonder what else could be released that fix my bug for a short time.
CONTEXT
I have a web app using WebGL and BabylonJS. It works really fine on all devices and all browsers except on iOS. With Safari, I always end by having the error message "A problem occurred with this web page so it was reloaded".
It happened on an iPad Air 2 (2Go RAM) running iOS 10.3.3 but I saw the same bug on every iOS device I could have in my hands. That's why I consider it as iOS only related.
I'm aware about iOS resource limits : https://stackoverflow.com/a/22193143/5053300
I tried to debug this for months, the bug appears randomly, sometimes quickly, sometimes slowly. There's nothing consistent so it's totally impossible to debug.
The best guess is that it's a memory issue because crashs seem to appear faster each time. And because the error message is quite always linked to memory issues.
I suspect textures and renderTargetTextures to take more and more memory (but I don't understand why it wouldn't be released, I don't keep useless references).
But there's something I'm sure : when I restart my device, it always work the first time (until I reload once, then begins the downward spiral).
If I clean my cache (via settings -> Safari) and my RAM (pressing home button when we are in shut down screen), the bug is still here. But if I restart my device, it disappears.
Something in memory is released and I would like to know what.
But I also could be totally wrong and it could be something else than memory, I'm open to all your suggestions.
Thanks in advance, it drives me crazy for months !
UPDATE
This is what JS Heap looks like :
UPDATE 2
A scene asking for 90 renderTargetTextures (379x890) each frame does work on iOS.
My app does work if I don't ask for any renderTargetTextures. But it crashes more or less quickly if I only ask for one small renderTargetTexture each frame.
What conclusions can I get from this observation ? Does it corroborate or deny the idea of a memory issue ?
UPDATE 3
There's no clue the following code really is the cause of the issue, but commenting/uncommenting it generaly makes disappear/appear the bug.
var texture = generateTexture();
function generateTexture() {
var rt1 = new BABYLON.RenderTargetTexture("rt1", { width: scene.getEngine().getRenderWidth(), height: scene.getEngine().getRenderHeight() }, scene, false, true, scene.getEngine().TEXTURETYPE_UNSIGNED_BYTE, false);
rt1.wrapU = BABYLON.Texture.CLAMP_ADDRESSMODE;
rt1.wrapV = BABYLON.Texture.CLAMP_ADDRESSMODE;
rt1.renderList.push(sphere);
rt1.onBeforeRender = function() {
sphere.material = std1;
};
scene.customRenderTargets.push(rt1);
return rt1;
}
I may insist on the fact that this code works in the link posted in update 2 so I don't think this code is relevant. The only thing is, commenting this part of the app seems to remove the random bug.
This code asks the 3D engine to render into one intermediary texture before to render to the screen. So this code impacts each frame.
In the link, I ask the engine to render into textures 90 times before to render to the screen.
UPDATE 4: PROBLEM SOLVED
The issue didn't depend on what is released on iOS restart, so I can't answer to my own question. But I can say that iOS doesn't like dynamic lighting.
From now on, all our projects will be limited to one light on iOS devices.
More lights, more computations each frame. It's too much for iOS that kills apps that ask for too much memory in a short time.
Without seeing your code I cannot guarantee any solution but I can offer some information about memory issues.
If you are sure that it never crashes on the first load, and it still crashes after clearing your device's memory, then it is unlikely to be a memory issue. That being said, if you are not sure of this, I would suggesting reading more about IOS memory usage. While the total memory for each device is known, the actual usable memory is much smaller. Furthermore, the memory shown in that post must be shared among all apps on the device, further limiting the memory availability.
I have found that even on a 2GB device, IOS will kill an app using ~200-300 MB of RAM because it was allocated too quickly. IOS takes both the amount and the speed of allocations into account when deciding whether or not to kill an app. Since each tab in Safari runs separately, it is likely that the same mechanism is at work. It is possible that your web app loads so much data into RAM so quickly that IOS refuses to carry out the request.
I would recommend reconsidering your assumptions about memory usage and profiling the app again after reading about the true memory limits of IOS. While it may not solve the issue, it will at least confirm whether memory is the issue.
I am working on a little side project, a game of sorts that I built with D3, and I am currently trying optimize performance. Over time it really becomes a problem. Looking at the dev tools, it seems like the biggest issue is the Garbage Collector uses more and more CPU over time. It starts at around 20% and scales up to over 50% (at which point the game gets maybe a frame a second).
I have a couple of questions:
What are general best practices to minimize Garbage Collector usage?
Does the steadily increasing CPU usage indicate any particular issue with my code?
One tip I've already found is to save and reuse old objects rather than removing and recreating the, which I will work on next, but I want to make sure I'm not missing anything else.
I've been trying to optimize an angular site, and I'm getting a huge amount of delay in the responsiveness of my page when switching between certain routes. Each page displayed is not massive, but it has a fair number of elements in, and a reasonable number of bindings. I've already done what I can with bindonce, so I went and looked in the debugger with Chrome and I see most of my time appears to be spent doing GC.
What's strange is there seems to be huge gaps between each GC, and I'm trying to figure out what exactly those are.
I'm guessing it's when it's actually removing the items and the little bars are when it's doing the mark and sweep, but I'm not as familiar with this level of depth of analyzing JS. Most of my work has been in C++/C#/Java.
In half a second more than 20MB of garbage was collected. GC is pretty busy. This also means that your software is pretty busy as well, producing at least the same memory usage through certain objects.
In order to better understand where the garbage came from, profiling heap allocations might prove useful at this point.
Under Profiles, you might take snapshots of heap allocations and see what type of objects were created, which objects consumed the most memory etc.
My site is pretty standard ecom site, it isn't a JS backed standalone app or anything, it's just a site which uses JS for standard stuff, as well as some jquery plugins to do a few things.
I'm trying to do some JS memory evaluation on my site. I've done this by looking at the Chrome Task Manager and through Heap Snapshots.
Initailly my site on first load sits between 35MB (i.e 35,000K) and 40MB on the task manager. This is the largest of any tab, if I have several tabs of other websites open at the same time.
If I refesh the page it jumps up to 55-60, another refresh sees it jump to 65-70MB.
On a normal page in a workflow, it fluctuates between 45-65 (sometimes 75 depending on what you're doing). Clicking around and doing the workflow from page to page sees the memory jump up to 85-100, and increases as you continue through the site.
I've tried to do a few things like check for:
detacted nodes
heap snapshots & looking at the deltas
amix's MemoryLeakChecker checking size of objects
I'd need a deeper dive to look for circular references or closure problems.
Heap snapshots don't reveal much, most of the top lists are (array), (string), (system). The snapshots sit between 4.8MB, 5.1MB, 5.8MB, 6.8MB and increase.
I've got a few questions as result:
How do I understand the different metrics between snapshot memory and task manager memory
Are there any good tutorials (apart from the ones on the Google Developers site)?
How much memory is considered acceptable? Given in the task manager my site is always the highest?
Do I have a memory leak? Apart from the steps I've described above (which I haven't found anything concrete from) is there any other ways I can find leaks?
Can you suggest any tools apart from the Chrome Dev Tools (a lot of the tools mentioned on Google for Firefox are not compatible with the latest version, eg: Leak Monitor for FF)
As a side note, most of my functions are low key operations, and don't exceed 200ms (based on a CPU profile). What is a good benchmark I should be aiming for? Is 200ms high?
What you are describing is not a memory leak, it's a garbage that Chrome knows of and that will be removed whenever Chrome decides it's time to do it. To explain this, lets have a closer look at the scenario you have described.
Making memory to 'leak'
First lets open up a new incognito window (just to be sure that browser extensions are not affecting our results) and navigate to google.com.
Then, lets open the Task Manager and enable "JavaScript Memory" column (by right-clicking on the Task Manager window). We need this column to be sure that the memory we will be 'leaking' is being, in fact, allocated by JavaScript. We end up with something like this:
Now, as you suggested, we should reload the page couple of times and observe the memory of our tab going up:
So far, so good - everything works exactly as you described it.
Wait a second...
However, lave your cursor inactive for half a minute, or go to another tab and you will observe a huge memory usage drop on our 'Tab:google'. Why is that? What happened there? Who cleaned up our 'leaked' memory for us?
The Memory Usage Drop
To investigate that, lets repeat what we have done so far, so that 'Tab:google' uses a lot of memory again. Then, lets open Chrome Developer Tools and start recording on the 'Timeline' tab. After that, lets change a tab for couple of seconds and when memory drops stop 'recording' on the 'Timeline'. You should end up with this:
In the last couple of seconds of our recording mysterious 'GC Events' appeared. Exactly in the same time when the memory was released. Coincidence? Nope.
GC Events
GC stands for the Garbage Collector. It's a mechanism that "attempts to reclaim garbage, or memory occupied by objects that are no longer in use by the program". So it turns out that memory of our tab was polluted by garbage and GC was capable of getting rid of these garbage for the whole time (you can even force garbage collection using button at the bottom of the 'Timeline' tab). So why it decided not to? Why it waited for us to stop interacting with the page or change the tab?
Lazy Garbage Collector
The short answer is that garbage collection has to 'freeze' the execution of all scripts before any work can be done. Also, it can take significant amount of CPU time to execute. This can result in lag, choppy animations, unresponsive controls etc. That's why Chrome waits for the right moment to call the garbage collection. And the best moment to do it is when user is not looking.
In addition, please note that 'GC Events' come in series, there are always couple of them with short breaks in between. These breaks are meant for 'normal' JavaScript to execute making the garbage collection less noticeable.
Live Objects
Take a look at "JavaScript Memory" tab at the top two screenshots in this post again. You will notice that this column contains two numbers. First one is memory "reserved for JavaScript VM
heap", the other one is "how much memory live (reachable) objects
comprise" (source). When benchmarking your applications you should worry only about the second value, all the rest will be handled by GC.
An example of a leak
A real JavaScript leak can happen ie. in a web chat application. If, over time, it will use more and more 'live' memory while always displaying only last 10 messages then we can talk about a leak. Such leak, will eventually crash a tab (or a browser).
Conclusion
For scripts running on the page, reloading the page (or going to another location) is equal to restarting your computer while your ANSI C app is running. After that, you should think about all the memory allocated by your scripts as wiped out. The only reason why, in practice, this may not happen immediately after reloading the page is that browser is waiting for the right moment to clean up. And you, as a web developer, should not be concerned about it.
If you still think that your page are leaking you can use the answer from this question to track down the leaked objects.