I've always been under the impression that as long as you have free memory, whether its 100% or 10%, the speed of your processes should not be affected.
However, I recently ran into a situation where it seems that my processes get a lot slower when it uses up a greater percentage of the memory available.
It could be a problem with the code itself, but I'm hoping to get a quick sanity check that I haven't been living a lie before delving deeper into the code iteself.
It all really depends upon how the app is coded and what it is doing. For some apps, it won't make any difference whether free memory is 10% or 100% as long as there's enough for it to do its job.
For other apps, they may encounter memory fragmentation, they may cause disk swapping, they may even adjust their own behavior because of less available memory (using smaller buffers, forcing data to disk, etc...). In a garbage collected system (like nodejs), a lower memory condition may cause more frequent garbage collection too.
The single biggest performance impact from running lower on memory will be if the app causes the OS to page memory to disk. This is where the virtual memory being used exceeds the actual physical memory and the OS has to substitute some disk space for memory that is allocated. The OS tries to swap memory to disk that hasn't been accessed recently in the hopes that it won't be needed again soon, but sometimes that just doesn't work very efficiently and you get a lot of hard disk thrashing, constantly reading/write memory to/from disk. Since disks are thousands of times slower than physical memory, this can massively slow things down.
There are also cases of app design where some operations in an app like Photoshop that will simply run faster with more memory available to use because the algorithms will adapt to use the larger amount of memory to make the operation run faster when working on large objects. A nodejs app or library could be doing the same thing. For example an image processing algorithm may be designed to work on images larger than will fit in memory so it has to decide how much memory is "safe" to allocate and then work on the image in chunks. With a smaller amount of memory available, the work gets done less efficiently in smaller chunks.
A more common reason why things get slower over time is because of some sort of internal fragmentation or leaks that make regular housekeeping chores (like allocating memory) less efficient. This may occur either at the heap level or at the app level. This is why some admins schedule long running processes (like servers) to be automatically restarted every once in awhile - to clear up any of this fragmentation or small leaks and regularly start afresh.
If it's a major problem, extensive debugging may be able to explain where any major impacts are coming from, but this is not trivial debugging as it involves lots of measuring, gathering data, adjusting what you're looking at based on what you find, etc... all while trying to not influence the very thing you're trying to find/measure.
Related
I am making an app, which needs a lot of GPU memory sometimes (many huge textures). I could make a system, where I keep frequently used textures in GPU memory, and I upload and delete the rest, only when they are needed.
In general, there are always more textures than the GPU memory, and the more memory I can use - the faster my program is. I don't want to restrict myself e.g. to 100 MB or 1 GB of memory, when there could be 4x more memory free to use. But if I try to allocate too much, the browser will kill my program.
I see, that in WebGL, there is no direct way to tell, how much memory is available. What would be your strategy, how to solve such issue?
Background: I came from Microsoft world, in which I used to have websites stored on IIS. Experience taught me to recycle my application pool once a day in order to eliminate weird problems due to fragmentation. Recycling the app pool basically means to restart your application without restarting the entire IIS. I also watched a lecture that explained how Microsoft had reduced the fragmentation a lot in .Net 4.5.
Now, I'm deploying a Node.js application to production environment and I have to make sure that it works flawlessly all the time. I originally thought to make my app restarted once a day. Then I did some research in order to find some clues about fragmentation problems in Node.js. The only thing I've found is a scrap of paragraph from an article describing GC in V8:
To ensure fast object allocation, short garbage collection pauses, and
the “no memory fragmentation V8” employs a stop-the-world,
generational, accurate, garbage collector.
This statement is really not enough for me to give up building a restart mechanism for my app, but on the other hand I don't want to do some work if there is no problem.
So my quesion is:
Should or shouldn't I restart my app every now and then in order to prevent fragmentation?
Implementing a server restart before you know that memory consumption is indeed a problem is a premature optimization. As such, I don't think you should do it until you actually find that it is a problem. You will likely find more important issues to optimize for as opposed to memory consumption.
To figure out if you need a server restart, I recommend doing the following:
Set up some monitoring tools like https://newrelic.com/ that let's your monitor your performance.
Monitor your memory continuously. Try to see if there is steady increase in the amount of memory consumed, or if it levels off.
Decide upon an acceptable threshold before you need to act. For example once your app consumes 60% of system memory you need to start thinking about a server restart and decide upon the restart interval.
Decide if you are ok with having "downtime" while restarting the sever or not. If you don't want downtime, you may need to build a proxy layer to direct traffic.
In general, I'd recommend server restarts for all dynamic, garbage collected languages. This is fairly common in those types of large applications. It is almost inevitable that a small mistake somewhere in your code base, or one of the libraries you depend on will leak memory. Even if you fix one leak, you'll get another one eventually. This may frustrate your team, which will basically lead to a server restart policy, and a definition of what is acceptable in regards to memory consumption for your application.
I agree with #Parris. You should probably figure out whether you actually need have a restart policy first. I would suggest using pm2 docs here. Even if you don't want to sign up for keymetrics, its a pretty good little process manager and real quick to set up. You can get a report of memory usage from command line. Looks something like this.
Also, if you start in cluster mode like above, you can call pm2 restart my_app and the first one will probably be up again before the last one is taken offline (this is an added benefit, the real reason for having 8 processes is to utilize all 8 cores). If you are adamant about downtime, you could restart them 1 by 1 acording to id.
I agree with #Parris this seems like a premature optimization. Also, restarting is not a solution to the underlying problem, it's a treatment for the symptoms.
If memory errors are a prevalent issue for your node application then I think that some thought as to why this fragmentation occurs in your program in the first place could be a valuable effort. Understanding why memory errors occur after a program has been running for a long period of time, and refactoring the architecture of your program to solve the root of the problem, is a better solution in my eyes than just addressing the symptoms.
I believe two things will benefit you.
immutable objects will help a lot, they are a lot more predictable than using mutable objects, and will not be affected by the length of time the project has been live. Also, since immutable objects are read only blocks of memory they are faster than mutable objects which the server has to spend resources deciding whether to read, or write on the memory block which stores the object. I currently use the library called IMMUTABLE and it works well for me. There are other one's as well like Deep Freeze, however, I have never used it.
Make sure to manage your application's processes correctly, memory leaks are the second big contributor to this problem that I have experienced. Again, this is solved by thinking about how your application is structured, and how user events are handled, making sure once a process is not being used by the client that it is properly removed from the heap, if it is not then the heap keeps growing until all memory is consumed causing the application to crash(refer to the below graphic to see V8's memory Scheme, and where the heap is). Node is a C++ program, and it's controlled by Google's V8 and Javascript.
You can use Node.js's process.memoryUsage() to monitor memory usage. When you identify how to manage your heap V8 offers two solutions, one is Scavenge which is very quick, but incomplete. The other is Mark-Sweep which is slow and frees up all non-referenced memory.
Refer to this blog post for more info on how to manage your heap and manage your memory on V8 which runs Node.js
So the responsible approach to your implementation is to keep a close eye on open processes, a deep understanding of the heap, and how to free non-referenced memory blocks. Creating your project with this in mind also makes the project a lot more scaleable as well.
I am working on a little side project, a game of sorts that I built with D3, and I am currently trying optimize performance. Over time it really becomes a problem. Looking at the dev tools, it seems like the biggest issue is the Garbage Collector uses more and more CPU over time. It starts at around 20% and scales up to over 50% (at which point the game gets maybe a frame a second).
I have a couple of questions:
What are general best practices to minimize Garbage Collector usage?
Does the steadily increasing CPU usage indicate any particular issue with my code?
One tip I've already found is to save and reuse old objects rather than removing and recreating the, which I will work on next, but I want to make sure I'm not missing anything else.
I've been trying to optimize an angular site, and I'm getting a huge amount of delay in the responsiveness of my page when switching between certain routes. Each page displayed is not massive, but it has a fair number of elements in, and a reasonable number of bindings. I've already done what I can with bindonce, so I went and looked in the debugger with Chrome and I see most of my time appears to be spent doing GC.
What's strange is there seems to be huge gaps between each GC, and I'm trying to figure out what exactly those are.
I'm guessing it's when it's actually removing the items and the little bars are when it's doing the mark and sweep, but I'm not as familiar with this level of depth of analyzing JS. Most of my work has been in C++/C#/Java.
In half a second more than 20MB of garbage was collected. GC is pretty busy. This also means that your software is pretty busy as well, producing at least the same memory usage through certain objects.
In order to better understand where the garbage came from, profiling heap allocations might prove useful at this point.
Under Profiles, you might take snapshots of heap allocations and see what type of objects were created, which objects consumed the most memory etc.
Can browsers enforce any sort of limit on the amount of data that can be stored in JavaScript objects? If so, is there any way to detect that limit?
It appears that by default, Firefox does not:
var data;
$("document").ready(function() {
data = [];
for(var i = 0; i < 100000000000; i++) {
data.push(Math.random());
}
});
That continues to consume more and more memory until my system runs out.
Since we can't detect available memory, is there any other way to tell we are getting close to that limit?
Update
The application I'm developing relies on very fast response times to be usable (it's the core selling point). Unfortunately, it also has a very large data set (more than will fit into memory on weaker client machines). Performance can be greatly improved by preemptively loading data strategically (guessing what will be clicked). The fallback to loading the data from the server works when the guesses are incorrect, but the server round trip isn't ideal. Making use of every bit of memory I can makes the application as performant as possible.
Right now, it works to allow the user to "configure" their performance settings (max data settings), but users don't want to manage that. Also, since it's a web application, I have to handle users setting that per computer (since a powerful desktop has a lot more memory than an old iPhone). It's better if it just uses optimal settings for what is available on the systems. But guessing too high can cause problems on the client computer too.
While it might be possible on some browsers, the right approach should be to decide what limit is acceptable for the typical customer and optionally provide a UI to define their limit.
Most heavy web apps get away with about 10MB JavaScript heap size. There does not seem to be a guideline. But I would imagine consuming more than 100MB on desktop and 20MB on mobile is not really nice. For everything after that look into local storage, e.g. FileSystem API (and you can totally make it PERSISTENT)
UPDATE
The reasoning behind this answer is the following. It is next to never user runs only one application. More so with counting on the browser having only one tab open. Ultimately, consuming all available memory is never a good option. Hence determining the upper boundary is not necessary.
Reasonable amount of memory user would like to allocate to the web app is a guess work. E.g. highly interactive data analytics tool is quite possible in JS and might need millions of data points. One option is to default to less resolution (say, daily instead of each second measurements) or smaller window (one day vs. a decade of seconds). But as user keeps exploring the data set, more and more data will be needed, potentially crippling the underlying OS on the agent side.
Good solution is to go with some reasonable initial assumption. Let's open some popular web applications and go to dev tools - profiles - heap snapshots to take a look:
FB: 18.2 MB
GMail: 33 MB
Google+: 53.4 MB
YouTube: 54 MB
Bing Maps: 55 MB
Note: these numbers include DOM nodes and JS Objects on the heap.
It seems to be then, people come to accept 50MB of RAM for a useful web site. (Update 2022: nowadays averaging closer to 100MB.) Once you build your DOM Tree, fill your data structures with test data and see how much is OK to keep in RAM.
Using similar measurements while turning device emulation in Chrome, one can see the consumption of the same sites on tablets and phones, BTW.
This is how I arrived at 100 MB on desktop and 20 MB on mobile numbers. Seemed to be reasonable too. Of course, for occasional heavy user it would be nice to have an option to bump max heap up to 2 GB.
Now, what do you do if pumping all this data from the server every time is too costly?
One thing is to use Application Cache. It does create mild version management headaches but allows you to store around 5 MB of data. Rather than storing data though, it is more useful to keep app code and resources in it.
Beyond that we have three choices:
SQLite - support was limited and it seems to be abandoned
IndexDB - better option but support is not universal yet (can I use it?)
FileSystem API
Of them, FileSystem is most supported and can use sizable chunk of storage.
In Chrome the answer is Sure!
Go to the console and type:
performance.memory.jsHeapSizeLimit; // will give you the JS heap size
performance.memory.usedJSHeapSize; // how much you're currently using
arr = []; for(var i = 0; i < 100000; i++) arr.push(i);
performance.memory.usedJSHeapSize; // likely a larger number now
I think you'll want something like the following:
const memory = navigator.deviceMemory
console.log (`This device has at least ${memory}GiB of RAM.`)
You can check out the following: https://developer.mozilla.org/en-US/docs/Web/API/Navigator/deviceMemory
Note: This feature is not supported across all browsers.
Since a web app can't have access to any system-related information (like the available amount of memory), and since you would prefer not having to ask users to manually set their performance settings, you must rely on a solution that allows you to get such information about the user's system (available memory) without asking them. Seems impossible ? Well, almost...
But I suggest you do the following : make a Java applet that will automatically get the available memory size (e.g. using Runtime.exec(...) with an appropriate command), provided your applet is signed, and return that information to the server or directly to the web page (with JSObject, see http://docs.oracle.com/javafx/2/api/netscape/javascript/JSObject.html).
However, that would assume your users can all run a Java applet within their browsers, which is not always the case. Therefore, you could ask them to install a small piece of software on their machines that will measure how much memory your app should use without crashing the browser, and will send that information to your server. Of course, you would have to re-write that little program for every OS and architecture (Windows, Mac, Linux, iPhone, Android...), but it's simpler that having to re-write the whole application in order to gain some performance. It's a sort of in-between solution.
I don't think there is an easy solution. There will be some drawbacks, whatever you choose to do. Remember that web applications don't have the reputation of being fast, so if performance is critical, you should consider writing a traditional desktop application.