Looks like i found memory leak in my code, but i'm not sure, and i don't have so many experience with nodejs memory leaks.
Can someone explain me, can this code produce memory leak?
var tasks = [];
// each 10 seconds
tasks.push(function () {
console.log('hello, world!');
});
// each minute
while (tasks.length) {
var task = tasks.shift();
task();
}
UPD: Missed while loop in my code, updated now.
My question is, will scope of my anonymous function from array cleared from memory?
Well, not a memory leak but you're putting new elements in your array 6 times faster than you are retrieving them.
As a result, you will actually be using only one out of 5 pushed functions, and your array will keep growing.
If you let it run long enough, you'll end up with a massive array that can never be emptied.
EDIT: After you added the while loop, the array is not growing anymore, and it shouldn't have any memory leak coming from this part of your code. It does not mean there is none in your project. Make sure any value created in your pushed functions can properly be garbage collected (i.e. that you did not keep a reference on it somewhere).
Related
I use console.time to show the time of the function. But I found that it shows different running time of the same function.
I have simplified my function as below:
const findIP = (res) => {
let arr = []
arr = res.split(',')
}
console.time('1')
findIP('1,2,3,4,5,6,7,8,9,0')
console.timeEnd('1')
console.time('2')
findIP('1,2,3,4,5,6,7,8,9,0')
console.timeEnd('2')
The time difference between the two is very large.
I have tried to run several times. And it still cost different time.
To quote the answer in the the following link:
If you run shorten multiple times, the V8 engine has a JIT compiler that will optimize that piece of code so it runs faster the next time.
https://stackoverflow.com/a/54601440
Try changing the argument value, for example
console.time('1')
findIP('1,2,3,4,5,6,7,8,9,0')
console.timeEnd('1')
console.time('2')
findIP('1,2,3,4,43,6,7,8,9,4')
console.timeEnd('2')
you will see approx equal time
Reason of that difference is: The browser cache
Simple Definition
browser cache is a temporary storage area in memory or on disk that holds the most recently downloaded Web pages and/or calculated result.
for (i=0;i<channelName.length;i++) {
if (channelName[i]=="channel"||channelName[i]=="user") {
checkUserDuplicate(channelName[i]);
}
}
This loop is causing an "out of memory" crash in all browsers. can anyone see why? It seems to be crashing at the IF statement and then causing an infinite loop somehow.
If you're wondering what the code does, it finds the keywords "channel" and "user" in an array of undefined length, then gets the string at the next position.
Any help would be much appreciated as I have been sitting here puzzled for 2 hours.
EDIT: channelName is a URL like http://www.youtube.com/user/username
this is the function:
function checkUserDuplicate(channelName) {
var idarray=[];match=0;$('.channels').each(function(){idarray.push(this.id)});
for (i=0;i<idarray.length;i++) {
var current=channelName.toLowerCase();compare=idarray[i].toLowerCase();
if (current==compare) {callError(channelName+" already exists in this collection");match=1;}
} if (match==0) {checkExists(channelName);}
}
It's a mess :)
In your for loop, if you don't specify var i = 0;, it's globally accessible then. And in your other functions, you may modify the value of i and cause it doesn't increment as expected ends up with an infinite loop.
Sorry I didn't read the code since it's pretty messy, but that could be the reason
In the checkUserDuplicate function it counts the amount of DIVs within a scope of classes. If there is no classes or DIVs the count is 0, and passed on to the for loop and creates a recursive loop.
I've just finished writing a script for parsing csv data. Having recently installed JShint, it's been badgering me about the re-use of variables. I've been using JS a fair bit lately, but I come from a python background where it's normal to reuse variables. I'm wondering what issues there are with reusing variables in the following two examples:
Loop with a Switch
The following loop steps through the rows on a csv file, and when it passes a certain value in a row, it switches variable "currentSwitch" from false to true. After currentSwitch is tripped, the loop starts to push stuff to an array.
for (f=0; f < data.length; f++){
if (data[f][0] === code){
if (currentSwitch === true){
dataListByCode.push(data[f]);
}
}
else if ((data[f][0]).slice(0,4) === "UNIN"){
var currentSwitch = true;
}
}
Processing Data with Broken Out Functions
I've got a few functions for processing data that it makes sense to keep separate. In the following code, I process with one function, then I process with another.
var dataListByCode = addDivideData(dataListByCode);
var dataListByCode = addBeforeEntriesArray(dataListByCode, invNumber, matterNumber, client, workType);
Can anyone tell me if this is not in line with best practice? Is there anything that could go wrong with either of these (or scenarios like them)?
You don't need to redeclare currentSwtich
var currentSwitch = true;
In fact it really doesn't make any sense to redeclare this variable in the middle of the loop and in most cases it's almost certainly not what you actually want.
Just initialize/declare it once at the beginning of your loop
var currentSwtich;
// or
var currentSwitch = false;
and drop the var when you set it to true:
currentSwitch = true;
Basically what you are doing is creating a brand new variable with the same name as the old one, and throwing away the old one. This isn't really what you want normally.
There is no analogous concept in python because python doesn't require you to declare variables.
The major problem with reusing variables is that:
a.) in bigger code blocks it can get very confusing, especially if you added/removed code ~20 times, and kept reusing same ~5 variables for multiple things
b.) any programmer that knows nothing about code(read: you after couple months/years) will have a much more difficult time grasping the code.
The lower function snippet can be expressed as:
var dataListByCode = addBeforeEntriesArray(addDivideData(dataListByCode), invNumber, matterNumber, client, workType);
which is not that problematic. The breaking up of functions is useless in this case, and if you have many inline function chains that is usually sign that you need to rethink the object/function design.
I would like to understand what kind of code causes memory leaks in JavaScript and created the script below. However, when I run the script in Safari 6.0.4 on OS X the memory consumption shown in the Activity Monitor does not really increase.
Is something wrong with my script or is this no longer an issue with modern browsers?
<html>
<body>
</body>
<script>
var i, el;
function attachAlert(element) {
element.onclick = function() { alert(element.innerHTML); };
}
for (i = 0; i < 1000000; i++) {
el = document.createElement('div');
el.innerHTML = i;
attachAlert(el);
}
</script>
</html>
The script is based on the Closure section of Google's JavaScript style guide:
http://google-styleguide.googlecode.com/svn/trunk/javascriptguide.xml?showone=Closures#Closures
EDIT: The bug that caused the above code to leak has apparently been fixed: http://jibbering.com/faq/notes/closures/#clMem
But my question remains: Would someone be able to provide a realistic example of JavaScript code that leaks memory in modern browsers?
There are many articles on the Internet that suggest memory leaks can be an issue for complex single page applications but I have a hard time finding an examples that I can run in my browser.
You're not keeping the element you've created around and referenced anywhere - that's why you're not seeing the memory usage increase. Try attaching the element to the DOM, or store it in an object, or set the onclick to be a different element that sticks around. Then you'll see the memory usage skyrocket. The garbage collector will come through and clean up anything that can no longer be referenced.
Basically a walkthrough of your code:
create element (el)
create a new function that references that
element
set the function to be the onclick of that element
overwrite the element with a new element
Everything is centric around the element existing. Once there isn't a way to access the element, the onclick can't be accessed anymore. So, since the onclick can't be accessed, the function that was created is destroyed.. and the function had the only reference to the element.. so the element is cleaned up as well.
Someone might have a more technical example, but that's the basis of my understanding of the javascript garbage collector.
Edit: Here's one of many possibilities for a leaking version of your script:
<html>
<body>
</body>
<script>
var i, el;
var createdElements = {};
var events = [];
function attachAlert(element) {
element.onclick = function() { alert(element.innerHTML); };
}
function reallyBadAttachAlert(element) {
return function() { alert(element.innerHTML); };
}
for (i = 0; i < 1000000; i++) {
el = document.createElement('div');
el.innerHTML = i;
/** posibility one: you're storing the element somewhere **/
attachAlert(el);
createdElements['div' + i] = el;
/** posibility two: you're storing the callbacks somewhere **/
event = reallyBadAttachAlert(el);
events.push(event);
el.onclick = event;
}
</script>
</html>
So, for #1, you're simply storing a reference to that element somewhere. Doesn't matter that you'll never use it - because that reference is made in the object, the element and its callbacks will never go away (or at least until you delete the element from the object). For possibility #2, you could be storing the events somewhere. Because the event can be accessed (i.e. by doing events[10]();) even though the element is nowhere to be found, it's still referenced by the event.. so the element will stay in memory as well as the event, until it's removed from the array.
update: Here is a very simple example based on the caching scenario in the Google I/O presentation:
/*
This is an example of a memory leak. A new property is added to the cache
object 10 times/second. The value of performance.memory.usedJSHeapSize
steadily increases.
Since the value of cache[key] is easy to recalculate, we might want to free
that memory if it becomes low. However, there is no way to do that...
Another method to manually clear the cache could be added, but manually
adding memory checks adds a lot of extra code and overhead. It would be
nice if we could clear the cache automatically only when memory became low.
Thus the solution presented at Google I/O!
*/
(function(w){
var cache = {}
function getCachedThing(key) {
if(!(key in cache)) {
cache[key] = key;
}
return cache[key];
}
var i = 0;
setInterval(function() {
getCachedThing(i++);
}, 100);
w.getCachedThing = getCachedThing
})(window);
Because usedJSHeapSize does not update when the page is opened from the local file system, you might not see the increasing memory usage. In that case, I have hosted this code for you here: https://memory-leak.surge.sh/example-for-waterfr
This Google I/O'19 presentation gives examples of real-world memory leaks as well as strategies for avoiding them:
Method getImageCached() returns a reference to an object, also caching a local reference. Even if this reference goes out of the method consumer's scope, the referenced memory cannot be garbage collected because there is a still a strong reference inside the implementation of getImageCached(). Ideally, the cached reference would be eligible for garbage collection if memory got too low. (Not exactly a memory leak, but a situation where there is memory that could be freed at the cost of running the expensive operations again).
Leak #1: the reference to the cached image. Solved by using weak references inside getImageCached().
Leak #2: the string keys inside the cache (Map object). Solved by using the new FinalizationGroup API.
Please see the linked video for JS code with line-by-line explanations.
More generally, "real" JS memory leaks are caused by unwanted references (to objects that will never be used again). They are usually bugs in the JS code. This article explains four common ways memory leaks are introduced in JS:
Accidental global variables
Forgotten timers/callbacks
Out of DOM references
Closures
An interesting kind of JavaScript memory leak documents how closures caused a memory leak in the popular MeteorJS framework.
2020 Update:
Most CPU-side memory overflow is no longer working on modern v8 engine based browser. However, we can overflow the GPU-side memory by running this script
// Initialize canvas and its context
window.reallyFatCanvas = document.createElement('canvas');
let context = window.reallyFatCanvas.getContext('2d');
// References new context inside context, in loop.
function leakingLoop() {
context.canvas.width = document.body.clientWidth;
context.canvas.height = document.body.clientHeight;
const newContext = document.createElement('canvas').getContext('2d');
context.context = newContext;
context.drawImage(newContext.canvas, 0, 0);
// The new context will reference another context on the next loop
context = newContext;
}
// Use interval instead of while(true) {...}
setInterval(leakingLoop,1);
EDIT: I rename every variables (and constants) so it makes a lot of sense. Here is the explanation.
Based on my observation, canvas context seems sync with Video Memory. So if we put reference of a canvas object which also reference another canvas object and so on, the Video RAM fills a lot more than DRAM, tested on microsoft edge and chrome.
This is my third attempt of screenshot:
I have no idea why my laptop always freeze seconds after taking screenshot while running this script. Please be careful if you want to try that script.
I tried to do something like that and got exception out of memory.
const test = (array) => {
array.push((new Array(1000000)).fill('test'));
};
const testArray = [];
for(let i = 0; i <= 1000; i++) {
test(testArray);
}
The Easiest Way Is:
while(true){}
A small example of code causing a 1MB memory leak:
Object.defineProperty(globalThis, Symbol(), {value: new Uint8Array(1<<20).slice(), writable: false, configurable: false})
After you run that code, the only way to free the leaked memory is to close the tab you ran it on.
If all you want is to create a memory leak, then the easiest way IMO is to instantiate a TypedArray since they hog up a fixed size of memory and outlive any references. For example, creating a Float64Array with 2^27 elements consumes 1GiB (1 Gibibyte) of memory since it needs 8 bytes per element.
Start the console and just write this:
new Float64Array(Math.pow(2, 27))
This is how I have things setup:
container.html
database1.js (contains large array called database1)
database2.js (contains large array called database2)
Here's a sample of the array (shortened from 6000+ rows to 2):
var database1=[[
"2010-01-03 07:45","2010-01-03 11:00","534","A","","","","","Installed washing machine","0","1","1","Indeed","",""],[
"2010-03-20 15:00","2010-03-20 16:00","571","F","","","","","Installed oven","0","5","1","Indeed","",""],[
etc,etc,etc...]];
When I append database1.js to the head on the container the memory used for IE in windows jumps from 7,7MB to 21,9MB. This is fine, I can now loop though the array called database1 (first row and column is at database1[0][0]).
The problem is that I need to re-append the database file from time to time and in doing so the memory usage increases (I thought the database array would be overwritten).
So the first re-append pushes memory usage in IE up to 30,4MB but then continues to increase on each re-append: 30,4MB > 33,9MB > 39,5MB > 42,1MB etc.
To prevent this from happening I now clear each item in the array before I re-append the database1 file.
for ( var i=0, il=database1.length; i<il; i++ ){
delete database1[i];
}
//append database1.js to head code here
This does help. The memory usage doesn't decrease to the initial 7,7MB but does however decrease from 21,9MB to 14,1MB. The next re-append increases the memory to 25,9MB (clearing with the loop: 18,8MB). The next re-append increases the memory to 29,3MB (clearing with the loop: 24,5MB).
I'm glad the memory usage is not climbing as fast but perhaps this can be optimized further? Unfortunately reloading the HTML page is not an option.
Memory management in JavaScript is aided by the garbage collector, a language feature that periodically deallocates memory no longer needed.
The delete operator essentially denotes an area of memory as being no longer needed. The memory will be deallocated when the garbage collector next runs.
Garbage collection is not predictable - delete $var or $var = null will allow the garbage collector to deallocate memory eventually. This may not be instant. This is why memory usage is not climbing as fast as you'd expect, and this is why delete $var is not reducing memory usage as fast as you'd expect.
IE does present a means for forcing the garbage collector to run:
if (typeof(CollectGarbage) == "function") {
CollectGarbage();
}
Your script may slow down whilst this is happening, particularly if there is a large volume of memory to deallocate.
It's best to avoid forcing garbage collection - let the garbage collector run at the point it determines is most appropriate. CollectGarbage() is also undocumented - the behaviour may change or the function may be removed at any time.
In short:
your use of delete database1[i] is the best you can do
this will deallocate memory eventually
memory usage will return to the expected level eventually
you can force garbage collection in IE, but this process will eventually happen on its own