I'm working on a graph visuzalization using D3 in a backbone view. I allow the user to pinch-zoom the graph, smoothly transitioning using webkit transforms, and redrawing on release. To keep the code simple, I'm just redrawing the graph at the new scale, rather than recaluclating the new positions and sizes for the elements (this was my original approach, but my team requested the redraw route).
[ I spoke with Bostock via twitter. This is actually not the preferred way of doing things ]
The thing I am noticing is that for each redraw, I'm dumping tons of dom nodes that aren't cleaned up.
This isn't related to circular references within event handlers/closures, as I've disabled all but my labels (these have no handlers attached), and the same behavior occurs.
I've tried aggressively removing the elements from the graph, but the dom nodes still seem to leak.
Here's some relevant code. 'render' is called for a new set of labels. Upon finishing zooming, 'close' is called on the old graph, and a new one is created with another view instantiation and call to 'render' :
render: function() {
// create the svg offscreen/off dom
//document.createElementNS(d3.ns.prefix.svg, "svg")
var svg = this.svg = d3.select(this.el)
.append("svg:svg")
.attr('width', this.VIEW_WIDTH)
.attr('height', this.VIEW_HEIGHT)
this._drawTimeTicks.call(this, true);
return this;
},
_drawTimeTicks: function(includeLabels) {
var bounds = this.getDayBounds();
var min = bounds.start;
var date = new Date(min);
var hour = 1000 * 60 * 60;
var hourDiff = 60 * this.SCALE;
var graphX = (date.getTime() - min) / 1000 / 60;
var textMargin = 7;
var textVert = 11;
// Using for loop to draw multiple vertical lines
// and time labels.
var timeTicks = d3.select(this.el).select('svg');
var width = timeTicks.attr('width');
var height = timeTicks.attr('height');
for (graphX; graphX < width; graphX += hourDiff) {
timeTicks.append("svg:line")
.attr("x1", graphX)
.attr("y1", 0)
.attr("x2", graphX)
.attr("y2", height)
.classed('timeTick');
if (includeLabels) {
timeTicks.append("svg:text")
.classed("timeLabel", true)
.text(this.formatDate(date))
.attr("x", graphX + textMargin)
.attr("y", textVert);
}
date.setTime(date.getTime() + hour);
}
close: function() {
console.log("### closing the header");
this.svg.selectAll('*').remove();
this.svg.remove();
this.svg = null;
this.el.innerHTML = '';
this.unbind();
this.remove();
}
As you can see, I'm not doing anything tricky with event handlers or closures. With a few zoom interactions I can leak dozens of dom nodes that are never reclaimed by GC.
Is this a memory leak, or is d3 doing something behind the scenes to optimize future graph construction/updates? Is there some better way to destroy a graph that I'm not aware of?
Any ideas?
D3 doesn't keep any hidden references to your nodes, so there's no internal concept of "DOM node cleanup". There are simply selections, which are arrays of DOM elements, and the DOM itself. If you remove an element from the DOM, and you don't keep any additional references to it, it should be reclaimed by the garbage collector.
(Aside: it's not clear whether the leak you are referring to is elements remaining in the DOM or orphaned elements not being reclaimed by the garbage collector. In the past, some older browsers had bugs garbage-collecting circular references between DOM elements and JavaScript closures, but I'm not aware of any such issues that affect modern browsers.)
If you are updating the DOM, the most performant way of doing this is (generally) using a data-join, because this allows you to reuse existing elements and only modify the attributes that need to change. Using a key function for the data-join is also a good idea for object constancy; if the same data is displayed both before and after the update, then you may not need to recompute all of its attributes, and the browser does less work to update.
In certain cases, there are alternatives to arbitrary updates that are faster, such as updating the transform attribute of an enclosing G element rather than updating the positions of descendant elements. As another example, you can do geometric zooming within an SVG element simply by changing the viewBox attribute. But geometric zooming is a very limited case; in general, the most efficient update depends on what precisely is changing. Use a data-join so that you can minimize the number of elements you append or remove, and minimize the number of attributes you need to recompute.
A couple other things I'll point out…
You could use a data-join to create multiple ticks at the same time, rather than using a for loop. For loops are almost never used with D3 since the data-join can create multiple elements (and hierarchical structures) without loops. Even better, use the axis component (d3.svg.axis) and a time scale (d3.time.scale) with a time format (d3.time.format).
Recent versions of D3 don't require the "svg:" namespace, so you can append "text", "line", etc.
I can't think of any situation where selectAll("*").remove() makes sense. The "*" selector matches all descendants, so this would remove every single descendant from its parent. You should always try to remove the top-most element—the SVG container, here—rather than redundant removals of child elements.
Related
I am using realtime data to draw some lines using SVG on a webpage. To manage the data I am using Angular.js and to manage the visualization I use D3.js.
I set up a controller in angular that holds the data(lines). The data consists of some arrays of points (dictionary with x/y coordinates). Some lines are known at initialization, others are updated according to live data.
I set up an angular directive ('topView') which contains an SVG element. For each line at initialization, I add it as a path using:
var routeLeftLine = container.select("#routes").append("path");
var routeLeftLineData = scope.val.route.left; // -> 1000+ points in there
routeLeftLine
.attr("d", lineFunction(routeLeftLineData))
.attr("stroke", "black")
.attr("stroke-width", 1)
.attr("fill", "none");
For each line I want to keep updating (1), I set up an angular directive, for instance:
<surface-cable val="data.cable"></surface-cable>
where data is my data object on the controller and data.cable the array of points.
The directive looks like this:
OCMSWeb.directive('surfaceCable', function ( /* dependencies */ ) {
return {
restrict: 'AE',
scope: {
val: '='
},
templateNamespace: 'svg',
replace: true,
template: '<g/>',
link: function (scope, element, attrs) {
var cableLine = d3.select(element[0]).append("path");
scope.$watch('val', function () {
var cableLineData = simplify(scope.val, 1, false); // size grows in time
cableLine
.attr("d", lineFunction(cableLineData))
.attr("stroke", "rgb(240,144,32)")
.attr("stroke-width", 1)
.attr("fill", "none");
}, true);
}
};
});
The structure works fine when I am updating the data using a timer, the change is reflected in the SVG.
The problem arises when I increase the number of points(>1000... I'll need even more in the future) that are in a line (both the non-changing, and the updated line have this effect) the performance degrades. The updating of the line becomes terribly slow, even when the elements to redraw do not contain many elements yet.
I can't find the cause. Does SVG/d3/angular render all elements in a svg again?
Is my way of binding data inefficient? Should I skip d3 all together?
I have tried to profile the javascript performance, and about 80-90% of the CPU time seems to go towards calls of angular $apply (which, I think, scans the DOM for changes?). Why does $apply take so long if an element (the line is a <path> element) has many data-points?
With this architecture, 1000 lines means 1000 directives, 1000 watches, and 1000 value comparisons every time you change anything on your scope, whether or not those values have actually changed. I doubt the root problem here is your d3 code, though reseting attributes for stroke, stroke-width, and fill unnecessarily certainly doesn't help.
Generally speaking, the better way to do this would be to have a single directive that takes an array of lines and handles the layout of all of your cable paths in the SVG. If your are looking at 10s of thousands of paths, then you might want to look at rendering them on canvas instead of SVG.
Although #ethan-jewett did not fully answer my question, you did
point me in the right direction though.
Because I linked the 'data' dictionary of my controller (containing both static and dynamic data) to the directive, I presume that angular does check all values in there for change. By moving my static data out of this 'data' dictionary, it does not get checked and makes this setup considerably faster.
Profiling still yields that angular gets slow when I increase the size of the dynamic data, and I assume this is for the same reason (angular needing to check all data for change). I am not sure on how I'll tackle this: I'll investigate whether D3.js has a more efficient mechanism for detecting changes in the data, or I'll split my long arrays in a static and a dynamic part (since they represent paths/cables, only the end of the cable can actually change. At some point a large part of the cable can be considered static.).
In famo.us, there are some easy ways to perform animations/interactions using modifiers on a surface. For instance, dragging and animating surfaces have pretty straight forward examples in the famo.us guides.
...assume require('') statements above here...
var mainContext = Engine.createContext();
var draggable = new Draggable({...});
var surface = new Surface({...});
var transitionableTransform = new TransitionableTransform();
var modifier = new Modifier({
origin: [.5, .5],
transform: transitionableTransform
});
surface.pipe(draggable);
surface.on('click', function () {
transitionableTransform.setScale([3, 3, 1], {duration: 300});
});
mainContext.add(draggable).add(surface);
However, in more complicated scenarios you might want to coordinate multiple animations, starting/stopping/reversing as needed depending on the interaction. In those cases, things as simple as adding transforms with a duration might work at first, but aren't guaranteed to not fall out of sync the more the user interacts with them.
The #render method appears to be a common place to put some types of coordinated animation. My limited understanding of it is it identifies the 'specs' of nodes that are being rendered, and is called on each frame of the render loop. So you might be able to identify each step of a particular animation, then depending on how it's interacted with be able to stop mid animation and change as needed.
For instance, Flipper seems to work this way
(src/views/Flipper.js):
Flipper.prototype.render = function render() {
var pos = this.state.get(); //state is a transitionable
var axis = this.options.direction;
var frontRotation = [0, 0, 0];
var backRotation = [0, 0, 0];
frontRotation[axis] = Math.PI * pos;
backRotation[axis] = Math.PI * (pos + 1);
if (this.options.cull && !this.state.isActive()) {
if (pos) return this.backNode.render();
else return this.frontNode.render();
}
else {
return [
{
transform: Transform.rotate.apply(null, frontRotation),
target: this.frontNode.render()
},
{
transform: Transform.rotate.apply(null, backRotation),
target: this.backNode.render()
}
];
}
};
Is there any documentation on the role #render should play when animating? Or how exactly the render method is supposed to work (for instance, the correct way to construct the specs that get returned). Is render supposed to be more low-level, and if so should a different construct be used?
The only way I've seen the render method used so far is to return specs from pre-existing elements. Personally, I've used it only when creating my own "Views", where I add a RenderNode to my class and create a pass-through render method that simply calls the RenderNode's render method. That's enough to pass custom objects into .add functions and have them work. I learned of that here:
How to remove nodes from the Render Tree?
As for understanding the construction of RenderSpecs themselves, I'm not aware of any docs. The best way to get a sense of it would be to read through the _parseSpec function of SpecParser:
https://github.com/Famous/core/blob/master/SpecParser.js#L92
From that it appears that any of the following can be used as a RenderSpec:
An Entity id (assigned to every Surface upon creation)
An object containing any of:
opacity
transform
origin
size
An array of RenderSpecs
If you want to take control of rendered nodes, write a custom View with a render function. The Flipper class is a simple example (and the RenderController is a complex example of this pattern)
How Famo.us renders:
Every RenderNode has a render function which creates a
renderSpec.
The renderSpec contains information about aModifier or
Surface.
2.1 The Modifier specs are used to calculatethe final CSS properties.
2.2 The Surface spec are coupled to DOMelements.
Every tick of the Engine, the renderSpec is rendered using the RenderNode.commit function.
The commit function uses the ElementAllocator (from the Context) to allocate/deallocate DOM elements. (Which actually recycles DOM nodes to conserve memory).
Therefore: Just return the correct renderSpec in your custom View, and famo.us will manage memory and performance for you.
Notes:
You don’t need to use the View class – an object with a render function will suffice. The View class simply adds events and options which is a nice way to create encapsulated, reusable components.
When a Surface element is allocated in the DOM, the deploy event is fired.
When a Surface element is deallocated, the recall event is fired.
As copied from http://famousco.de/2014/07/look-inside-rendering/
I am building something quite similar to this. What I would love is to make every node either their size as defined in the json file, OR, if it has no size attribute but a children attribute in json, the sum of all of its children's sizes. How would one go about doing that? I have tried various methods but short of adding things up and hardcoding it in JSON, which is a bit lame, I haven't found anything that really would have worked ;( Any suggestions, hive mind?
If your data is a tree structure, you could use a Partition Layout to initialize positions and sizes of nodes. The d.value returned by partition for parent nodes is by default the sum of values for all children nodes, assuming you've properly set the value accessor function to return the data variable that you want to use for size for leaf nodes.
Although the standard display in partition examples is to have space-filling rectangles or arcs instead of nodes and links, it still has all the basic functionality of the other hierarchy layouts. So once you've run the layout on your root to generate your array of nodes, you can run the links function to calculate the links.
If you still want a force-based layout instead of a static tree, you can just pass in your nodes and links to the force layout and start it up.
In sum:
var root = /*root object read from JSON data*/;
var w,h; /*set appropriately */
var partition = d3.layout.partition()
.size([w,h])
.value(function(d){return d.size;})
.children(function(d){return d.children;})
//optional: this is the default, change as needed
.sort(null);
//optional: turns off sorting,
//the default is to sort children by descending size
var nodes = partition(root);
var links = partition.links(nodes);
initialize();
var force = d3.layout.force()
.nodes(nodes)
.links(links)
.size([w,h])
/*and any other customization*/
.start();
svg.on("tick", update);
One thing to note. The x value created by partition layout is designed to be the corner of a rectangle instead of the centre of a circle. So if you position your nodes based on the original x value, you'll end up with parents off to the far left of their children. If you're running everything through a force-based layout afterwards, it will sort itself out eventually, but you can centre them from the beginning by setting d.x = d.x + d.dx/2 and d.y = d.y + d.dy/2 on all your nodes during initialization (e.g., using an .each() call in your enter() chain). And of course, use d.value to initialize your node size (with an appropriate scale).
In this example: http://bl.ocks.org/mbostock/1747543:
...Mike shows us how to avoid collision among nodes so that no two nodes overlap each other.
I wonder if it is possible to avoid collision between nodes and edges so that no node 'clips' or overlaps an edge unless it is connected by that edge.
The following example using D3 force-direct shows that node L overlaps with the edge connecting I and A, and similarly, node M overlaps with the edge connecting L and D. How do we prevent such cases?
If your graph doesn't have too many nodes, you can fake it. Just insert one or more nodes for each link, and set their position along the link in the tick handler. Check out http://bl.ocks.org/couchand/7190660 for an example, but the changes to Mike Bostock's version amount to basically just:
var linkNodes = [];
graph.links.forEach(function(link) {
linkNodes.push({
source: graph.nodes[link.source],
target: graph.nodes[link.target]
});
});
and
// force.on('tick', function() {
linkNodes.forEach(function(node) {
node.x = (node.source.x + node.target.x) * 0.5;
node.y = (node.source.y + node.target.y) * 0.5;
});
This will introduce a pretty serious performance overhead if you have very many nodes and edges, but if your graph doesn't get much larger than your example it would hardly be noticed.
You may also want to fiddle with the relative force of the real nodes versus the link nodes.
Take this one step further and you get the nice curved links of http://bl.ocks.org/mbostock/4600693.
I want to be able to change the radius of each node in my graph that i am creating using d3.js. However, i want to change the radius of each node, one at a time, and i want to able to control the delay between each change along with the sequence of the nodes.
For now this is what i have in terms of code:
var nodes = svg.selectAll(".node");
nodes.each(function() {
d3.select(this).
transition().
delay(100).
attr("r", "5")
});
You can replicate this simply by using the code at this link: http://bl.ocks.org/mbostock/4062045. The code that i have pasted above is simply an addition to the code at the aforementioned link.
When i run this, all the nodes in my graph transition simultaneously, i.e. grow in size (radius) simultaneously. I however want them to transition i.e. grow in size (radius), one at a time. I repeat that i want to be able to control:
the delay between the transition of each node and
the order of nodes that undergo the transitions.
Any pointers, tutorials, or even other stackoverflow answers would be great. I would ideally want some code examples.
The closest i have come to in terms of online references is this subsection of a tutorial on d3.js transitions: http://bost.ocks.org/mike/transition/#per-element. However, it lacks a concrete code example. I, being new to d3.js and javascript in general, am not able to pick it up without concrete code examples.
You can do this quite easily by calculating a delay based on each node's index. Mike Bostock has an example of such an implementation here. This is the relevant code:
var transition = svg.transition().duration(750),
delay = function(d, i) { return i * 50; };
transition.selectAll(".bar")
.delay(delay)
.attr("x", function(d) { return x0(d.letter); }); // this would be .attr('r', ... ) in your case
To control the order of the transition, all you would then have to do is sort the array so that the elements' indices reflect the animation flow you want. To see how to sort an array, refer to the documentation on JavaScript's array.sort method and also see the Arrays > Ordering section of the D3 API reference.