I am using realtime data to draw some lines using SVG on a webpage. To manage the data I am using Angular.js and to manage the visualization I use D3.js.
I set up a controller in angular that holds the data(lines). The data consists of some arrays of points (dictionary with x/y coordinates). Some lines are known at initialization, others are updated according to live data.
I set up an angular directive ('topView') which contains an SVG element. For each line at initialization, I add it as a path using:
var routeLeftLine = container.select("#routes").append("path");
var routeLeftLineData = scope.val.route.left; // -> 1000+ points in there
routeLeftLine
.attr("d", lineFunction(routeLeftLineData))
.attr("stroke", "black")
.attr("stroke-width", 1)
.attr("fill", "none");
For each line I want to keep updating (1), I set up an angular directive, for instance:
<surface-cable val="data.cable"></surface-cable>
where data is my data object on the controller and data.cable the array of points.
The directive looks like this:
OCMSWeb.directive('surfaceCable', function ( /* dependencies */ ) {
return {
restrict: 'AE',
scope: {
val: '='
},
templateNamespace: 'svg',
replace: true,
template: '<g/>',
link: function (scope, element, attrs) {
var cableLine = d3.select(element[0]).append("path");
scope.$watch('val', function () {
var cableLineData = simplify(scope.val, 1, false); // size grows in time
cableLine
.attr("d", lineFunction(cableLineData))
.attr("stroke", "rgb(240,144,32)")
.attr("stroke-width", 1)
.attr("fill", "none");
}, true);
}
};
});
The structure works fine when I am updating the data using a timer, the change is reflected in the SVG.
The problem arises when I increase the number of points(>1000... I'll need even more in the future) that are in a line (both the non-changing, and the updated line have this effect) the performance degrades. The updating of the line becomes terribly slow, even when the elements to redraw do not contain many elements yet.
I can't find the cause. Does SVG/d3/angular render all elements in a svg again?
Is my way of binding data inefficient? Should I skip d3 all together?
I have tried to profile the javascript performance, and about 80-90% of the CPU time seems to go towards calls of angular $apply (which, I think, scans the DOM for changes?). Why does $apply take so long if an element (the line is a <path> element) has many data-points?
With this architecture, 1000 lines means 1000 directives, 1000 watches, and 1000 value comparisons every time you change anything on your scope, whether or not those values have actually changed. I doubt the root problem here is your d3 code, though reseting attributes for stroke, stroke-width, and fill unnecessarily certainly doesn't help.
Generally speaking, the better way to do this would be to have a single directive that takes an array of lines and handles the layout of all of your cable paths in the SVG. If your are looking at 10s of thousands of paths, then you might want to look at rendering them on canvas instead of SVG.
Although #ethan-jewett did not fully answer my question, you did
point me in the right direction though.
Because I linked the 'data' dictionary of my controller (containing both static and dynamic data) to the directive, I presume that angular does check all values in there for change. By moving my static data out of this 'data' dictionary, it does not get checked and makes this setup considerably faster.
Profiling still yields that angular gets slow when I increase the size of the dynamic data, and I assume this is for the same reason (angular needing to check all data for change). I am not sure on how I'll tackle this: I'll investigate whether D3.js has a more efficient mechanism for detecting changes in the data, or I'll split my long arrays in a static and a dynamic part (since they represent paths/cables, only the end of the cable can actually change. At some point a large part of the cable can be considered static.).
Related
Posting both question & answer here to save somebody else the same trouble later...
When I create two sunburst charts using d3.layout.partition, the first sunburst's slice proportions are overwritten by the second sunburst's slice proportions upon resize of the slices.
The two charts pass different .value accessor functions into the partition layout, e.g.
d3.layout.partition()
.sort(null)
.value(function(d) { return 1; });
vs.
d3.layout.partition()
.sort(null)
.value(function(d) { return d.size; });
And they generate their own list of nodes that are not shared between the two sunbursts. However, if I re-call the d3.svg.arc generator to resize to larger radius (but not change overall proportions), the slice angles are suddenly overwritten.
See the example here: http://bl.ocks.org/explunit/ab8cf15534f7fec5ac6d
The problem is that while partition.nodes() seems to generate a new data structure (e.g if you give it some .key functions, it writes the extra properties (e.g. .x, .y, .dx, dy) to the underlying data and does not make a copy of the data. Thus if the data structure is shared between the two charts, these .x, .y, .dx, dy properties will bleed through to the other graphs.
This seems like a bug to me, but in reading this old GitHub issue it seems to be treated as "by design". Perhaps it will be reconsidered in future versions.
One workaround is to use something like Lodash/Underscore cloneDeep or Angular's copy to make each chart have it's own copy of the data.
makeSunburst(null, _.cloneDeep(root), countAccessorFn);
makeSunburst(null, _.cloneDeep(root), sizeAccessorFn);
See example here: http://bl.ocks.org/explunit/e9efb830439247eea1be
An alternative to copying the whole dataset for each chart would be to simply recompute the partition before re-rendering.
Instead of having makeSunburst() be a function of the accessor, make it a function of the partition. Pass a different partition function to each chart:
// create separate partition variables
var countPartition = d3.layout.partition().sort(null).value(countAccessorFn);
var sizePartition = d3.layout.partition().sort(null).value(sizeAccessorFn);
// make the charts as a function of partition
charts.push(makeSunburst(root, countPartition));
charts.push(makeSunburst(root, sizePartition));
Then before applying the transition, simply update the nodes variable to reflect the associated partition:
addToRadius: function(radiusChange) {
radius += radiusChange;
ringRadiusScale.range([0, radius]);
// update the data before re-rendering each chart
nodes = partition.nodes(dataRoot);
path.transition().attr('d', arc);
}
Now when you update each chart, it is using the correct partition.
Here's an updated example.
I have a reusable module based on the d3.layout.pack graph example.
I added a transition on exit on the node elements but it seems like the transition works only for one data set and it's not working for the other.
Basically, to simulate the data update I am calling a function with setInterval this way:
function test(){
d3.select('#vis')
.datum(data2)
.call(cluster);
}
setInterval(test, 1500);
...and I added the transition this way:
c.exit().transition()
.duration(700)
.attr("r", function(d){ return 0; })
.remove();
You can find the data update section in the bottom of the file and find the exit transition handling on line 431.
Could you please check what's wrong?
The behaviour you're seeing is caused by the way data is matched in D3. For one of the data sets, all the data elements are matched by existing DOM elements and hence the exit selection is empty -- you're simply updating position, dimensions, etc. of the existing elements.
The way to "fix" this is to tell D3 explicitly how you want data to be matched -- for example by changing line 424 to
.data(nodes, function(d) { return d.name; });
which will compare the name to determine whether a data element is represented by a DOM element. Modified jsfiddle here.
Folks -
I'm now trying to trigger a second chart based on which series in the first chart is clicked on.
Based on which is chosen, one of two data sets are sent to the function:
.on("mouseup", function(d) {return d.myCat == 0 ? updateData(yesXYZData) : updateData(nonXYZData)})
This part works, but I'm getting one big stack in the target div, not the iteration I am expecting.
function updateData(whichDataSet) {...
I've tried putting the updateData() function into the window.onload function, duping or reusing various elements (since the domain and range for the X axis are the same, I expect to reuse).
[Note- I have taken Lars Kothoff's advice regarding numbers in the data object. Also, I will create a better data structure later, using crossfilter.js and native d3.js data manipulation- for now I need a working prototype demonstrating functionality.]
here is the gist:
https://gist.github.com/RCL1/6906892
Thanks in advance!
-RL
line 242 of the gist. I needed to use a non-filtered version of the data to calculate x axis (totalAll.map).
I want to be able to change the radius of each node in my graph that i am creating using d3.js. However, i want to change the radius of each node, one at a time, and i want to able to control the delay between each change along with the sequence of the nodes.
For now this is what i have in terms of code:
var nodes = svg.selectAll(".node");
nodes.each(function() {
d3.select(this).
transition().
delay(100).
attr("r", "5")
});
You can replicate this simply by using the code at this link: http://bl.ocks.org/mbostock/4062045. The code that i have pasted above is simply an addition to the code at the aforementioned link.
When i run this, all the nodes in my graph transition simultaneously, i.e. grow in size (radius) simultaneously. I however want them to transition i.e. grow in size (radius), one at a time. I repeat that i want to be able to control:
the delay between the transition of each node and
the order of nodes that undergo the transitions.
Any pointers, tutorials, or even other stackoverflow answers would be great. I would ideally want some code examples.
The closest i have come to in terms of online references is this subsection of a tutorial on d3.js transitions: http://bost.ocks.org/mike/transition/#per-element. However, it lacks a concrete code example. I, being new to d3.js and javascript in general, am not able to pick it up without concrete code examples.
You can do this quite easily by calculating a delay based on each node's index. Mike Bostock has an example of such an implementation here. This is the relevant code:
var transition = svg.transition().duration(750),
delay = function(d, i) { return i * 50; };
transition.selectAll(".bar")
.delay(delay)
.attr("x", function(d) { return x0(d.letter); }); // this would be .attr('r', ... ) in your case
To control the order of the transition, all you would then have to do is sort the array so that the elements' indices reflect the animation flow you want. To see how to sort an array, refer to the documentation on JavaScript's array.sort method and also see the Arrays > Ordering section of the D3 API reference.
I'm working on a graph visuzalization using D3 in a backbone view. I allow the user to pinch-zoom the graph, smoothly transitioning using webkit transforms, and redrawing on release. To keep the code simple, I'm just redrawing the graph at the new scale, rather than recaluclating the new positions and sizes for the elements (this was my original approach, but my team requested the redraw route).
[ I spoke with Bostock via twitter. This is actually not the preferred way of doing things ]
The thing I am noticing is that for each redraw, I'm dumping tons of dom nodes that aren't cleaned up.
This isn't related to circular references within event handlers/closures, as I've disabled all but my labels (these have no handlers attached), and the same behavior occurs.
I've tried aggressively removing the elements from the graph, but the dom nodes still seem to leak.
Here's some relevant code. 'render' is called for a new set of labels. Upon finishing zooming, 'close' is called on the old graph, and a new one is created with another view instantiation and call to 'render' :
render: function() {
// create the svg offscreen/off dom
//document.createElementNS(d3.ns.prefix.svg, "svg")
var svg = this.svg = d3.select(this.el)
.append("svg:svg")
.attr('width', this.VIEW_WIDTH)
.attr('height', this.VIEW_HEIGHT)
this._drawTimeTicks.call(this, true);
return this;
},
_drawTimeTicks: function(includeLabels) {
var bounds = this.getDayBounds();
var min = bounds.start;
var date = new Date(min);
var hour = 1000 * 60 * 60;
var hourDiff = 60 * this.SCALE;
var graphX = (date.getTime() - min) / 1000 / 60;
var textMargin = 7;
var textVert = 11;
// Using for loop to draw multiple vertical lines
// and time labels.
var timeTicks = d3.select(this.el).select('svg');
var width = timeTicks.attr('width');
var height = timeTicks.attr('height');
for (graphX; graphX < width; graphX += hourDiff) {
timeTicks.append("svg:line")
.attr("x1", graphX)
.attr("y1", 0)
.attr("x2", graphX)
.attr("y2", height)
.classed('timeTick');
if (includeLabels) {
timeTicks.append("svg:text")
.classed("timeLabel", true)
.text(this.formatDate(date))
.attr("x", graphX + textMargin)
.attr("y", textVert);
}
date.setTime(date.getTime() + hour);
}
close: function() {
console.log("### closing the header");
this.svg.selectAll('*').remove();
this.svg.remove();
this.svg = null;
this.el.innerHTML = '';
this.unbind();
this.remove();
}
As you can see, I'm not doing anything tricky with event handlers or closures. With a few zoom interactions I can leak dozens of dom nodes that are never reclaimed by GC.
Is this a memory leak, or is d3 doing something behind the scenes to optimize future graph construction/updates? Is there some better way to destroy a graph that I'm not aware of?
Any ideas?
D3 doesn't keep any hidden references to your nodes, so there's no internal concept of "DOM node cleanup". There are simply selections, which are arrays of DOM elements, and the DOM itself. If you remove an element from the DOM, and you don't keep any additional references to it, it should be reclaimed by the garbage collector.
(Aside: it's not clear whether the leak you are referring to is elements remaining in the DOM or orphaned elements not being reclaimed by the garbage collector. In the past, some older browsers had bugs garbage-collecting circular references between DOM elements and JavaScript closures, but I'm not aware of any such issues that affect modern browsers.)
If you are updating the DOM, the most performant way of doing this is (generally) using a data-join, because this allows you to reuse existing elements and only modify the attributes that need to change. Using a key function for the data-join is also a good idea for object constancy; if the same data is displayed both before and after the update, then you may not need to recompute all of its attributes, and the browser does less work to update.
In certain cases, there are alternatives to arbitrary updates that are faster, such as updating the transform attribute of an enclosing G element rather than updating the positions of descendant elements. As another example, you can do geometric zooming within an SVG element simply by changing the viewBox attribute. But geometric zooming is a very limited case; in general, the most efficient update depends on what precisely is changing. Use a data-join so that you can minimize the number of elements you append or remove, and minimize the number of attributes you need to recompute.
A couple other things I'll point out…
You could use a data-join to create multiple ticks at the same time, rather than using a for loop. For loops are almost never used with D3 since the data-join can create multiple elements (and hierarchical structures) without loops. Even better, use the axis component (d3.svg.axis) and a time scale (d3.time.scale) with a time format (d3.time.format).
Recent versions of D3 don't require the "svg:" namespace, so you can append "text", "line", etc.
I can't think of any situation where selectAll("*").remove() makes sense. The "*" selector matches all descendants, so this would remove every single descendant from its parent. You should always try to remove the top-most element—the SVG container, here—rather than redundant removals of child elements.