Develop Reference

D3 Sankey Node Width

I am new to JavaScript and not sure how to write this into a loop. How do I change the node width in the D3 library for Sankey diagrams so that it is not one constant value and instead updates based on the node names?
For example, I have this data:
"nodes": [
{"name":"A" },
{"name":"B" },
{"name":"C" }
],
And I want the node width of A=10, B=20, C=30.
Here is the relevant code from https://bl.ocks.org/tomshanley.
var sankey = d3.sankeyCircular()
.nodeWidth(10)
.nodePadding(40) //note that this will be overridden by nodePaddingRatio
.nodePaddingRatio(0.15)
.size([width, height])
.nodeId(function (d) {
return d.name;
})
.nodeAlign(d3.sankeyJustify)
.iterations(42)
.circularLinkGap(3);
Thank you!

To respect the data-driven approach it would be better to have the width inside your data array:
"nodes": [ {"name":"A", "width": 10 }, ...]
Then it will become:
.nodeWidth(d => d.width)
Otherwise you will need a way to map the name to the width, e.g. with:
var nameToWidth = {"A": 10, ...};
Then it will become:
.nodeWidth(d => nameToWidth[d.name])

With D3.js is it better to re-draw or "move" objects?

I've been experimenting with animation.
It's very simple to animate an object across the canvas by clearing the entire canvas and redrawing it in a new location every frame (or "tick"):
// Inside requestAnimationFrame(...) callback
// Clear canvas
canvas.selectAll('*').remove();
// ... calculate position of x and y
// x, y = ...
// Add object in new position
canvas.append('circle')
.attr('cx', x)
.attr('cy', y)
.attr('r', 10)
.attr('fill', '#ffffff');
Is this a bad practice or am I doing it right?
For instance, if you were making a screen full of objects moving around, is it better practice to animate them by updating their attributes (e.g., x, y coordinates) in each frame?
Or, perhaps there is some other method I'm entirely unaware of, no?
Note: my animation might include 100-200 objects in view at a time.

It is better to move them, because that is the only way you can animate without errors.
In d3.js the idea is that the objects are data-bound. Clearing and redrawing the 'canvas' is not the correct approach. Firstly its not a canvas, its a web page, and any clearing and redrawing is handled by the browser itself. You job is to bind data to SVG, basically.
You need to make use of the d3 events, enter, exit, update which handles how the SVG behaves when the databound underlying data is modified and let d3 handle the animations.
the most simple example is here: https://bost.ocks.org/mike/circles/
select your elements, and store the selction in a variable
var svg= d3.select("svg");
var circles = svg.selectAll('circle');
now we need to databind something to the circle.
var databoundCircles = circles.data([12,13,14,15,66]);
This data can be anything. Usually I would expect a list of object, but these are simple numbers.
handle how things 'are made' when data appears
databoundCircles.enter().append('circle');;
handle what happens to them when data is removed
databoundCircles.exit().remove()
handle what happens when the data is updated
databoundCircles.attr('r', function(d, i) { return d * 2; })
this will change the radius when the data changes.
And recap from that tutorial:
enter - incoming elements, entering the stage.
update - persistent elements, staying on stage.
exit - outgoing elements, exiting the stage.
so in conclusion: don't do it like you are. Make sure you are using those events specifically to handle the lifecycle of elements.
PRO TIP: if you're using a list of objects make sure you bind the data by id, or some unique identifier, or the animations might behave unusually over time. Remember you are binding data to SVG you are not just wiping and redrawing a canvas!
d3.selectAll('circle').data([{id:1},{id:2}], function(d) { return d.id; });
Make note the optional second argument, that tells us how to bind the data! very important!
var svg = d3.select("svg");
//the data looks like this.
var data = [{
id: 1,
r: 3,
x: 35,
y: 30
}, {
id: 2,
r: 5,
x: 30,
y: 35
}];
//data generator makes the list above
function newList() {
//just make a simple array full of the number 1
var items = new Array(randoNum(1, 10)).fill(1)
//make the pieces of data. ID is important!
return items.map(function(val, i) {
var r = randoNum(1, 16)
return {
id: i,
r: r,
x: randoNum(1, 200) + r,
y: randoNum(1, 100) + r
}
});
}
//im just making rando numbers with this.
function randoNum(from, to) {
return Math.floor(Math.random() * (to - from) + from);
}
function update(data) {
//1. get circles (there are none in the first pass!)
var circles = svg.selectAll('circle');
//2. bind data
var databoundCircles = circles.data(data, function(d) {
return d.id;
});
//3. enter
var enter = databoundCircles.enter()
.append('circle')
.attr('r', 0)
//4. exit
databoundCircles.exit()
.transition()
.attr('r', 0)
.remove();
//5. update
//(everything after transition is tweened)
databoundCircles
.attr('fill', function(d, i){
var h = parseInt(i.toString(16));
return '#' + [h,h,h].join('');
})
.transition()
.duration(1000)
.attr('r', function(d, i) {
return d.r * 4
})
.attr('cx', function(d, i) {
return d.x * 2;
})
.attr('cy', function(d, i){
return d.y * 2
})
;
}
//first time I run, I use my example data above
update(data);
//now i update every few seconds
//watch how d3 'keeps track' of each circle
setInterval(function() {
update(newList());
}, 2000);
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.4.11/d3.min.js"></script>
<svg width="500" height="300">
</svg>

Is this a bad practice or am I doing it right?
Yes, it is a bad practice. In a normal circumstance I like to call it lazy coding: clearing the SVG (or whatever) and painting the dataviz again.
But, in your case, it's even worse: you will end up writing a huge amount of code (not exactly laziness, though), ignoring d3.transition(), which can easily do what you want. And that takes us to your second question:
Or, perhaps there is some other method I'm entirely unaware of, no?
Yes, as I just said, it's called transition(): https://github.com/d3/d3-transition
Then, at the end, you said:
Note: my animation might include 100-200 objects in view at a time.
First, modern browsers can handle that very well. Second, you still have to remove and repaint manually all that elements. If you benchmark the two approaches, maybe this is even worse.
Thus, just use d3.transition().
You can change the data (or the attributes) of the elements anytime you want, and "moving" (or transitioning) them to the new value calling a transition. For instance, to move this circle around, I don't have to remove it and painting it again:
var circle = d3.select("circle")
setInterval(() => {
circle.transition()
.duration(900)
.attr("cx", Math.random() * 300)
.attr("cy", Math.random() * 150)
.ease(d3.easeElastic);
}, 1000)
<script src="https://d3js.org/d3.v4.min.js"></script>
<svg>
<circle r="10" cx="100" cy="50" fill="teal"></circle>
</svg>

Wrong result in dimplejs scatterplot

I'm trying to understand how to use with dimplejs but the result is not what i ment.
JSFiddleCode
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="http://dimplejs.org/dist/dimple.v2.0.0.min.js"></script>
<script type="text/javascript">
var svg = dimple.newSvg("#chartContainer", 590, 400);
d3.csv("carsData.csv", function (data) {
// change string (from CSV) into number format
data.forEach(function(d) {
if(d["Sports Car"]==1)
d.Category = "Sports Car";
else if(d["SUV"]==1)
d.Category = "SUV";
else
d.Category = "Other";
d.HP = +d.HP;
d["Engine Size (l)"] = +d["Engine Size (l)"];
});
// Latest period only
//dimple.filterData(data, "Date", "01/12/2012");
// Create the chart
var myChart = new dimple.chart(svg, data);
myChart.setBounds(60, 30, 420, 330)
// Create a standard bubble of SKUs by Price and Sales Value
// We are coloring by Owner as that will be the key in the legend
myChart.addMeasureAxis("x", "HP");
myChart.addMeasureAxis("y", "Engine Size (l)");
myChart.addSeries("Category", dimple.plot.bubble);
var myLegend = myChart.addLegend(530, 100, 60, 300, "Right");
myChart.draw();
// This is a critical step. By doing this we orphan the legend. This
// means it will not respond to graph updates. Without this the legend
// will redraw when the chart refreshes removing the unchecked item and
// also dropping the events we define below.
myChart.legends = [];
// This block simply adds the legend title. I put it into a d3 data
// object to split it onto 2 lines. This technique works with any
// number of lines, it isn't dimple specific.
svg.selectAll("title_text")
.data(["Click legend to","show/hide owners:"])
.enter()
.append("text")
.attr("x", 499)
.attr("y", function (d, i) { return 90 + i * 14; })
.style("font-family", "sans-serif")
.style("font-size", "10px")
.style("color", "Black")
.text(function (d) { return d; });
// Get a unique list of Owner values to use when filtering
var filterValues = dimple.getUniqueValues(data, "Category");
// Get all the rectangles from our now orphaned legend
myLegend.shapes.selectAll("rect")
// Add a click event to each rectangle
.on("click", function (e) {
// This indicates whether the item is already visible or not
var hide = false;
var newFilters = [];
// If the filters contain the clicked shape hide it
filterValues.forEach(function (f) {
if (f === e.aggField.slice(-1)[0]) {
hide = true;
} else {
newFilters.push(f);
}
});
// Hide the shape or show it
if (hide) {
d3.select(this).style("opacity", 0.2);
} else {
newFilters.push(e.aggField.slice(-1)[0]);
d3.select(this).style("opacity", 0.8);
}
// Update the filters
filterValues = newFilters;
// Filter the data
myChart.data = dimple.filterData(data, "Category", filterValues);
// Passing a duration parameter makes the chart animate. Without
// it there is no transition
myChart.draw(800);
});
});
the scatterplot result is only 3 and i dont know why.
the x is the HP and the y is horse power.
more questions:
1. how can i change the axis unit.
2. how can i control the size of each bubble.
3. how to fix the wrong results.
heres the result picture:
The csv file has 480 rows.
maybe the addseries is wrong (i dont know what it is)?

Dimple aggregates the data for you based on the first parameter of the addSeries method. You have passed "Category" which has 3 values and therefore creates 3 bubbles with summed values. If instead you want a bubble per vehicle coloured by category you could try:
myChart.addSeries(["Vehicle Name", "Category"], dimple.plot.bubble);
To change the axis unit you can use axis.tickFormat though the change above will reduce scale so you might find you don't need to.
To control bubble size based on values in your data you need to add a "z" axis. See this example.
If you want to just set a different marker size for your scatter plot you can do so after the draw method has been called with the following:
var mySeries = myChart.addSeries("Category", dimple.plot.bubble);
var myLegend = myChart.addLegend(530, 100, 60, 300, "Right");
myChart.draw();
// Set the bubble to 3 pixel radius
mySeries.shapes.selectAll("circle").attr("r", 3);
NB. A built in property for this is going to be included in the next release.

Compare/Diff new data with previous data on d3.js update

I'd like to represent the difference between the current data set and the previous data set, as calculated by the client.
Imagine I already have three circles, bound to the data [1, 2, 3]. Now I'd like to update the data and do something based on the difference between the new values and the old?
var new_data = [2, 2, 2]; // This is the new data I'd like to compare with the old
svg.selectAll("circle").data(new_data)
.transition().duration(2000)
.attr("fill", "red") // e.g. I'd like to colour the circles red if the change
// is negative, blue if positive, black if no change.
.attr("r", function(d) { return d * 10; });
Here's a JSFiddle with the above code set into an example.

You have two options for saving the old data attached to an element in order to identify changes after a new data join.
The first option, as you suggested, is to use data attributes. This SO Q&A describes that approach. Things to consider:
all your data values will get coerced to strings
you'll need a separate method call/attribute for each aspect of the data
you're manipulating the DOM, so it could slow things down if you've got a lot of elements or lot of data for each
the data is now part of the DOM, so can be saved with the image or accessed by other scripts
The second option is to store the data as a Javascript property of the DOM object for the element, in the same way that d3 stores the active data as the __data__ property. I've discussed this method in this forum post.
The general approach:
selection = selection.property(" __oldData__", function(d){ return d; } );
//store the old data as a property of the node
.data(newData, dataKeyFunction);
//over-write the default data property with new data
//and store the new data-joined selection in your variable
selection.enter() /*etc*/;
selection.attr("fill", function(d) {
// Within any d3 callback function,
// you can now compare `d` (the new data object)
// with `this.__oldData__` (the old data object).
// Just remember to check whether `this.__oldData__` exists
// to account for the just-entered elements.
if (this.__oldData__) { //old data exists
var dif = d.value - this.__oldData__.value;
return (dif) ? //is dif non-zero?
( (dif > 0)? "blue" : "red" ) :
"black" ;
} else {
return "green"; //value for new data
}
});
selection.property("__oldData__", null);
//delete the old data once it's no longer needed
//(not required, but a good idea if it's using up a lot of memory)
You can of course use any name for the old data property, it's just convention to throw a lot of "_" characters around it to avoid messing up any of the browser's native DOM properties.

As of D3 v4 you can use the built-in support for local variables. The internal implementation is basically the same as suggested by AmeliaBR's answer, but it frees you from having to do the storing of old data on your own. When using d3.local() you can set a value scoped to a specific DOM node, hence the name local variable. In below snippet this is done for each circle by the line
.each(function(d) { previousData.set(this, d) }); // Store previous data locally...
You can later on retrieve that value for any particular node it was stored upon:
.attr("fill", function(d) {
var diff = previousData.get(this) - d; // Retrieve previously stored data.
return diff < 0 ? "red" : diff > 0 ? "blue" : "black";
})
This full code might look something like this:
var old_data = [1, 2, 3]; // When the data gets updated I'd like to 'remember' these values
// Create a local variable for storing previous data.
var previousData = d3.local();
var svg = d3.select("body").append("svg")
.attr("width", 500)
.attr("height", 200);
var p = d3.select("body")
.append("p")
.text("Old data. Click on the circles to update the data.");
var circle = svg.selectAll("circle")
.data(old_data)
.enter().append("circle")
.attr("fill", "black")
.attr("r", function(d) { return d * 10; })
.attr("cx", function(d){ return d * 40; })
.attr("cy", function(d){ return d * 40; })
.each(function(d) { previousData.set(this, d) }); // Store previous data locally on each node
svg.on("click", function(d) {
p.text("Updated data.");
var new_data = [2, 2, 2]; // This is the new data I'd like to compare with the old
circle.data(new_data)
.transition().duration(2000)
.attr("fill", function(d) {
var diff = previousData.get(this) - d; // Retrieve previously stored data.
return diff < 0 ? "red" : diff > 0 ? "blue" : "black";
})
.attr("r", function(d) { return d * 10; });
});
<script src="https://d3js.org/d3.v4.js"></script>

D3.js - Retrieving DOM subset given data subset

I'm using d3.js to create a large number of svg:ellipse elements (~5000). After the initial rendering some of the data items can be updated via the backend (I'll know which ones) and I want to change the color of those ellipses (for example).
Is there a fast way to recover the DOM element or elements associated with a data item or items? Other than the obvious technique if recomputing a join over the full set of DOM elements with the subset of data?
var myData = [{ id: 'item1'}, { id: 'item2' }, ... { id: 'item5000' }];
var create = d3.selectAll('ellipse).data(myData, function(d) { return d.id; });
create.enter().append('ellipse').each(function(d) {
// initialize ellipse
});
// later on
// this works, but it seems like it would have to iterate over all 5000 elements
var subset = myData.slice(1200, 1210); // just an example
var updateElements = d3.selectAll('ellipse').data(subset, function(d) { return d.id; });
updateElements.each(function(d) {
// this was O(5000) to do the join, I _think_
// change color or otherwise update
});
I'm rendering updates multiple times per second (as fast as possible, really) and it seems like O(5000) to update a handful of elements is a lot.
I was thinking of something like this:
create.enter().append('ellipse').each(function(d) {
d.__dom = this;
// continue with initialization
});
// later on
// pull the dom nodes back out
var subset = myData.slice(1200, 1210).map(function(d) { return d.__dom; });
d3.selectAll(subset).each(function(d) {
// now it should be O(subset.length)
});
This works. But it seems like this would be a common pattern, so I'm wondering if there is a standard way to solve this problem? I actually want to use my data in multiple renderings, so I would need to be more clever so they don't trip over each other.
Basically, I know that d3 provides a map from DOM -> data via domElement.__data__. Is there a fast and easy way to compute the reverse map, other than caching the values myself manually?
I need to get from data -> DOM.

As long as you keep the d3 selection reference alive (create in your example), D3 is using a map to map the data keys to DOM nodes in the update so it's actually O(log n).
We can do some testing with the D3 update /data operator method vs a loop method over the subset:
var d3UpdateMethod = function() {
svg.selectAll("ellipse").data(subset, keyFunc)
.attr("style", "fill:green");
}
var loopMethod = function() {
for (var i=0; i < subset.length; i++) {
svg.selectAll(".di" + i)
.attr("style", "fill:green");
}
}
var timedTest = function(f) {
var sumTime=0;
for (var i=0; i < 10; i++) {
var startTime = Date.now();
f();
sumTime += (Date.now() - startTime);
}
return sumTime / 10;
};
var nextY = 100;
var log = function(text) {
svg.append("text")
.attr("x", width/2)
.attr("y", nextY+=100)
.attr("text-anchor", "middle")
.attr("style", "fill:red")
.text(text);
};
log("d3UpdateMethod time:" + timedTest(d3UpdateMethod));
log("loopMethod time:" + timedTest(loopMethod));
I also created a fiddle to demonstrate what I understand you're trying to do here.
Another method to make it easy to track the nodes that are in your subset is by adding a CSS class to the subset. For example:
var ellipse = svg.selectAll("ellipse").data(data, keyFunc).enter()
.append("ellipse")
.attr("class", function (d) {
var cl = "di" + d.i;
if (d.i % 10 == 0)
cl+= " subset"; //<< add css class for those nodes to be updated later
return cl;
})
...
Note how the "subset" class would be added only to those nodes that you know are in your subset to be updated later. You can then select them later for an update with the following:
svg.selectAll("ellipse.subset").attr("style", "fill:yellow");
I updated the fiddle to include this test too and it's nearly as fast as the directMethod.