Develop Reference

Scatterplot update: same number of data points, but new data points

Apologies in advance that my question does not include code, but rather is a high level question on D3 and how to build my app correctly. I will attempt to make my question as clear and concise as possible:
I am building a React / D3 app that creates a scatter graph of NBA team logos, that allows users to click buttons to choose variables for the X and Y axis. The user can also filter the graph to include only certain teams (those in a particular division of the NBA).
Here is a quick demo gif of the app that features the main problem I am having:
.
.
.
.
and here is the link to my app for anyone interested.
What is working correctly
When I change the X or Y axis button (2nd half of the gif), the team logos correctly slide to their new locations.
What is working incorrectly
When I change the division (1st half of gif), it changes the 5 team logos that are showing, which is correct. However, the animation (which I show a few times in the gif) is incorrect. The old logos should simply disappear in place, and the new logos should simply appear in place. Instead, the logos change and slide.
I understand why the animation is doing this - D3 sees 5 data points before the update, and 5 data points after the update, but doesn't distinguish that my points are unique (different team logos). Since the updated data points have new (x,y) locations (different stats for each team), it simply animates the old logos to the locations of the new logos.
My proposed fix
I think the structure of my app is holding be back with regards to fixing this. Currently, I have a container component that loads the data, filters the teams (based on the division selected), and then passes the filtered data (an array of objects with the team stats) into a graph component that creates the logo scatter graph.
If, on the other hand, I pass the full object (of all 30 teams) to the graph component, then could I fix this problem by simply having D3 change the "fill" of the markers to transparent when they are filtered out? This way, there are always 30 logos being plotted, although 25 would be invisible, and the logos displaying should animate correctly.
Thanks in advance for your thoughts!
Edit: please let me know if the post is unclear in any way and I will try to clarify. I try to avoid posting Qs without code, but this is a fairly high level Q that focuses on how the D3 general update pattern works, and how I can build a graph with a specific animation that works within the general update pattern framework.
Edit2: The radio buttons are built in the container component here. Using my API to grab the data from my database, and then using these radio buttons to filter the data, are all done in the container component. I am considering bringing these radio buttons into the graph component and building them with D3. I think I may have to.
Edit3: Should have shared earlier, here is the D3 code that makes these markers:
const update = svg.select('g.points')
.selectAll("rect")
.data(graphData);
// Second exit and remove
update.exit().remove();
// Third Update
update
.enter()
.append("rect")
.attr("transform", d => `translate(${(xScale(+d[xColname]) - logoRadius)}, ${(yScale(+d[yColname]) - logoRadius)})`)
.attr("x", 0).attr("y", 0)
.attr("height", logoRadius)
.attr("width", logoRadius)
.attr("fill", d => `url(#teamlogo-${d.teamAbbrev})`)
.attr("opacity", 0.95);
// And Fourth transition
update
.transition()
.duration(750)
.delay((d, i) => i * 15)
.attr("transform", d => `translate(${(xScale(+d[xColname]) - logoRadius)}, ${(yScale(+d[yColname]) - logoRadius)})`)
.attr("x", 0).attr("y", 0)
.attr("height", logoRadius)
.attr("width", logoRadius)
.attr("fill", d => `url(#teamlogo-${d.teamAbbrev})`)
.attr("opacity", 0.95);

The issue here seems to be just the lack of a key function during the data join.
You asked in the comments section:
Doesn't the D3 general update pattern just look for the number of objects in the data array?
If you set up a key function the answer is no. The thing is that...
If a key function is not specified, then the first datum in data is assigned to the first selected element, the second datum to the second selected element, and so on. A key function may be specified to control which datum is assigned to which element, replacing the default join-by-index, by computing a string identifier for each datum and element. (source)
So, if you don't set up a key function, because you have always just 5 teams you don't effectively have working enter and exit selections when you change the division, but just an update one: as you're binding the data by their order, D3 thinks that Chicago Bulls and Atlanta Hawks are the same team.
Solution: set up a key function.
It can be simple as using the team abbreviation (supposing they are unique):
const update = svg.select('g.points')
.selectAll("rect")
.data(graphData, function(d){
return d.teamAbbrv;
});
PS: Just a question not related to your problem: why are you appending rects here? Since you have logoRadius, doesn't appending circles seem more natural? On top of that, the data representation would be more accurate, since the center of the circle, regardless its size, is at the correct datum coordinate. That's not the case with a rectangle, in which the coordinates (x, y) represent its top left corner.

D3 exit selecting wrong data

I'm trying to build kind of real time graph using D3.js. Code is available at https://plnkr.co/edit/hrawv8CTBIsJf2QWTBMb?p=preview.
The source data represent user authentication results from different organizations. For each organization there is a name, ok count and fail count. The graph should be dynamically (getting the data in loop) updated based on data.
The code is based on https://bl.ocks.org/mbostock/3808234.
There are few problems and few things i'm not sure about.
exit function only selects the red bars based on data update:
// JOIN new data with old elements
// specify function for data matching - correct?
var boxes = svg.selectAll(".box").data(data, function(d) {
return d.inst_name;
});
// EXIT old elements not present in new data
// this works somehow strange
// it does select all red boxes
boxes.exit().transition(t).remove();
Why does exit() select only red bars and not all? From my point of understanding the d3 documentation exit() should only select such elements that do not have any new data. Shouldn't that be all bars in case of infinite loop and constant data file?
This obviously breaks the graph quite a lot (see plunker). I need the exit to select only bars, which are not available in data file anymore. See example below.
initial state of data file:
inst_name,ok,fail
inst1,24,-1
inst2,23,-3
...
updated state of data file:
inst_name,ok,fail
inst1,26,-1
inst14,22,-4
...
The bars (both blue and red) for inst2 from intial state should be removed (and replaced by data of inst14) when the data is updated. Why is this not working?
I've read, that new data are matched against older using index. I've specified that inst_name should be used:
var boxes = svg.selectAll(".box").data(data, function(d) {
return d.inst_name;
});
Is this necessary (I've used it everywhere when inserting data)?
Also the transition for removing the elements does not work. What is the problem?
I'm also not sure if specifying data is necessary when adding new bars:
var boxes = svg.selectAll(".box").data(data, function(d) {
return d.inst_name;
});
.....
// add new element in new data
svg.selectAll(".blue")
.data(data, function(d) { // is this necessary ?
return d.inst_name;
}) // use function for new data matching against inst_name, necessary?
.enter().append("rect")
.transition(t)
.attr("class", function(d) {
return "blue box "
})
.attr("x", function(d) {
return x(d.inst_name);
})
.attr("width", x.bandwidth())
.attr("y", function(d) {
return y(d.ok);
})
.attr("height", function(d) {
return height - y(d.ok + min);
})
Thanks for help.
EDIT
The underlying data get changed by script (this was not written clearly in the original post), so it can change independently of the graph state. The data should be only growing.

You've asked a lot of questions.
Why does exit() select only red bars and not all? From my point of understanding the d3 documentation exit() should only select such elements that do not have any new data. Shouldn't that be all bars in case of infinite loop and constant data file?
First, you build two sets of bars (blue [ok] and red [fail]). When you data bind these you give them the same key function, which identifies them by inst_name. You then do your data update, which now selects all the bars at once with:
svg.selectAll(".box")
You again data-bind with the same key function. Your data has 10 values in the array but you just selected 20 bars. The second 10 bars exit (the red ones) because to d3 they are not in your 10 data-points
The bars (both blue and red) for inst2 from intial state should be removed (and replaced by data of inst14) when the data is updated. Why is this not working?
I don't see that in your plunker, you are giving it the same data over and over.
Also the transition for removing the elements does not work. What is the problem?
You haven't given the transition anything to do. It'll run it, then at the end remove the rects. What you need is something for it to transition, like "height":
boxes.exit().transition(t).attr('height', 0).remove();
This will shrink them to 0 height.
So how do we clean up your code?
First, I would operate on g elements each one paired to an item in your data array. You then place both bars in the g that belong to that data point. Take a look here, I've started to clean-up your code (incomplete, though, hopefully it gets you going).

Append different shapes

I am trying to make a force graph with 2 types of shapes as nodes: rect and circle, the shape information is in d.shape. There are several threads out there, but the solutions are not very clear to me.
I tried first to use merge method, which does not work: in this jsbin, var circlesANDrects = rects shows rectangles and var circlesANDrects = circles shows circles, whereas var circlesANDrects = circles.merge(rects) does not show both.
Does anyone know how to fix this?
Otherwise, I think the idea solution would be to use one block and append different shapes according to the shape information:
var circlesANDrects = svg.append("g").selectAll("rect circle")
.data(force.nodes())
.enter()
<!-- a function that appends different shapes according to shape information -->
Does anyone know how to add cases / condition to append?

You can pass a function as an argument to append. According to the API:
If the specified type is a string, appends a new element of this type (tag name) as the last child of each selected element [...] Otherwise, the type may be a function which is evaluated for each selected element
The problem is, if you use a function, you cannot simply return "circle" or "rect", like this:
.append(function(d){
if(d.shape == "rect"){
return "rect";
} else {
return "circle";
}
});//this don't work...
Instead, you have to return the DOM element, something like this:
.append(function(d){
if(d.shape == "rect"){
return document.createElementNS("http://www.w3.org/2000/svg", "rect");
} else {
return document.createElementNS("http://www.w3.org/2000/svg", "circle");
}
});//this works...
As it is a little complicated, an easier solution (but not exactly following what you asked) is simply using a symbol here:
var circlesOrRects = svg.append("g").selectAll(".foo")
.data(force.nodes())
.enter()
.append("path")
.attr("d", d3.svg.symbol()
.type(function(d) { return d.shape == "rect" ? "circle" : "square"; }))
.call(force.drag);
Here is your Bin: https://jsbin.com/povuwulipu/1/edit

So, by the understanding the tick method in force layout.
tick method is a call back of force when each updation.
So, we need to update both rect and circle separately in your case. Because, You created and stored the circle and rect into two variables.
the working version of your code is here: https://jsbin.com/disayafube/1/edit?html,output

Object Consistency when updating shape attributes

I'm very new to D3 and Javascript, so forgive me if my code looks a little ugly or poorly organized.
I have been working on a plot that utilizes 3 metrics: an x and y axis, and the radius of the circle as a data metric for the plot. The data I am reading is a two dimensional array, with each row being a different metric, and each column being a new data point. I have successfully implemented a method to change the radius of the circle dynamically by picking a different metric from a drop box, but this was after struggling endlessly with a very particular issue - my data was being assigned to the wrong circle!
When I initially create my circles, I first use sort() to sort the circles in descending order from the default radius metric (in my code, its "impactcpu"). This was done to fix an issue where larger circles that were drawn after smaller circles were obstructing the smaller circles, so I wanted to "paint" the largest circles first.
I was able to get past this issue by first sorting my calculated data array before assignign it to the circles, which preserved the default order. However, I am now trying to do something similar with my X and Y axis. While my dropdown menu is correctly assigning metric values to circles, it is doing so to the WRONG circles. I have yet to figure out a solution to this issue, as re-sorting the array before assignign is like I was doing for the radius isn't working (which I expected). Are there any suggestions as to how I could ensure the right data point is assigned to the correct circle? Preferably one that wouldn't require an overhaul of the rest of my code :)
Please take a look at my jsfiddle for an example of my above situation:
http://jsfiddle.net/kingernest/YDQR4/3/
Example of how I am creating my circles initially:
var circles = svg.selectAll("circle")
.data(dataset, function(d) { return d.id })
.enter()
.append("circle")
.sort(function(a, b){ //Sort by radius size, helps reduce obstruction of circles
return d3.descending(a[14], b[14]);
})
.attr("cx", function(d){ //x axis is Req IO, col index 9
return xScale(d[9]);
})
.attr("cy", function(d){ //y axis is Req CPU, col index 8
return yScale(d[8]);
})
.attr("r", function(d){ //radius is based off Impact CPU, col 14
console.log("Rad: " + d[14])
return d[14] * 1.5;
})
.attr("class", "dataCircle")
etc
How I am currently altering my radius:
function changeRad() {
console.log(this.value);
var myRadData = [];
var index = metricHash[this.value];
var weight; //to adjust data to fit appropriately in graph
switch(this.value)
{
case "impactcpu":
weight = 1.5;
break;
case "spool":
weight = .0000001; //spool is normally a very large value
break;
case "pji":
weight = 8;
break;
case "unnecio":
weight = 12;
break;
case "duration":
weight = .0002;
break;
default: alert("Invalid value: " + this.value);
break;
}
for(var i=0; i < dataset.length; i++)
{
console.log(dataset[i][index]);
myRadData.push(dataset[i][index] * weight);
}
myRadData.sort(function(a,b){return b-a});
d3.selectAll("circle")
.data(myRadData)
.transition().duration(500)
.attr("r", function(d){
return d;
});
circles.data(dataset); //reassign old data set (with all data values)
}

There are two things I see with your code:
Inconsistent key function on your data binding - you correctly use it on your initial creation of the circles(.data(dataset, function(d) { return d.id })), but do not reference it when updating them, adding the same key on the updates will make sure that you are updating the same elements.
DOM sorting - Your use of selection.sort when initially creating your circles seems logical and appropriate(.sort(function(a, b){ return d3.descending(a[14], b[14]); })) I would recommending extending this to your update functions, rather than re-binding data.
I have made these quick updates to your code and it appears to solve your issues:
http://jsfiddle.net/AbHfk/3/

I'm not sure I grasp your entire code (it's quite long), but I think the solution lies along these lines:
1 - When you initially create the circles, use a keys function. Good, you're doing this:
.data(dataset, function(d) { return d.id; })
2 - Give the circles an ID attribute using the same function:
.attr("ID", function(d) { return d.id; })
3 - Then when you need to modify a particular circle individually you can select it like so:
svg.select('#' + myCircleID).attr('blahblah', somevalue)
I also notice that you've lost the ID attribute as you build up the myRadData array. This will prevent the code from joining them to the correct circles. Since you have an ID attribute at the beginning, you're better off using the keys function throughout, rather than trying to use sorting to make things line up.
If you want a more specific answer I think you need to boil the example down to the simplest possible form that reproduces the issue.

Adding and Removing Nodes in D3js Force Graph

I am loading json from database and creating a json file which loads fine. Now I don't know which steps to take for making the nodes responsive in a Force-Directed Graph. I need to remove and add new nodes and their links.
force.nodes(json.nodes)
.links(json.links)
.start();
initNodes(json);
How can I make this more dynamic or update it without resetting the whole visualization?
I have seen this question a couple of times not being answered so I hope someone can post and give a guide.

Adding nodes/links to my D3 force graph was very confusing until I better understood the way I was adding the initial set of nodes.
Assuming a <g> is what you'd like to use for your nodes:
// Select the element where you'd like to create the force layout
var el = d3.select("#svg");
// This should not select anything
el.selectAll("g")
// Because it's being compared to the data in force.nodes()
.data(force.nodes())
// Calling `.enter()` below returns the difference in nodes between
// the current selection and force.nodes(). At this point, there are
// no nodes in the selection, so `.enter()` should return
// all of the nodes in force.nodes()
.enter()
// Create the nodes
.append("g")
.attr("id", d.name)
.classed("manipulateYourNewNode", true);
Now let's make that function that will add a node to the layout once the graph has been initialized!
newNodeData is an object with the data you'd like to use for your new node.
connectToMe is a string containing the unique id of a node you'd like to connect your new node to.
function createNode (newNodeData, connectToMe) {
force.nodes().push(newNodeData);
el.selectAll("g")
.data(force.nodes(), function(datum, index) { return index })
The function given as the optional second argument in .data() is run once for each node in the selection and again for each node in force.nodes(), matching them up based on the returned value. If no function is supplied, a fallback function is invoked, which returns the index (as above).
However, there's most likely going to be a dispute between the index of your new selection (I believe the order is random) and the order in force.nodes(). Instead you'll most likely need the function to return a property that is unique to each node.
This time, .enter() will only return the node you're trying to add as newData because no key was found for it by the second argument of .data().
.enter()
.insert("g", "#svg")
.attr("id", d.name)
.classed("manipulatYourNewNode", true);
createLink(connectToMe, newNodeData.name);
force.start();
}
The function createLink (defined below) creates a link between your new node and your node of choice.
Additionally, the d3js API states that force.start() should be called after updating the layout.
Note: Calling force.stop() at the very beginning of my function was a huge help for me when I was first trying to figure out how to add nodes and links to my graph.
function createLink (from, to) {
var source = d3.select( "g#" + from ).datum(),
target = d3.select( "g#" + to ).datum(),
newLink = {
source: source,
target: target,
value: 1
};
force.links().push(newLink);
The code below works under the assumptions that:
#links is the wrapper element that contains all of your link elements
Your links are represented as <line> elements:
d3.select("#links")
.selectAll("line")
.data(force.links())
.enter()
.append("line");

You can see an example of how to append new nodes and relationships here:
http://bl.ocks.org/2432083
Getting rid of nodes and relationships is slightly trickier, but you can see the process here:
http://bl.ocks.org/1095795