I'm using d3.js to create a large number of svg:ellipse elements (~5000). After the initial rendering some of the data items can be updated via the backend (I'll know which ones) and I want to change the color of those ellipses (for example).
Is there a fast way to recover the DOM element or elements associated with a data item or items? Other than the obvious technique if recomputing a join over the full set of DOM elements with the subset of data?
var myData = [{ id: 'item1'}, { id: 'item2' }, ... { id: 'item5000' }];
var create = d3.selectAll('ellipse).data(myData, function(d) { return d.id; });
create.enter().append('ellipse').each(function(d) {
// initialize ellipse
});
// later on
// this works, but it seems like it would have to iterate over all 5000 elements
var subset = myData.slice(1200, 1210); // just an example
var updateElements = d3.selectAll('ellipse').data(subset, function(d) { return d.id; });
updateElements.each(function(d) {
// this was O(5000) to do the join, I _think_
// change color or otherwise update
});
I'm rendering updates multiple times per second (as fast as possible, really) and it seems like O(5000) to update a handful of elements is a lot.
I was thinking of something like this:
create.enter().append('ellipse').each(function(d) {
d.__dom = this;
// continue with initialization
});
// later on
// pull the dom nodes back out
var subset = myData.slice(1200, 1210).map(function(d) { return d.__dom; });
d3.selectAll(subset).each(function(d) {
// now it should be O(subset.length)
});
This works. But it seems like this would be a common pattern, so I'm wondering if there is a standard way to solve this problem? I actually want to use my data in multiple renderings, so I would need to be more clever so they don't trip over each other.
Basically, I know that d3 provides a map from DOM -> data via domElement.__data__. Is there a fast and easy way to compute the reverse map, other than caching the values myself manually?
I need to get from data -> DOM.
As long as you keep the d3 selection reference alive (create in your example), D3 is using a map to map the data keys to DOM nodes in the update so it's actually O(log n).
We can do some testing with the D3 update /data operator method vs a loop method over the subset:
var d3UpdateMethod = function() {
svg.selectAll("ellipse").data(subset, keyFunc)
.attr("style", "fill:green");
}
var loopMethod = function() {
for (var i=0; i < subset.length; i++) {
svg.selectAll(".di" + i)
.attr("style", "fill:green");
}
}
var timedTest = function(f) {
var sumTime=0;
for (var i=0; i < 10; i++) {
var startTime = Date.now();
f();
sumTime += (Date.now() - startTime);
}
return sumTime / 10;
};
var nextY = 100;
var log = function(text) {
svg.append("text")
.attr("x", width/2)
.attr("y", nextY+=100)
.attr("text-anchor", "middle")
.attr("style", "fill:red")
.text(text);
};
log("d3UpdateMethod time:" + timedTest(d3UpdateMethod));
log("loopMethod time:" + timedTest(loopMethod));
I also created a fiddle to demonstrate what I understand you're trying to do here.
Another method to make it easy to track the nodes that are in your subset is by adding a CSS class to the subset. For example:
var ellipse = svg.selectAll("ellipse").data(data, keyFunc).enter()
.append("ellipse")
.attr("class", function (d) {
var cl = "di" + d.i;
if (d.i % 10 == 0)
cl+= " subset"; //<< add css class for those nodes to be updated later
return cl;
})
...
Note how the "subset" class would be added only to those nodes that you know are in your subset to be updated later. You can then select them later for an update with the following:
svg.selectAll("ellipse.subset").attr("style", "fill:yellow");
I updated the fiddle to include this test too and it's nearly as fast as the directMethod.
Related
I've been experimenting with animation.
It's very simple to animate an object across the canvas by clearing the entire canvas and redrawing it in a new location every frame (or "tick"):
// Inside requestAnimationFrame(...) callback
// Clear canvas
canvas.selectAll('*').remove();
// ... calculate position of x and y
// x, y = ...
// Add object in new position
canvas.append('circle')
.attr('cx', x)
.attr('cy', y)
.attr('r', 10)
.attr('fill', '#ffffff');
Is this a bad practice or am I doing it right?
For instance, if you were making a screen full of objects moving around, is it better practice to animate them by updating their attributes (e.g., x, y coordinates) in each frame?
Or, perhaps there is some other method I'm entirely unaware of, no?
Note: my animation might include 100-200 objects in view at a time.
It is better to move them, because that is the only way you can animate without errors.
In d3.js the idea is that the objects are data-bound. Clearing and redrawing the 'canvas' is not the correct approach. Firstly its not a canvas, its a web page, and any clearing and redrawing is handled by the browser itself. You job is to bind data to SVG, basically.
You need to make use of the d3 events, enter, exit, update which handles how the SVG behaves when the databound underlying data is modified and let d3 handle the animations.
the most simple example is here: https://bost.ocks.org/mike/circles/
select your elements, and store the selction in a variable
var svg= d3.select("svg");
var circles = svg.selectAll('circle');
now we need to databind something to the circle.
var databoundCircles = circles.data([12,13,14,15,66]);
This data can be anything. Usually I would expect a list of object, but these are simple numbers.
handle how things 'are made' when data appears
databoundCircles.enter().append('circle');;
handle what happens to them when data is removed
databoundCircles.exit().remove()
handle what happens when the data is updated
databoundCircles.attr('r', function(d, i) { return d * 2; })
this will change the radius when the data changes.
And recap from that tutorial:
enter - incoming elements, entering the stage.
update - persistent elements, staying on stage.
exit - outgoing elements, exiting the stage.
so in conclusion: don't do it like you are. Make sure you are using those events specifically to handle the lifecycle of elements.
PRO TIP: if you're using a list of objects make sure you bind the data by id, or some unique identifier, or the animations might behave unusually over time. Remember you are binding data to SVG you are not just wiping and redrawing a canvas!
d3.selectAll('circle').data([{id:1},{id:2}], function(d) { return d.id; });
Make note the optional second argument, that tells us how to bind the data! very important!
var svg = d3.select("svg");
//the data looks like this.
var data = [{
id: 1,
r: 3,
x: 35,
y: 30
}, {
id: 2,
r: 5,
x: 30,
y: 35
}];
//data generator makes the list above
function newList() {
//just make a simple array full of the number 1
var items = new Array(randoNum(1, 10)).fill(1)
//make the pieces of data. ID is important!
return items.map(function(val, i) {
var r = randoNum(1, 16)
return {
id: i,
r: r,
x: randoNum(1, 200) + r,
y: randoNum(1, 100) + r
}
});
}
//im just making rando numbers with this.
function randoNum(from, to) {
return Math.floor(Math.random() * (to - from) + from);
}
function update(data) {
//1. get circles (there are none in the first pass!)
var circles = svg.selectAll('circle');
//2. bind data
var databoundCircles = circles.data(data, function(d) {
return d.id;
});
//3. enter
var enter = databoundCircles.enter()
.append('circle')
.attr('r', 0)
//4. exit
databoundCircles.exit()
.transition()
.attr('r', 0)
.remove();
//5. update
//(everything after transition is tweened)
databoundCircles
.attr('fill', function(d, i){
var h = parseInt(i.toString(16));
return '#' + [h,h,h].join('');
})
.transition()
.duration(1000)
.attr('r', function(d, i) {
return d.r * 4
})
.attr('cx', function(d, i) {
return d.x * 2;
})
.attr('cy', function(d, i){
return d.y * 2
})
;
}
//first time I run, I use my example data above
update(data);
//now i update every few seconds
//watch how d3 'keeps track' of each circle
setInterval(function() {
update(newList());
}, 2000);
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.4.11/d3.min.js"></script>
<svg width="500" height="300">
</svg>
Is this a bad practice or am I doing it right?
Yes, it is a bad practice. In a normal circumstance I like to call it lazy coding: clearing the SVG (or whatever) and painting the dataviz again.
But, in your case, it's even worse: you will end up writing a huge amount of code (not exactly laziness, though), ignoring d3.transition(), which can easily do what you want. And that takes us to your second question:
Or, perhaps there is some other method I'm entirely unaware of, no?
Yes, as I just said, it's called transition(): https://github.com/d3/d3-transition
Then, at the end, you said:
Note: my animation might include 100-200 objects in view at a time.
First, modern browsers can handle that very well. Second, you still have to remove and repaint manually all that elements. If you benchmark the two approaches, maybe this is even worse.
Thus, just use d3.transition().
You can change the data (or the attributes) of the elements anytime you want, and "moving" (or transitioning) them to the new value calling a transition. For instance, to move this circle around, I don't have to remove it and painting it again:
var circle = d3.select("circle")
setInterval(() => {
circle.transition()
.duration(900)
.attr("cx", Math.random() * 300)
.attr("cy", Math.random() * 150)
.ease(d3.easeElastic);
}, 1000)
<script src="https://d3js.org/d3.v4.min.js"></script>
<svg>
<circle r="10" cx="100" cy="50" fill="teal"></circle>
</svg>
When using Hamster.js we have to define the parameter array as
params = {"array":my_array}
my_array is formed by elements with many attributes and I need to change the attributes x and y.
In the function I pass to Hamster.js I defined:
function map_node_hamster() {
params.array.forEach(function (d) {
d.x = some_calculation_here;
d.y = other_calculation_here;
}
}
but after calling the hamster.run() function, the elements of my original array stay intact.
As I need performance, I thought Hamsters could just change the values that my array is pointing. I actually do not know much about Javascript and how it manages arrays.
I'm calling the run function like this:
console.log("Before hamster:");
console.log(network.nodes);
p = {'array': network.nodes, "w": w, "h":h};
hamsters.run(p, map_node_hamster, function(output){console.log(output); return output;}, hamsters.maxThreads, true);
console.log("After hamster:");
console.log(network.nodes);
And the elements of network.nodes are intact after hamsters.run().
How could I change elements of array inside run function? Or.. How would be the right way to do the changes?
As the vector nodes is large, copying, sorting, and things like this will decrease performance and maybe it will be worst than single thread/non-parallel version.
It seems like the answer is to create an index array and change nodes in the callback function (output).
Despite the behavior of my "d3 graphic representation of network" is really not what I expected... maybe the code is actually right in the sense of answering my question (how to change objects from array in parallel computing with Hamsters.js). That is:
p = {'array': network.indexes,
"nodes": network.nodes,
"w": w,
"h": h};
hamsters.run(p, map_node_hamster, function (output) {
output.forEach(function (d) {
network.nodes[d.i].x = d.xy[0];
network.nodes[d.i].y = d.xy[1];
});
return output;
}, cores, true);
And... changing the function to work like this:
function map_node_hamster() {
pfor (var i = 0; i < params.array.length; i++) {
var result;
var d = params.nodes[params.array[i]];
var d = params.nodes[params.array[i]];
result = {x: d.x calculation, y: d.y calculation};
rtn.data.push({"i": params.array[i], "xy": [result.x, result.y]});
}
}
I'm using d3 library and I want to create different elements by checking some values. If I do:
elements.append("rect").attr(...);
What happens if I want to create different elements? I tried:
elements.append(function (d) {
if (d.foo) {
return "rect";
}
return "circle";
});
This seems not to work.
What's the alternative to this?
As you've already pointed out, .append() accepts a function that returns the DOM element to append. This looks as follows.
var data = ["circle", "rect"];
d3.select("svg").selectAll(".shape").data(data)
.enter()
.append(function(d) {
return document.createElementNS(d3.ns.prefix.svg, d);
});
Of course you also need to set all the attributes correctly, which is the crux of the method -- in principle you don't need lots of if/switch statements to handle each element type, but in practice you do because the attributes you need to set differ (unless you want to set everything for everything).
Complete demo here.
Even append function accepts a function, I'm not sure how that should work (maybe someone brings the light here). However the following solution is human readable and easy to implement:
elements.each(function (d) {
var el = d3.select(this);
var type = "circle";
if (el.foo) {
type = "rect";
}
var aType = el.append(type);
if (type === "rect") {
aType.attr("rx", 5)
.attr("ry", 5)
.attr("height", ...)
.attr("width", ...);
} else if (type === "circle") {
aType.attr("r", ...);
}
});
I'd like to represent the difference between the current data set and the previous data set, as calculated by the client.
Imagine I already have three circles, bound to the data [1, 2, 3]. Now I'd like to update the data and do something based on the difference between the new values and the old?
var new_data = [2, 2, 2]; // This is the new data I'd like to compare with the old
svg.selectAll("circle").data(new_data)
.transition().duration(2000)
.attr("fill", "red") // e.g. I'd like to colour the circles red if the change
// is negative, blue if positive, black if no change.
.attr("r", function(d) { return d * 10; });
Here's a JSFiddle with the above code set into an example.
You have two options for saving the old data attached to an element in order to identify changes after a new data join.
The first option, as you suggested, is to use data attributes. This SO Q&A describes that approach. Things to consider:
all your data values will get coerced to strings
you'll need a separate method call/attribute for each aspect of the data
you're manipulating the DOM, so it could slow things down if you've got a lot of elements or lot of data for each
the data is now part of the DOM, so can be saved with the image or accessed by other scripts
The second option is to store the data as a Javascript property of the DOM object for the element, in the same way that d3 stores the active data as the __data__ property. I've discussed this method in this forum post.
The general approach:
selection = selection.property(" __oldData__", function(d){ return d; } );
//store the old data as a property of the node
.data(newData, dataKeyFunction);
//over-write the default data property with new data
//and store the new data-joined selection in your variable
selection.enter() /*etc*/;
selection.attr("fill", function(d) {
// Within any d3 callback function,
// you can now compare `d` (the new data object)
// with `this.__oldData__` (the old data object).
// Just remember to check whether `this.__oldData__` exists
// to account for the just-entered elements.
if (this.__oldData__) { //old data exists
var dif = d.value - this.__oldData__.value;
return (dif) ? //is dif non-zero?
( (dif > 0)? "blue" : "red" ) :
"black" ;
} else {
return "green"; //value for new data
}
});
selection.property("__oldData__", null);
//delete the old data once it's no longer needed
//(not required, but a good idea if it's using up a lot of memory)
You can of course use any name for the old data property, it's just convention to throw a lot of "_" characters around it to avoid messing up any of the browser's native DOM properties.
As of D3 v4 you can use the built-in support for local variables. The internal implementation is basically the same as suggested by AmeliaBR's answer, but it frees you from having to do the storing of old data on your own. When using d3.local() you can set a value scoped to a specific DOM node, hence the name local variable. In below snippet this is done for each circle by the line
.each(function(d) { previousData.set(this, d) }); // Store previous data locally...
You can later on retrieve that value for any particular node it was stored upon:
.attr("fill", function(d) {
var diff = previousData.get(this) - d; // Retrieve previously stored data.
return diff < 0 ? "red" : diff > 0 ? "blue" : "black";
})
This full code might look something like this:
var old_data = [1, 2, 3]; // When the data gets updated I'd like to 'remember' these values
// Create a local variable for storing previous data.
var previousData = d3.local();
var svg = d3.select("body").append("svg")
.attr("width", 500)
.attr("height", 200);
var p = d3.select("body")
.append("p")
.text("Old data. Click on the circles to update the data.");
var circle = svg.selectAll("circle")
.data(old_data)
.enter().append("circle")
.attr("fill", "black")
.attr("r", function(d) { return d * 10; })
.attr("cx", function(d){ return d * 40; })
.attr("cy", function(d){ return d * 40; })
.each(function(d) { previousData.set(this, d) }); // Store previous data locally on each node
svg.on("click", function(d) {
p.text("Updated data.");
var new_data = [2, 2, 2]; // This is the new data I'd like to compare with the old
circle.data(new_data)
.transition().duration(2000)
.attr("fill", function(d) {
var diff = previousData.get(this) - d; // Retrieve previously stored data.
return diff < 0 ? "red" : diff > 0 ? "blue" : "black";
})
.attr("r", function(d) { return d * 10; });
});
<script src="https://d3js.org/d3.v4.js"></script>
I'm working on a simple d3 example where I use d3 to place some new divs on a page, add attributes, and add data-driven styles. The part that is tripping me up is when I want to use d3 to update some styles using new data. I've pasted the code from a jsFiddle ( http://jsfiddle.net/MzPUg/15/ ) below.
In the step that originally creates the divs, I use a key function to add indexes to the elements and in the update step (the part that isn't working) I also use a key function. But what isn't clear from the d3 documentation is how the actual data join works (e.g. where are indexes stored in the DOM elements? what if there are duplicate indexes?, etc.).
So, there are obvious gaps in my knowledge, but keeping it simple here can anyone shed light on why this example is not working? Any additional info on the precise nature of data joins in d3 would be frosting on the cake. (I've already seen http://bost.ocks.org/mike/join/.)
//add a container div to the body and add a class
var thediv = d3.select("body").append("div").attr("class","bluediv");
//add six medium-sized divs to the container div
//note that a key index function is provided to the data method here
//where do the resulting index value get stored?
var mediumdivs = thediv.selectAll("div")
.data([10,50,90,130,170,210],function(d){return d})
.enter().append("div")
.style("top",function(d){return d + "px"})
.style("left",function(d){return d + "px"})
.attr("class","meddiv")
//UPDATE STEP - NOT WORKING
//Attempt to update the position of two divs
var newdata = [{newval:30,oldval:10},{newval:80,oldval:50}]
var mediumUpdate = mediumdivs.data(newdata,function(d){return d.oldval})
.style("left",function(d){return d.newval + "px"})
As far as I know, you do not update the elements that already exist. Instead, you tell D3 which elements to draw and it determines what to remove or update on the screen.
I updated your JSFiddle with working code. I have also added the code below.
//add a container div to the body and add a class
var thediv = d3.select("body").append("div").attr("class", "bluediv");
function update(data) {
var mediumdivs = thediv.selectAll("div").data(data, function(d) {
return d;
});
// Tell D3 to add a div for each data point.
mediumdivs.enter().append("div").style("top", function(d) {
return d + "px";
}).style("left", function(d) {
return d + "px";
}).attr("class", "meddiv")
// Add an id element to allow you to find this div outside of D3.
.attr("id", function(d) {
return d;
});
// Tell D3 to remove all divs that no longer point to existing data.
mediumdivs.exit().remove();
}
// Draw the scene for the initial data array at the top.
update([10, 50, 90, 130, 170, 210]);
// Draw the scene with the updated array.
update([30, 80, 90, 130, 170, 210]);
I am not sure of D3's inner workings of how it stores indexes, but you can add an id attribute to the divs you create to create unique indexes for yourself.
In the above answer an update step is needed for transition of divs with the same key. illustrative jsfiddle showing what happens with/without update function.
Update function is just selection.stuff, rather than selection.enter().stuff :
//add a container div to the body and add a class
var updateDiv = d3.select("#updateDiv").attr("class", "bluediv");
var noUpdateDiv = d3.select("#noUpdateDiv").attr("class", "bluediv");
function update(selection,data,zeroTop,withUpdate) {
//add six medium-sized divs to the container div
//note that a key index function is provided to the data method here
//where do the resulting index value get stored?
var mediumdivs = selection.selectAll("div").data(data, function(d) {
return d;
});
if(withUpdate){
mediumdivs.style("top", function(d) {
if(zeroTop){
return 0
}else{
return d + "px";
}
}).style("left", function(d) {
return d + "px";
}).attr("class", "meddiv");
}
mediumdivs.enter().append("div").style("top", function(d) {
if(zeroTop){
return 0
}else{
return d + "px";
}
}).style("left", function(d) {
return d + "px";
}).attr("class", "meddiv");
mediumdivs.exit().remove();
}
//with the update function we maintain 3 of the old divs, and move them to the top
update(updateDiv,[10, 50, 90, 130, 170, 210],false,true);
update(updateDiv,[10,50,90],true,true);
//without the update function divs are maintained, but not transitioned
update(noUpdateDiv,[10, 50, 90, 130, 170, 210],false,false);
update(noUpdateDiv,[10,50,90],true,false);
The other answers given so far use the strategy of removing and recreating divs. This isn't necessary. The problem with Al R.'s original code was just in the way it used the data key. The same data key function is used both for the old data and for the data that's newly passed in. Since in Al R.'s example, the old data was a simple array of numbers, and the new data was an array of objects with properties, no data was selected in the mediumUpdate line.
Here's one way to make the selection work:
var newdata = [10, 50];
var newdatamap = {10:30, 50:80};
var mediumUpdate = mediumdivs.data(newdata, function(d){return d;})
.style("left",function(d){return newdatamap[d] + "px";});
Here's a jsfiddle, which also changes the color of the selected divs to make the effect obvious.