I'm currently rendering a US map along with every district's border. The grabbing the features of the topojson, we have an array of ~13,000 rows.
I'm also joining data to the topojson file, and running through a csv of ~180,000 rows. I believe I've optimized this process of joining data by ID enough using memoization, where the CSV is essentially turned into a hash, and each ID is the key to it's row data.
This process^ is run 24 times in Next JS through SSG to further the user experience, and so all 24 versions of this map is calculated before the first visit of this deployment. I'm sadly timing out during deployment phase for this specific web page^.
I've inspected the program and seem to find that painting/filling each district may be what's causing the slowdown. Are there any tips you all use to optimize rendering an SVG map of many path elements? Currently the attributes to this map:
1: tooltip for each district styled in tailwind
2: around 5 properties turned to text from topojson file w/ joined data to display upon hover, displayed by tooltip
3: filled dynamically with this snippet which runs a function based on the district's property type
.attr('fill', function (d) {return figureOutColor(d['properties'].type);})
4: adding a mouseover, mousemove, and mouseout event handler to each district.
Edit: Code snippet of adding attrs to my map
export const drawMap = (svgRef: SVGSVGElement, allDistricts: any[]) => {
const svg = d3.select(svgRef);
svg.selectAll('*').remove();
const projection = d3.geoAlbersUsa().scale(900).translate([400, 255]);
const path = d3.geoPath().projection(projection);
const tooltip = d3
.select('body')
.append('div')
.attr(
'class',
'absolute z-10 invisible bg-white',
);
svg
.selectAll('.district')
.data(allDistricts)
.enter()
.append('path')
.attr('class', 'district stroke-current stroke-0.5')
.attr('transform', 'translate(0' + margin.left + ',' + margin.top + ')')
.attr('d', path)
.attr('fill', function (d) {
return figureOutColor(d['properties'].type);
})
.on('mouseover', function (d) {
return tooltip
.style('visibility', 'visible')
.text(`${d.properties.name});
})
.on('mousemove', function (data) {
return tooltip.style('top', d3.event.pageY - 40 + 'px').style('left', d3.event.pageX + 'px');
})
.on('mouseout', function (d) {
d3.select(this).classed('selected fill-current text-white', false);
return tooltip.style('visibility', 'hidden');
});
I'm trying to create a cartogram using cartogram.js and d3.js. I've used the examples found in the cartogram.js repo and here to put together a script that generates a world map inside an SVG using the d3.geo.mercator() projection and now I'm trying to distort the map using the cartogram.js library however I'm getting the following error:
d3.js:8756 Error: <path> attribute d: Expected number, "MNaN,NaNLNaN,NaNL…".
(anonymous function) # d3.js:8756
tick # d3.js:8956
(anonymous function) # d3.js:8936
d3_timer_mark # d3.js:2166
d3_timer_step # d3.js:2147
Here's my code I'm using to distort the map:
var dataLoaded = new Event("dataLoaded"),
svg = d3.select("svg"),
proj = d3.geo.mercator(),
path = d3.geo.path()
.projection(proj),
countries = svg.append("g")
.attr("id", "countries")
.selectAll("path"),
carto = d3.cartogram()
.projection(proj)
.properties(function(d) {
return d.properties
}),
mapData = d3.map(),
geometries,
topology
function init() {
d3.csv("data/data.csv", function(data) {
data.forEach(function (d) {
mapData.set(d.COUNTRY, d.VALUE)
})
})
d3.json("data/world.json", function(data) {
topology = data
geometries = topology.objects.countries
var features = carto.features(topology, geometries)
countries = countries
.data(features)
.enter()
.append("path")
.attr("fill", function (e) {
return "#000000"
})
.attr("d", path)
document.dispatchEvent(dataLoaded)
})
}
document.addEventListener("dataLoaded", function() {
$("#container").css("visibility", "visible").hide().fadeIn("fast")
$("header").css("visibility", "visible").hide().fadeIn("slow")
carto.value(function(d) {
return +mapData.get(d.properties.name)
})
countries.data(carto(topology, geometries).features)
countries.transition()
.duration(750)
.attr("d", carto.path);
})
init()
and the CSV file containing the data I want to use to distort the map:
COUNTRY,VALUE
Afghanistan,90
Albania,390
Algeria,90
Andora,110
Angola,10
Antigua,2400
Argentina,320
Armenia,40
Australia,6600
Austria,1300
Axerbaijan,0
Bahamas,1900
Bahrain,90
Bangladesh,50
Barbados,8100
Belarus,20
Belgium,260
Belize,480
Benin,0
Bhutan,170
Bolivia,90
Bosnia,70
Botswana,110
Brazil,1300
Brunei,40
Bulgaria,3600
Burkina Faso,0
Burundi,0
Cabo Verde,0
Cambodia,720
Cameroon,10
Canada,4400
Central African Republic,0
Chad,10
Chile,320
China,1600
Combodia,0
Comoros,10
Congo,20
Costa Rica,2900
Cote d'Ivoire,0
Croatia,9900
Cuba,14800
Cyprus,8100
Czech Republic,70
Denmark,320
Dijbouti,0
Dominica,0
Dominican Republic,4400
Ecuador,90
Egypt,6600
El Salvador,10
Equatorial Guinea,0
Eritrea,10
Estonia,110
Ethiopia,70
Fiji,1900
Finland,720
France,2900
Gabon,10
Gambia,2400
Georgia,70
Germany,880
Ghana,210
Greece,14800
Grenada,720
Guatemala,40
Guinea,0
Guinea - Bissau,0
Guyana,50
Haiti,90
Honduras,110
Hungary,170
Iceland,8100
India,2900
Indonesia,390
Iran,390
Iraq,140
Ireland,1900
Israel,590
Italy,9900
Jamaica,6600
Japan,3600
Jordan,480
Kazakhstan,40
Kenya,1000
Kiribati,10
Kosovo,10
Kuwait,40
Kyrgyzstan,10
Laos,70
Latvia,110
Lebanon,70
Lesotho,0
Liberia,10
Libya,30
Liechtenstein,10
Lithuania,70
Luxembourg,50
Macedonia,70
Madagascar,0
Malawi,40
Malaysia,1300
Maldives,12100
Mali,40
Malta,12100
Marshall Islands,10
Mauritania,10
Mauritius,6600
Mexico,18100
Micronesia,20
Moldova,20
Monaco,590
Mongolia,110
Montenegro,880
Morocco,4400
Mozambique,90
Myanmar,90
Namibia,210
Nauru,10
Nepal,0
Netherlands,50
New Zealand,1900
Nicaragua,50
Niger,10
Nigeria,90
North Korea,390
Norway,1600
Oman,590
Pakistan,110
Palau,50
Palestine,10
Panama,210
Papua New Guinea,40
Paraguay,10
Peru,1000
Philippines,590
Poland,880
Portugal,12100
Qatar,210
Romania,320
Russia,480
Rwanda,20
Saint Kitts and Nevis,0
Saint Lucia,90
Saint Vincent and the Grenadines,0
Samoa,90
San Marino,70
Sao Tome and Principe,10
Saudi Arabia,110
Senegal,70
Serbia,50
Seychelles,1600
Sierra Leone,20
Singapore,880
Slovakia,70
Slovenia,390
Solomon Islands,10
Somalia,70
South Africa,1900
South Korea,140
South Sudan ,0
Spain,14800
Sri Lanka,3600
Sudan,20
Suriname,10
Sweden,720
Switzerland,1300
Syria,590
Taiwan,50
Tajikistan,10
Tanzania,260
Thailand,14800
Timor-Leste,0
Togo,10
Tonga,50
Trinidad and Tobago,140
Tunisia,4400
Turkey,9900
Turkmenistan,10
Tuvalu,30
Uganda,50
Ukraine,70
United Arab Emirates,20
United Kingdom,50
United States of America,3600
Uruguay,50
Uzbekistan,30
Vanuatu,30
Vatican City,30
Venezuela,170
Vietnam,2400
Yemen,20
Zambia,90
Zimbabwe,70
I don't have any experience using d3.js prior to this project so I would appreciate any feedback/guidance you can give me.
I'm using version 3.5.17 of d3, fyi.
Thanks.
UPDATE - 9/8/2016 15:22 BST
As per #Mark's suggestion, I've implemented d3-queue, although the problem still persists. If I've done anything wrong with this implementation, however, I'd be grateful for any insight anyone can give me! :)
var svg = d3.select("svg"),
proj = d3.geo.mercator(),
path = d3.geo.path()
.projection(proj),
countries = svg.append("g")
.attr("id", "countries")
.selectAll("path"),
carto = d3.cartogram()
.projection(proj)
.properties(function(d) {
return d.properties
}),
queue = d3.queue()
.defer(csv)
.defer(json)
.awaitAll(ready),
mapData = d3.map(),
geometries,
topology
function json(callback) {
d3.json("data/world.json", function(data) {
topology = data
geometries = topology.objects.countries
var features = carto.features(topology, geometries)
countries = countries
.data(features)
.enter()
.append("path")
.attr("fill", function (e) {
return "#000000"
})
.attr("d", path)
callback()
})
}
function csv(callback) {
d3.csv("data/data.csv", function(data) {
data.forEach(function (d) {
mapData.set(d.COUNTRY, +d.VALUE)
})
callback()
})
}
function ready() {
$("#container").css("visibility", "visible").hide().fadeIn("fast")
$("header").css("visibility", "visible").hide().fadeIn("slow")
carto.value(function(d) {
if (mapData.has(d.properties.name)) {
return +mapData.get(d.properties.name)
}
})
countries.data(carto(topology, geometries).features)
countries.transition()
.duration(750)
.attr("d", carto.path);
}
UPDATE 2 - 9/8/2016 18:05 BST
Here is the latest version of the script on Plunker which can be used for testing, courtesy of #Mark: http://plnkr.co/edit/iK9EZSIfwIXjIEBHhlep?p=preview
It seems my initial error has been fixed although the resulting cartogram isn't displaying correctly.
UPDATE 3 - 10/8/2016 20:45 BST
#Mark's answer helped clarify a lot of my issues and I had a partially functioning cartogram as a result however to fix the issue detailed here, I regenerated my map file using the --stitch-poles false parameter and and after doing this I am once again receiving the following error:
d3.js:8756 Error: <path> attribute d: Expected number, "MNaN,NaNLNaN,NaNL…".
#Mark's initial fix for this error is still in place therefore I'm quite confused as to why this has resurfaced. You can see my latest code here and my new map topojson file here. Thanks again.
Okay, I'm making progress. It turns out that after fixing your .value function the reason you don't get a catrogram is the your values are too disparate. Why this throws off cartogram.js, I'm unsure, but the problem can be easily solved by introducing a scale.
With your data:
s = d3.scale.linear().range([1,100]).domain(d3.extent(data, function(d){ return +d.VALUE}));
And then in your .value accessor:
carto.value(function(d,i) {
if (mapData.has(d.properties.name)) {
return s(mapData.get(d.properties.name));
} else {
return 1;
}
});
Alas, though, all your problems aren't fixed. It seems that countries that "wrap" the projection (ie Russia and Fiji) get distorted by the paths generated by cartogram.js. Here's a fix though, discussed at length here
Regardless of that, here's what we've got so far.
Problem:
I'm attempting to create an interactive map of the US in which state, county and national boundaries are displayed. Counties are shaded based on data, and hovering over a state should highlight all counties in the state, and the state should be clickable. I want to achieve this by having a SVG with the county shapes inside of state shapes, inside of a US shape.
I can generate a county map based on a CENSUS county shape file, and I can shade the states based on data in an external CSV by prepping the file with TopoJSON command line and using the following code in D3:
<!DOCTYPE html>
<meta charset="utf-8">
<style>
path {
fill: none;
stroke-linejoin: round;
stroke-linecap: round;
}
</style>
<body>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="http://d3js.org/topojson.v1.min.js"></script>
<script>
var width = 960,
height = 600;
var path = d3.geo.path()
.projection(d3.geo.albersUsa());
var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height);
d3.json("counties_pa.json", function(error, us) {
if (error) return console.error(error);
var color = d3.scale.threshold()
.domain([1, 10, 50, 100, 500, 1000, 2000, 5000])
.range(["#fff7ec", "#fee8c8", "#fdd49e", "#fdbb84", "#fc8d59", "#ef6548", "#d7301f", "#b30000", "#7f0000"]);
svg.append('g').attr('class','counties').selectAll("path").data(topojson.feature(us, us.objects.cb_2014_us_county_20m).features).enter().append('path').attr('d',path).attr('style',function(d){return 'fill:'+color(d.properties.population / d.properties.area * 2.58999e6);});
});
</script>
This is mostly visually acceptable (except it doesn't have discrete state / national boundaries) - but is functionally inadequate. In order to apply CSS to the counties on a state hover, the counties need to be within a state shape, or grouped somehow.
What i've tried:
Using topojson-merge in the command line to merge the counties into state shapes, and then render the state shapes separately - this helps with having discrete state borders - but I haven't figured a way to nest the counties into the respective state shapes.
What i'm working out now:
Somehow combining a state TopoJSON file and a county TopoJSON file and nesting the counties in the states, then rendering with D3.
Somehow using d3 to take non-nested state and county data and just nest it on the client on the client level.
In the end I would like to learn about the most effective and quickest rendering process to achieve my desired functionality.
Thanks for your help in advance.
I took a punt on your data sources, and here is what it looks like you're trying to achieve: http://bl.ocks.org/benlyall/55bc9474e6d531a1c1fe
Basically, I have generated a TopoJSON file using the following command line:
topojson -o counties_pa.json --id-property=+GEOID -p -e POP01.txt --id-property=+STCOU -p population=+POP010210D,area=ALAND,state=+STATEFP,county=+COUNTYFP cb_2014_us_county_20m.shp cb_2014_us_state_20m.shp
Some explanation on this:
-o counties_pa.json sets the name of the output file
--id-property=+GEOID will use that property in the input file as the id of each output geometry
-p means include all properties from the input file
-e POP01.txt will pull external data in from the file POP01.txt. This file is a csv file generated from the POP01.xls spreadsheet available from http://www.census.gov/support/USACdataDownloads.html#POP
--id-property=+STCOU means that the id property from the external file (POP01.txt) is in the STCOU column. This is used to match up with matching ids in the input file (which are in the GEOID property as explained above)
-p population=+POP010210D,area=ALAND,state=+STATEFP,county=+COUNTYFP explicitly lists the properties that I want in the output file, so anything extra won't be included. POP010210D is the column name for the population as at the 2010 census, so I just used that for demonstration purposes.
cb_2014_us_county_20m.shp cb_2014_us_state_20m.shp are the two input files. One for county shapes and one for state shapes. They will each be added to the output file in seperate properties named after their filenames.
I did it this way, as you seemed to be colouring your county areas based on population density, so both population and area needed to be in the output file. The population was pulled from the POP01 spreadsheet and linked to each county based on the GEOID (which is just the state number concatentated with the county number).
I was just looking for a quick and easy way to recreate your dataset, and then add the state boundaries to it so I could post the answer. Not sure how closely this matches your original data, but it seems to work for demonstration purposes.
From that, I took your code above and updated it to:
<!DOCTYPE html>
<meta charset="utf-8">
<style>
path {
fill: none;
stroke-linejoin: round;
stroke-linecap: round;
}
path.state {
fill: none;
stroke: black;
stroke-width: .5px;
}
</style>
<body>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="http://d3js.org/topojson.v1.min.js"></script>
<script>
var width = 960,
height = 600;
var path = d3.geo.path()
.projection(d3.geo.albersUsa());
var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height);
d3.json("counties_pa.json", function(error, us) {
if (error) return console.error(error);
var color = d3.scale.threshold()
.domain([1, 10, 50, 100, 500, 1000, 2000, 5000])
.range(["#fff7ec", "#fee8c8", "#fdd49e", "#fdbb84", "#fc8d59", "#ef6548", "#d7301f", "#b30000", "#7f0000"]);
svg.append('g')
.attr('class','counties')
.selectAll("path")
.data(topojson.feature(us, us.objects.cb_2014_us_county_20m).features).enter()
.append('path')
.attr('d', path)
.attr("id", function(d) { return "county-" + d.id; })
.attr("data-state", function(d) { return d.properties.state; })
.attr('style',function(d) {
return 'fill:'+color(d.properties.population / d.properties.area * 2.58999e6);
})
.on("mouseover", hoverCounty)
.on("mouseout", outCounty);
svg.append('g')
.attr('class', 'states')
.selectAll("path")
.data(topojson.feature(us, us.objects.cb_2014_us_state_20m).features).enter()
.append("path")
.attr("class", "state")
.attr("id", function(d) { return "state-" + d.id; })
.attr("d", path);
});
function hoverCounty(county) {
d3.selectAll("path[data-state='" + county.properties.state + "']").style("opacity", .5);
}
function outCounty(county) {
d3.select(".counties").selectAll("path").style("opacity", null);
}
</script>
The new and interesting bits of code are:
Add a data-state attribute to each county to determine which state it belongs to:
.attr("data-state", function(d) { return d.properties.state; })
Add the state boundaries (I combined states to the TopoJSON file in the topojson command line)
svg.append('g')
.attr('class', 'states')
.selectAll("path")
.data(topojson.feature(us, us.objects.cb_2014_us_state_20m).features).enter()
.append("path")
.attr("class", "state")
.attr("id", function(d) { return "state-" + d.id; })
.attr("d", path);
});
Added hover handlers so you can see how I'm determining the grouping of counties into states:
function hoverCounty(county) {
d3.selectAll("path[data-state='" + county.properties.state + "']").style("opacity", .5);
}
function outCounty(county) {
d3.select(".counties").selectAll("path").style("opacity", null);
}
Tied these hover handlers to each county so they get executed at the appropriate times:
.on("mouseover", hoverCounty)
.on("mouseout", outCounty);
I've put together a choropleth map using d3, helped by examples written by Mike Bostock. I'm new to d3 (and HTML, JavaScript, CSS to be honest).
I've got as far as creating the map and the legend, and being able to switch between different data sets. The map and source code can be viewed here on bl.ocks.org
Glasgow Index of Deprivation 2012
The problem I'm having now is working out how to replace the map and legend content when switching between the different datasets. As you can see at the moment, when a different dataset is selected, the content is simply added on top of the existing content.
I've tried following the advice given by Stephen Spann in this answer, and the code he provided in a working fiddle. But to no avail.
As I understand, I should add the g append to the svg variable in the beginning like so...
var svg = d3.select("body")
.append("svg")
.attr("width", width)
.attr("height", height)
.append("g");
Then select it when updating like so...
var appending = svg.selectAll("g")
.attr("class", "S12000046_geo")
.data(topojson.feature(glasgowdep, glasgowdep.objects.S12000046_geo).features);
// add new elements
appending.enter().append("path");
// update existing elements
appending.style("fill",
function (d) {
return color(choro[d.id]);
})
.style("stroke", "#cfcfcf")
.attr("d", path)
// rollover functionality to display tool tips
.on("mouseover", function (d) {
tip.show(d)
d3.select(this)
.transition().duration(200)
.style("fill", "red");
})
.on("mouseout", function () {
tip.hide()
d3.select(this)
.transition().duration(200)
.style("fill",
function (d) {
return color(choro[d.id]);
});
})
// build the map legend
var legend = d3.select('#legend')
.append('ul')
.attr('class', 'list-inline');
var keys = legend.selectAll('li.key')
.data(color.range());
var legend_items = ["Low", "", "", "", "", "", "", "", "High"];
keys.enter().append('li')
.attr('class', 'key')
.style('border-top-color', String)
.text(function (d, i) {
return legend_items[i];
});
// remove old elements
appending.exit().remove();
A solution could be the following: at your code in http://bl.ocks.org/niallmackenzie/8a763afd14e195154e63 try adding the following line just before you build the map legend (line 220 in index.html):
d3.select('#legend').selectAll('ul').remove();
Every time you update your data, you empty first the #legend.
Thanks to the advice from Lars and the solution proposed by nipro, the following works. By adding the following code just above the section that builds the legend, the legend is emptied first before it gets updated:
d3.select('#legend')
.selectAll('ul')
.remove();
// build the map legend
var legend = d3.select('#legend')
...
And by doing the same for the main map, we can first empty the map before updating it:
d3.select("g")
.selectAll("path")
.remove();
// build the choropleth map
var appending = svg.selectAll("g")
...
The full working updated code can been seen on bl.ocks.org here.
I am trying to modify the following code found on one of the sites to display both the original and updated data in the graph. I want the updated data be in different color and still show the original data and see the change. Can anyone point me the error.
<title>d3 example</title>
<style>
.original{
fill: rgb(7, 130, 180);
}
.updated{
fill: rgb(7,200,200);
}
</style>
<script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js"></script>
<script type="text/javascript" src="http://d3js.org/d3.v2.min.js"></script>
<script type="text/javascript">
// Suppose there is currently one div with id "d3TutoGraphContainer" in the DOM
// We append a 600x300 empty SVG container in the div
var chart = d3.select("#d3TutoGraphContainer").append("svg").attr("width", "600").attr("height", "300");
// Create the bar chart which consists of ten SVG rectangles, one for each piece of data
var rects = chart.selectAll('rect').data([1 ,4, 5, 6, 24, 8, 12, 1, 1, 20])
.enter().append('rect')
.attr("stroke", "none")
//.attr("fill", "rgb(7, 130, 180)")
.attr('class','original')
.attr("x", 0)
.attr("y", function(d, i) { return 25 * i; } )
.attr("width", function(d) { return 20 * d; } )
.attr("height", "20");
// Transition on click managed by jQuery
rects.on('click', function() {
// Generate randomly a data set with 10 elements
var newData = [1,2,3,4];
//for (var i=0; i<10; i+=1) { newData.push(Math.floor(24 * Math.random())); }
var newRects = d3.selectAll('rects.original');
newRects.data(newData)
.transition().duration(2000).delay(200)
.attr("width", function(d) { return d * 20; } )
//.attr("fill", newColor);
.attr('class','updated');
});
</script>
I want to know if I can get control of the original data using d3.selectAll('rects.original')
If you select "rects.original" and bind data to it, you create a join with an Update, Exit and Enter selection. Im not sure i fully understand what you are trying to achieve, but if you want new data to be drawn independently of the old rects and data, you have to create a new selection for it:
var newRects = chart.selectAll("rect.new")
.data(newData)
.enter()
(...)
and draw it.
Beware that overlapping in SVG means that underlying Elements will not be displayed anymore.
Im not sure what you mean by "getting control over the original Data", it is still bound to the selection you bound it to. If you want to modify it, you have to modify the data, rebind it, and then apply a transition on the update selection.