Develop Reference

Construct multi column table from key,value pairs

I have the following JSON coming from a database table
[{"name":"field1","value":Jim,"tieRef":1},{"name":"field2","value":120.11,"tieRef":1},
{"name":"field3","value":AAA,"tieRef":1},{"name":"field1","value":Stacy,"tieRef":2},
{"name":"field2","value":34.10,"tieRef":2},{"name":"field3","value":BBB,"tieRef":2}]
And I need to construct an HTML table which looks like the below. The values related to the same row should be identified by the tieRef attribute.
field1 | field2 | field3
-------------------------
JIm | 120.11 | AAA
-------------------------
Stacy | 34.10 | BBB
Column names should be extracted from the "name" attribute. For extracting and summarizing the values under a single column I used the d3.js as listed below
var transformed = d3.nest()
.key(function(d) { return d.name; })
.object(data);
By using this I was able to transform the original JSON in to the following
{"field1":[
{"name":"field1","value":"Jim","tieRef":1},
{"name":"field1","value":"Stacy","tieRef":2}],
"field2":[
{"name":"field2","value":120.11,"tieRef":1},
{"name":"field2","value":34.1,"tieRef":2}],
"field3":[
{"name":"field3","value":"AAA","tieRef":1},
{"name":"field3","value":"BBB","tieRef":2}]}
But I'm not sure this the correct transformation needed to make it easier for constructing the HTML table. Please suggest a better transformation of the JSON so I can loop through and construct the table
Does anyone know how to achieve this in an elegant way?

Key on tieRef and then on name worked really well for transforming the JSON. first do the below using d3
var data = [{"name":"field1","value":"Jim","tieRef":1},
{"name":"field2","value":120.11,"tieRef":1},
{"name":"field3","value":"AAA","tieRef":1},
{"name":"field1","value":"Stacy","tieRef":2},
{"name":"field2","value":34.10,"tieRef":2},
{"name":"field3","value":"BBB","tieRef":2}];
var transform = d3.nest()
.key(function(d) { return d.tieRef; })
.key(function(d) { return d.name; })
.object(data);
This will transform the JSON in to the below.
{"1":
{"field1":
[{"name":"field1","value":"Jim","tieRef":1}],
"field2":
[{"name":"field2","value":120.11,"tieRef":1}],
"field3":
[{"name":"field3","value":"AAA","tieRef":1}]
},
"2":
{"field1":
[{"name":"field1","value":"Stacy","tieRef":2}],
"field2":
[{"name":"field2","value":34.1,"tieRef":2}],
"field3":
[{"name":"field3","value":"BBB","tieRef":2}]
}
}
This is then iterated by the row number 1,2,.. and constructed the table or further transform the JSON

Reshape data for D3 stacked bar chart

I have some csv data of the following format, that I have loaded with d3.csv():
Name, Group, Amount
Bill, A, 25
Bill, B, 35
Bill, C, 45
Joe, A, 5
Joe, B, 8
But as I understand from various examples, I need the data like this to use it in a stacked bar chart:
Name, AmountA, AmountB, AmountC
Bill, 25, 35, 45
Joe, 5, 8, NA
How can I transform my data appropriately in the js script? There is also the issue of missing data, as you can see in my example.
Thanks for any help.

Yes, you are correct that in order to use d3.stack your data needs re-shaping. You could use d3.nest to group the data by name, then construct an object for each group - but your missing data will cause issues.
Instead, I'd do the following. Parse the data:
var data = `Name,Group,Amount
Bill,A,25
Bill,B,35
Bill,C,45
Joe,A,5
Joe,B,8`;
var parsed = d3.csvParse(data);
Obtain an array of names and an array of groups:
// obtain unique names
var names = d3.nest()
.key(function(d) { return d.Name; })
.entries(parsed)
.map(function(d) { return d.key; });
// obtain unique groups
var groups = d3.nest()
.key(function(d) { return d.Group; })
.entries(parsed)
.map(function(d) { return d.key; });
(Note, this is using d3.nest to create an array of unique values. Other utility libraries such as underscore have a simpler mechanism for achieving this).
Next, iterate over each unique name and add the group value, using zero for the missing data:
var grouped = names.map(function(d) {
var item = {
name: d
};
groups.forEach(function(e) {
var itemForGroup = parsed.filter(function(f) {
return f.Group === e && f.Name === d;
});
if (itemForGroup.length) {
item[e] = Number(itemForGroup[0].Amount);
} else {
item[e] = 0;
}
})
return item;
})
This gives the data in the correct form for use with d3.stack.
Here's a codepen with the complete example:
https://codepen.io/ColinEberhardt/pen/BQbBoX
It also makes use of d3fc in order to make it easier to render the stacked series.

How do you retrieve the key values of a nested data set in D3

I've used the nest() in d3js to group a bunch of insects by Pain Index, but am having trouble accessing a property. There's a similar question on SO, but I keep getting undefined instead of the property name and value. This is the data I'm dealing with:
Array[6]
0: Object
1: Object
2: Object
3: Object
4: Object
5: Object //expanded below
key: "4"
values: Array[3]
0: Object
Insect: "Tarantula Hawk" // I'm trying to access this object inside an object
PainIndex: "4"
I thought something like this would work since d.values gives you an array organized by PainIndex, but this prints undefined in my console:
var eachPain = d3.values(data_group).map(function(d){console.log(d.values); return d.values.Insect})
I'm curious to know how to access either the Insect or PainIndex properties. Any help is very apprecitaed

values is an array. Looks like you want the first instance. Try this:
var eachPain = d3.values(data_group).map(function(d){ return d.values[0].Insect });
EDIT:
var eachPain = d3.values(data_group).map(function(d) {
return d.values.map(function(v) { return v.Insect; }).join(', ');
});

Just to add some more information, besides the one in the accepted answer:
If you want to get a specific insect, and if you know it's position in the array (as you seem to do), this is what you need:
var someInsect = data_group[i].values[j].insect;
//outer object ----^ ^---- inner object
In this variable, the index i refers to the outer object, with all insects with a given level pain. Then, inside it, the index j refers to the inner object, with a specific insect.
For instance, check this following demo (I'm using a <pre> to load the data), where I get the data and nest it the way you did. In this demo, to get the tarantula hawk, I'm using:
var someInsect = data_group[0].values[1].insect;
var data = d3.csvParse(d3.select("#data").text());
var data_group = d3.nest()
.key(d => d.painIndex)
.entries(data);
var someInsect = data_group[0].values[1].insect;
console.log(someInsect);
pre {
display: none;
}
<script src="https://d3js.org/d3.v4.min.js"></script>
<pre id="data">insect,painIndex
fire ant,4
honey wasp,1
warrios wasp,3
paper wasp,1
velvet ant,2
tarantula hawk,4
bullet ant,4</pre>
Of course, that variable will change according to the insect you want.
Another possibility is getting the pain level of any insect you want by name. In your nested array, that can be done with:
var filtered = [].concat.apply([], data_group.map(function(d) {
return d.values
})).filter(function(d) {
return d.insect === myInsect
})[0].painIndex;
Where myInsect holds the name of the insect you'll use in the filter.
Check this other demo, where we get the pain value of the tarantula hawk:
var data = d3.csvParse(d3.select("#data").text());
var data_group = d3.nest()
.key(d => d.painIndex)
.entries(data);
var myInsect = "tarantula hawk"
var filtered = [].concat.apply([], data_group.map(function(d) {
return d.values
})).filter(function(d) {
return d.insect === myInsect
})[0].painIndex;
console.log(filtered);
pre {
display: none;
}
<script src="https://d3js.org/d3.v4.min.js"></script>
<pre id="data">insect,painIndex
fire ant,4
honey wasp,1
warrios wasp,3
paper wasp,1
velvet ant,2
tarantula hawk,4
bullet ant,4</pre>
Of course, all of this (getting the key values, get the value depending on other value etc...) would be way easier if you used your original array, instead of the nested one.

ignoring a group with the D3 nest function

I have some data where, in a given column of a csv, there are six possible values:
1,2,3,4,5,NaN.
I am currently trying to group the data using the d3.nest and rollup functions. My goal is to group the data but exclude "NaN" values in the final output.
This is my current code:
var nested = d3.nest()
.key(function(d){return d[question];
})
.rollup(function(leaves){
var total = data.length
var responses = leaves.length;
return {
'responses' : responses,
'percent' : responses/total
};
})
.entries(data)
As you can see, I would like to return both a count of each of the categories as well as the percentage of the total that they represent. After removing NaN, I would also like the removal of NaN represented in percentage values of all of the other categories so that they sum to 100%.

The easiest way to do this is to remove the rows the contain NaN before passing the data to d3.nest():
var filtered = data.filter(function(d) { return d.question !== 'NaN'; });

Using D3.nest() to create an array of objects by column header

I have a CSV file with the format:
time,group1,group2,group3
0,45,30,30,
1,30,25,31,
2,50,45,30,
I want to structure it as an array of objects such that :
[{group1:array[3]}, {group2:array[3]},...]
where each array[3] is itself an array of objects that pairs
the value in the time column with the value in its respective group column
i.e. :
group1 [{time:0, value:45},{time:1, value:30},...] group2 [{time:0, value:30},...]
D3.csv parses by row and I'm not sure how to iterate through the resulting array of objects with d3.nest or if there's a way to set up the data structure within the d3.csv accessor function. (I apologize if I'm using terms improperly)

You don't have to use d3.nest() to do this. You could just loop over the parsed csv file to put the data in the format you need. For instance:
var csv = "time,group1,group2,group3\n0,45,30,30\n1,30,25,31\n2,50,45,30";
var data = d3.csv.parse(csv);
var result = [ {group1: []}, {group2: []}, {group3: []}]
data.forEach(function(d) {
result[0]['group1'].push({time: +d.time, value: +d['group1']})
result[1]['group2'].push({time: +d.time, value: +d['group2']})
result[2]['group3'].push({time: +d.time, value: +d['group3']})
});
This isn't the most flexible example (since the columns are hardcoded) but hopefully it will give you an idea of how to go about it.