I have the following JSON coming from a database table
[{"name":"field1","value":Jim,"tieRef":1},{"name":"field2","value":120.11,"tieRef":1},
{"name":"field3","value":AAA,"tieRef":1},{"name":"field1","value":Stacy,"tieRef":2},
{"name":"field2","value":34.10,"tieRef":2},{"name":"field3","value":BBB,"tieRef":2}]
And I need to construct an HTML table which looks like the below. The values related to the same row should be identified by the tieRef attribute.
field1 | field2 | field3
-------------------------
JIm | 120.11 | AAA
-------------------------
Stacy | 34.10 | BBB
Column names should be extracted from the "name" attribute. For extracting and summarizing the values under a single column I used the d3.js as listed below
var transformed = d3.nest()
.key(function(d) { return d.name; })
.object(data);
By using this I was able to transform the original JSON in to the following
{"field1":[
{"name":"field1","value":"Jim","tieRef":1},
{"name":"field1","value":"Stacy","tieRef":2}],
"field2":[
{"name":"field2","value":120.11,"tieRef":1},
{"name":"field2","value":34.1,"tieRef":2}],
"field3":[
{"name":"field3","value":"AAA","tieRef":1},
{"name":"field3","value":"BBB","tieRef":2}]}
But I'm not sure this the correct transformation needed to make it easier for constructing the HTML table. Please suggest a better transformation of the JSON so I can loop through and construct the table
Does anyone know how to achieve this in an elegant way?
Key on tieRef and then on name worked really well for transforming the JSON. first do the below using d3
var data = [{"name":"field1","value":"Jim","tieRef":1},
{"name":"field2","value":120.11,"tieRef":1},
{"name":"field3","value":"AAA","tieRef":1},
{"name":"field1","value":"Stacy","tieRef":2},
{"name":"field2","value":34.10,"tieRef":2},
{"name":"field3","value":"BBB","tieRef":2}];
var transform = d3.nest()
.key(function(d) { return d.tieRef; })
.key(function(d) { return d.name; })
.object(data);
This will transform the JSON in to the below.
{"1":
{"field1":
[{"name":"field1","value":"Jim","tieRef":1}],
"field2":
[{"name":"field2","value":120.11,"tieRef":1}],
"field3":
[{"name":"field3","value":"AAA","tieRef":1}]
},
"2":
{"field1":
[{"name":"field1","value":"Stacy","tieRef":2}],
"field2":
[{"name":"field2","value":34.1,"tieRef":2}],
"field3":
[{"name":"field3","value":"BBB","tieRef":2}]
}
}
This is then iterated by the row number 1,2,.. and constructed the table or further transform the JSON
Related
I used d3.nest() to group my data into category.
d3.nest()
.key(function(d) { return d[indexCateCol]; })
.entries(irisData);`
However, I'd like to know how can I make an array of value from the attribute that I am interested in.
(3) [Object, Object, Object]
0:Object
key:"setosa"
values:Array(50)
0:Array(5)
0:5.1
1:3.5
2:1.4
3:0.2
4:"setosa"
length:5
1:Array(5)
2:Array(5)`
In other words, what I want to produce is arrays of column that I'm interested for each Species like [5.1, 4.9, 4.7, 4.6, ...] for "Sepal.Length" or from '0: 5.1' from each array in value.
I believe that I can write a loop to get those arrays but there are any more JavaScript-ish ways to do this.
If you want to get the array of values for each species (from what I understand based on the data you're showing), you can do :
var myNest = d3.nest()
.key(function(d) { return d[indexCateCol]; })
.entries(irisData);`
/* Get the values of setosa as an array of arrays*/
var arrSetosa = myNest.setosa.values;
/* Get the 1st value of the 3rd "column" */
var val = arrSetosa[2][0];
I have some csv data of the following format, that I have loaded with d3.csv():
Name, Group, Amount
Bill, A, 25
Bill, B, 35
Bill, C, 45
Joe, A, 5
Joe, B, 8
But as I understand from various examples, I need the data like this to use it in a stacked bar chart:
Name, AmountA, AmountB, AmountC
Bill, 25, 35, 45
Joe, 5, 8, NA
How can I transform my data appropriately in the js script? There is also the issue of missing data, as you can see in my example.
Thanks for any help.
Yes, you are correct that in order to use d3.stack your data needs re-shaping. You could use d3.nest to group the data by name, then construct an object for each group - but your missing data will cause issues.
Instead, I'd do the following. Parse the data:
var data = `Name,Group,Amount
Bill,A,25
Bill,B,35
Bill,C,45
Joe,A,5
Joe,B,8`;
var parsed = d3.csvParse(data);
Obtain an array of names and an array of groups:
// obtain unique names
var names = d3.nest()
.key(function(d) { return d.Name; })
.entries(parsed)
.map(function(d) { return d.key; });
// obtain unique groups
var groups = d3.nest()
.key(function(d) { return d.Group; })
.entries(parsed)
.map(function(d) { return d.key; });
(Note, this is using d3.nest to create an array of unique values. Other utility libraries such as underscore have a simpler mechanism for achieving this).
Next, iterate over each unique name and add the group value, using zero for the missing data:
var grouped = names.map(function(d) {
var item = {
name: d
};
groups.forEach(function(e) {
var itemForGroup = parsed.filter(function(f) {
return f.Group === e && f.Name === d;
});
if (itemForGroup.length) {
item[e] = Number(itemForGroup[0].Amount);
} else {
item[e] = 0;
}
})
return item;
})
This gives the data in the correct form for use with d3.stack.
Here's a codepen with the complete example:
https://codepen.io/ColinEberhardt/pen/BQbBoX
It also makes use of d3fc in order to make it easier to render the stacked series.
I have an svg (can be viewed here) to which I would like to present some data on click. The Data is in json format and I would like to match each id of svg element with key in json data if both of these are same then do something. For example if Json key is key: china and element id is china then present information of china from json. I have already extracted the desired format of data I just cannot figure out how to match these keys with the element ids.
This is how I am accessing the dataset
var countriesByName = d3.nest()
.key(function (d) {
return d.Country_Names;
})
.entries(data);
// creating dropdown
var data = JSON.stringify(countriesByName)
var data = JSON.parse(data);
//this is the key I would like to match with element ids:
var keys = function( d ) {
return d.key;
}
From this the format of json changes to
[{"key":"Albania",
"values": [{"Continent":"Europe",
"Country_Names":"Albania",
"Total":"3.8",
"Change_total":"-38.7",
"Main activity electricity and heat production":"0.1",
"Main activity electricity plants":"",
"Main activity CHP plants":"","Unallocated autoproducers / Other energy industry own use":"0.1",
"Other":"1.4",
"Manufacturing industries and construction":"1",
"Iron and steel":"0",
"Chemical and petrochemical":"0",
"Machinery":"",
"Mining and quarrying":"",
"Food and tobacco":"0.1",
"Paper, pulp and printing":"",
"Construction":"0",
"Transport":"2.2",
"Road":"2.2",
"Domestic aviation":"",
"Rail":"0",
"Domestic navigation":"0.1",
"Residential":"0.2",
"Commercial and public services":"0",
"Agriculture/forestry":"0.2",
"Sub-bituminous coal / Lignite":"",
"Other bituminous coal":"",
"Natural gas":"0",
"Motor gasoline excl. bio":"0.3",
"Gas/diesel oil excl. bio":"2.2"}]}
fiddle: https://jsfiddle.net/n5v84svm/47/
You can get the JSON data from the remote URL, then you can filter with any key value to get the particular data object form the data source. Please check the below code. Apologies if My solution is wrong for your problem.
//Get Data from the Remote URL
var countryData=$.get("https://gist.githubusercontent.com/heenaI/cbbc5c5f49994f174376/raw/c3f7ea250a2039c9edca0b12a530f108d6304e1c/data.json");
countryData=JSON.parse(countryData.responseText);
//this is the key I would like to match with element ids:
var keys = function( d ) {
var keyData=data.filter(function(value) { return value.Country_Names == d;});
return keyData;
}
I have a CSV file with the format:
time,group1,group2,group3
0,45,30,30,
1,30,25,31,
2,50,45,30,
I want to structure it as an array of objects such that :
[{group1:array[3]}, {group2:array[3]},...]
where each array[3] is itself an array of objects that pairs
the value in the time column with the value in its respective group column
i.e. :
group1 [{time:0, value:45},{time:1, value:30},...] group2 [{time:0, value:30},...]
D3.csv parses by row and I'm not sure how to iterate through the resulting array of objects with d3.nest or if there's a way to set up the data structure within the d3.csv accessor function. (I apologize if I'm using terms improperly)
You don't have to use d3.nest() to do this. You could just loop over the parsed csv file to put the data in the format you need. For instance:
var csv = "time,group1,group2,group3\n0,45,30,30\n1,30,25,31\n2,50,45,30";
var data = d3.csv.parse(csv);
var result = [ {group1: []}, {group2: []}, {group3: []}]
data.forEach(function(d) {
result[0]['group1'].push({time: +d.time, value: +d['group1']})
result[1]['group2'].push({time: +d.time, value: +d['group2']})
result[2]['group3'].push({time: +d.time, value: +d['group3']})
});
This isn't the most flexible example (since the columns are hardcoded) but hopefully it will give you an idea of how to go about it.
I have an array of nested JSON structures where they have varying depth and not the same set of keys everywhere:
[
{
"name":"bob",
"salary":10000,
"friends":[
{
"name": "sarah",
"salary":10000
},
{
"name": "bill",
"salary":5000
}
]
},
{
"name":"marge",
"salary":10000,
"friends":[
{
"name": "rhonda",
"salary":10000
},
{
"name": "mike",
"salary":5000,
"hobbies":[
{
"name":"surfing",
"frequency":10
},
{
"name":"surfing",
"frequency":15
}
]
}
]
},
{
"name":"joe",
"salary":10000,
"friends":[
{
"name": "harry",
"salary":10000
},
{
"name": "sally",
"salary":5000
}
]
}
]
I wanted to use D3 to render this as nested html tables. For example the friends column will have tables showing the name, and salary of the friends of the individual referenced in the row. Sometimes one of these tables will have another level of a sub table.
I imagine the way to do this is by recursively creating tables. I wrote a python program that takes a JSON structure like this, and renders tables within tables, and the easiest way to do that was recursively. I see on the d3.js documentation there is a .each() thing you can call, which I am sure is what I need, I just need a little boost getting there (https://github.com/mbostock/d3/wiki/Selections#wiki-each).
So is there a nice way to do this in D3? I found this great example for rendering a 2d matrix of data as a table Creating a table linked to a csv file. With that tutorial I was able to get the outer most level of this data-structure rendered as a table, but I am stuck on how to go into levels recursively as needed, as of now they just show up as "Object" in the table since I am not treating them differently from normal strings and numbers.
Also I found this other question/answer that is similar to my question, but I really don't understand javascript well enough to see where/how the recursion is happening and readapt the solution to fit my needs: How do I process data that is nested multiple levels in D3?. Any advice or pointers to tutorials on recursively or iteratively processing nested tree like JSON data-structures in D3 would be much appreciated!
A recursive function would probably be good approach. See code below for one possible implementation (assuming your data is stored in jdata). See the comments in the code for some explanation and see this Gist for a live version: http://bl.ocks.org/4085017
d3.select("body").selectAll("table")
.data([jdata])
.enter().append("table")
.call(recurse);
function recurse(sel) {
// sel is a d3.selection of one or more empty tables
sel.each(function(d) {
// d is an array of objects
var colnames,
tds,
table = d3.select(this);
// obtain column names by gathering unique key names in all 1st level objects
// following method emulates a set by using the keys of a d3.map()
colnames = d // array of objects
.reduce(function(p,c) { return p.concat(d3.keys(c)); }, []) // array with all keynames
.reduce(function(p,c) { return (p.set(c,0), p); }, d3.map()) // map with unique keynames as keys
.keys(); // array with unique keynames (arb. order)
// colnames array is in arbitrary order
// sort colnames here if required
// create header row using standard 1D data join and enter()
table.append("thead").append("tr").selectAll("th")
.data(colnames)
.enter().append("th")
.text(function(d) { return d; });
// create the table cells by using nested 2D data join and enter()
// see also http://bost.ocks.org/mike/nest/
tds = table.append("tbody").selectAll("tr")
.data(d) // each row gets one object
.enter().append("tr").selectAll("td")
.data(function(d) { // each cell gets one value
return colnames.map(function(k) { // for each colname (i.e. key) find the corresponding value
return d[k] || ""; // use empty string if key doesn't exist for that object
});
})
.enter().append("td");
// cell contents depends on the data bound to the cell
// fill with text if data is not an Array
tds.filter(function(d) { return !(d instanceof Array); })
.text(function(d) { return d; });
// fill with a new table if data is an Array
tds.filter(function(d) { return (d instanceof Array); })
.append("table")
.call(recurse);
});
}