I'd like to visualize data pulled in from the census data API, specifically from the ACS survey.
The data is returned in a non-standard version of JSON, as a two-dimensional array. It basically looks like this:
[
[
“POPULATION”,
“DATE”,
“ANOTHERTHING”
],
[
“ALABAMA”,
“2000”,
“MORESTUFF”
],
[
“ALASKA”,
“2000”,
“OTHERSTUFF”
],
…
]
I'm unfamiliar with working with this kind of JSON data, which almost looks more like a CSV, where the keys are written in the first line, and the values in every line after the first.
Is anyone familiar with how to parse and work with this data in D3, without having to go convert it first (i.e. https://gist.github.com/sarfarazansari/7e8ae05168b80b36016eb1c561a82f73)? (I'd like to draw from the data API directly).
Any help or guidance would be much appreciated.
First of all: do not use smart quotes (“) in a JSON (or in your code, whatever it is). I reckon that your real JSON is correct, this just happened because you used a text editor, like MS Word, to write this question. Also, there is no such a thing like non-standard JSON, because there is no standard JSON to begin with. You just have a JSON which is an array of arrays, nothing special here.
That being said, you can use that exact data structure to create your charts... However, specially if you're a beginner, it's a good idea to stick with the most common (here we can say standard) way to organise the data in D3 codes, which is an array of objects.
We can easily convert that array of arrays in an array of objects, which will be more comfortable to you.
First, supposing that your array is named data, let's extract the headers:
var keys = data.shift();
That not only creates a headers array, but it will remove the first inner array from the data array.
Now, we iterate over each inner array, creating an object with all the key/value pairs we need:
var newData = data.map(function(d) {
var obj = {};
d.forEach(function(e, i) {
obj[keys[i]] = e;
});
return obj;
});
There are shorter ways to do this, the one above is quite verbose but it is more didactic.
Here is a demo using the data you shared:
var data = [
[
"POPULATION",
"DATE",
"ANOTHERTHING"
],
[
"ALABAMA",
"2000",
"MORESTUFF"
],
[
"ALASKA",
"2000",
"OTHERSTUFF"
]
];
var keys = data.shift();
var newData = data.map(function(d) {
var obj = {};
d.forEach(function(e, i) {
obj[keys[i]] = e;
});
return obj;
});
console.log(newData)
<script src="https://d3js.org/d3.v5.min.js"></script>
I make a chart by using c3js library. Normally when I click legend, its only shows legend data.
Now, I want to add new Simple Moving Average data without remove existing data when I click a legend.
Here is the sample image of what I want,
If I click dept1 legend, I want to add new SMV value to chart.
Here is what I tried,
function show_svm(data) {
chart.load({
bindto: '#chart',
x: 'date',
xFormat: m_d_format,
json: data,
keys: {
x: 'date',
value: ['Dept1','Dept2','Dept3', 'Dept4','SMV']
}
});
}
In the function argument data is like this,
{
'date':["2015-11-23","2015-11-24","2015-11-25","2015-11-26","2015-11-27","2015-11-30","2015-12-01","2015-12-02","2015-12-03","2015-12-04","2015-12-08","2015-12-09","2015-12-11","2015-12-14","2015-12-15","2015-12-16","2015-12-17","2015-12-18","2015-12-19","2015-12-21","2015-12-22","2015-12-23","2015-12-24","2015-12-25","2015-12-28","2015-12-29","2015-12-30","2016-01-04","2016-01-05","2016-01-06","2016-01-07","2016-01-08","2016-01-09","2016-01-11","2016-01-12","2016-01-13","2016-01-14","2016-01-15","2016-01-18","2016-01-19","2016-01-20","2016-01-21","2016-01-22","2016-01-23","2016-01-24","2016-01-25","2016-01-26","2016-01-27","2016-01-28","2016-01-29","2016-02-01","2016-02-02","2016-02-03","2016-02-04","2016-02-05","2016-02-08"],
'Dept1':["100.00","50.00","57.14","71.43","100.00","42.86","85.71","85.71","71.43","66.67","33.33","100.00","50.00","57.14","66.67","71.43","100.00","57.14","0.00","28.57","57.14","66.67","57.14","57.14","71.43","57.14","57.14","71.43","71.43","71.43","85.71","83.33","100.00","42.86","57.14","57.14","66.67","100.00","71.43","100.00","71.43","14.29","28.57","100.00","50.00","28.57","57.14","57.14","57.14","100.00","42.86","57.14","71.43","71.43","42.86","71.43"],
'SMV':[null,null,null,null,null,null,null,70.407142857143,73.468571428571,74.83,69.387142857143,69.387142857143,70.407142857143,66.325714285714,63.605714285714,63.605714285714,68.367142857143,71.768571428571,57.482857142857,54.421428571429,54.421428571429,54.421428571429,52.38,46.257142857143,48.298571428571,56.461428571429,60.542857142857,62.584285714286,63.264285714286,65.305714285714,69.387142857143,71.087142857143,77.21,75.17,73.128571428571,71.087142857143,70.407142857143,72.448571428571,70.748571428571,70.748571428571,74.83,68.708571428571,64.627142857143,69.388571428571,62.245714285714,56.122857142857,50,47.958571428571,54.08,64.284285714286,56.121428571429,57.141428571429,63.264285714286,65.305714285714,63.265714285714,65.307142857143]
}
This data are came from ajax.
But, this SVM data cannot render on chart and I don't get any error. I don't know why.
The another thing I want to know is, can I use javascript object to json of c3js chart like this: json:data,. Is it correct way or not?. I'm very appreciate for any suggestion.
You have just a few mistakes in your code. First, you used the attribute json. For using that attribute, your data needs to be in the following form:
var data = [
{ 'date': '2015-01-01', 'Dept1': null, 'SMV': 3},
{ 'date': '2015-01-02', 'Dept1': 1, 'SMV': 4}
];
Your data however looks different. You separated all the date, the Dept1 and the SMV data into separate arrays. The way to go here is the following: The data should contain one array of several arrays. The first argument is defined as the key of the dataset. An example dataset looks then like this:
var data = [
['date', '2015-01-01', 2015-01-02'],
['Dept1', null, 1],
['SMV', 3, 4]
];
You then also do not need the keys attribute anymore.
Lastly, you forgot to add the c3 and d3 dependencies in your jsFiddle, what led to the error c3 is not defined. You can find a fixed example of your jsFiddle here: jsFiddle.
I have a JSON file with multiple columns, and would like to keep the display of any of those as a map relatively open, based on the choice of the user. Now, I can't see any examples of using other column names for display in Highmaps than "value". Is it so? Or is there any way to name within the Javascript code the column that should be used by Highmaps for the display of the data?
That is, the column names are (for example) these:
LC_Ctry_Limit,LC_Capita_Limit,LC_Ctry_Footprint,LC_Capita_Footprint,LC_Score
Now, in order to be able to display the data from LC_Score, I'd need to rename the column (hard coded into the JSON-file) into "value". Thus:
LC_Ctry_Limit,LC_Capita_Limit,LC_Ctry_Footprint,LC_Capita_Footprint,value
But what is, if the user wants to display instead another column? Do I need to create five different JSON files, for each column, and name it "value" then? That seems rather inflexible to me.
My Highmaps-code looks like this, based on loading first a GeoJSON and then a JSON file, and I guess there, it should be something about a column-name-specifier:
series : [
{
data : data,
mapData: geojson,
joinBy: ['Country_40','Country'],
name: 'LC_Score',
borderWidth: 0.2,
COLUMN-TO-BE-USED: lorem
}]
Thanks for any hints!
You could set values of value property before setting the data in chart's options/configuration. This selection can be based on value passed into function that will return prepared array for configuration.
Example: http://jsfiddle.net/bmv0y8dn/
var columnToBeUsed = 'lorem';
$('#container').highcharts('Map', {
series: [{
data: (function (columnToBeUsed) {
var len = data.length,
tab = [];
for (var i = 0; i < len; i++) {
data[i].value = data[i][columnToBeUsed];
}
return data;
})(columnToBeUsed),
...
Series data can be changed dynamically using Series.setData, so you could change value's value and call setData() with new data for dynamic data change.
I'm working on a visualization utilizing the crossfilter.js library, and I'm a bit confused on how to create some dimensions from nested attributes within my dataset. For instance, each instance of the dataset has multiple dates associated with it, resulting in a data structure that looks like this:
[ { name: 'instance_name',
dates: ['2014-11-11', '2013-07-06', '2011-02-04'],
category: 'category 1' },
{ name: 'instance_name2',
dates: ['2012-01-01', '2013-03-07'],
category: 'category 2' } ]
I'd like to be able to create dimensions that will allow for filtering based on, say, the dates and the category and dimensions are a straightforward way to do this with crossfilter. However, I'm not sure how to parse the dates. I've tried first creating a date dimension using something like:
var cf = crossfilter(data);
var dateDim = cf.dimension(function(d) { return d.dates; });
and then tried to store just the dates as a variable using the .map() method like so:
var date = dateDim.top(Infinity).map(function(d) { return d.dates; });
The above does retrieve just the dates and stores them as a variable, however (a) this is just an array of dates each of which is a string, and (b) this doesn't get me any closer to linking the dateDim to other crossfilter dimensions I'd like to create for the visualization. Any help would be greatly appreciated. Thanks for reading.
My recommendation is be to flatten your structure before loading it in to Crossfilter. So your 1st record will become 3 records (1 for each date) and your 2nd record will become 2 records. You can then parse the dates and treat them as a dimension in Crossfilter without too much trouble. The downside is that counting because a problem, but that is manageable with custom grouping functions.
I have an array declared as so:
$scope.test=['blah', 'blah2'];
And I wanted to use ng-grid as a test to merely display the data. I did the following:
$scope.sourceGridOptionsApprovers = {
plugins: [gridLayoutPlugin],
data : 'test',
columnDefs: [
{field: 'test', displayName: 'Approvers', width:'35%',
cellFilter: 'stringArrayFilter'}
]
};
I wanted to filter the array so I could display the contents of test into one of my columns in ng-grid. I appended the following to my angular controller:
.filter('stringArrayFilter', function(){
return function(myArray) {
//console.log(myArray);
return myArray.join(', ');
};
});
But when I try to display the contents of the array after I filter it, everything shows up as undefined, and I'm thinking it may be the way I passed in the array into the filter, but I'm not entirely sure. Any help would be appreciated.
Here is a plunkr of my example: http://plnkr.co/edit/Ba7hoGFhI7cWWaD3itZx?p=preview
First of all i think ngGrid strictly requires a json object with properties to work on. Without properties the cell cannot be matched to specific data so the data for the cell is 'undefined' => which results in 'undefined' error messages.
So your data need to be like:
$scope.test=[{name: 'blah'}, {name: 'blah2'}];
Secondly the cellfilter is applied per row/cell and not over your whole array. With your example data the single cell data contains no arrays so you will get 'no method join' error messages.
The data should be structured like that with the use of your filter:
$scope.test = [{name: ['blah', 'blah2']}, {name: ['blah3', 'blah4']}];
Here is a full working example:
http://plnkr.co/edit/6EsriMwBNILKzwk3JIR1?p=preview