I have built a custom D3 chart that used an old REST api, and I am looking for help on how to adapt the D3 input to the new one.
At http://codepen.io/anon/pen/bEgMdV I use the dataset 'dataset' and I have tried to use the 'data' which has 'genes' and 'dataset' in one.
So far, I have tried changing
var gs = svg.selectAll("g")
.data(d3.values(dataset)).enter().append("g");
gs.selectAll("path").data(function(d) {
return pie(d.data);
to
var gs = svg.selectAll("g")
.data(d3.values(data.values)).enter().append("g");
gs.selectAll("path").data(function(d) {
return pie(d[0]);
In order to get the numbers of the array in 'values'.
I feel I am somewhat close to getting this. Can someone help?
One of the ways is to convert the new data into the old data structure, so that you don't need to change the code.
This for loop will change you new data to the expected old data structure.
genes = [];
dataset = [{data:[], gene:"LPAR"}];
data.values.forEach(function(s){
genes.push(s[0]);//iterate and add the gene
dataset[0].data.push(s[1])//iterate and add the values.
});
Working code here
Hope this helps!
Related
I'm stuck with a quite simple problem and need help.
I have a big CSV file with 50 columns which i absolutely can't modifie.
Now i want to make a chart where i only need 5-6 columns out of it.
My idea was now to make a new "data2" which contains only these 5-6 columns (with key and evertything) and work with this data2.
But i'm not able to create this data2.
To filter which columns i need i wanted to work with regex. Something like this:
d3.keys(data[0]).filter(function(d) { return d.match(/.../); })
But how do i create the new data2 then? I'm sure i need to work with d3.map but even with the api i'm not able to understand how it works correctly.
Can someone help me out?
Firstly, your question's title is misleading: you're not asking about making a smaller CSV file, since the file itself is not changed. You're asking about changing the data array created by D3 when that CSV was parsed.
That brings us to the second point: you don't need to do that. Since you already lost some time/resources loading the CSV and parsing that CSV, the best idea is just keeping it the way it is, and using only those 5 columns you want. If you try to filter some columns out (which means deleting some properties from each object in the array) you will only add more unnecessary tasks for the browser to execute. A way better idea is changing the CSV itself.
However, if you really want to do this, you can use the array property that d3.csv creates when it loads an CSV, called columns, and a for...in loop to delete some properties from each object.
For instance, here...
var myColumns = data.columns.splice(0, 4);
... I'm getting the first 4 columns in the CSV. Then, I use this array to delete, in each object, the properties regarding all other columns:
var filteredData = data.map(function(d) {
for (var key in d) {
if (myColumns.indexOf(key) === -1) delete d[key];
}
return d;
})
Here is a demo. I'm using a <pre> element because I cannot use a real CSV in the Stack snippet. My "CSV" has 12 columns, but my filtered array keeps only the first 4:
var data = d3.csvParse(d3.select("#csv").text());
var myColumns = data.columns.splice(0, 4);
var filteredData = data.map(function(d) {
for (var key in d) {
if (myColumns.indexOf(key) === -1) delete d[key];
}
return d;
})
console.log(filteredData)
pre {
display: none;
}
<script src="https://d3js.org/d3.v4.min.js"></script>
<pre id="csv">foo,bar,baz,foofoo,foobar,foobaz,barfoo,barbar,barbaz,bazfoo,bazbar,bazbaz
1,2,5,4,3,5,6,5,7,3,4,3
3,4,2,8,7,6,5,6,4,3,5,4
8,7,9,6,5,6,4,3,4,2,9,8</pre>
I'm looking for some advice on how to effectively use large amounts of data with d3.js. Lets say for instance, I have this data set taken from a raw .csv file (converted from excel);
EA
,Jan_2016,Feb_2016,Mar_2016
Netherlands,11.7999,15.0526,13.2411
Belgium,25.7713,24.1374
France,27.6033,23.6186,20.2142
EB
,Jan_2016,Feb_2016,Mar_2016
Netherlands,1.9024,2.9456,4.0728
Belgium,-,6.5699,7.8894
France,5.3284,4.8213,1.471
EC
,Jan_2016,Feb_2016,Mar_2016
Netherlands,3.1499,3.1139,3.3284
Belgium,3.0781,4.8349,5.1596
France,16.3458,12.6975,11.6196
Using csv I guess the best way to represent this data would be something like;
Org,Country,Month,Score
EA,Netherlands,Jan,11.7999
EA,Belgium,Jan,27.6033
EA,France,Jan,20.2142
EA,Netherlands,Feb,15.0526
EA,Belgium,Feb,25.9374
EA,France,Feb,23.6186
EA,Netherlands,Mar,13.2411
EA,Belgium,Mar,24.1374
EA,France,Mar,20.2142
This seems very long winded to me, and would use up a lot of time. I was wondering if there was an easier way to do this?
From what I can think of, I assume that JSON may be the more logical choice?
And for context of what kind of chart this data would go into, I would be looking to create a pie chart which can update the data depending on the country/month selected and comparing the three organisations scores each time.
(plnk to visualise)
http://plnkr.co/edit/P3loEGu4jMRpsvTOgCMM?p=preview
Thanks for any advice, I'm a bit lost here.
I would say the intermediary step you propose is a good one for keeping everything organized in memory. You don't have to go through a csv file though, you can just load your original csv file and turn it into an array of objects. Here is a parser:
d3.text("data.csv", function(error, dataTxt) { //import data file as text first
var dataCsv=d3.csv.parseRows(dataTxt); //parseRows gives a 2D array
var group=""; // the current group header ("organization")
var times=[]; //the current month headers
var data=[]; //the final data object, will be filled up progressively
for (var i=0;i<dataCsv.length;i++) {
if (dataCsv[i].length==1 ) { //group name
if ( dataCsv[i][0] == "")
i++; //remove empty line
group = dataCsv[i][0]; //get group name
i++;
times = dataCsv[i];//get list of time headings for this group
times.shift(); // (shift out first empty element)
} else {
country=dataCsv[i].shift(); //regular row: get country name
dataCsv[i].forEach(function(x,j){ //enumerate values
data.push({ //create new data item
Org: group,
Country: country,
Month: times[j],
Score: x
})
})
}
}
This gives the following data array:
data= [{"Org":"EA","Country":"Netherlands","Month":"Jan_2016","Score":"11.7999"},
{"Org":"EA","Country":"Netherlands","Month":"Feb_2016","Score":"15.0526"}, ...]
This is IMO the most versatile structure you can have. Not the best for memory usage though.
A simple way to nest this is the following:
d3.nest()
.key(function(d) { return d.Month+"-"+d.Country; })
.map(data);
It will give a map with key-values such as:
"Jan_2016-Netherlands":[{"Org":"EA","Country":"Netherlands","Month":"Jan_2016","Score":"11.7999"},{"Org":"EB","Country":"Netherlands","Month":"Jan_2016","Score":"1.9024"},{"Org":"EC","Country":"Netherlands","Month":"Jan_2016","Score":"3.1499"}]
Use entries instead of mapto have an array instead of a map, and use a rollup function if you want to simplify the data by keeping only the array of scores. At this point it is rather straightforward to plug it into any d3 drawing tool.
PS: a Plunker with the running code of this script. Everything is shown in the console.
I'm creating a flot graph using some php-script. The php generates the data and uses json_encodeto pass this data to some javascript-flot code where I parse using jQuery.parseJson.
I was using the data-array filled with (x,y) values. Plotting this doesn't seem to work. If I encapsulate the array within an object flot is plotting it without problems. Why doesn't the first method work? I've added a jsFiddle below.
var data = '[["201518","1"],["201519","3"],["201520","6"]]',
data2 = '{"data":[["201518","1"],["201519","3"],["201520","6"]]}';
var set = jQuery.parseJSON(data),
set2 = jQuery.parseJSON(data2);
var placeholder = $('#placeholder');
$.plot(placeholder, [set2.data]);
//$.plot(placeholder, set); <= not working? Why?
jsfiddle
You need to pass an array:
$.plot(placeholder, [set])
// instead of `$.plot(placeholder, set)`
Two problems. First you need numbers and not strings when passing as an array (see here where it says
Note that to simplify the internal logic in Flot both the x and y
values must be numbers (even if specifying time series, see below for
how to do this). This is a common problem because you might retrieve
data from the database and serialize them directly to JSON without
noticing the wrong type. If you're getting mysterious errors, double
check that you're inputting numbers and not strings.
Second (as pointed out in another answer), you need the [array] around set. The following works:
$(document).ready(function () {
var data = '[[201518,1], [201519,3], [201520,6]]',
data2 = '{"data":[["201518","1"],["201519","3"],["201520","6"]]}';
var set = jQuery.parseJSON(data),
set2 = jQuery.parseJSON(data2);
var placeholder = $('#placeholder');
//$.plot(placeholder, [set2.data]);
$.plot(placeholder, [set]);
});
Im using knockoutJS in following way:
var messagesFromServer = getJSONData().messages; //this get msgs from server
ko.mapping.fromJS(messagesFromServer, {}, myobject.viewModel.Messages);
Then i am basically calling this every three seconds to update html table, and it works just fine, new rows are added if new data found from server. Now i would like to add custom callback when something has actually changed, for example when new messages are found.
How should i implement this?
thanks in adv,
-hk
You could convert the two objects into json, then compare them json strings.
var messagesFromServer = getJSONData().messages; //this get msgs from server
var newString = ko.toJSON(messagesFromServer);
var oldString = ko.toJSON(myobject.viewModel.Messages);
if(newString != oldString ) {
// something new
}
ko.mapping.fromJS(messagesFromServer, {}, myobject.viewModel.Messages);
See ko.toJSON doc
I hope it helps.
If the messages is array, you can use ko.utils.compareArrays to detect the changes and raise custom events yourself. Here is code example for comparing ko.observableArray(). Look for Comparing two arrays
I'm using the jqPlot library to build charts. I'm using some JS to fetch a JSON file, build a string using values from the JSON file, convert it to a nested array (the only format that jqPlot likes) and then passing to jqplot. jqPlot is reading the arry just fine and is plotting the correct values, but its adding a 0 value at the end.
Here's the string code:
$(function () {
$.getJSON("test.json", chartData);
function chartData(data) {
$.each(data.values, function(index,val){
chartValues += val + ",";
});
};
here's the code that converts it into a nested array:
var temp = new Array();
temp = chartValues.split(',');
var temp2 = new Array(temp);
alert(temp2);
So when temp2 is passed to jqplot it adds a zero, but when I pass it an identical nested array called test that is declared manually, it doesn't add the zero. Here they are for comparison:
var test = [[12,32,21,23,34,43,52,86,25,]];
and here's temp2
[[12,32,21,23,34,43,52,86,25,]]
Any ideas? I'd also appreciate any help with my logic in this, as I feel like I could be creating the nested array more elegantly.
I'm not sure 100% at this point but I think in .....86,25,]] is not right. That might be the reason to add a zero value. Try eliminating this. Another thing is that you can access the data arrays in json files directly using basic access methods. Try at json org.
Removing the last character in the string (before converting to an array) was the solution in this case.
newStr = chartValues.substring(0, chartValues.length-1);