I have this CSV file (with 4 rows and 6 columns of data). While parsing the values:
var headers = d3.keys(data[0])
.filter(function(key) { return key !== ""; });
for(i=0;i<headers.length;i++)
data.forEach(function (d){
d[headers[i]] = +d[headers[i]];
})
data.forEach(function (d){
console.log(d[headers[4]]);
})
But once I used the console log command to check the data for every column I realised that some data are missing. I noticed that if the values are the same(two 5 for example) somehow they one is skipped. How can I change that? Any ideas?
Related
I've got an array question for processing in google sheets. I'm trying to map a value from one sheet into another. Thanks for any help. I've got explanation and code below...
PROBLEM: this gives me blank rows (policies2[i]).
DESIRED OUTCOME: new values from entry_top in the last row of policies2.
I'm working with two sheets. policies.values is the array from a normal google sheet of tabular values. policies2 is a copy of policies. entry_top is the array of values from a sheet with unstructured data, except that headers and values are beside each other.
So if "Line" is a header in policies, it would find "Line" in a cell/array node in entry_top and then get the next cell/array value (next column/index/to the right), take that value and put it in the last row in the same column in policies (match by header).
So the code below loops through policies.values[0], the header values. Then loops through every cell in entry_top, by rows (e) and columns (ee). Then if the headers match (h and ee), it puts the next value ([ei][eei+1]) in the matching policies2 column in the last row ([policies2.length-1][hi]).
policies.values[0].map((h, hi) => {
entry_top.map((e, ei) => {
e.map((ee, eei) => {
if (ee == h) policies2[policies2.length - 1][hi] = entry_top[ei][eei + 1];
});
});
});
MRE: oddly this example works. so not sure what above is causing an issue...
function testdp() {
policies = {
values:[[1,2,3],[4,5,6],[7,8,9]]
}
policies2=[[1,2,3],[4,5,6],[7,8,9],[,,,]];
entry_top = [[,,1,'add',,],[,'a','b',2,'add2',,'c'],['c',,,3,'add3',,]]
Logger.log(policies.values);
Logger.log(policies2);
Logger.log(entry_top);
policies.values[0].forEach((h,hi)=>{
entry_top.forEach((e,ei)=>{
e.forEach((ee,eei)=>{
if (ee == h) policies2[policies2.length - 1][hi] = entry_top[ei][eei + 1];
})
})
});
Logger.log(policies2);
// [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0], [7.0, 8.0, 9.0], [add, add2, add3]]
}
this all works just fine. I was logging the wrong result.
I am trying to create a simple "plug-n-play" map template, that allows user to put a csv file with geoids and values and then see the values as a choropleth.
Right now I am merging two datasets (map and values) using double loop, but wondering if there is any other option:
This chunk of code stays within the function that loads geodata (fresh_ctss) :
d3.csv("data/communities_pop.csv", function(error, comms)
{
csv = comms.map(function(d)
{
//each d is one line of the csv file represented as a json object
// console.log("Label: " + d.CTLabel)
return {"community": d.community, "population" :d.population,"label": d.tract} ;
})
csv.forEach(function(d, i) {
fresh_ctss.forEach(function(e, j) {
if (d.label === e.properties.geoid) {
e.properties.community = parseInt(d.community)
e.properties.population = parseInt(d.population)
}
})
})
You'll definitely need two loops (or a nested loop) - the most optimal way would be to just limit how much iteration needs to happen. Right now, the first loop goes through every csv row. The following nested loop goes through every csv row (as new different object) and then, as many times as there are rows in the csv, through every item in fresh_ctss.
If you mapped the rows into an object instead of an array, you could iterate through the rows once (total) and then once through the elements of fresh_ctss (again, total). Code below assumes that there are no tract duplicates in comms:
all_comms = {}
comms.forEach(function(d) {
all_comms[d.tract] = {"community": d.community, "population": d.population}
})
fresh_ctss.forEach(function(e) {
comm = all_comms[e.properties.geoid]
e.properties.community = parseInt(comm.community)
e.properties.population = parseInt(comm.population)
}
I have some data where, in a given column of a csv, there are six possible values:
1,2,3,4,5,NaN.
I am currently trying to group the data using the d3.nest and rollup functions. My goal is to group the data but exclude "NaN" values in the final output.
This is my current code:
var nested = d3.nest()
.key(function(d){return d[question];
})
.rollup(function(leaves){
var total = data.length
var responses = leaves.length;
return {
'responses' : responses,
'percent' : responses/total
};
})
.entries(data)
As you can see, I would like to return both a count of each of the categories as well as the percentage of the total that they represent. After removing NaN, I would also like the removal of NaN represented in percentage values of all of the other categories so that they sum to 100%.
The easiest way to do this is to remove the rows the contain NaN before passing the data to d3.nest():
var filtered = data.filter(function(d) { return d.question !== 'NaN'; });
This question already has answers here:
How can I access and process nested objects, arrays, or JSON?
(31 answers)
Closed 9 years ago.
I have entered the following code in the Chrome console:
d3.csv("dataset32.txt")
.row(function(d) { return {time: +d.time, value: +d.val}; })
.get(function(error, rows) { console.log(rows); });
This returns an array of 160 objects which has the data of my CSV file.
How do I reference the data in these objects?
The "dataset32.txt" is CSV data which looks like this:
time,val
0,1.8988762857143
0.0625,0
0.125,-2.6204492857143
0.1875,-0.34179771428571
The console prints out the result of the above commands as follows:
[Object, Object, Object…]
[0 … 99]
0: Object
time: 0
value: 1.8988762857143
__proto__: Object
So how do I reference the data inside these objects: "time" and "value"?
Here's a straight forward method for importing your csv file using d3.
d3.csv("path to the csv", function(data, error) { }).
More info about csv.
Here's a function that's help you.
Step 1 : Importing your CSV file.
d3.csv("path to your csv file", function(data, error) {
// You're function goes in here.
})
Step 2 : Targeting your data.
// This can be done using the data variable assigned in the function.
data.forEach(function(d) {
// Function goes in here.
})
Step 3 : Targeting your columns.
// Suppose you have 2 columns namely time and val.
data.forEach(function(d) {
// You can target your columns using this fucntion.
// function(d) {return d.colname}
// Here I'm parsing your time and val col numbers to actual numbers after importing.
d.time = +d.time;
d.val = +d.val;
})
Documentation for d3.
Hope this helps.
I'm trying to read in a csv file with D3 and I'm a little stuck. The way my csv file is formatted is that the first line is a merged cell containing a year then the next line will contain the data descriptions (name, age etc).
Currently I have the following:
var resourceList = [{description: "All Yearly Data",
name: "yearlyData",
path: "data.csv"};
d3.csv(resourceInfo.path, function(error, d) {
theData.resources[resourceInfo.name].processed = true;
theData.resources[resourceInfo.name].error = error;
theData.resources[resourceInfo.name].data = d;
theData.numProcessed += 1;
});
This reads the first line in as the data descriptions and then the following lines as actual data. What I want to do is have an multidimensional array which I could go through by year. Is it possible to skip lines while parsing to make sure I can manage that or no?
Thanks!
one way of getting at this would be to use filter:
d3.csv(resourceInfo.path, function(error, d) {
var newData = d.filter(function(obs) { return INSERT YOUR FILTER CONDITION HERE;});
...
see also:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter?redirectlocale=en-US&redirectslug=JavaScript%2FReference%2FGlobal_Objects%2FArray%2Ffilter