Develop Reference

How to get D3 csv data into a different variable [duplicate]

This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Closed 4 years ago.
I am trying to extract imdb data from a csv and put two elements of it into an array of objects. There are 10 fields. I only want to use 2 in my chart. I am trying this:
var mydata;
// Grab the data
d3.csv('./movies.csv', function(csv){
csv.forEach(function(d){
mydata += {rating: d.imdbRating,
winsNoms: d.WinsNoms };
});
});
console.log(mydata);
Can somebody explain exactly what is going on here and why I get undefined when I try to output mydata in the final line? Finally, how do I change it around so it works?

You should push objects into an array. See adjustments below:
var mydata = [];
// Grab the data
d3.csv('./movies.csv', function(csv){
csv.forEach(function(d){
mydata.push({
rating: d.imdbRating,
winsNoms: d.WinsNoms
});
});
});
console.log(mydata);

You’ll need to first declare an empty array. Then you can push items into it.
To only use part of the set, you can use “slice” to truncate the array.
var mydata = [];
// Grab the data
d3.csv('./movies.csv', function(csv){
csv.forEach(function(d){
mydata.push({rating: d.imdbRating,
winsNoms: d.WinsNoms });
});
});
console.log(mydata);

Read a JSON list in list [duplicate]

This question already has answers here:
How can I access and process nested objects, arrays, or JSON?
(31 answers)
Closed 6 years ago.
I have a json like this :
{
"Project
[id=1, dateDebut=2017-01-13, dateFin=2017-01-18, description=qsd, sponsor=qsd ]"
:
[
{"id":1,"title":"qsd ","description":"qsdqsd","dateFin":"2017-01-26"},
{"id":2,"title":"sss ","description":"sss","dateFin":"2017-01-26"}
]
}
originated from : return new ObjectMapper.write(Map<Project,List<Task>> projectTasks = new LinkedMultiValueMap<>()) ;
EDIT : this is the real response :
{"Project [id=1, name=qsdsqd, type=null, done=false, dateDebut=2017-01-13, dateFin=2017-01-18, description=qsd, sponsor=qsd, client=qsd, showable=true]":
[{"id":1,"title":"qsd ","description":"qsdqsd","dateFin":"2017-01-26","dateDebut":"2017-01-14","period":null,"done":false,"status":"Actif","priority":"Normal"},
{"id":2,"title":"task 2 ","description":"qsdqsd","dateFin":"2017-01-26","dateDebut":"2017-01-14","period":null,"done":false,"status":"Actif","priority":"Normal"}]}
How can I read the list of tasks in the client side ?

First of all, your JSON is not valid. Are you sure that is a line break between the word Project and [id...]. A valid JSON would be:
{
"Project [id=1, dateDebut=2017-01-13, dateFin=2017-01-18, description=qsd, sponsor=qsd, ]":
[
{"id":1,"title":"qsd ","description":"qsdqsd","dateFin":"2017-01-26"},
{"id":2,"title":"sss ","description":"sss","dateFin":"2017-01-26"}
]
}
You can have object key names like that. But i'ts not very friendly to retrieve data.
If you cannot change your data schema (or just don't want), you can iterate over the Object with the
Object.keys(obj).forEach ( (key) => {
console.log('key: ' + key);
console.log('value: ' + obj[key]);
/* you can iterate over your value (tasks) here */
obj[key].forEach( (task) => {
console.log('task1: ', task);
});
}); //where obj is your json
Or you can access the first object property with:
obj[Object.keys(obj)[0]]; //where obj is your json
EDIT As pointed by #André Dion, forEachis best suited to iteration, not map. And we're assuming your response is already parsed from the server (by yourself or by a lib like jquery). If not, you should do a JSON.parse(response); to retrieve the object.

You may try this:
Assume above response in stored in var response.
for(var project in response) { // this will get every project
for(var i=0; i<project.length; i++) { // this will iterate over the array for each project which are your tasks.
console.log("task" + project[i]);
console.log(project[i]["id"]); // access each tasks id, similar for other attributes
}
}

D3js skips data

I have this CSV file (with 4 rows and 6 columns of data). While parsing the values:
var headers = d3.keys(data[0])
.filter(function(key) { return key !== ""; });
for(i=0;i<headers.length;i++)
data.forEach(function (d){
d[headers[i]] = +d[headers[i]];
})
data.forEach(function (d){
console.log(d[headers[4]]);
})
But once I used the console log command to check the data for every column I realised that some data are missing. I noticed that if the values are the same(two 5 for example) somehow they one is skipped. How can I change that? Any ideas?

ScatterChart in NVD3 – Reading the data from csv file

I am trying to read in data from csv file and want to visualise this data with a scatterChart in NVD3.
I would have linked to a JSfiddle or something similar but I don't know how to include a csv file in these online JavaScript IDEs. Is that possible?
The csv file has the following format:
country,y,x
Algeria,91.8,15.7
Bahrain,98.2,49.3
Jordan,99.1,55.0
Kuwait,98.6,57.4
Lebanon,98.7,58.6
My best guess for the code to read the csv file with is:
var scatterdata = [
{
key : "Group1",
values : []//{x:"",y:""}
}
];
d3.csv("literacyScatterCountrynames.csv", function (error, csv) {
if (error) return console.log("there was an error loading the csv: " + error);
console.log("there are " + csv.length + " elements in my csv set");
scatterdata[0].values["x"] = csv.map(function(d){return [+d["x"] ]; });
scatterdata[0].values["y"] = csv.map(function(d){return [+d["y"] ]; });
I see my data in the DOM and it looks about right but the chart is not shown and instead it says 'No Data Available.' in bold letters where the chart should be.
Neither here at StockOverflow, nor in the NVD3 documentation on Github, nor in the helpful website on NVD3 charts by cmaurer on GitHub could I find more information on how to do this.

Turning your csv into JSON would work, but isn't necessary. You've just got your data formatting methods inside-out.
You seem to be expecting an object containing three arrays, one for each column in your table. The D3 methods create (and the NVD3 methods expect) an array of objects, one for each row.
When you do
scatterdata[0].values["y"] = csv.map(function(d){return [+d["y"] ]; });
You're creating named properties of the values array object (all Javascript arrays are also objects), but not actually adding content using array methods, so the length of that array is still zero and NVD3 sees it as an empty array -- and gives you the "no data" warning.
Instead of using the mapping function as you have it, you can use a single mapping function to do number formatting on the data array, and then set the result directly to be your values array.
Like so:
var scatterdata = [
{
key : "Group1",
values : []//{x:"",y:""}
}
];
d3.csv("literacyScatterCountrynames.csv", function (error, csv) {
if (error) return console.log("there was an error loading the csv: " + error);
console.log("there are " + csv.length + " elements in my csv set");
scatterdata[0].values = csv.map(function(d){
d.x = +d.x;
d.y = +d.y;
return d;
});
console.log("there are " + scatterdata[0].values.length + " elements in my data");
//this should now match the previous log statement
/* draw your graph using scatterdata */
}
The mapping function takes all the elements in the csv array -- each one of which represents a row from your csv file -- and passes them to the function, then takes the returned values from the function and creates a new array out of them. The function replaces the string-version of the x and y properties of the passed in object with their numerical version, and then returns the correctly formatted object. The resulting array of formatted objects becomes the values array directly.
Edit
The above method creates a single data series containing all the data points. As discussed in the comments, that can be a problem if you want a category name to show up in the tooltip -- the NVD3 tooltip automatically shows the series name as the tooltip value. Which in the above code, would mean that every point would have the tooltip "Group1". Not terribly informative.
To format the data to get useful tooltips, you need each point as its own data series. The easiest way to make that happen, and have the output in the form NVD3 expects, is with d3.nest. Each "nested" sub-array will only have one data point in it, but that's not a problem for NVD3.
The code to create each point as a separate series would be:
var scatterdata;
//Don't need to initialize nested array, d3.nest will create it.
d3.csv("literacyScatterCountrynames.csv", function (error, csv) {
if (error) return console.log("there was an error loading the csv: " + error);
console.log("there are " + csv.length + " elements in my csv set");
var nestFunction = d3.nest().key(function(d){return d.country;});
//create the function that will nest data by country name
scatterdata = nestFunction.entries(
csv.map(function(d){
d.x = +d.x;
d.y = +d.y;
return d;
})
); //pass the formatted data array into the nest function
console.log("there are " + scatterdata.length + " elements in my data");
//this should still match the previous log statement
//but each element in scatterdatta will be a nested object containing
//one data point
/* draw your graph using scatterdata */
}

You could place the data into a variable, as Mike describes here:
name value
Locke 4
Reyes 8
Ford 15
Jarrah 16
Shephard 23
Kwon 42
is represented this way:
var data = [
{name: "Locke", value: 4},
{name: "Reyes", value: 8},
{name: "Ford", value: 15},
{name: "Jarrah", value: 16},
{name: "Shephard", value: 23},
{name: "Kwon", value: 42}
];

d3: skipping lines when reading in csv file

I'm trying to read in a csv file with D3 and I'm a little stuck. The way my csv file is formatted is that the first line is a merged cell containing a year then the next line will contain the data descriptions (name, age etc).
Currently I have the following:
var resourceList = [{description: "All Yearly Data",
name: "yearlyData",
path: "data.csv"};
d3.csv(resourceInfo.path, function(error, d) {
theData.resources[resourceInfo.name].processed = true;
theData.resources[resourceInfo.name].error = error;
theData.resources[resourceInfo.name].data = d;
theData.numProcessed += 1;
});
This reads the first line in as the data descriptions and then the following lines as actual data. What I want to do is have an multidimensional array which I could go through by year. Is it possible to skip lines while parsing to make sure I can manage that or no?
Thanks!

one way of getting at this would be to use filter:
d3.csv(resourceInfo.path, function(error, d) {
var newData = d.filter(function(obs) { return INSERT YOUR FILTER CONDITION HERE;});
...
see also:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter?redirectlocale=en-US&redirectslug=JavaScript%2FReference%2FGlobal_Objects%2FArray%2Ffilter