I am very new to Javascript and D3. So maybe a very simple question.
I have a csv file that I need to import.
The columns are the following.
Date, Value A(score), Value A(rank), Value B(score), Value B(rank), ....Value Z(rank).
I need to graph this using D3 for Date vs Value A(score) and Value B(score).
I am currently using the code below.
const dataset = d3.csv("boardgame_ratings.csv")
dataset1.then(function(data){
const slices = data.columns.slice(1).map(function(id){
return {
id: id,
values: data.map(function(d){
return {
date: timeConv(d.date),
measurement: +d[id]
};
})
};
})
I omit the actual graphing, because I do not have issue with that.
What I am having an issue is that I need to graph only the "score" section. Not the "rank" section. the code that I wrote graphs every column.
I am having trouble because JS is asynchronous.
How would I implement the following?
Extract the "score" only and date columns from the CSV file.
Declare the new data as dataset.
implement the code.
I'd say the problem here is your code for drawing the chart (as you mentioned, "the code that I wrote graphs every column"). This is not ideal because you need to have control of what you draw, so a quick fix would be creating an array of keys telling what columns to use. Also, filtering out columns is unnecessary because when you fetch the CSV you're getting the whole file: if you don't need some columns, just ignore them. Therefore, filtering them out will only add burden to the browser.
All that said, d3.csv accepts a row function, so eliminating some columns is trivial:
const dataset = d3.csv("boardgame_ratings.csv", d => ({foo: d.foo, bar: d.bar}));
Here I'm keeping only the columns foo and bar, ignoring all other columns.
Related
I want to export chart data from a website.
Using console.log(Highcharts.charts[0].series[0].data) I can see the data I need.
I want the values stored under Highcharts.charts[0].series[0].data[n].category for n in [0, 1, ..., N_data_points-1] (These values are the time-stamps of the data points in the plot.)
When I try to use JSON.stringify(Highcharts.charts[0].series[0].data) to write the whole data attribute I get Uncaught TypeError: Converting circular structure to JSON.
How can I get that data[n].category values for all n data points?
Note that the x axis data can be obtained straightforwardly via Highcharts.charts[0].series[0].processedXData, but these are not the dates, just consecutive integers.
Also, I don't necessarily need to do this via console. If it's possible to get the data via, say, a Python script, that would do as well.
You can adapt the python/selenium script here.
Most of the stuff remains as is, except for obvious changes (change the url, etc).
You can get the category data of each data point with:
dates = []
for i in range(len(values)):
date = driver.execute_script('return Highcharts.charts[0].series[0].data[{}].category'.format(i))
dates.append(date[0])
EDIT: as pointed out in a comment, you can get the category data without a for loop using:
categories = driver.execute_script('return Highcharts.charts[0].userOptions.xAxis[0].categories')
I have a dataset in csv and it looks like below.
country,col1,col2,col3
Germany,19979188,11233906,43.7719591
UK,3839766,1884423,50.92349378
France,1363608,796271,41.60557873
Italy,957516,557967,41.72765781
I'd like to drop col1, col2 off while keeping country and col3. If possible, I'd like to wrap it into a function where I can pass column list that I'd like to drop/keep.
Using pandas, which I'm familiar with, I can easily do it. e.g. data.drop(['col1', 'col2'], axis = 1). But I found d3 way or js way in general is based on each row so couldn't come up with an idea to drop columns.
I was thinking of d3.map() taking desirable columns only. But I was stuck to build a general function that the column list can be passed in.
Could anyone have thoughts?
D3 fetch methods, like d3.csv, will retrieve the whole CSV and will create an array of objects based on that CSV. Because of that, filtering out some columns is useless. Actually, it's worse than useless: you'll spend time and resources with an unnecessary operation.
Therefore, the only useful solution is, if you have access and own that CSV, creating a new CSV without those columns. That way you'll have a smaller file, faster to load. Otherwise, if you cannot change the CSV itself, don't bother: just load the whole thing and use the columns you want (which will be properties in the objects), ignoring the others.
Finally, if you have a lot of data manipulation it might be interesting reducing the size of the objects in the data array. If that's your case, use a row function to return only the properties you want. For instance:
d3.csv(url, function (d){
return {country: d.country, col3: d.col3}
}).then(etc...)
I have the following snippet of code which updates my chart type from one type to another.
chart.series.forEach(function(serie){
serie.update({
type: type,
stacking: stacking,
})
})
I can pass different values in from type to stacking.
If I want my new chart to use a different array of values, the arrays of which are already defined and contain data from the database; how would I do it?
The reason I wish to do this is because I have two values, which are entered in their own arrays, say for example, 40 and 50 from a database. These are then stacked in a column chart (It's the only way I can stack).
I've created another array to store both these values in the one array when deciding to show it in a pie chart (not stacked).
I've tried adding 'data:arrayName' in as a property but this prevents my chart from rendering. Any idea guys?
Thank you.
EDIT: My problem probably only required one object to call. I don't have that much going on so the solution in the possible duplicate isn't really helpful to me. The duplicate code is trying to generate a new array on the fly using loops, I already have an array in place, I'm just figuring out how to update the array once the chart has changed to another type.
I’m trying to use data from multiple GoogleSheets to produce a single HighChart graph.
I’d like to do this without moving all the data into one area of a single spreadsheet, particularly as I want to use the drilldown option which would make it difficult to collect all the data together.
I thought I could pass the columns as an array and then reference the array in the data property of my chart, but I’m struggling to do that with even one column from one sheet.
I have searched for answers online, but I have not found anything relating to highcharts getting data from multiple sources.
Previous Research:
Using GoogleSheets Data as an array before creating the chart: (Removed Link) - The problem is that I could only use one GoogleSheets reference here as the chart object sits inside the data object.
API documentation - (Removed Link) – tells me I can access the columns but that’s not the part I’m having problems with
Querying the Chart: (Removed Link) - I have actually considered making hidden charts, then interrogating the data and making new charts from those, but that seems like a very long way round and I’m still not sure I could grab all the data I need.
Using two GoogleSheets for separate charts on the same page: (Removed Link) I have done this.
Please could you help me to understand how I can access the properties and methods of this object outside of the object itself?
Thank you.
My Code:
//Function to produce a data array ***Not Working - Cannot extract array from object method***
function getData(){
Highcharts.data({
googleSpreadsheetKey: '12x66_QEkTKg_UzvpHEygdTmfnu7oH81pSQYn78Hxt80',
googleSpreadsheetWorksheet: 4,
startColumn: 16,
endColumn: 22,
startRow: 63,
endRow: 76,
parsed: function (columns) {
var dataTest2 = [];
var columnLength = columns[1].length;
for (i = 0; i < columnLength; i = i + 1) {
dataTest2.push(columns[1][i]);
}
alert(dataTest2); //This works here, but not if I move it outside of the method, even if I return it.
var testReturn = this.googleSpreadsheetKey; //trying to return any property using "this" - I've also tried this.googleSpreadsheetKey.value but still undefined
return testReturn; //returns "Undefined"
}
});
}
You could use Google Sheets webQuery. Basically, this is a method to export the Spreadsheet's data into a given format such as csv, json, etc. In your case, the link should look like this:
https://docs.google.com/spreadsheet/tq?key=12x66_QEkTKg_UzvpHEygdTmfnu7oH81pSQYn78Hxt80&gid=4&tq=select%20A,%20B&tqx=reqId:1;out:csv;%20responseHandler:webQuery
Please note that here "tg?key" is the key of your Google Sheet, and "&gid=" is NOT 4, this only tells Highcharts to selected Sheet 4, but for Google Sheets look at the source link and copy the numbers which go after "&gid=". Furthermore, "&tq=" is used to select the columns of the Google Sheet, which in the link above selects "Column A" and "Column B". To find out more on how to select columns and query the output refer to:
https://developers.google.com/chart/interactive/docs/querylanguage?csw=1#Setting_the_Query_in_the_Data_Source_URL
Lastly, "&tqx=" is used to output your data in the format you want. The above link uses "out:csv" which will output the data as comma-separated values. This could as well be Json if you like. Have a look at this documentation:
https://developers.google.com/chart/interactive/docs/dev/implementing_data_source#requestformat
In order to implement this output into your javascript code which you would then use to built your chart, you could use external Javascript libraries to handle these outputs. If you output your data as CSV, you can use "papaparse.js" to parse the CSV into Json which you can be then read by highcharts and allows you to built the chart. Refer to this documentation:
http://papaparse.com/docs#remote-files
An alternative to this would be, to output your Google Sheets directly as Json, then use jquery to make an Ajax Call and load the JSON-encoded data into your Javascript code. Precisely, you could perhaps use jQuery.getJSON() to get the data. Look at this link for more details on how to get Json:
http://api.jquery.com/jquery.getjson/
Finally, it is up to you on which format you choose to output the data, I prefer using Json as it saves you the extra step of having to convert the CSV into Json. Whatever suits you best and is easier for you to understand.
Once you have your data, you may have to parse your Json objects with Json.parse(), and then organize your data into an array with .push(). As #jlbriggs stated, organize your data first before you built the chart. Afterwards, you can make two, three or more Ajax calls to import data from different sources. I would not use many as this will impact in your loading and slow down data transfer.
NB: It is important to format the data accordingly for Highcharts to recognize the data, so use parseFloat() to convert strings into numbers, Date.UTC() to convert strings into date, etc.
Hope this helps you.
I am currently taking my first steps in D3 data visualisation. So far, the D3 tutorials have helped me a lot. The pie chart tutorial http://bl.ocks.org/mbostock/3887235 though does not further explain the necessary data structure for pie charts.
My data is more complex than the label/value structure of the example. I have the annual total import data and import data of a specific good stored in JSON:
var data = [{"year":"2001","total_import":"100000","import_specific_good":"25000"},{"year":"2002",...}];
If I understand the tutorial correctly pie() iterates over the SAME entry of each DIFFERENT object.
What if I need specific DIFFERENT values of the SAME object?
I am not interested in a pie showing all annual total imports as portions, but the annual import of a specific good as a portion of the annual total import. My values would be 1. (total_import - import_specific_good) and 2. import_specific_good.
Is my proposed data structure correct for what I want to do? Or do I have to restructure everything so that the values for every year are stored in a separate variable?
var data_2001 = [{"label":"Total Import","value":"100000"},{"label":"Import of Specific Good","value":"25000"}];
var data_2002 = [{"label": ...}];
You don't have to use a specific data structure -- you can (and will need to anyway) modify the example so you can use anything you like! So you can use your first JSON just fine. As you have only 2 values to show, you could simply construct the structure to pass to .data() on the fly.
// for pie chart
...data([json[index].total_import - json[index].import_specific_good,
json[index].import_specific_good])...
...
// similarly for the labels
I would advise to store the numbers as numbers (e.g. without quotes) in your JSON though -- otherwise you'll have to convert them to numbers in Javascript.