I just started programming with Highcharts, and wonder if it's possible to get any sort of line chart based on this table: http://www2.nve.no/h/hd/plotreal/Q/0027.00025.000/knekkpunkt.html
The table updates each half hour, and I want to program a line chart that updates accordingly.
My problem is that I have no idea if this is possible. The table data are remote, on a server I don't have access to. The data is public and open, though.
Would be grateful for some help!
One great open source project you'd want to check out is D3.js. It's a feature-rich library for generating complex visualizations (using SVG elements and the DOM) from raw data. Here is a demo I made using a snippet of your data that is powered by D3:
http://jsfiddle.net/2s6Y3/
//Define two functions that will 'map' x- and y-values into their respective axis domain
var x_scale_fn = d3.scale.linear().domain([Math.min.apply(null, times), Math.max.apply(null, times)]).range([0, width]);
var y_scale_fn = d3.scale.linear().domain([Math.max.apply(null, levels), Math.min.apply(null, levels)]).range([0, height]);
//Create a 'line' object that can read our records and turn them into x-y coordinates
var line = d3.svg.line()
.x_scale_fn(function(d, i) { return x(times[i]); })
.y_scale_fn(function(d, i) { return y(levels[i]); });
//Create a new SVG element (our chart) and give it some basic attributes
var graph = d3.select(".chart").append("svg:svg")
.attr("width", width + margins[1] + margins[3])
.attr("height", height + margins[0] + margins[2])
.append("svg:g")
.attr("transform", "translate(" + margins[3] + ", " + margins[0] + ")");
//Create our chart axes
var x_axis = d3.svg.axis().scale(x),
y_axis = d3.svg.axis().scale(y).orient("left");
//Render the x-axis
graph.append("svg:g")
.attr("transform", "translate(0," + height + ")")
.attr("class", "x axis")
.call(x_axis)
.selectAll("text")
.attr("transform", function(d) { return "rotate(45deg)"; });
//Render the y-axs
graph.append("svg:g")
.attr("transform", "translate(-25,0)")
.attr("class", "y axis")
.call(y_axis);
//Render our data (line) onto the chart
graph.append("svg:path").attr("d", line(data));
If you want to create a project that offers a persistent, up-to-date visualization of the water data, you'll need to set up a scraper that will periodically read data from the page and format it to be used by D3 (i.e. - JSON). This could be done with a number of different tools, but one suggestion would be using Python and urllib2.
This will be possible, but not fun at all. That data is all one big preformatted string. Getting access to a table of those values will make things way easier for you. Otherwise, you'll need to look into writing some RegEx to parse that monster node's text.
Your link seems to not be working.
If you are talking about an HTML table, take a look at the demo here:
http://www.highcharts.com/demo/column-parsed
Thanks for all the help, this made me realize that my skills doesn't add up to what it takes to get this working.
Related
Sorry for the lengthy post, but I just wanted to share all of the trouble shooting and progress I made up until this point. This error has been persistent for over a week.
I have a line chart supplemented with three buttons representing three days of data. I have the buttons mapped to three .txt files and a d3 event listener:
var fileMap = {
'Day 1':'2018-05-17.txt',
'Day 2':'2018-05-18.txt',
'Day 3':'2018-05-19.txt'
}
d3.selectAll('.button').on('click', function(d) {
var dayValue = this.innerHTML;
var thisFile = fileMap[dayValue];
createChart(thisFile);
});
So the idea is you can click the button and the graph will update itself. I kept running into trouble while implementing the .transition() call. I ended up using this, which works:
before calling the main createGraph function, I have a variable that counts how many times we create a graph:
var graphCount = 0;
Then within the scope of the createGraph function I have a counter:
graphCount +=1;
Then I have the logic necessary to draw the graph (initial state) or simply transition (if already drawn):
if (graphCount>1) {
xScale.domain(d3.extent(data, function(d) {return (d.date)}));
yScale.domain(d3.extent(data, function(d) {return (d.y2)}));
d3.selectAll(".line")
.data([data])
.transition() // change the line
.duration(750)
.attr("d", graphLine);
d3.selectAll("g.y.axis").transition() // change the y axis
.duration(750)
.call(yAxis);
} else {
svg.append('path')
.attr('transform', 'translate(' + margins.left + ',' + margins.top + ')')
.datum(data)
.attr('class', 'line')
.attr('d', graphLine);
var yAxisNodes = svg.append('g')
.attr('class', 'y axis')
.attr('transform', 'translate(' + margins.left + ',' + margins.top + ')')
.call(yAxis);
}
Maybe it's not the most elegant way to handle update functionality. I admit it seems a little crude, but at least it does work.
Red Flags:
X Axis: First off, the x axis isn't visible in my graph, but that's by design. I have a tooltip that shows the x axis value, but I have omiited the tooltip for simplicity.
The X axis is a time dimension, the .txts use unix time, which I convert using new Date(((split[0]*300)+rawTime)*1000). Meaning that instead of hour:minutes (as I originally wanted) all my d.dates are full fledged dates. Here is an example slice:
data[0]->Object->date: Thu May 17 2018 09:35:00 ….
I have a few timeFormat variables as well, but oddly when I try to call d3.time.format("%H:%M") the graph isn't drawn. So in my code they are not called. Since I don't need a visible x axis, I have let this go. The graph works fine when I don't use timeFormat and use the full fledged date from new Date().
The problem: Whenever a button is clicked, the graph updates successfully -- new data is read from the respective .txt but for some reason I'm getting over 40 errors reading:
Error: attribute d: Expected number, "12342542342352452, Nan".
Which makes no sense because the graph is plotted fine. As noted earlier, the unix dates have all been successfully parsed into javascript date objects. I passed the data to the console to examine just to be safe, and yes, it seems there are no parsing errors. All the dates and all the y axis values are there.
I'm not sure what to do, since a few button clicks will result in the log showing 100+ errors. I don't think that's good for browswer stability, but on the other hand, the graph appears to work just fine.
Question: Why am I getting 40+ errors in the log? My data set is around 80 observations long. I noticed it only throws errors during transitions. The initial graph creation doesn't have any errors.
Please, please make the errors go awayyyy.
I have a very minimal example here:
https://plnkr.co/edit/Zq9Yd6Hf7mlTlegMXiIG
Hunches
I also thought it could be the xAxis itself. But I passed
d3.extent(data, function(d) {return d.date
into the console log, and everything checked out again. Very strange.
The errors are due only to an empty last row in the txt files, giving you a last object like this in the arrays:
{y2: NaN, date: Wed May 16 2018 23:35:00}
Of course, you're not getting anywhere with that NaN.
Solution: remove those empty rows. After that your arrays have the correct number of objects (77 versus 78 in the running code you linked)
Here is the updated Plunker: https://plnkr.co/edit/XdOcqsdCG3kXw8cvja9y?p=preview
PS: regarding that graphCount and your supposition...
Maybe it's not the most elegant way to handle update functionality.
Yes, it's certainly not the best way. There is a much better approach: the famous enter-update-exit pattern! Get rid of that awkward counter and the if...else statement!
My intention is to draw multiple small barcharts in one svg with different domains, building upon the example.
The main problem I am stuck on seems to be the problem of extracting values for a particular key from the output of d3.nest and defining the domain corresponding to each key. The problem arises when plotting all the values, for which the dates are drawn in the domain. Since not each key has a value corresponding to all possible dates, there is a tail plotted on the right, which breaks the order. Could you please tell me how it can be fixed, how can the inputs without a corresponding output be removed from the plotted set of values?
Here is the result:
https://plnkr.co/edit/KUnMSfJwWan3JaIpZutZ/
The data is in the following form:
CSC 20160919 4.0
MAT 20160923 16.0
In the example sample of data given, there are two dates, and two dates are going to be plotted for each small subgraph. It would be great to separate the dates corresponding to a particular key in such a way that only those dates for which there is a corresponding score are plotted.
There is a deficiency: the domains must be separate, while in the example above they are not.
Could you please tell me how I can plot the graphs for each key with the date corresponding to a particular key as defined in d3.nest?
The relevant part of the code seems to be here:
x.domain(data.map(function(d) { return d.date; }));
and here:
svg.selectAll(".bar")
.data(function(d) {return d.values;})
.enter()
.append("rect")
.attr("class", "bar")
.attr("x", function(d) { return x(d.date); })
.attr("width", x.rangeBand())
.attr("y", function(d) { return y(d.score); })
.attr("height", function(d) { return height - y(d.score); })
.attr("fill", function(d) {return color(d.score)})
APPENDIX
The following questions can be useful:
D3.js binding an object to data and appending for each key
X,Y domain data-binding in multiple grouped bar charts in d3.js
You can also find the following links helpful:
Grouped Bar Chart
Tutorial: Grouped Bar Chart
Grouping Data
HYPOTHESES
Can we nest the data twice, first by course, then by date, and then plot keys' keys on the axes? Is nesting the best way to approach the problem at all?
Here is my solution (warning: I'm the first one to admit that my code is too complicated. Let's see if someone else comes with a simpler solution).
First, we create an empty object:
var xScale = {};
And populate this new object with two functions, one for each different scale.
courses.forEach(function(d) {
xScale[d.key] = d3.scale.ordinal()
.rangeRoundBands([0, width], .1)
.domain(d.values.map(function(e) {
return e.date
}));
});
After this, we'll have two different scales:
xScale.HIS//this is the scale for the first SVG
xScale.PHY//this is the scale for the second SVG
Now you can use those scales in your bars, but not directly. We have to define what scale is going to be used in what SVG.
To do that, firstly, we'll get the key property in the data for each SVG, and then using this value to access the correct object inside xScale, which contains the proper scale:
.attr("x", function(d) {
var scale = d3.select(this.parentNode).datum().key;
return xScale[scale](d.date);
})
.attr("width", function() {
var scale = d3.select(this.parentNode).datum().key;
return xScale[scale].rangeBand()
})
The most complicated part is calling the axes. Here, we'll use each to call xAxis two times, using the first argument (the datum) to get the value of key:
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + height + ")")
.each(function(d) {
return d3.select(this).call(xAxis.scale(xScale[d.key]))
.selectAll("text")
.style("text-anchor", "end")
.attr("dx", "-.8em")
.attr("dy", "-.55em")
.attr("transform", "rotate(-90)");
})
All together, this is your updated plunker: https://plnkr.co/edit/XFKzQm0zp0MkhzK4Qt5d?p=preview
I'm helping guide a project that's combining some visualizations in D3. In our specific example we are producing a graph of pies by using a regular D3 Pie chart with the sankey layout visualization. The affect of this is to produce something like:
The development is aimed to try and keep this as modular as possible, therefore the very first step was to produce an updating pie chart that could be used stand alone or plugged into another visualization. This is currently encapuslated into a pieObject which looks something like this:
var pieObject = function( d, target ){
var pie = {};
// other code to handle init
pie.update = function(data) {
// render code
};
};
Where it gets a little confusion is in the tree visualization, when I need to start handling updates. Here is how a new pie is added:
sankey.nodes(data.nodes)
.links(data.links)
.layout(32);
var node = svg.append("g")
.selectAll(".node")
.data(data.nodes)
.enter()
.append("g")
.attr("class", "node")
.attr("transform", function (d) { return "translate(" + d.x + "," + d.y + ")"; });
.each( function( d ) {
var pie = new pieObject( d, this );
})
If I however want to deal with an existing node, I'm not sure how I should go about accessing the pieObject? There are a couple of options I can think of, but I'm wondering if there's a general approach commonly used?
Store the pieObject on the element d
Store the pieObject in an array or JavaScript object with a lookup from a field on the d
Out of the examples I've outlined, I prefer option #1. But I'm also re-loading my entire data-set from JSON (e.g. new JSON request returns existing data + new data) so I believe when I call sankey.nodes(data.nodes).links(data.links).layout(32); that this is going to lose any additional information I've stored on the d object?
EDIT
I've put together a JSFiddle to help illustrate my problem. The code is a little lengthy and I don't own it as yet so don't know all the ins & outs but here's a breakdown:
1-#214 - Sankey code which produces the Tree layout
215-#451 - Code for a Pie Chart
453-#475 - Code to add the viz to the page
Specifically the area of creating the pies, and trying to figure out how to update them is in the render function between lines #129-#149
I'd recommend starting with Mike Bostock's tutorial on re-usable charts, which does pretty much what you are looking for: http://bost.ocks.org/mike/chart/
Specifically, the key to this type of thing is to use the selection.call() function to insert your chart into another chart/layout in a re-usable way. Then when you need to update your embedded chart, just do the same selection.call() again. Hopefully that gets you started.
I have this data structure:
[
{x:1,comp:63.79,h:21.74,l:7.25,edu:8.7},
{x:2,comp:63.79,h:21.74,l:7.25,edu:8.7},
{x:3,comp:57.99,h:21.74,l:7.25,edu:8.7},
{x:4,comp:56.54,h:21.74,l:7.25,edu:8.7},
{x:5,comp:50.74,h:21.74,l:7.25,edu:8.7},
{x:6,comp:36.24,h:21.74,l:0,edu:8.7},
{x:7,comp:34.8,h:0,l:7.25,edu:8.7},
{x:8,comp:30.45,h:0,l:7.25,edu:8.7},
{x:9,comp:34.8,h:0,l:0,edu:8.7},
{x:10,comp:34.8,h:0,l:7.25,edu:8.7}
];
I am trying to make a pie chart representation.
I tried learning from the d3 tutorials but, those are not so useful in this scenario.
The tutorials only work for linear structures like [10,20,30,40].
But in my case, I have to make one pie for each record and plus each record has 4 parameters to include in a pie.
I realize (as per tutorial understanding) that I need to make arc generators for each line per property something like that. But cant find a way to do so. Also, it seems unfeasible.
Specially bcoz, you I will have to write some thing like (as per my attempt on a linear structure):
var pie = d3.layout.pie()
.value(function(d){ return d; });
var arcs = group.selectAll(".arc")
.data(pie(data))
.enter()
.append("g")
.attr("class", "arc");
arcs.append("path")
.attr("d", arc)
.attr("fill", function(d){ return color(d.data); });
Is there a way out....I was thinking I should revamp the data structure or at least change it before using it.
Thanks is Advance !
I am trying to use the following code taken from this example to pan my time-series line-chart sideways, as new data is added to the graph:
d3.select("path")
.datum(globalData)
.attr("class", "line")
.attr("d", valueline(globalData))
.transition()
.ease("linear")
.attr("transform", "translate(" + x(-dx) + ")");
But this does't work - my line disappears. I am trying to understand what the units of the translate vector need to be for this to work. In my case dx would be the difference between the first and last x-values of the new data.
The units used in translate are screen pixels in the default coordinate system.