Show gap of missing data with Highstock - javascript

Using Highstock to chart a sorted time serie: [[timestamp, value], ...]
The datasource is sampled at irregular intervals. As result the distances between two points (in the time axis) varies.
If two adjacent points are separated for more than 5 minutes I want to show a gap in the chart.
Using the gapSize option doesn't work, because it doesn't allows to specify the 'size' of the gap as a function of time.
Showing gaps is already a part of Highstock, I just need a way to specify it as a fixed amount of time (5 minutes). Ideas?
Btw, beside that the plot works great.

Here's a slightly unclean way to "manipulate" gapSize to work so that it's value is the amount of milliseconds required to create a gap.
(function (H) {
// Wrap getSegments to change gapSize functionality to work based on time (milliseconds)
H.wrap(H.Series.prototype, 'getSegments', function (proceed) {
var cPR = this.xAxis.closestPointRange;
this.xAxis.closestPointRange = 1;
proceed.apply(this, Array.prototype.slice.call(arguments, 1));
this.xAxis.closestPointRange = cPR;
});
}(Highcharts));
This utilizes that gapSize is only used within the getSegments function (see source), and it works based on the closestPointRange of the axis. It wraps the getSegments, sets closestPointRange to 1, calls the original method and then resets closestPointRange to its original value.
With the code above you could do gaps for 5 minutes like this:
plotOptions: {
line: {
gapSize: 300000 // 5 minutes in milliseconds
}
}
See this JSFiddle demonstration of how it may work.

Halvor Strand function wrapper did not work for me as long as getSegments is not part of highstock source code anymore to calculate that gap. Anyway, you can find an approximation to solve the problem combining this other topic and the previows answer like this:
(function(H) {
H.wrap(H.Series.prototype, 'gappedPath', function(proceed) {
var gapSize = this.options.gapSize,
xAxis = this.xAxis,
points = this.points.slice(),
i = points.length - 1;
if (gapSize && i > 0) { // #5008
while (i--) {
if (points[i + 1].x - points[i].x > gapSize) { // gapSize redefinition to be the real threshold instead of using this.closestPointRange * gapSize
points.splice( // insert after this one
i + 1,
0, {
isNull: true
}
);
}
}
}
return this.getGraphPath(points);
});
}(Highcharts))
setting gapSize in plotOptions to the desired size (in ms) like Halvor said:
plotOptions: {
line: {
gapSize: 300000 // 5 minutes in milliseconds
}
}

In case anyone comes across this and is spending hours trying to figure out why gapSize is not working like me. Make sure your time series data is sorted, only then will the gaps appear in the graph.
Another issue I ran into was my data series was in this format
[
{x: 1643967900000, y: 72},
{x: 1643967600000, y: 72},
{x: 1643967300000, y: 72}
]
However this does not seem to work with gapSize and needs to be in the format below
[
[1643967900000, 72],
[1643967600000, 91],
[1643967300000, 241]
]

Related

Plotly: How to discard older points?

TL;DR I want to display a long-running strip chart with Plotly.js. I don't know how to discard old points.
Details
The following updater from my CodePen at https://codepen.io/Michael-F-Ellis/pen/QvXPQr does almost what I want. It shows a set of 20 samples in 2 traces that update continuously at 500 msec intervals. At the end of the demo, it plots all the points to show they still exist.
var cnt = 0;
var interval = setInterval(function() {
// Add next point to each trace
Plotly.extendTraces('graph', {
y: [[rand()], [rand()]]
}, [0, 1])
// Display only 20 most recent points
Plotly.relayout('graph', { 'xaxis.range': [cnt-20, cnt]})
cnt = cnt+1;
if(cnt === 100) {
// Before ending the run, show all points
// to demonstrate they still exist in Plotly.
Plotly.relayout('graph', { 'xaxis.range': [0, cnt]});
clearInterval(interval);
}
}, 500);
The problem is that I do want to delete older points. The real application needs to run essentially forever on a system with limited memory. I'm looking for a Plotly call that will drop the oldest N trace points. It needs to be reasonably efficient as performance of the target system is also limited.
Thanks!
https://codepen.io/Michael-F-Ellis/pen/YxeEwm
The above seems workable from a behavioral standpoint. Here's the revised updating routine:
Plotly.plot('graph', data);
var cnt = 0;
var max = 20;
var interval = setInterval(function() {
// Add next point to each trace
Plotly.extendTraces('graph', {
y: [[rand()], [rand()]]
}, [0, 1])
// Keep only 'max' most recent points
if(cnt > max) {
data[0].y.shift();
data[1].y.shift();
}
cnt = cnt+1;
if(cnt === 100) {
// Before ending the run, show all points
// to demonstrate that only 'max' points
// still exist in Plotly.
Plotly.relayout('graph', { 'xaxis.range': [0, cnt]});
clearInterval(interval);
}
}, 500);
The solution is to keep the data object in a var outside of Plotly and use shift() to drop old points from the beginning of the array as new points are added.
I'm open to another solution, especially if there are known memory or performance problems with this approach.

Chart.js - Setting max Y axis value and keeping steps correct

I'm using Chart.js 2.6. I have a chart to which I've added custom pagination to step through the dataset, as it is quite large. My pagination and everything works great, I simply grab the next chunk of data in my set and update the chart.config.data with the new data object, and then call .update() on the chart. However, in order to make the chart make sense, I needed to keep the left (Y-axis) scale the same when the user is paginating through. Normally Chart.js would rebuild it based on the data in the chart, but I want it to always reflect the same values.
I've set the max value on the yAxes object of the chart to the maximum value in my data set. I've also set the beginAtZero option to true, and the maxTicksLimit to 10. However, even though my Yaxis does stay the same, it doesn't always look that great (see below screenshot). In this example, my max is set to 21,000 in the chart. Does anyone have any suggestions as to how I can either provide a better max (rounding up to next 5,000, 500, 100, etc based on the value) or some way to get it to create the Y axis without crunching the top number the way it does now?
Here is the function I currently use to determining the max data value to set as the max value in the Yaxes object in the chart. the plugin.settings.chartData variable represents an array of the data values used in the chart. I am trying to get it to increment correctly to the next 1000, 500, etc based on what the maxValue is, but as you can see my math is not correct. In the screenshot example, the maxValue is coming back as 20,750 and my function is rounding it up to 21,000. In this example it SHOULD round it up to the next increment which would be 25,000.
var determineMaxDataValue = function() {
var maxValue = Math.max.apply(Math, plugin.settings.chartData);
var step = maxValue > 1000 ? 1000 : 500;
plugin.settings.maxDataValue = (Math.ceil(maxValue / step) * step);
};
I too had the same problem. You needn't write any special function for determining the max value in the Yaxes. Use 'suggestedMax' setting. Instead for setting 'max' as maximum value in your graph, set suggestMax as the maximum value in your graph. This never works if you have set 'stepsize'.
options: {
scales: {
yAxes: [{
ticks: {
suggestedMax: maxvalue+20
}
}]
}
}
20 is added, so that the tooltip on max value will be clearly visible.
For more info, refer http://www.chartjs.org/docs/latest/axes/cartesian/linear.html#axis-range-settings
Figured it out. Instead of supplying the max value on the Y Axis as I have been, I instead implemented the afterBuildTicks callback and updated the ticks to have the correct increments.
yAxes: [{
afterBuildTicks: function(scale) {
scale.ticks = updateChartTicks(scale);
return;
},
beforeUpdate: function(oScale) {
return;
},
ticks: {
beginAtZero:true,
// max:plugin.settings.maxDataValue,
maxTicksLimit: 10
}
}]
my updateChartTicks function loops over the existing ticks and determines the correct increment amount between the ticks. Then I use that value to add my final "tick" which will always be greater than the largest data in the dataset.
var updateChartTicks = function(scale) {
var incrementAmount = 0;
var previousAmount = 0;
var newTicks = [];
newTicks = scale.ticks;
for (x=0;x<newTicks.length;x++) {
incrementAmount = (previousAmount - newTicks[x]);
previousAmount = newTicks[x];
}
if (newTicks.length > 2) {
if (newTicks[0] - newTicks[1] != incrementAmount) {
newTicks[0] = newTicks[1] + incrementAmount;
}
}
return newTicks;
};

Efficiently detect missing dates in array and inject a null (highcharts and jquery)

I'm using highcharts.js to visualize data series from a database. There's lots of data series and they can potantially change from the database they are collected from with ajax. I can't guarantee that they are flawless and sometimes they will have blank gaps in the dates, which is a problem. Highcharts simply draws a line through the entire gap to the next available date, and that's bad in my case.
The series exists in different resolutions. Hours, Days and Weeks. Meaning that a couple of hours, days or weeks can be missing. A chart will only show 1 resolution at a time on draw, and redraw if the resolution is changed.
The 'acutal' question is how to get highcharts to not draw those gaps in an efficient way that works for hous, days and weeks
I know highcharts (line type) can have that behaviour where it doesn't draw a single line over a gap if the gap begins with a null.
What I tried to do is use the resolution (noted as 0, 1, 2 for hour day or week), to loop through the array that contains the values for and detect is "this date + 1 != (what this date + 1 should be)
The code where I need to work this out is here. Filled with psudo
for (var k in data.values) {
//help start, psudo code.
if(object-after-k != k + resolution){ //The date after "this date" is not as expected
data.values.push(null after k)
}
//help end
HC_datamap.push({ //this is what I use to fill the highchart later, so not important
x: Date.parse(k),
y: data.values[k]
});
}
the k objects in data.values look like this
2015-05-19T00:00:00
2015-05-20T00:00:00
2015-05-21T00:00:00
...and more dates
as strings. They can number in thousands, and I don't want the user to have to wait forever. So performance is an issue and I'm not an expert here either
Please ask away for clarifications.
I wrote this loop.
In my case my data is always keyed to a date (12am) and it moves either in intervals of 1 day, 1 week or 1 month. Its designed to work on an already prepared array of points ({x,y}). Thats what dataPoints is, these are mapped to finalDataPoints which also gets the nulls. finalDataPoints is what is ultimately used as the series data. This is using momentjs, forwardUnit is the interval (d, w, or M).
It assumes that the data points are already ordered from earliest x to foremost x.
dataPoints.forEach(function (point, index) {
var plotDate = moment(point.x);
finalDataPoints.push(point);
var nextPoint = dataPoints[index+1];
if (!nextPoint) {
return;
}
var nextDate = moment(nextPoint.x);
while (plotDate.add(1, forwardUnit).isBefore(nextDate)) {
finalDataPoints.push({x: plotDate.toDate(), y: null});
}
});
Personally, object with property names as dates may be a bit problematic, I think. Instead I would create an array of data. Then simple loop to fill gaps shouldn't be very slow. Example: http://jsfiddle.net/4mxtvotv/ (note: I'm changing format to array, as suggested).
var origData = {
"2015-05-19T00:00:00": 20,
"2015-05-20T00:00:00": 30,
"2015-05-21T00:00:00": 50,
"2015-06-21T00:00:00": 50,
"2015-06-22T00:00:00": 50
};
// let's change to array format
var data = (function () {
var d = [];
for (var k in origData) {
d.push([k, origData[k]]);
}
return d;
})();
var interval = 'Date'; //or Hour or Month or Year etc.
function fillData(data, interval) {
var d = [],
now = new Date(data[0][0]), // first x-point
len = data.length,
last = new Date(data[len - 1][0]), // last x-point
iterator = 0,
y;
while (now <= last) { // loop over all items
y = null;
if (now.getTime() == new Date(data[iterator][0]).getTime()) { //compare times
y = data[iterator][1]; // get y-value
iterator++; // jump to next date in the data
}
d.push([now.getTime(), y]); // set point
now["set" + interval](now.getDate() + 1); // jump to the next period
}
return d;
}
var chart = new Highcharts.StockChart({
chart: {
renderTo: 'container'
},
series: [{
data: fillData(data, interval)
}]
});
Second note: I'm using Date.setDay() or Date.setMonth(), of course if your data is UTC-based, then should be: now["setUTC" + interval].

Force ChartJS to show Doughnut chart with null values

I have a grid with 4 doughtnut charts on each column for different periods of time: last 90 days, last 60 days, last 7 days and today.
The problem with today is that it doesn't always show data, especially in the morning. Is there a way to force ChartJS to show the chart even if it doesn't have any data?
Here's an example: http://jsfiddle.net/6xV78/219/
var pieData = [
{
value: 0,
color:"#3F9F3F"
},
{
value : 0,
color : "#222"
}
];
I found a quick work-around, not sure how "good" or "valid" way it is but it's working for me. If the values are null/zero I replaced them with -1 to retain the looks of the chart and then use the custom tooltip function to override the output.
{
...
data: [earned == 0 ? -1 : earned, pending == 0 ? -1 : pending]
...
},
options: {
tooltip: {
callbacks: {
label: function (tooltipItem, data) {
const value = data['datasets'][0]['data'][tooltipItem['index']];
return '$' + (value == -1 ? 0 : value);
}
}
}
}
Obviously I have 2 slices and when both are 0 the chart is displayed with 2 equal halves both showing $0 income (both earned and pending).
*Do note that this will still take 1 into account when others aren't null so you need to take care of that on your end. I added a method to verify if ALL values are null and that's the only case I display it like this.
A pie chart with all values equal to zero is ill-defined. Because the angle (and the area) of each slice is proportionate to the ratio of the slice's respective value over the sum of all values. If all values are equal to zero, then their sum is zero. Division by zero should be rightfully avoided by the library (hopefully by design), resulting in the no-pie-chart case you encounter. It should be the programmer's responsibility to detect the all-zeros case before populating the chart, if such a case has a possibility of occurring. Then the programmer could either display a "No data yet. What are you doing up so early? Go to sleep." message, if all values are zero. Or, maybe, if they terribly need to keep the looks consistent (having a chart there all the time), they could show a special all-black no-data pie chart with the following dummy data:
var pieNoData = [
{
value: 1,
color: "#000000"
}
];
There is no shame in disclosing that no data exists yet, though.

Flot: Plot rolling 1 hour chart

I have this code to plot a chart which is invoked at a 5 second interval. How can I set the X axis to plot for a rolling 1 hour period?
/**
* Plot chart from retrieved quote data.
*/
function plotData() {
for(var i = 0; i < Quotes.length; ++i) {
if(dataSets[i].length == 7) dataSets[i].shift();
var timestamp = new Date().getTime();
dataSets[i].push([timestamp, Quotes[i].unitprice]);
}
var data = [];
for(var i = 0; i < Quotes.length; ++i) {
data.push({label: Quotes[i].stock, data: dataSets[i]});
}
$.plot('#livetrades-chart', data,
{ xaxis: { axisLabel: 'Time', axisLabelUseCanvas: true,
mode: 'time', timeformat: '%d/%m/%Y %H:%M:%S', timezone: TIME_ZONE },
yaxis: { axisLabel: 'Stock Price', axisLabelUseCanvas: true, tickDecimals: 2 }
});
}
Thanks.
The real-time updates example demonstrates rolling data, where each time a new point is added the oldest one is shifted off the array. What you want to do is basically identical, except with a time axis.
Edit: I still don't understand what the question is; your screenshot shows a time axis, and if you have an hour of data in your array (as opposed to the five seconds shown) then it will show an hour on the axis.
I think maybe you're confused about having to configure the x-axis in some way. You don't: if you provide data whose x-values are spaced an hour apart, the axis will fit to match it. The only thing you might need to tweak is the timeformat (see the Time Series section of the docs for more info) option, if you want the ticks to appear with only H:M:S rather than Y/M/D.
So as far as DNS suggests your cuestion is how did you setup the flot to updates also the xaxis.
here is a runing plunkr for that scenario.
http://plnkr.co/edit/TWpWhL?p=preview

Categories

Resources