how to pass big data to google scatter chart - javascript

I am relatively new to JavaScript and Django and I am struggling with passing big data to my google chart.
I have a chart representing velocities for a given date and distance. In my django views I create list of distances, dates and according velocities. I also generate there a list with sorted values occuring in velocity list and a list with colors according to velocity's value.
I want to have a chart with velocity map with applied colortable like this :
http://i.imgur.com/9Tyv8Rn.jpg
So I used scatter chart with velocity series. The chart is dynamic, it's diffrent for every item selected by a user.
JS to generate rows and columns :
// Define data table rows:
var rows = [];
var rows_list = [];
var vl_max = vel_list.length;
for (i=0; i < vl_max; i+=1) {
var date_tmp = new Date(date_list[i].split(',')[0],date_list[i].split(',')[1]-1,date_list[i].split(',')[2]);
var date = [date_tmp];
var vel_tmp = vel_list[i];
var vtemp_max = vel_tmp.length;
var tooltip_dsname = dsname_list[i];
var tooltip_track = track_list[i];
for (j=0; j < vtemp_max; j+=1) {
var cell = [{v : date_tmp}];
for (k=0; k < vr_max; k+=1) {
var vel_full = vel_tmp[j];
var vel = vel_full.toFixed(1);
if (vel == vel_range[k]) {
// tooltip:
var dist = dist_list[j]/1000;
var yyyy = date_tmp.getFullYear().toString();
var mm = (date_tmp.getMonth()+1).toString(); // getMonth() is zero-based
var dd = date_tmp.getDate().toString();
var tooltip_date = yyyy + "-" + (mm[1]?mm:"0"+mm[0]) + "-" + (dd[1]?dd:"0"+dd[0]);
var tooltip = "<b>dataset: </b>"+tooltip_dsname+"<br><b>date: </b>"+tooltip_date+"<br><b>track: </b>"+tooltip_track+"<br><b>distance: </b>"+dist+" k"+mapunit+"<br><b> velocity: </b>"+vel_full.toFixed(2)+" m/d";
var color = color_list[k]
var style = "point { shape-type: square; fill-color: "+color+";}"
} else {
var dist = NaN;
var tooltip = "empty" ;
var style = "empty" ;
}
cell.push({v: dist},{v: tooltip},{v:style});
}
rows_list.push({c: cell});
}
};
Here is JSfiddle for chart generation with smaller data :
http://jsfiddle.net/joannao89/t26ooyrt/2/
The problem that I have is while the chart is working for smaller data, once I want to load it for a long distance and a wide date range, the browser keeps on popping us this line : "A website is slowing down your browser, what would you like to do ? {stop} {wait}"
I know that this is probably the problem of too large amount of rows, my website generates also 3 other charts like this, with the same data but in another X-Y axis combination (for example time on X-axis, velocity on Y-axis and distance as series) and it works perfectly fine. That's why I would like to pass the data to the chart in some faster way, but I have no clue how.
I already tried to use setTimeout, but it doesn't change a lot. I also tried doing a little less coding on JS side and more in django views, but it also didn't help.
So any suggestions about how to solve this will be very appreciated!

Related

flot multiple line graph animation

I have multiple series on a graph and would like to animate them but it is not working. I am using flot and animator plugin.
https://jsfiddle.net/shorif2000/L0vtrgc2/
var datasets = [{"label":"IT","curvedLines":{"apply":true},"animator":{"start":0,"steps":7,"duration":3000,"direction":"right"},"idx":0,"data":[{"0":1433156400000,"1":"095.07"},{"0":1435748400000,"1":"097.80"},{"0":1438426800000,"1":"096.72"},{"0":1441105200000,"1":"097.62"},{"0":1443697200000,"1":"097.68"},{"0":1446379200000,"1":"098.49"},{"0":1448971200000,"1":"098.59"},{"0":1451649600000,"1":"098.69"}]},{"label":"Network","curvedLines":{"apply":true},"animator":{"start":0,"steps":7,"duration":3000,"direction":"right"},"idx":1,"data":[{"0":1433156400000,"1":"095.07"},{"0":1435748400000,"1":"097.80"},{"0":1438426800000,"1":"096.72"},{"0":1441105200000,"1":"097.62"},{"0":1443697200000,"1":"097.68"},{"0":1446379200000,"1":"098.49"},{"0":1448971200000,"1":"098.59"},{"0":1451649600000,"1":"098.69"}]},{"label":"Success Rate","curvedLines":{"apply":true},"animator":{"start":0,"steps":7,"duration":3000,"direction":"right"},"idx":2,"data":[[1433156400000,98.58],[1433156400000,null],[1433156400000,95.18],[1433156400000,null],[1435748400000,null],[1438426800000,null],[1441105200000,null],[1443697200000,null],[1446379200000,null],[1448971200000,null],[1451649600000,null]]}];
var options = {"series":{"lines":{"show":true},"curvedLines":{"active":true}},"xaxis":{"mode":"time","tickSize":[1,"month"],"timeformat":"%b %y"},"grid":{"clickable":true,"hoverable":true},"legend":{"noColumns":3,"show":true}};
$.plotAnimator($('#CAGraph'), datasets, options);
Problem I have is when I add curved lines it does not work. https://github.com/MichaelZinsmaier/CurvedLines
Without curvedLines plugin (like in the fiddle in your question):
1) If you have multiple data series and use animator, it will only animate the last series. All other series are drawn instantly. (You can see this in your fiddle when you comment out the third data series.)
2) Your last data series has only two points at the same date, so there is nothing to animate (this leads also to problems with the curvedLines plugin for this series).
To animate multiple data series one by one see this answer to another question.
With curvedLines plugin:
3) The curvedLines plugin doesn't work together with the animator plugin (probably because the animator plugin generates a new partial data series for each step). But we can work around this issue with these steps:
a) plot a curvedLines chart without animator,
b) read the data points from this chart and replace the original data,
c) change the options (deactivate curvedLines since the new data is already curved and adjust the step count to the new data),
d) plot the animated chart with the new data.
See this fiddle for a working example with one data series. Relevant code:
var plot = $.plot($('#CAGraph'), datasets, options);
var newData = plot.getData()[0].datapoints.points;
datasets[0].data = [];
for (var i = 0; i < newData.length; i = i+2) {
datasets[0].data.push([newData[i], newData[i+1]]);
}
datasets[0].animator.steps = (newData.length / 2) - 1;
options.series.curvedLines.active = false;
var ani = $.plotAnimator($('#CAGraph'), datasets, options);
Full solution:
Combining the two parts above we get a fiddle which animates two curved lines one by one (the third data series is left out because of the issues mentioned under 2)). Relevant code:
var chartcount = datasets.length;
var chartsdone = 0;
var plot = $.plot($('#CAGraph'), datasets, options);
for (var i = 0; i < chartcount; i++) {
var newData = plot.getData()[i].datapoints.points;
datasets[i].data = [];
for (var j = 0; j < newData.length; j = j + 2) {
datasets[i].data.push([newData[j], newData[j + 1]]);
}
datasets[i].animator.steps = (newData.length / 2) - 1;
}
options.series.curvedLines.active = false;
var ani = $.plotAnimator($('#CAGraph'), [datasets[0]], options);
$("#CAGraph ").on("animatorComplete", function() {
chartsdone++;
if (chartsdone < chartcount) {
ani = $.plotAnimator($('#CAGraph'), datasets.slice(0, chartsdone + 1), options);
}
});

Programmatically creating labels for Chart.js in angular?

I am dynamically creating a line graph for a movie. The y axis is a number of values that is collected over the course of the movies length. The length of the movie is known before. My problem is that ChartJs needs meaningful labels. It also puts each label at the same point as a y axis coordinate so I have to have the same amount of labels as points on the y axis.
This is the code I tried to use to programmatically created an array of labels based on the length of the data that I am feeding ChartJS
var makeLabels = function(){
var numberOfSegments = $scope.data.length;
$scope.labelArray = [];
for (var i = 0; i = numberOfSegments; i++) {
var label = i * 10;
var labelString = label.toString();
$scope.labelArray.push(labelString);
};
}
Image
this is a graph without all of the y axis data.
I fixed it do display the way I want to. The function makeLabels() gets the length of the movie and adds a blank "" label for every label that is not a multiple of 10. I removed the grid lines with scaleShowGridLines: false in the ChartJsProvider.setOptions method.
var makeLabels = function(){
var segments = $scope.movieLength / 1;
s
for (var i = 0; i <= segments; i++) {
var modulus = i % 10;
if(modulus === 0){
$scope.labelsArray.push(i.toString());
} else {
$scope.labelsArray.push("");
};
};

Amcharts: Interpolating step line chart values, show balloon of interpolated value

I'm a newbie at amCharts, trying to create a pretty simple step line chart.
I have data where the value stays constant a long time, and only records changes to the value, for example:
{
"timeOfDay": "18:20",
"value": 100
}, {
"timeOfDay": "19:40",
"value": 80
}, {
"timeOfDay": "21:40",
"value": 20
}, {
"timeOfDay": "22:40",
"value": 50
} // and so on
The chart should draw a horizontal line until the next change point, and then again until the next and so on, which I've managed to do. However, at the moment, it's only showing the balloon text at the data points and not where the cursor is, like this:
Here I'm hovering around time 20:15 though the screenshot taker didn't capture my cursor. The balloon text is displayed at the nearest data point. I'd want the balloon text to show up either at my cursor, or over the graph on the spot my cursor is in, and for it to show the time at the place where the cursor is in.
I know I could generate new data points in between the existing ones, but would rather not, since it would be pretty heavy of an operation since I want it to show the time with an accuracy of one second - that'd be thousands of data points an hour, and I'm trusting amCharts has an interpolation option for times hidden somewhere - I tried to search the docs / google for such an option, but didn't find one so far.
EDIT: Current chart code here: http://pastebin.com/BEZxgtCb
UPDATE: Managed to extract the time with the following functions and attaching an event listener to the chartCursor object - still can't get it to show anywhere else but on the predefined points on the graph though.
var parseDate = function(dateObj) {
return dateObj.getHours() + ":" + dateObj.getMinutes() + ":" + dateObj.getSeconds();
}
var handleMove = function(event) {
var time = event.chart.categoryAxis.coordinateToDate(event.x);
event.chart.graphs[0].balloonText = parseDate(time);
}
the balloon only shows up on actual data points which mean you need to inject them manually to show the balloon across the "interpolated" time. Following walkthrough your data, calculates the distance between the previous points and inject the amount of minutes in between.
var prevTime = undefined;
var prevValue = undefined;
var interpolate = [];
for ( var i1 = 0; i1 < chart.dataProvider.length; i1++ ) {
var item = chart.dataProvider[i1];
var curTime = AmCharts.stringToDate(item.timeOfDay,"JJ:NN")
var curVal = item.value;
if ( prevTime ) {
var distance = (Number(curTime) - Number(prevTime)) / 1000 / 60;
for ( var i2 = 0; i2 < distance; i2++ ) {
var newDate = new Date(prevTime);
newDate.setMinutes(prevTime.getMinutes() + i2);
if ( Number(newDate) < Number(curTime) ) {
interpolate.push({
timeOfDay: AmCharts.formatDate(newDate,"JJ:NN"),
value: prevValue
});
}
}
}
// NEW ONE
interpolate.push(item);
// FOR NEXT TIME
prevTime = curTime;
prevValue = curVal;
}
chart.dataProvider = interpolate;
Unfortunately there is no option to achieve that functionality.

All series data are equal highcharts

I have function for updating sensors. Idea is simple, i add new points from sensors (1 sensor = 1 line) and after appending I update charts. But something went wrong and all series equal to themself
seriesArr - just array of sensors like {"sensorId1", sensorId2 etc}
chartObj - object of charts like chart 1: { chart: highcharts,
seriesArr: seriesArr }
Simple explanations:
chartObject.chart.series[0].points = chartObject.chart.series[1].points = chartObject.chart.series[2].points = etc
My code:
updateAllCharts: function() {
var allCharts = this.charts;
console.log('charts');
for (var chart in allCharts) {
var chartObject = allCharts[chart];
for (var i = 0; i < chartObject.seriesArr.length; i++) {
var elemId = chartObject.seriesArr[i][0];
var val = this.sensors[elemId].get('value');
console.log(val);
if (val === undefined) { // if sensor cant get data
continue;
}
var now = new Date;
var x = Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), now.getUTCDate(),
now.getUTCHours(), now.getUTCMinutes(), now.getUTCSeconds(), now.getUTCMilliseconds());
var y = parseFloat(val);
chartObject.chart.series[i].addPoint([x, y], false, false);
console.log(x, y);
console.log(chartObject.chart.series[i].data);
}
}
for (var chart in allCharts) {
var chartObject = allCharts[chart];
chartObject.chart.redraw();
}
}
Screen:
UPDATE:
ProcessedDataX and Y arrays are changing, but there problem with points array, I always get some weird (maybe cached) points. WTF.
Playground
If you wanna play, i've setted up here jsfiddle, but actually it probably doesn't work. Can't add points to highchart object with setInterval.
JSFIDDLE
UPDATE2:
Actually it works fine in jsfiddle but i don't know wtf is going on in my project.
UPDATE3:
Found, that the same last point adds to each series. For example, series[i].lastPoints = series[n] , where n = last iteration.
Works, only if i REDRAW EVERYTIME AFTER ADDING EACH POINT. so bad solution, so bad.
just put true in parameter after Point object
chartObject.chart.series[z].addPoint(Point, true, false);
To all chart.redraw() you need to set isDirty flag
so try to use this code:
for (var chart in allCharts) {
var chartObject = allCharts[chart];
chartObject.yAxis[0].isDirty = true;
chartObject.chart.redraw();
}

How to set the dynamic or static tick size in a Rickshaw.js plot?

I am creating a Rickshaw.js-powered graph much like in this example: http://code.shutterstock.com/rickshaw/tutorial/example_07.html based on my own data that is returned via an AJAX call. The data is either measured in bytes (typical values range in a few gigabytes or hundreds of MBs) or seconds (anywhere between 10s and 50 minutes). I tried using a Rickshaw.Fixtures.Number.formatBase1024KMGTP formatter for the bytes and wrote my own for the seconds, which does its part well. The problem is that I need to position the tick lines in a smart way - preferably dynamically, but even static settings (e.g. place a tick every 1024*1024*1024=1 GB or every 60 s) would be fine.
I tried setting the tickSize to 1024^3 like so:
var y_axis = new Rickshaw.Graph.Axis.Y({
graph: graph,
tickSize: 1073741824 // 1 GB
});
y_axis.render();
but I ended up seeing no ticks at all. What am I doing wrong and what would be the right way?
Basically, you need to adapt the tickOffsets() function of the Axis.X, Axis.Y and Axis.Time classes in Rickshaw.
tickSize will not help you with that as - like #Old Pro stated correctly - it indicates the size of the bold tick lines in pixels. It has nothing to do with spacing.
For Time-based Axes
My solution essentially consists of replacing the standard tickOffsets() function in those files
this.tickOffsets = function() {
var domain = this.graph.x.domain();
var unit = this.fixedTimeUnit || this.appropriateTimeUnit();
var count = Math.ceil((domain[1] - domain[0]) / unit.seconds);
var runningTick = domain[0];
var offsets = [];
for (var i = 0; i < count; i++) {
var tickValue = time.ceil(runningTick, unit);
runningTick = tickValue + unit.seconds / 2;
offsets.push( { value: tickValue, unit: unit } );
}
return offsets;
};
by a custom routine. This is gonna do the trick:
this.tickOffsets = function() {
var domain = this.graph.x.domain();
var unit = this.fixedTimeUnit || this.appropriateTimeUnit();
var tickSpacing = args.tickSpacing || unit.seconds;
var count = Math.ceil((domain[1] - domain[0]) / tickSpacing);
var runningTick = domain[0];
var offsets = [];
for (var i = 0; i < count; i++) {
var tickValue = time.ceil(runningTick, unit);
runningTick = tickValue + tickSpacing;
offsets.push( { value: tickValue, unit: unit } );
}
return offsets;
};
With that in place, you can write something like
var time = new Rickshaw.Fixtures.Time();
var timeUnit = time.unit('year');
var x_axis = new Rickshaw.Graph.Axis.ExtendedTime(
{
graph: graph,
tickSpacing: 60*60*24*365*13, // 13 years
timeUnit: timeUnit
} );
to have ticks spaced out evenly at every 13 years.
For Value-based Axes
For value-based Axes, you would need to extend the render() function to include a facility that "manually" sets the ticks for the axis. I did it like this:
this.render = function() {
if (this.graph.height !== this._renderHeight) this.setSize({ auto: true });
var axis = d3.svg.axis().scale(this.graph.y).orient(this.orientation);
if (this.tickSpacing) {
var tickValues = [];
var min = Math.ceil(axis.scale().domain()[0]/this.tickSpacing);
var max = Math.floor(axis.scale().domain()[1]/this.tickSpacing);
for (i = min * this.tickSpacing; i < max; i += 1) {
console.log(i);
tickValues.push(i * this.tickSpacing);
}
axis.tickValues(tickValues);
}
axis.tickFormat( args.tickFormat || function(y) { return y } );
if (this.orientation == 'left') {
var berth = this.height * berthRate;
var transform = 'translate(' + this.width + ', ' + berth + ')';
}
if (this.element) {
this.vis.selectAll('*').remove();
}
this.vis
.append("svg:g")
.attr("class", ["y_ticks", this.ticksTreatment].join(" "))
.attr("transform", transform)
.call(axis.ticks(this.ticks).tickSubdivide(0).tickSize(this.tickSize));
var gridSize = (this.orientation == 'right' ? 1 : -1) * this.graph.width;
this.graph.vis
.append("svg:g")
.attr("class", "y_grid")
.call(axis.ticks(this.ticks).tickSubdivide(0).tickSize(gridSize));
this._renderHeight = this.graph.height;
};
The important part here are the statements in the if (this.tickSpacing) clause. They compute ticks given by the tickSpacing variable in the config array, and assign them to the axis in the axis.tickValues(tickValues) statement. Note that this.tickValues is assigned in the this.tickSpacing = args.tickSpacing statement in the initialize() function, not stated above.
Try it yourself
Have a look at this jsfiddle, where the complete code is available. This will certainly give you some pointers. If you want, you can create your own jsfiddle with your values and tell me if you need anything else.
tickSize is the size of the ticks in pixels. Not what you want to be setting to a huge number.
Set ticks to the number of ticks you want on the graph and Rickshaw (actually d3) will do some magic to give you pretty values of ticks that generate about that number of ticks on the graph.
If you want further control you're going to have to dig into d3, where you will be able to explicitly set the tick values using axis.tickValues(). I'd probably copy the existing Rickshaw.Graph.Axis.Y code and create my own Y axis class that includes access to tickValues or the ability to use my own scale. It's a little unclean in that Rickshaw creates the Y scale in the graph.render() function, so you can't easily override the Y scale, but the Y scale Rickshaw creates does have the range set from the graph data, which is information you will want when creating your own tick values.

Categories

Resources