I am trying to use my own data in a nvD3 stacked area chart. The sample data format from the Angular nvD3 site has a format like this:
[{
"key":"Series 1",
"values":[[1025409600000,0],[1028088000000,-6.3382185140371]]
},
{
"key":"Series 2",
"values":[[1025409600000,0],[1028088000000,0]]
}]
I have data coming from my database in this format:
[{
"Loc_Cnt":6,"Num_Cars":552,"Num_Employees":34,"active_month":"2017-10-01T00:00:00"
},
{
"Loc_Cnt":4,"Num_Cars":252,"Num_Employees":14,"active_month":"2017-11-01T00:00:00"
}]
I am trying to graph from my data, three series (Series 1: Flt_Cnt, Series 2: Num_Cars, Series 3: Num_Employees). For each series, the X axis value being the active_month date, and the Y axis value being the series value.
How can I either A) convert my data to look like the sample data easily, or B) use my data as is in the AngularJs nvd3 chart? I feel a .forEach on the array would not be efficient for larger data sets, and not as easy to read. I tried to use d3.nest in some way, but haven't been able to get a correct format. Thanks for your help!
It's not elegant, but I brute forced a way to my solution. If there are any better solutions, please do let me know.
var Loc_Cnt = [];
var Num_Cars = [];
var Num_Employees = [];
var obj = {};
//arr is the array of values in my format
arr.forEach(function (element) {
//need the date in milisecond format
var date = new Date(element.active_month);
var time = date.getTime();
//load corresponding arrays
Loc_Cnt.push([time, element.Loc_Cnt]);
Num_Cars.push([time, element.Num_Cars]);
Num_Employees.push([time, element.Num_Employees]);
});
//load each key/values pair into new object
obj["Loc_Cnt"] = Loc_Cnt;
obj["Num_Cars"] = Num_Cars;
obj["Num_Employees"] = Num_Employees;
//d3.entries creates an object of key/VALUEs
arrRollup = d3.entries(obj);
//change the key word values to value
var i;
for (i = 0; i < arrRollup.length; i++) {
arrRollup[i].values = arrRollup[i]['value'];
delete arrRollup[i].value;
}
Related
I'm working on a project to brush up on my JS, and I'm finding Highcharts rather challenging. What I've done is set up project that pulls in current JSON data from an API, and displays results on a map. When you click on a state, you can view the historical data for that state. if you shift click, you can view more than one state.
My x-axis is formatted with dates from each state, as not each state began tracking data at the same time. When multiple states are selected, the information is incorrect because even though they all have the same date for the most recent data point (today), the first data point in the date array varies. For instance, if you click New York, their first data point starts on 3/04, but if you click Connecticut, the first data point starts on 3/07.
Is there a way I can reconcile this? Can I have my categories start from the most recent data point and work backwards, so the data points for today are concurrent?
Here's my pen: https://codepen.io/maxpalmer/pen/rNVRzVX?editors=0010
Stackoverflow is requiring I post some code, so here is the function I wrote that reassembles the api data into an array for each state for the area chart:
for (i = 0; i < stateAbbrevs.length; i++){
var values = new Array(), categories = new Array();
// var categories = new Array();
var state = stateAbbrevs[i];
var stateObj = jsonData.filter(obj => obj.state == state);
for (x = 0; x < stateObj.length; x++) {
var value = stateObj[x].positive;
var date = formatDate(stateObj[x].date);
var name = stateNames[i];
values.push(value);
categories.push(date);
}
values.reverse();
categories.reverse();
historicData[state] = {
name: name,
data: values,
categories: categories
};
}
}```
Ah, found it in the myriad Highcharts documentation. Comes down to reversing my data array, and then reversing it in the highcharts x-axis options.
https://api.highcharts.com/highcharts/xAxis.reversed
I am using NEX-GDDP for obtaining daily climatology (Precipitation, Temp min and Temp max) data of the 21 GCM models in the period 2018-01-01 to 2099-12-31, for certain points. I made this script, for one model in one scenario
//Dataset
var dataset = ee.ImageCollection('NASA/NEX-GDDP')
.filter(ee.Filter.date('2018-01-01', '2099-12-31'))
.filterMetadata('scenario','equals','rcp45')
.filterMetadata('model','equals','MPI-ESM-LR')
//Points of interest
var Acomayo = ee.Geometry.Point([-71.689166667, -13.921388889]),
var Machupicchu = ee.Geometry.Point([-72.545555556, -13.166666667]),
var Urubamba = ee.Geometry.Point([-72.129116546, -13.323123791]),
var Pisac = ee.Geometry.Point([-71.849444444, -13.415833333]),
var Ccatcca = ee.Geometry.Point([-71.56, -13.609722222]),
var GranjaKcayra = ee.Geometry.Point([-71.875, -13.556666667]),
var Pomacanchi = ee.Geometry.Point([-71.5357971, -14.027777778]),
var Sicuani = ee.Geometry.Point([-71.236944444, -14.253333333]);
var pts = ee.FeatureCollection(ee.List([ee.Feature(Acomayo),ee.Feature(Machupicchu),ee.Feature(Urubamba),ee.Feature(Pisac)
,ee.Feature(Ccatcca),ee.Feature(GranjaKcayra),ee.Feature(Pomacanchi),ee.Feature(Sicuani)]));
//Export to table .CSV
// Empty Collection to fill
var ft = ee.FeatureCollection(ee.List([]));
//Without removal of null values ----------------------------------
//Function to extract values from image collection based on point file and export as a table
var fill = function(img, ini) {
// type cast
var inift = ee.FeatureCollection(ini);
// gets the values for the points in the current img
var ft2 = img.reduceRegions(pts, ee.Reducer.first(),30);
// gets the date of the img
var date = img.date().format("yyyy/MM/dd");
var scenario = img.get("scenario");
var model = img.get("model");
// writes the date in each feature
var ft3 = ft2.map(function(f){return f.set("date", date, "scenario", scenario, "model", model)});
// merges the FeatureCollections
return inift.merge(ft3);
};
// Iterates over the ImageCollection
var newft = ee.FeatureCollection(dataset.iterate(fill, ft));
//print(newft);
// Export
Export.table.toDrive({
collection: newft,
description: 'GCM_diario',
folder: 'Downscalling_Diario',
fileFormat: 'csv'
});
The scripts work fine for two days and two points, but for the current points and period of time I need, it still working after 5 hrs. To reduce the amount of data I think these ideas:
Average the daily data of the 21 GCMs models in the product, and make it one ImgaeCollection, so
there is only a need to separate by scenario.
Export the ImageCollection of every variable (Pp, Tmin, Tmax) to a NetCDF only the area that contains points (don't know if it is
possible).
geometry = ee.Geometry.Polygon(
[[[-72.77555636882136, -12.867571480133547],
[-72.77555636882136, -14.670820732958893],
[-70.69914035319636, -14.670820732958893],
[-70.69914035319636, -12.867571480133547]]], null, false);
If there is another way to download this data I open to do it.
Using .iterate() can be very memory intensive and is prone to memory errors. A more straight forward approach to this would be to select a specific point you want to focus on, loop over all of the days of interest, and use .reduceRegion() to get the information desired. You can then export the time series as a CSV and convert it to whichever format you want.
Here is an example that gets all variables for all models and scenarios:
// specify start and end date
// Change as needed
var startDate = ee.Date('2018-01-01');
var endDate = ee.Date('2019-01-01');
// get the dataset between date range and extract band on interest
var dataset = ee.ImageCollection('NASA/NEX-GDDP')
.filter(ee.Filter.date(startDate,endDate));
// get projection and band information
var firstImage = dataset.first();
var bandNames = firstImage.bandNames();
var proj = firstImage.projection();
var point = ee.Geometry.Point([-71.689166667, -13.921388889])
// calculate number of days to map and extract data for
var n = endDate.difference(startDate,'day').subtract(1);
// map over each date and extract all climate model values
var timeseries = ee.FeatureCollection(
ee.List.sequence(0,n).map(function(i){
var t1 = startDate.advance(i,'day');
var t2 = t1.advance(1,'day');
var dailyColl = dataset.filterDate(t1, t2);
var dailyImg = dailyColl.toBands();
// rename bands to handle different names by date
var bands = dailyImg.bandNames();
var renamed = bands.map(function(b){
var split = ee.String(b).split('_');
return split.slice(0,2).cat(split.slice(-1)).join('_');
});
// extract the data for the day and add time information
var dict = dailyImg.rename(renamed).reduceRegion({
reducer: ee.Reducer.mean(),
geometry: point,
scale: proj.nominalScale()
}).combine(
ee.Dictionary({'system:time_start':t1.millis(),'isodate':t1.format('YYYY-MM-dd')})
);
return ee.Feature(point,dict);
})
);
print(timeseries);
// get properties to chart (all climate models)
var props = timeseries.first().propertyNames().removeAll(['system:time_start','system:index','isodate']);
// Make a chart of the results.
var chart = ui.Chart.feature.byFeature(timeseries, 'system:time_start', props.getInfo());
print(chart);
Map.addLayer(point);
Map.centerObject(point,6);
// export feature collection to CSV
Export.table.toDrive({
collection: timeseries,
description: 'NEX-GDDP-timeseries',
fileFormat: 'CSV',
});
You may run into memory errors in the code editor when working with long date ranges (2018-2099) but the export should work. Also, keep in mind that the Earth Engine exports are a little simplistic so exporting point by point would be the best approach, 1) to avoid memory errors and 2) keep the resulting CSV to one point. You can merge all the points together and export the time series per point in one file but that may be difficult to work with once exported...
Here is a working link: https://code.earthengine.google.com/139432f76ae3f6a81b1459762325ef7f
The data series in my HighCharts chart only includes dates from the past few days, but the chart's x-axis and zoom bar show a date range all the way back to 1970.
How can I limit the presented date range so it only goes back as far as the first date present in the series data?
Example
HTML
<div id="chart_container" style="height: 500px></div>
JavaScript
$(function () {
var timestamps = [1481000484000,1481108510000,1481215541000,1481316568000,1481417583000];
var series_raw_data = [100,300,750,400,200];
// prepare data
var points = [[],[]];
for (var i = 0; i < timestamps.length; i++) {
points.push([timestamps[i], series_raw_data[i]]);
}
// create chart
$('#chart_container').highcharts('StockChart', {
series: [{
data: points
}]
});
});
Here's Fiddle1 which shows the behavior.
I also tried setting the xAxis 'min' option to the first timestamp, and setting the axis type to 'datetime', but those didn't help - Fiddle2.
The reason why it happens is your points array.
If fact, after filling, it looks like this:
points = [ [], [], [x, y], [x, y]]
Those two empty arrays create unwanted behaviour.
Fix the initial array and it works
var points = [];
example: https://jsfiddle.net/hbwosk3o/3/
I am trying to create something like this resizable HighChart.
The difference is that i am loading my data from a blob.
This is the graph that i receive:
This is part of the received data, from the console.log(lines);:
[{ date: '7/13/2016 8:35:00 AM', value: 60 },{ date: '7/13/2016
8:36:00 AM', value: 45 },...]
This is my code: https://jsfiddle.net/end5xc7m/
series: [{
turboThreshold: 20000,
type: 'area',
name: 'Values to date',
data: data}
I believe this is where i am getting the problem from, in the function visitorData.
I am not having the data projected onto the graph.
As jlbriggs noted, this is due to a formatting issue. Unless you you're using categories to plot your axes, Highcharts visualizations will not draw if data are input as strings.
I've updated your fiddle with a few fixes: https://jsfiddle.net/brightmatrix/end5xc7m/2/
function processData(allText) {
var allTextLines = allText.split(/\r?\n/);
var lines = [];
for (var i = 0; i < allTextLines.length - 1; i++) {
var currentLine = allTextLines[i].split(',');
var thisLineValue = [Date.parse(currentLine[0]),parseInt(currentLine[1])];
lines.push(thisLineValue);
}
return lines;
}
Here's what I changed:
What you want to pass along to your chart is a set of arrays like [x,y], where these variables are either dates or numbers. Building the values using curly braces and + concatenation operators turns these into strings. So, instead, I created a temporary array called thisLineValue and pushed that to your lines array.
Next, within that temporary array, I used Date.parse() to turn your date values into timestamps. This is happily understood by Highcharts for datetime axes, as you set with your x-axis.
For your y-axis value, I used parseInt() to avoid those values being turned into strings as well.
Finally, I removed the toString() function when you return the lines array to the chart. Again, this keeps the values in the format the chart is expecting (dates and numbers).
I hope this is helpful for you!
The data:
A 'premiums.tsv' file, with two columns, 'BiddingDate' and 'CategoryA'
What I'm trying to do:
Using d3, display in a table or divs just the latest BiddingDate and the CategoryA value that corresponds to that latest BiddingDate.
I've got a chart on the page, that works fine. I can also get a table of all the values. That works fine too. But I just can't figure out how to isolate the data corresponding to the latest date value and then display it. Would really appreciate any help. Thanks!
I solved it by using data.map to map the data into a new array, then simply calling the last item in the array.
I only ever have 241 data points, but for datasets of variable size, you can just use .length to figure out the latest data point.
var allDates = [];
allDates = data.map(function(d) {return d.BiddingMonth;});
var latestDate = allDates[240];
var allCatA = [];
allCatA = data.map(function(d) {return d.CategoryAPremium;});
var latestCatA = allCatA[240];
Then all I had to do was to print latestDate and latestCatA wherever I wanted.
d3.select("#lateDate").text("$ " + latestDate);
d3.select("#lateA").text("$ " + latestCatA);