Develop Reference

Parsing file - Improve load time

May I know If I could parse differently, so that performance and load time can be improved. I deal with file size that is in MBs.
My file with multiple objects is parsed as below:
I display multiple graphs in a single page.
'chart1' to 'chart5' arrays are used for each graph's Y-axis.
'xAxisTime' array is used for x-axis (common x-axis for all graphs).
eg: I get 50000 values in each array (chart1, chart2...xAxisTime).
data = fileWithMultipleObjects;
objects= data.split('\n');
for (var i= 0; i < objects.length - 2; i++) {
var obj = JSON.parse(objects[i])
if (obj.type=== "A") {
chart1.push(obj.c1)
chart2.push(obj.c2)
}
else {
chart1.push(null)
chart2.push(null)
}
if (obj.type=== "B") {
chart3.push(obj.c3)
chart4.push(obj.c4)
chart5.push(obj.c5)
}
else {
chart3.push(null)
chart4.push(null)
chart5.push(null)
}
//common for all charts - xAxis data
if (obj.date === undefined) {
obj.date = null
}
if (obj.date!== null) {
var date= obj.date
xAxisTime.push(date)
}
}
Reason for pushing null values when no data:
I need to show all 5 graphs in line with the time the job was run.
Also, this helps me use "this.point.y" (var index = this.point.y), which helps me display tooltips on plotline. Fiddle showing tooltip(when clicked on green line) with "this.point.y" :https://jsfiddle.net/xqb0cn5r/4/
This is how my graphs look. Fiddle: https://jsfiddle.net/xgpL1w3t/. Only charts that meets if condition are rendered. chart1 and chart5 are rendered in this case. Thank you.
Edited:
Below is the data structure of my file:
{"C1":55.77,"C2":11367.25,"type":"A","date":"10/24/2022 12:05:37.236"}
{"C3":55.77,"C4":11367.25,"type":"B","date":"10/24/2022 12:05:37.236","C5":445.21}
{"C1":55.77,"C2":11367.25,"type":"A","date":"10/24/2022 12:05:37.236"}
{"C1":55.77,"C2":11367.25,"type":"A","date":"10/24/2022 12:05:37.240"}
{"C3":55.77,"C4":11367.25,"type":"B","date":"10/24/2022 12:05:37.250","C5:445.25}
{"C3":55.77,"C4":11367.25,"type":"B","date":"10/24/2022 12:05:37.275","C5":445.26}
I display like 8 to 10 charts in real scenario with around 50000 points each. Common x-axis for all. Out of 50000, only 2000 to 4000 are real data, remaining are all null values. I push null, so that I can keep graphs in line with the whole time(x-axis) the job was run.
Right now, it takes time to load charts. As so much data to parse and load. I believe performance can be be improved if my data is parsed differently for the same scenario. I would appreciate any suggestions.
Performing single json parse as below:
var convertStringToJsonFormat= "[" + data.split('\n').join(", ")
JsonFormat = convertStringToJsonFormat.slice(0, -2);
JsonFormat = JsonFormat + "]"
data = JSON.parse(JsonFormat)

How to match up category labels with multiple series?

I'm working on a project to brush up on my JS, and I'm finding Highcharts rather challenging. What I've done is set up project that pulls in current JSON data from an API, and displays results on a map. When you click on a state, you can view the historical data for that state. if you shift click, you can view more than one state.
My x-axis is formatted with dates from each state, as not each state began tracking data at the same time. When multiple states are selected, the information is incorrect because even though they all have the same date for the most recent data point (today), the first data point in the date array varies. For instance, if you click New York, their first data point starts on 3/04, but if you click Connecticut, the first data point starts on 3/07.
Is there a way I can reconcile this? Can I have my categories start from the most recent data point and work backwards, so the data points for today are concurrent?
Here's my pen: https://codepen.io/maxpalmer/pen/rNVRzVX?editors=0010
Stackoverflow is requiring I post some code, so here is the function I wrote that reassembles the api data into an array for each state for the area chart:
for (i = 0; i < stateAbbrevs.length; i++){
var values = new Array(), categories = new Array();
// var categories = new Array();
var state = stateAbbrevs[i];
var stateObj = jsonData.filter(obj => obj.state == state);
for (x = 0; x < stateObj.length; x++) {
var value = stateObj[x].positive;
var date = formatDate(stateObj[x].date);
var name = stateNames[i];
values.push(value);
categories.push(date);
}
values.reverse();
categories.reverse();
historicData[state] = {
name: name,
data: values,
categories: categories
};
}
}```

Ah, found it in the myriad Highcharts documentation. Comes down to reversing my data array, and then reversing it in the highcharts x-axis options.
https://api.highcharts.com/highcharts/xAxis.reversed

HighChart with more than 50 data points breaks

I am trying to create a bubble chart using the JS HighChart in Angular2+. Whenever there are more than 50 data points (bubbles), the graph breaks. There are the correct number of bubbles in the correct positions (x,y plots) with all different colors but the sizes are all the same even though the z-values are all different. (I am outputing the z-values in a tooltip and the z-values are accurate)
This function is how I am passing in data to the high-chart configuration.
setSeries() {
this.objData = []
this.Data.forEach(element => {
var x= element['xVal'];
var y = element['yVal'];
var z = element['zVal'].toFixed(0);
var name = element['seriesName'].trim();
var newData =[{
x:x,
y:y,
z:+z,
}]
// SetSeriesData is how i am creating the obj to pass into series=[] in highchart configuration
if(i<50) //If I comment this condition, the graph breaks. Right now, the graph is working properly
this.setSeriesData(sumData, name, this.objData)
i++
})
this.options.series = this.objData;
this.generateChart();
}
This is my setSeriesData function.
setSeriesData(graphData: any, dataName: any, objData: any){
var obj = {};
obj['name'] = dataName;
obj['data'] = graphData;
obj['events'] = {click: function(e) {
//takes me to another link
}};
objData.push(obj)
}
In the above function, I configured the chart so that when you click the bubble, it takes you to another page. When the data points >50, this click functionality is not working either. In addition, the fillOpacity is not correct.
Just a few things to point out
1. I am using Angular 2+
2. The discovered issues are, fillOpacity, click, and size based on z-value.
3. It works perfectly when the data points are less than 50
How can I fix this?

Angular nvD3 Stacked Area Chart Data Series Format

I am trying to use my own data in a nvD3 stacked area chart. The sample data format from the Angular nvD3 site has a format like this:
[{
"key":"Series 1",
"values":[[1025409600000,0],[1028088000000,-6.3382185140371]]
},
{
"key":"Series 2",
"values":[[1025409600000,0],[1028088000000,0]]
}]
I have data coming from my database in this format:
[{
"Loc_Cnt":6,"Num_Cars":552,"Num_Employees":34,"active_month":"2017-10-01T00:00:00"
},
{
"Loc_Cnt":4,"Num_Cars":252,"Num_Employees":14,"active_month":"2017-11-01T00:00:00"
}]
I am trying to graph from my data, three series (Series 1: Flt_Cnt, Series 2: Num_Cars, Series 3: Num_Employees). For each series, the X axis value being the active_month date, and the Y axis value being the series value.
How can I either A) convert my data to look like the sample data easily, or B) use my data as is in the AngularJs nvd3 chart? I feel a .forEach on the array would not be efficient for larger data sets, and not as easy to read. I tried to use d3.nest in some way, but haven't been able to get a correct format. Thanks for your help!

It's not elegant, but I brute forced a way to my solution. If there are any better solutions, please do let me know.
var Loc_Cnt = [];
var Num_Cars = [];
var Num_Employees = [];
var obj = {};
//arr is the array of values in my format
arr.forEach(function (element) {
//need the date in milisecond format
var date = new Date(element.active_month);
var time = date.getTime();
//load corresponding arrays
Loc_Cnt.push([time, element.Loc_Cnt]);
Num_Cars.push([time, element.Num_Cars]);
Num_Employees.push([time, element.Num_Employees]);
});
//load each key/values pair into new object
obj["Loc_Cnt"] = Loc_Cnt;
obj["Num_Cars"] = Num_Cars;
obj["Num_Employees"] = Num_Employees;
//d3.entries creates an object of key/VALUEs
arrRollup = d3.entries(obj);
//change the key word values to value
var i;
for (i = 0; i < arrRollup.length; i++) {
arrRollup[i].values = arrRollup[i]['value'];
delete arrRollup[i].value;
}

Choropleth Alternative Method and more

In my previous post1 and post2, i managed to fix the choropleth map/legend issue + draw circles problems when drawing a map.
When i follow this must-do tutorial about choropleth and when i search on internet i always find the same logic
d3.csv("my.csv", function(data) {
d3.json(myjson, function(json) {
for (var i = 0; i < data.length ; i++) {
//Grab state name
var dataState = data[i].nom;
//Grab data value, and convert from string to float
var dataValue = data[i].population;
//Find the corresponding state inside the GeoJSON
for (var j = 0; j < json.features.length; j++) {
var jsonState = json.features[j].properties.nom;
if (dataState == jsonState) {
//Copy the data value into the JSON
json.features[j].properties.CA = dataValue;
//Stop looking through the JSON
break;
}
}
}
So in my case, i have a map with 75 path (1 path=region) and my csv file have 75 rows (1 row = 1 path)
Now i'm trying to do things a little differently
My new csv has N rows (N > 75, let's say 200) and for each row, a store (properties+lat+lon) is affected to a path ==> i can have 5 stores/path e.g
Here are my questions :
1) How do i write my choropleth code differently ==> I'd like to scan the csv file and return for each distinct path the sum of specific properties (here "income") in order to write it on my json file ???
2) When i click on a specific region/path, i'd like to display on a new div (in my case #output) the json file corresponding to my region (basicaly i have 75 json files "region1.json", "region2.json" and so on...") with circles inside (one circle = one store, in my csv file "name" column") ==> How do i retrieve this "on click value" and call the correct/corresponding json file ????
3) Finally, if i click on a displayed specific circle of the #output div, i'd like to have on a third div a chart ==> How do i writte correctly my 3rd div so it is correctly displayed (css, other ?? ==> it can be applied to #output too) ??
Thank you so much for reading this request and for your availability and help
Here's the plunker file (do not mind the sales.csv file, i just used it to try displaying something when i click on the path
Thanks again

d3 nest roll up is the solution
d3.csv("source-data.csv", function(error, csv_data) {
var data = d3.nest()
.key(function(d) { return d.date;})
.rollup(function(d) {
return d3.sum(d, function(g) {return g.value; });
}).entries(csv_data);
});
More info here