I'm using d3 to import a CSV file for data visulisation. My CSV file has header rows that I need to convert to dates, and data that needs to be converted to integers:
ISBN,2019-08,2019-09,2019-10
9782749205465,107,488,218
9789423069313,95,87,186
I can import the CSV file and parse the data as integers using:
d3.csv("/assets/2019-20-download-stats.csv").then(function (data) {
data.forEach(function(d) {
d.ISBN = +d.ISBN;
d['201908'] = +d['201908'];
d['201909'] = +d['201909'];
d['201910'] = +d['201910'];
});
});
but the header rows are all strings in the output array, e.g.:
{
"201908": 107,
"201909": 488,
"201910": 218,
"ISBN": 9782749205465,
}
How do I format the header rows when importing the data?
This is almost certainly a XY problem. I can't see any situation where your data should have objects with dates as keys. On top of that, pay attention to the fact that JavaScript will automatically convert your date objects to strings if you pass them as an object's key. If you (for whatever reason) really want a date as key, you should use a Map.
All that being said, here is what you should do: in the row conversion function (you have none in your code), check for a date...
if (!isNaN(new Date(key).getTime()))
... and, if you find one, use new Date to generate the key:
foo[new Date(key)] = +d[key]
Here is a demo using d3.csvParse (which would be exactly the same for your d3.csv):
const csv = `ISBN,2019-08,2019-09,2019-10
9782749205465,107,488,218
9789423069313,95,87,186`;
const data = d3.csvParse(csv, row);
function row(d) {
const newObject = {};
for (var key in d) {
if (!isNaN(new Date(key).getTime())) {
newObject[new Date(key)] = +d[key]
} else {
newObject[key] = +d[key]
}
};
return newObject;
};
console.log(data)
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/5.7.0/d3.min.js"></script>
Related
I have a CSV file which needs to be converted into a Javascript object / JSON file. Doesn't really matter which since I'll be be handling the data in JS anyway and either is fine.
Data in csv:-
Name,Time,HeightOptions/WidthRange,HeightOptions/Options
EGBL,Today,12,12.13.14.15.16
,,26,12.13.14.15.16
Desired Output:-
{
Name:"EGBL",
Time:"Today",
HeightOptions : [
{WidthRange:"12",
Options:[12,13,14,15,16]},
{WidthRange:"26",
Options:[12,13,14,15,16]
}]
}
This is what I have came up with:
const CSV = (csv) => {
var attrs = csv.splice(0, 1);
console.log("attrsattrs", attrs);
var result = csv.map(function (row) {
var obj = {};
var rowData = row.split(",");
attrs[0].split(",").forEach(function (val, idx) {
obj = constructObj(val, obj, rowData[idx]);
});
return obj;
});
function constructObj(str, parentObj, data) {
if (str.split("/").length === 1) {
parentObj[str] = data;
return parentObj;
}
var curKey = str.split("/")[0];
if (!parentObj[curKey]) parentObj[curKey] = {};
parentObj[curKey] = constructObj(
str.split("/").slice(1).join("/"),
parentObj[curKey],
data
);
return parentObj;
}
console.log("resultresultresult", result);
};
But it returns like this:-
{
Name:"EGBL",
Time:"Today",
HeightOptions : [
{WidthRange:"12",
Options:"12.13.14.15.16"},
]
},{
Name:"",
Time:"",
HeightOptions : [
{WidthRange:"26",
Options:"12.13.14.15.16"
}]
}
So as you see code is reading through as rows and not combining in 1 object.
As far as possible I wish to have this done in vanilla JS without any other libraries.
Thanks folks.
Its the general way how the converter works. What you can do is, after getting the result in some JSON format, write another function that will convert to your final JSON format. Otherwise write your own CSV parser by reading it as raw text. Also you can play around by changing the csv format to different way so that you can get expected result (if possible).
I'm having an issue getting my data from my "react-hook-form" web form in the correct format for my api
I'm already cheating by entering in the [] date field of the form which isn't ideal but just trying to get passed this step for now.
I need to send it over like the below DesiredData, basically with array brackets around the entire thing and then the apostrophes '["1-2-2020", "1-3,2020"]' for the dates
DesiredData:[
{
name:"bob"
age:"20"
dates: ["1-2-2020", "1-3,2020"]
}]
CurrentData: {
name:"bob"
age:"20"
dates: '["1-2-2020", "1-3,2020"]'
}
var currentData = {
name:"bob",
age:"20",
dates: '["1-2-2020", "1-3,2020"]'
}
currentData.dates = JSON.parse(currentData.dates)
var desiredData = [currentData];
console.log(desiredData)
I have a CSV file where for a row some value is missing. So if a value is missing then we need to delete that row from the CSV file. I am facing a problem with doing that. Please help me with this.
We can use a CSV parsing library, such as the excellent Papa Parse to parse the data, then we can filter the rows based on the column that we wish to filter on.
For example:
const Papa = require('papaparse');
let csvData = `Col1,Col2,Col3\na1,b1,c1\na2,,c2\na3,b3,c3`;
let { data } = Papa.parse(csvData, { header: true });
console.log("Original csv data:");
console.log(csvData);
function filterEmptyValues(data, column) {
return data.filter(row => row[column]);
}
let filteredData = filterEmptyValues(data, "Col2");
let filteredCsv = Papa.unparse(filteredData);
console.log("\nFiltered csv:")
console.log(filteredCsv);
A javascript data object (JSON notation) has been created with the following content:
"[
{"range":"Shape","values":[{"idx":0,"val":"Random"},{"idx":1,"val":"Line"},{"idx":2,"val":"Square"},{"idx":3,"val":"Circle"},{"idx":4,"val":"Oval"},{"idx":5,"val":"Egg"}]},
{"range":"Color","values":[{"idx":0,"val":"Red"},{"idx":1,"val":"Blue"},{"idx":2,"val":"Yellow"},{"idx":3,"val":"Green"},{"idx":4,"val":"Cyan"}]}
]"
In a next step the index of an ordinal value has to be found in this object. The function should find the index of the value 'Blue' in the range 'Color'.
So the function should have the meta scripting form
f("Color")("Blue")=1
What is the most elegant form to create such a function in the context of D3 and javascript?
Depending on your use case, it might make sense to convert the data structure to a different structure more suitable for direct access. E.g. you could convert your structure to
var data = {
Shape: ['Random', 'Line', ...],
// ...
};
and access it with
data['Shape'].indexOf('Line') // or data.Shape.indexOf('Line')
Or go even one step further and convert to
var data = {
Shape: {
Random: 0,
Line: 1,
// ...
},
// ...
};
and access it with
data['Shape']['Line'] // or data.Shape.Line
What the best solution is depends on the actual use case.
Converting the structure dynamically is pretty straight forward. Here is an example to convert it to the first suggestion:
var newData = {};
data.forEach(function(item) {
newData[item.range] =
item.values.map(function(value) { return value.val; });
});
This would also reduce redundancy (e.g. idx seems to correspond with the element index).
Would this work for you ?
var dataJson = '[ \
{"range":"Shape","values":[{"idx":0,"val":"Random"},{"idx":1,"val":"Line"},{"idx":2,"val":"Square"},{"idx":3,"val":"Circle"},{"idx":4,"val":"Oval"},{"idx":5,"val":"Egg"}]},\
{"range":"Color","values":[{"idx":0,"val":"Red"},{"idx":1,"val":"Blue"},{"idx":2,"val":"Yellow"},{"idx":3,"val":"Green"},{"idx":4,"val":"Cyan"}]}\
]';
var data = JSON.parse(dataJson);
for (each in data){
if ( (data[each].range) === 'Color'){
for (eachVal in data[each].values){
if (data[each].values[eachVal].val === 'Blue'){
alert(data[each].values[eachVal].idx);
}
}
} ;
}
And here is the JSFiddle for you too.
I'm trying to read in a csv file with D3 and I'm a little stuck. The way my csv file is formatted is that the first line is a merged cell containing a year then the next line will contain the data descriptions (name, age etc).
Currently I have the following:
var resourceList = [{description: "All Yearly Data",
name: "yearlyData",
path: "data.csv"};
d3.csv(resourceInfo.path, function(error, d) {
theData.resources[resourceInfo.name].processed = true;
theData.resources[resourceInfo.name].error = error;
theData.resources[resourceInfo.name].data = d;
theData.numProcessed += 1;
});
This reads the first line in as the data descriptions and then the following lines as actual data. What I want to do is have an multidimensional array which I could go through by year. Is it possible to skip lines while parsing to make sure I can manage that or no?
Thanks!
one way of getting at this would be to use filter:
d3.csv(resourceInfo.path, function(error, d) {
var newData = d.filter(function(obs) { return INSERT YOUR FILTER CONDITION HERE;});
...
see also:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter?redirectlocale=en-US&redirectslug=JavaScript%2FReference%2FGlobal_Objects%2FArray%2Ffilter