Develop Reference

restructure CSV data to create correct format in JSON

I'm working with some CSV data. Right now the CSV has a column called 'characteristic' which is one of three types, and a column called 'value', which contains the numerical value for the characteristic.
I'd like to change the structure of the data so that the columns are the characteristics themselves, and the values fall directly under those columns.
Here are screenshots of the tables, for clarity:
Currently:
What I'd like:
I changed things manually to give an example. The actual table I'll need to change is thousands of lines, so I'm hoping I can do this programmatically in some way.
The reason I need to restructure is that I need to transform the CSV to JSON, and the JSON needs to look like this:
[
{
"country":"afghanistan",
"iso3":"afg",
"first_indicator":3,
"second_indicator":5,
"third_indicator":3
},
{
"country":"united states",
"iso3":"usa",
"first_indicator":8,
"second_indicator":6,
"third_indicator":7
},
{
"country":"china",
"iso3":"chn",
"first_indicator":6,
"second_indicator":0.7,
"third_indicator":2
}
]
So - is there any way to take my CSV as it is now (first screenshot), and transform it to the JSON I want, without doing it all manually?
I've done a lot of searching, and I think maybe I just don't know what to search for. Ideally I would use javascript for this, but any suggestions welcome.
Thank you.

I made a JSFiddle for you, something like this should be what you want.
JavaScript
function Country(name, short){
this["country"] = name;
this["iso3"] = short;
}
function getCountryByName(name) {
for(var i = 0; i < countries.length; i++){
var country = countries[i];
if(country["country"] == name){
return country;
}
}
return null;
}
var csv = "country,shortname,characteristics,value\nafghanistan,afg,first_characteristic,3\nunited states,usa,first_characteristic,8\nchina,chn,first_characteristic,6\nafghanistan,afg,second_characteristic,5\nunited states,usa,second_characteristic,6\nchina,chn,second_characteristic,0.7\nafghanistan,afg,third_characteristic,3\nunited states,usa,third_characteristic,7\nchina,chn,third_characteristic,2"
var rows = csv.split("\n");
var countries = [];
if(rows.length > 0){
var header = rows[0];
var columns = header.split(",");
var countryIndex = columns.indexOf("country");
var shortnameIndex = columns.indexOf("shortname");
var characteristicsIndex = columns.indexOf("characteristics");
var valueIndex = columns.indexOf("value");
for(var i = 1; i < rows.length; i++) {
var row = rows[i];
var columns = row.split(",");
var name = columns[countryIndex];
var short = columns[shortnameIndex];
var characteristic = columns[characteristicsIndex];
var value = columns[valueIndex];
var country = getCountryByName(name);
if(!country){
country = new Country(name, short);
countries.push(country);
}
country[characteristic.replace("characteristic", "indicator")] = +value;
}
}
console.log(countries);
console.log(JSON.stringify(countries));
Output from the last line is this:
[{"country":"afghanistan","iso3":"afg","first_indicator":"3","second_indicator":"5","third_indicator":"3"},
{"country":"united states","iso3":"usa","first_indicator":"8","second_indicator":"6","third_indicator":"7"},
{"country":"china","iso3":"chn","first_indicator":"6","second_indicator":"0.7","third_indicator":"2"}]

My suggestion is to convert the CSV to JSON first. You can use an online tool.
When you have the JSON you can write a Javascript code to modify the JSON in the format you want.

Razor MVC Populating Javascript array with Model Array

I'm trying to load a JavaScript array with an array from my model. Its seems to me that this should be possible.
Neither of the below ways work.
Cannot create a JavaScript loop and increment through Model Array with JavaScript variable
for(var j=0; j<255; j++)
{
jsArray = (#(Model.data[j])));
}
Cannot create a Razor loop, JavaScript is out of scope
#foreach(var d in Model.data)
{
jsArray = d;
}
I can get it to work with
var jsdata = #Html.Raw(Json.Encode(Model.data));
But I don't know why I should have to use JSON.
Also while at the moment I'm restricting this to 255 bytes. In the future it could run into many MBs.

This is possible, you just need to loop through the razor collection
<script type="text/javascript">
var myArray = [];
#foreach (var d in Model.data)
{
#:myArray.push("#d");
}
alert(myArray);
</script>

I was working with a list of toasts (alert messages), List<Alert> from C# and needed it as JavaScript array for Toastr in a partial view (.cshtml file). The JavaScript code below is what worked for me:
var toasts = #Html.Raw(Newtonsoft.Json.JsonConvert.SerializeObject(alerts));
toasts.forEach(function (entry) {
var command = entry.AlertStyle;
var message = entry.Message;
if (command === "danger") { command = "error"; }
toastr[command](message);
});

JSON syntax is pretty much the JavaScript syntax for coding your object. Therefore, in terms of conciseness and speed, your own answer is the best bet.
I use this approach when populating dropdown lists in my KnockoutJS model. E.g.
var desktopGrpViewModel = {
availableComputeOfferings: ko.observableArray(#Html.Raw(JsonConvert.SerializeObject(ViewBag.ComputeOfferings))),
desktopGrpComputeOfferingSelected: ko.observable(),
};
ko.applyBindings(desktopGrpViewModel);
...
<select name="ComputeOffering" class="form-control valid" id="ComputeOffering" data-val="true"
data-bind="options: availableComputeOffering,
optionsText: 'Name',
optionsValue: 'Id',
value: desktopGrpComputeOfferingSelect,
optionsCaption: 'Choose...'">
</select>
Note that I'm using Json.NET NuGet package for serialization and the ViewBag to pass data.

To expand on the top-voted answer, for reference, if the you want to add more complex items to the array:
#:myArray.push(ClassMember1: "#d.ClassMember1", ClassMember2: "#d.ClassMember2");
etc.
Furthermore, if you want to pass the array as a parameter to your controller, you can stringify it first:
myArray = JSON.stringify({ 'myArray': myArray });

I was integrating a slider and needed to get all the files in the folder and was having same situationof C# array to javascript array.This solution by #heymega worked perfectly except my javascript parser was annoyed on var use in foreach loop. So i did a little work around avoiding the loop.
var allowedExtensions = new string[] { ".jpg", ".jpeg", ".bmp", ".png", ".gif" };
var bannerImages = string.Join(",", Directory.GetFiles(Path.Combine(System.Web.Hosting.HostingEnvironment.ApplicationPhysicalPath, "Images", "banners"), "*.*", SearchOption.TopDirectoryOnly)
.Where(d => allowedExtensions.Contains(Path.GetExtension(d).ToLower()))
.Select(d => string.Format("'{0}'", Path.GetFileName(d)))
.ToArray());
And the javascript code is
var imagesArray = new Array(#Html.Raw(bannerImages));
Hope it helps

This would be better approach as I have implemented :)
#model ObjectUser
#using System.Web.Script.Serialization
#{
var javaScriptSearilizer = new JavaScriptSerializer();
var searializedObject = javaScriptSearilizer.Serialize(Model);
}
<script>
var searializedObject = #Html.Raw(searializedObject )
console.log(searializedObject);
alert(searializedObject);
</script>
Hope this will help you to prevent you from iterating model ( happy coding )

If it is a symmetrical (rectangular) array then
Try pushing into a single dimension javascript array;
use razor to determine the array structure; and
then transform into a 2 dimensional array.
// this just sticks them all in a one dimension array of rows * cols
var myArray = new Array();
#foreach (var d in Model.ResultArray)
{
#:myArray.push("#d");
}
var MyA = new Array();
var rows = #Model.ResultArray.GetLength(0);
var cols = #Model.ResultArray.GetLength(1);
// now convert the single dimension array to 2 dimensions
var NewRow;
var myArrayPointer = 0;
for (rr = 0; rr < rows; rr++)
{
NewRow = new Array();
for ( cc = 0; cc < cols; cc++)
{
NewRow.push(myArray[myArrayPointer]);
myArrayPointer++;
}
MyA.push(NewRow);
}

The valid syntax with named fields:
var array = [];
#foreach (var item in model.List)
{
#:array.push({
"Project": "#item.Project",
"ProjectOrgUnit": "#item.ProjectOrgUnit"
});
}

#functions
{
string GetStringArray()
{
var stringArray = "[";
for (int i = 0; i < Model.List.Count; i++)
{
if (i != Model.List.Count - 1)
{
stringArray += $"'{Model.List[i]}', ";
}
else
{
stringArray += $"'{Model.List[i]}']";
}
}
return stringArray;
}
}
<script>
var list = #Html.Raw(GetStringArray());
</script>

Maybe it could be interesting also this easy solution that can be easily applied also to javascript dictionaries:
<script type="text/javascript">
var myArray = [
#foreach (var d in Model.data)
{
#:"#d",
}
];
</script>
that translates into this (string1 to stringN are considered here the content of Model.data)
<script type="text/javascript">
var myArray = [
"string1",
"string2",
"string3",
...
"stringN",
];
</script>

<script>
var tempArray = [];
#foreach (var item in Model.Collection)
{
#:tempArray.push({ Field1: "#item.Field1", Field2: "#item.Field2" });
}
$("#btn").on("click", function () {
$.ajax({
url: '/controller/action',
type: 'POST',
contentType: 'application/json; charset=utf-8',
data: JSON.stringify(tempArray),
success: function (resp) {
alert(resp);
}
});
});
</script>
Controller/Action
parameter: ICollection <_Model> _items

Using AmCharts to create bar chart from JSON object retrieved from localStorage

I'm trying to create a bar chart using AmCharts using an array that is part of a larger JSON object that is being retrieved from localStorage. Basically, what I do is retrieve the JSON string, parse it back to JSON object, search take the specific array I'm dealing with ("results"), search that array for the result I'm looking for (via "id"), and then take the array I need ("scores") from that specific result. Then, I use that array as the input for the AmChart code.
I've tested the AmChart code with another sample array (defined in the code) and it works fine. However, when I try to retrieve from localStorage, I am unable to get the chart to display. Any thoughts as to why? The code in question is here:
//retrieve initial results response from localstorage
var search_results = localStorage.getItem("results_response");
//convert retrieved string back to JSON array
var search_results_object = JSON.parse(search_results); //search_results_objects is now a JSON array
var results = search_results_object.results; //pull out results array
//get venue_id of detail page from local storage
var ven_id = localStorage.getItem("country_id");
//search JSON response for venue ID
var chartDataResults = [];
var searchField = "id";
var searchVal = ven_id;
for (var i=0 ; i < results.length ; i++)
{
if (results[i][searchField] == searchVal) {
chartDataResults.push(results[i]);
}
}
var chartDataInput = chartDataResults.scores;
//this all works:
var chartData = chartDataInput;
var chart = new AmCharts.AmSerialChart();
chart.dataProvider = chartData;
chart.categoryField = "country";
chart.rotate = true;
var graph = new AmCharts.AmGraph();
graph.valueField = "score";
graph.type = "column";
chart.addGraph(graph);
var categoryAxis = chart.categoryAxis;
categoryAxis.autoGridCount = false;
categoryAxis.gridCount = chartData.length;
categoryAxis.gridPosition = "start";
categoryAxis.labelRotation = 90;
graph.fillAlphas = 0.8;
chart.write('chartdiv');
Most recently, the dev console in Chrome has told me "Uncaught TypeError: Cannot read property 'length' of undefined". The chart does not render, either. Thoughts? I appreciate any and all help.

Your code fails because of this:
var chartDataInput = chartDataResults.scores;
chartDataResults is an array by your declaration, not an object. According to the amDocs you could pass chartDataResults directly, given that it is in the correct format, eg [{title:"sample 1",value:130},{title:"sample 2",value:26}];.
If so - try to replace:
var chartDataInput = chartDataResults.scores;
//this all works:
var chartData = chartDataInput;
with
var chartData = chartDataResults;
If you chartDataResults array is not in the correct format, you need to construct it in that loop. Something like:
for (var i=0 ; i < results.length ; i++) {
if (results[i][searchField] == searchVal) {
var aResult = new Object();
aResult.title = results[i].title; // or results[i]['title']
aResult.value = parseInt(results[i].value); // or results[i]['value']
chartDataResults.push(aResult);
}
}
I am speculating that in your json objects you have .title and .value. If not - replace them accordingly. If you're not sure :) what properties they have - print them to the console and inspect them.

Javascript table string to array

I have a string that looks like:
<tr><td>Date</td><td>Value</td></tr>
<tr><td>2013-01-01</td><td>231.198</td></tr>
<tr><td>2013-02-01</td><td>232.770</td></tr>
<tr><td>2013-03-01</td><td>232.340</td></tr>
<tr><td>2013-04-01</td><td>231.485</td></tr>
<tr><td>2013-05-01</td><td>231.831</td></tr>
<tr><td>2013-06-01</td><td>232.944</td></tr>
<tr><td>2013-07-01</td><td>233.318</td></tr>
...which is of course essentially a table.
I'd like to dynamically convert this string into an array containing 2 arrays. One of dates, one of values.
[Edited in]
An array of objects with date and values would work too.

The following::
var input = // your string
var output = $(input).slice(1).map(function(i,el) {
var tds = $(el).find("td");
return { "date" : tds.eq(0).text(), "value" : tds.eq(1).text() };
}).get();
...will return an array of objects in this format:
[{"date":"2013-01-01","value":"231.198"}, {"date":"2013-02-01","value":"232.770"}, ... ]
If you'd like each value to be treated as a number you can convert it like so:
return { "date" : tds.eq(0).text(), "value" : +tds.eq(1).text() };
// add the unary plus operator ---------------^
Then the result will be:
[{"date":"2013-01-01","value":231.198}, {"date":"2013-02-01","value":232.77}, ... ]

While you've already accepted an answer, I thought I'd post a plain JavaScript solution (albeit largely because I spent time working on it, before Barmar pointed out that you're willing and able to use jQuery):
function cellContents(htmlStr, what) {
var _table = document.createElement('table');
_table.innerHTML = htmlStr;
var rows = _table.getElementsByTagName('tr'),
text = 'textContent' in document ? 'textContent' : 'innerText',
cells,
matches = {};
for (var w = 0, wL = what.length; w < wL; w++) {
matches[what[w]] = [];
for (var r = 1, rL = rows.length; r < rL; r++) {
cells = rows[r].getElementsByTagName('td');
matches[what[w]].push(cells[w][text]);
}
}
return matches;
}
var str = "<tr><td>Date</td><td>Value</td></tr><tr><td>2013-01-01</td><td>231.198</td></tr><tr><td>2013-02-01</td><td>232.770</td></tr><tr><td>2013-03-01</td><td>232.340</td></tr><tr><td>2013-04-01</td><td>231.485</td></tr><tr><td>2013-05-01</td><td>231.831</td></tr><tr><td>2013-06-01</td><td>232.944</td></tr><tr><td>2013-07-01</td><td>233.318</td></tr>";
console.log(cellContents(str, ['dates', 'values']));
JS Fiddle demo.

For a pure JavaScript solution you can try something like this (assuming str holds your string) :
var arrStr = str.replace(/<td>/g, "").replace(/<tr>/g, "").split("</td></tr>");
var arrObj = [];
var arrData
for (var i = 1; i < arrStr.length - 1; i++) {
arrData = arrStr[i].split("</td>");
arrObj.push({ Date: arrData[0], Value: arrData[1] })
}
It's a burte-force string replacement/split, but at the end arrObj will have array of objects.

if its a valid html table structure, wrap it between table tags, and use jquery to parse it.
then use jquery's selectors to find the columns.
e.g something like this ( pseudo code, havent tried it )
table = $(yourTableString);
dates = table.find("tr td:nth-child(1)");
values = table.find("tr td:nth-child(2)");

Using jQuery:
var table = $('<table>'+str+'</table>');
var result = {};
table.find('tr:gt(0)').each(function () {
var date = $(this).find("td:nth-child(1)").text();
var value = $(this).find("td:nth-child(2)").text();
result[date] = value;
}
:gt(0) is to skip over the header line. This will create an associative array object that maps dates to values. Assuming the dates are unique, this is likely to be more useful than two arrays or an array of objects.

Dynamic creation of JSON object from given parameters

I have a json object of this type:
jsonObject={visits:'3',
v1:{'timestamp':'1231313.311',
'time_year':'2011',
'time_month':'Jan',
'time_date':'15'
'X_value':'X'
},
v2:{'timestamp':'1231313.311',
'time_year':'2011',
'time_month':'Jan',
'time_date':'15'
'X_value':'Y'
},
v3:{'timestamp':'534224242.311',
'time_year':'2011',
'time_month':'Feb',
'time_date':'5',
'X_value':'Z'
}
}
Now i want to convert this into a format like...
JsonObject={'2011':{'Jan':{'15':['X','Y']
},
'Feb':{'5':'Z'
},
}
}
How can i do that...I ve tried iterating over the first JSONObject and taking out year,month,and date values but i am not able to create a new JSONObject from that data.
Thanks in advance.

If you've tried iterating the original object, you're probably not far from done. You just need to create the new object:
var newObject = {};
for(var key in jsonObject) {
if(key == 'visits')
continue;
var year = jsonObject[key]['time_year'];
var month = jsonObject[key]['time_month'];
var date = jsonObject[key]['time_date'];
if(!(year in newObject))
newObject[year] = {};
if(!(month in newObject[year]))
newObject[year][month] = {};
if(!(date in newObject[year][month]))
newObject[year][month][date] = [];
newObject[year][month][date].push(jsonObject[key]['X_value']);
}
JSFiddle demo, outputs into your browser console log.