Export JSON data to CSV file for Excel - javascript

Given an array of data objects
const data = [{
"id": "CT20",
"type": "a11y-unknown",
"urls": ["https://www.example.com/test/"]
},
{
"id": "BC192",
"type": "a11y-true",
"urls": [
"https://www.example.com/something/",
"https://www.example.com/another-thing/"
]
}
]
I'm trying to convert the objects to a CSV file that can be imported to Excel so that it shows as:
id | type | urls
CT20 | a11y-unknown| https://www.example.com/test/
I'm using the following to get the keys:
const keys = Object.keys(data[0]);
then map over the data like so:
const commaSeparatedString = [keys.join(","),data.map(row => keys.map(key => row[key]).join(",")).join("\n")].join("\n");
However, this returns the following:
'id,type,urls\nCT20,a11y-unknown,https://www.example.com/test/\nBC192,a11y-true,https://www.example.com/something/,https://www.example.com/another-thing/'
When imported to Excel as a CSV file, and delimited with \, it appears like this:
How can I correctly map the objects so that they are delimited and line break after each set of urls?
const data = [{
"id": "CT20",
"type": "a11y-unknown",
"urls": ["https://www.example.com/test/"]
},
{
"id": "BC192",
"type": "a11y-true",
"urls": [
"https://www.example.com/something/",
"https://www.example.com/another-thing/"
]
}
]
const keys = Object.keys(data[0]);
const commaSeparatedString = [keys.join(","),data.map(row => keys.map(key => row[key]).join(",")).join("\n")].join("\n");
console.log(commaSeparatedString)

You need to have fixed number of columns. So either JSON.stringify the urls array, or designate columns such as url1, url2, url3...
EDIT: naturally if you don't escape commas by enclosing them in quotes, it will break the CSV. Genrally speaking you should use a library for parsing CSV such as papaparse.
const data = [{
"id": "CT20",
"type": "a11y-unknown",
"urls": ["https://www.example.com/test/"]
},
{
"id": "BC192",
"type": "a11y-true",
"urls": [
"https://www.example.com/something/",
"https://www.example.com/another-thing/"
]
}
]
var keys = Object.keys(data[0]);
var arr = [keys, ...data.map(row => keys.map(key => {
return typeof row[key] === "string" ? row[key] : JSON.stringify(row[key])
}))];
// don't!
// .join(",")).join("\n")].join("\n");
// instead
var csv = Papa.unparse(arr);
console.log(csv)
<script src="https://cdnjs.cloudflare.com/ajax/libs/PapaParse/5.1.0/papaparse.min.js"></script>

As a general rule, the csv must be delimited with the same delimiter used when importing it, also for reliability all the fields should be included into quotes, so since in your case the delimiter is \, your attempt can be rewritten as below:
const data = [{
"id": "CT20",
"type": "a11y-unknown",
"urls": ["https://www.example.com/test/"]
},
{
"id": "BC192",
"type": "a11y-true",
"urls": [
"https://www.example.com/something/",
"https://www.example.com/another-thing/"
]
}
]
const keys = Object.keys(data[0]);
const commaSeparatedString = [keys.join("\\"),data.map(row => keys.map(key => `"${[row[key]].flat().join()}"`).join("\\")).join("\n")].join("\n");
console.log(commaSeparatedString)

Related

Looping through different files

I am trying to loop through some files but I have a problem when trying to put an if and get some data from there.
I have a folder named Channels with 654 txt files each have 1 json with 100 json nested in an array
Also I have a list of phone numbers which I take from a csv (2500 numbers)
What I need to do it's check if a phone number from the list matches with channel from any json of the 654 text files
I made this code but the problem here its if I just put 1 phone number which matches with 8 json, it's works fine, but If I check the same phone number adding other 3 new numbers to the list, the result its completely different and only matches with just one json
const fs = require('fs');
let csvFormatPhonesList = fs.readFileSync(`${__dirname}\\PhonesNumbersFile.csv`, 'utf8'); //<= This came one below the other like '+549XXXXXXXXX\n+549XXXXXXXXX\n+549XXXXXXXXX'
let phonesNumbersSplited = csvFormatPhonesList.split('\n');
const channelsFound = [];
const numbersOfFilesChannels = 654; //I have 654 txt. Each file have 100 jsons
for(let i=0; i<numbersOfFilesChannels; i++) {
let getOneSingleFile = JSON.parse(fs.readFileSync(`${__dirname}\\Channels\\channels${i}.txt`, 'utf8'));
phonesNumbersSplited.forEach((phone) => {
getOneSingleFile.channels.forEach((channel) => {
if (channel.friendly_name.includes(phone)){
let parseAttributes = JSON.parse(channel.attributes);
if (parseAttributes.long_lived == true) {
channelsFound.push(channel);
}
}
})
});
}
//****File of json example****//
//The array " channels " has 100 jsons inside
"channels": [
{
"unique_name": null,
"members_count": 1,
"date_updated": "2021-09-28T18:54:59Z",
"friendly_name": "whatsapp_+549XXXXXXXXX",
"created_by": "system",
"account_sid": "ACXXXXXXXXXXX",
"url": "https://chat.twilio.com/v2/Services/ISXXXXX/Channels/CHXXXXXXXX",
"date_created": "2021-09-28T13:06:16Z",
"sid": "CHXXXXXXXX",
"attributes": "{\"serviceNumber\":\"whatsapp_+549XXXXXXXXX\",\"task_sid\":\"WTXXXXX\",\"from\":\"whatsapp:+549XXXXXXXXX\",\"forwarding\":true,\"proxySession\":\"KCXXXXX\",\"twilioNumber\":\"whatsapp:+549XXXXXXXXX\",\"channel_type\":\"whatsapp\",\"status\":\"INACTIVE\",\"long_lived\":true}",
"service_sid": "ISXXXXX",
"type": "private",
"messages_count": 1,
"links": {
"webhooks": "https://chat.twilio.com/v2/Services/ISXXXXX/Channels/CHXXXXXXXX/Webhooks",
"messages": "https://chat.twilio.com/v2/Services/ISXXXXX/Channels/CHXXXXXXXX/Messages",
"invites": "https://chat.twilio.com/v2/Services/ISXXXXX/Channels/CHXXXXXXXX/Invites",
"members": "https://chat.twilio.com/v2/Services/ISXXXXX/Channels/CHXXXXXXXX/Members",
"last_message": "https://chat.twilio.com/v2/Services/ISXXXXX/Channels/CHXXXXXXXX/Messages/IMXXXXXXXXXX"
}
}
],
"meta": {
"page": 0,
"page_size": 100,
"first_page_url": "https://chat.twilio.com/v2/Services/ISXXXXX/Channels?PageSize=100&Page=0",
"previous_page_url": null,
"url": "https://chat.twilio.com/v2/Services/ISXXXXX/Channels?PageSize=100&Page=0",
"next_page_url": "https://chat.twilio.com/v2/Services/ISXXXXX/Channels?PageSize=100&Page=1",
"key": "channels"
}

How to Turn a Multiple Array Object into Query String Parameters in JavaScript

I have the following object below with multiple arrays.
{
"services": [
{
"id": "100",
"name": "PIX"
},
{
"id": "200",
"name": "Rendimentos"
}
],
"channels": [
{
"id": "300",
"name": "Chat"
}
]
}
The idea is to generate query strings, something like that.
services=100&services=200&channels=300
I know you can do it with map and join, but I would know if it was with a pure object, now this format below, I'm confused
You can use URLSearchParams() API.
Iterate your data and append key/value pairs or map an entries array to pass to the constructor
I have no idea what determines the expected output you have shown from the data displayed so am using a simpler data structure for demonstration purposes.
You can combine with URL() API to create full url string as shown below also
const data = [
{name:'foo', value:10},
{name:'bar', value:20}
]
// Loop and append key/values
const params = new URLSearchParams();
data.forEach(e => params.append(e.name, e.value));
console.log('params:', params.toString());
// Alternate approach passing entries array to constructor
const params2 = new URLSearchParams(data.map(e => [e.name,e.value]));
console.log('params2:',params2.toString())
//Adding to a URL
const url = new URL('http://example.com')
url.search = params
console.log('Full url:',url)
Using the updated array data in question:
const data={services:[{id:"100",name:"PIX"},{id:"200",name:"Rendimentos"}],channels:[{id:"300",name:"Chat"}]};
const entries = [];
Object.entries(data).forEach(([k,arr])=> arr.forEach(({id}) => entries.push([k,id])));
const params = new URLSearchParams(entries);
const url = new URL('http://example.com')
url.search = params;
console.log(url)
Looks like you're hung up on trying to iterate an object with map() or join(), which you can't do directly. Instead you can use Object.entries to convert the object into an array and iterate that. Since there is a nested map() you can flat() it before join()
let obj = {
"services": [{
"id": "100",
"name": "PIX"
},
{
"id": "200",
"name": "Rendimentos"
}
],
"channels": [{
"id": "300",
"name": "Chat"
}]
}
let queryString = Object.entries(obj).map(s => s[1].map(e => `${s[0]}=${e.id}`)).flat().join('&')
console.log(queryString)

How to convert JSON string having multiple rows to single row using javascript function

I have an output of REST API in following JSON format:
I need to convert the format to flat format so it can be passed as input to another API call.
{
"result": {
"data": [
{
"data": 2.824315071105957,
"dateTime": "2019-09-10T11:32:05.220Z",
"device": { "id": "b3" },
"diagnostic": { "id": "DiagnosticAccelerationForwardBrakingId" },
"controller": "ControllerNoneId",
"version": "00000000000363b0",
"id": "a5UyPzhknSC-N2wtLBph3BA"
},
{
"data": 0,
"dateTime": "2019-09-10T11:32:05.220Z",
"device": { "id": "b3" },
"diagnostic": { "id": "DiagnosticAccelerationSideToSideId" },
"controller": "ControllerNoneId",
"version": "00000000000363b1",
"id": "a5UyPzhknSC-N2wtLBph3BQ"
},
// ... 1000's of rows like this
]
}
}
I need to convert it in below format using a java-script
Desired format:
{"result":{ "data":[{"id":"b3","dateTime":"2019-09- 10T11:32:05.220Z","DiagnosticAccelerationSideToSideId":0,"DiagnosticAccelerationForwardBrakingId ":2.824315071105957},...
The rows needs to be merged with primary key as combination of ID and dateTime attributes. Please note the diagnostic id value becomes key for the required format and data value is the value of the key.
Is there any way to convert this JSON to above flat format.
Need to convert JSON having many rows for single data entry to single row format. Need one java-script function that can accept a string of rows format and convert or merge it and return the string in desired format
function String mergeRows(String flatDataJSONString) {
...
}
If the items are ordered (meaning i and i+1 are merged) than iterate with jumps of i += 2;
If its not ordered or the amount of items to be merged can be > 2 you use an object with unique key composed of the id and date, and override its data whenever a record match this key:
function merger (jsonStr) {
// convert str to obj
const jsonObj = JSON.parse(jsonStr);
const dataObj = {};
for (let i = 0; i < jsonObj.result.length; i++) {
const item = jsonObj.result[i];
// use unique key to merge by
const itemUniqueKey = item.device.id + item.dateTime;
// take last value or create empty object if not exists
const existingItem = dataObj[itemUniqueKey] || {};
// add some logic to merge item with existingItem as you need
...
// set the result back to dataObj to be used on next merges
dataObj[itemUniqueKey] = [merge result of item and existing item];
}
// take dataObj values, you don't need the keys any more
const dataArr = Object.values(dataObj);
const finalResult = {
result: {
data: dataArr
}
}
// convert back to json
return JSON.stringify(finalResult);
}
As stated in the comment you want first to have a clean json definition in order to stringify it. Please get to the following definition of your JSON first:
const json = {
"result": [
{
"data": 2.824315071105957,
"dateTime": "2019-09-10T11:32:05.220Z",
"device": { "id": "b3" },
"diagnostic": { "id": "DiagnosticAccelerationForwardBrakingId" },
"controller": "ControllerNoneId",
"version": "00000000000363b0",
"id": "a5UyPzhknSC-N2wtLBph3BA"
},
{
"data": 0,
"dateTime": "2019-09-10T11:32:05.220Z",
"device": { "id": "b3" },
"diagnostic": { "id": "DiagnosticAccelerationSideToSideId" },
"controller": "ControllerNoneId",
"version": "00000000000363b1",
"id": "a5UyPzhknSC-N2wtLBph3BQ"
}]
};
and then you will be able to perform like hereafter :
JSON.stringify(json)
Hope this helps !

how to transform json data into table

I have the below api call that returns data in JSON:
https://xama-was-service.herokuapp.com/api/socialone/databoards/10042?1=2019-02-01T00:00:00.000Z&2=test
That returns data as below:
[
[
{
"Empid": 2326,
"Empname": "Sam Smith",
"AbsenceId": 12840,
"Comment": "a001t000004FQgHAAW",
"AbsenceStartDate": "2019-05-31T00:00:00.000Z",
"AbsenceEndDate": "2019-05-31T00:00:00.000Z",
"JobId": 400004,
"AbsenceRequestId": ""
},
{
"Empid": 3387,
"Empname": "Joe bloggs",
"AbsenceId": 12842,
"Comment": "a001t000004FK67AAG",
"AbsenceStartDate": "2019-06-06T00:00:00.000Z",
"AbsenceEndDate": "2019-06-10T00:00:00.000Z",
"JobId": 700004,
"AbsenceRequestId": ""
}
]
]
I would like to move this into excel and also power bi but i cannot transform it into a table?
Can anyone advise how to format the returned data into a table or what code to use on the original call to help with this?
ideal end product would be as below but not sure how to achieve?
Thanks.
This will parse your data into a comma delimited string (CSV).
You just need to separate each row element with a comma ,, and each row with a new line character \n. Excel knows this format, though sometimes you may need to use the text to columns function to let it know the data is comma delimited.
const data = [
[
{
"Empid": 2326,
"Empname": "Sam Smith",
"AbsenceId": 12840,
"Comment": "a001t000004FQgHAAW",
"AbsenceStartDate": "2019-05-31T00:00:00.000Z",
"AbsenceEndDate": "2019-05-31T00:00:00.000Z",
"JobId": 400004,
"AbsenceRequestId": ""
},
{
"Empid": 3387,
"Empname": "Joe bloggs",
"AbsenceId": 12842,
"Comment": "a001t000004FK67AAG",
"AbsenceStartDate": "2019-06-06T00:00:00.000Z",
"AbsenceEndDate": "2019-06-10T00:00:00.000Z",
"JobId": 700004,
"AbsenceRequestId": ""
}
]
]
window.generateCSV = function () {
let CSVData = ''
// set the column names
for (const value of Object.keys(data[0][0])) {
CSVData = CSVData.concat(value + ',')
}
CSVData = CSVData.slice(0, CSVData.length - 1)
CSVData = CSVData.concat('\n')
// parse the data
for (const tbl of data) {
for (const row of tbl) {
for (const value of Object.values(row)) {
CSVData = CSVData.concat(value + ',')
}
CSVData = CSVData.slice(0, CSVData.length - 2)
CSVData = CSVData.concat('\n')
}
}
document.getElementById("csvdata").innerText = CSVData
}
<input type="button" value="generateCSV" onclick="generateCSV()">
<div id="csvdata">
</div>
After saving the output string to a .txt or .csv through notepad, I can open in excel to get this.

How to extract multiple hashtags from a JSON object?

I am trying to extract "animal" and "fish" hashtags from the JSON object below. I know how to extract the first instance named "animal", but I have no idea how to extract both instances. I was thinking to use a loop, but unsure where to start with it. Please advise.
data = '{"hashtags":[{"text":"animal","indices":[5110,1521]},
{"text":"Fish","indices":[122,142]}],"symbols":[],"user_mentions":
[{"screen_name":"test241","name":"Test
Dude","id":4999095,"id_str":"489996095","indices":[30,1111]},
{"screen_name":"test","name":"test","id":11999991,
"id_str":"1999990", "indices":[11,11]}],"urls":[]}';
function showHashtag(data){
i = 0;
obj = JSON.parse(data);
console.log(obj.hashtags[i].text);
}
showHashtag(data);
Use Array.prototype.filter():
let data = '{"hashtags":[{"text":"animal","indices":[5110,1521]},{"text":"Fish","indices":[122,142]}],"symbols":[],"user_mentions":[{"screen_name":"test241","name":"Test Dude","id":4999095,"id_str":"489996095","indices":[30,1111]}, {"screen_name":"test","name":"test","id":11999991, "id_str":"1999990", "indices":[11,11]}],"urls":[]}';
function showHashtag(data){
return JSON.parse(data).hashtags.filter(e => /animal|fish/i.test(e.text))
}
console.log(showHashtag(data));
To make the function reusable, in case you want to find other "hashtags", you could pass an array like so:
function showHashtag(data, tags){
let r = new RegExp(tags.join("|"), "i");
return JSON.parse(data).hashtags.filter(e => r.test(e.text))
}
console.log(showHashtag(data, ['animal', 'fish']));
To get only the text property, just chain map()
console.log(showHashtag(data, ['animal', 'fish']).map(e => e.text));
or in the function
return JSON.parse(data).hashtags
.filter(e => /animal|fish/i.test(e.text))
.map(e => e.text);
EDIT:
I don't really get why you would filter by animal and fish if all you want is an array with ['animal', 'fish']. To only get the objects that have a text property, again, use filter, but like this
let data = '{"hashtags":[{"text":"animal","indices":[5110,1521]},{"text":"Fish","indices":[122,142]}],"symbols":[],"user_mentions":[{"screen_name":"test241","name":"Test Dude","id":4999095,"id_str":"489996095","indices":[30,1111]}, {"screen_name":"test","name":"test","id":11999991, "id_str":"1999990", "indices":[11,11]}],"urls":[]}';
function showHashtag(data){
return JSON.parse(data).hashtags
.filter(e => e.text)
.map(e => e.text);
}
console.log(showHashtag(data));
For me, Lodash can be of great use here, which have different functions in terms of collections. For your case i'd use _.find function to help check the array and get any of the tags with the creteria passed in as second argument like so:
.find(collection, [predicate=.identity], [fromIndex=0])
source npm package
Iterates over elements of collection, returning the first element
predicate returns truthy for. The predicate is invoked with three
arguments: (value, index|key, collection).
with your case this should work
var data = '{ "hashtags": [ { "text": "animal", "indices": [ 5110, 1521 ] }, { "text": "Fish", "indices": [ 122, 142 ] } ], "symbols": [], "user_mentions": [ { "screen_name": "test241", "name": "Test \n Dude", "id": 4999095, "id_str": "489996095", "indices": [ 30, 1111 ] }, { "screen_name": "test", "name": "test", "id": 11999991, "id_str": "1999990", "indices": [ 11, 11 ] } ], "urls": [] }';
var obj = JSON.parse(data);
_.find(obj.hashtags, { 'text': 'animal' });
// => { "text": "animal", "indices": [ 5110, 1521 ] }
For simple parsing like this one, I would use the plain old obj.forEach() method, it is more readable and easy to understand, especially for javascript beginner.
obj = JSON.parse(data).hashtags;
obj.forEach(function(element) {
console.log(element['text']);
});

Categories

Resources