I am using Axios to execute a GET request to a public API, I need to combine the names if they are the same and add the values up to only show the top 20 (It's a large dataset) based on the highest to lowest amounts(ascending order).
Axios Response
[
{
name: "foo1",
value: "8123.30"
},
{
name: "foo1",
value: "2852.13"
},
{
name: "foo2",
value: "5132.23"
},
{
name: "foo1",
value: "1224.20"
},
{
name: "foo2",
value: "1285.23"
}
1200...
];
Expected Output
[
{ name: "foo1",
value: "12199.63" // from all combined "foo1" amounts in the dataset
},
{
name: "foo2",
value: "6417.46" // from all combined "foo2" amounts in the dataset
},
18..
]
I tried to do something like this....
const fetchData = () => {
return axios.get(url)
.then((response) => response.data)
};
function onlyWhatINeed() {
const newArr = []
return fetchData().then(data => {
const sortedData = data.sort((a, b) => parseFloat(a.value) - parseFloat(b.value));
// I need to loop through the dataset and add all the "values" up
// returning only the top 20 highest values in an array of those objects
newArr.push(sortedData)
})
}
But I am confused as to how to push this data to a new array of the sorted data (top 20 values in ascending order) and use this data in my web application. I am a bit new to creating REST APIs so if you could provide articles and/or resources so I can understand a little more that would be an awesome bonus!
You can combine the entries that share the same name using a map, then sort the map and keep the first twenty elements :
function onlyWhatINeed() {
const newArr = []
return fetchData().then(data => {
let map = new Map();
data.forEach(d => {
if(!map.has(d.name)) {
map.set(d.name, parseFloat(d.value));
} else {
map.set(d.name, map.get(d.name) + parseFloat(d.value));
}
})
return Array.from(map.entries()).sort((a, b) => a.value - b.value).slice(0, 20);
})
}
Since you're dealing with a large dataset, I recommend that you handle this server side instead of offloading the sorting to your clients.
async function fetchData(){
const { data } = await axios.get(url);
let newArr = []
data.forEach((e,i) => {
let index = newArr.findIndex(el => el.name === e.name);
if(index !== -1 ) newArr[index].value += parseFloat(e.value); //add to the value if an element is not unique
if(index === -1 ) newArr.push({...e, value: parseFloat(e.value)}); //push to the array if the element is unique and convert value to float
});
return newArr.sort((a,b) => a.value - b.value).slice(0,20);//returns an array of 20 elements after sorting
}
Please do more research on how to work with arrays and objects in general.
If you happen to already be using lodash, then here's a functional-style solution using lodash chaining. Probably not optimal performance, but could be useful for relatively small datasets.
const _ = require('lodash');
const data = [
{
name: "foo1",
value: "8123.30"
},
{
name: "foo1",
value: "2852.13"
},
{
name: "foo2",
value: "5132.23"
},
{
name: "foo1",
value: "1224.20"
},
{
name: "foo2",
value: "1285.23"
},
{
name: "foo3",
value: "1000.00"
},
{
name: "foo3",
value: "2000.00"
}
];
// 1. convert string values to floats
// 2. group by name
// 3. sum values by name
// 4. sort by descending value
// 5. take top 20
const output =
_(data)
.map(obj => ({
name: obj.name,
value: parseFloat(obj.value)
}))
.groupBy('name')
.map((objs, key) => ({
name: key,
value: _.sumBy(objs, 'value')
}))
.orderBy(['value'], 'desc')
.slice(0, 20)
.value();
console.log('output:', output);
Related
I am having what I think is a pretty trivial problem but somehow I can't find a solution to. I have a response body that looks like this:
{
"sizes": [
{
"43": 35
},
{
"42": 20
},
{
"38": 10
}
]
}
where the keys are shoe sizes and the value is quantity of each size. How do I access the sizes? What I currently have is this:
const sizesArray = response.data.sizes
const arr = Object.values(msizes);
console.log('arr', arr);
arr.map((i,a) => {
console.log('i',i);
console.log('a',a);
})
but i is then again a object {43: 35}
and a is just the index. I want somehow to assign the key to parameter called 'sizes' and the key to a parameter called quantity.
You can use Object.keys, a bit simpler than Object.entries
Example:
const data = { sizes: [{ "43": 35 }, { "42": 20 }, { "38": 10 }] };
const result = data.sizes.map((element, index) => {
let obj = Object.keys(element); // returns an array of keys
let key = obj[0]; // first element is the only key
let quantity = element[key]; // bracket notation, key is an string, not number
console.log("size", key);
console.log("quantity", quantity);
});
You can just iterate the sizes array, using reduce to append the keys of each object to an output array of sizes:
const data = { sizes: [{ "43": 35 }, { "42": 20 }, { "38": 10 }] }
const sizes = data.sizes.reduce((acc, s) => acc.concat(Object.keys(s)), [])
console.log(sizes)
If you want sizes and quantities, you can take a similar approach, just generate an object which accumulates both sets of values:
const data = { sizes: [{ "43": 35 }, { "42": 20 }, { "38": 10 }] }
const { sizes, quantities } = data.sizes
.reduce((acc, s) => {
acc.sizes = acc.sizes.concat(Object.keys(s))
acc.quantities = acc.quantities.concat(Object.values(s))
return acc
},
{ sizes : [], quantities : [] })
console.log(sizes)
console.log(quantities)
You were on the right track :)
Use Object.keys() to get an array of your keys (shoe-sizes). Then use the map()-function to create a new array. Use the index of map() to access the quantity in your response.
const sizesArray = response.data.sizes
const sizes = Object.keys(sizesArray);
const result = sizes.map((element, index) => ({
size: element,
quantity: sizesArray[index]
}));
console.log(result);
if I have an array of strings like:
['person,item,cost,amount',
'John,shoes,200,2']
how could I convert this into an object that resembles:
{
'John':[
{
item:'shoes',
cost:'200',
amount:'2',
totalPriceForItems:'400'
}
If I understand correctly, you may try something like this:
const convert = data => {
const [columnsText, ...items] = data;
const columns = columnsText.split(',');
return items.reduce((acc, text) => {
const { person, ...entries } = Object.fromEntries(text.split(',').map((value, i) => [columns[i], value]));
entries.totalPriceForItems = String(entries.cost * entries.amount);
if(acc[person]) {
acc[person].push(entries);
} else {
acc[person] = [entries];
}
return acc;
}, {});
};
const result = convert([
'person,item,cost,amount',
'John,shoes,200,2',
'Bob,glasses,50,3',
'John,shirts,100,5',
]);
console.log(result);
According to your comment,
I have 8 lines of 'John,shoes,200,2' but with different amounts in the
same array. The 'person,item,cost,amount' is only mentioned once at
the start of the array
What I understand is that you have a csv with headers and multiple rows.
If that is the case, then your data would resemble something like this:
data = [
'person,item,cost,amount',
'John,shoes,200,2',
'Adam,pants,60,1',
'Kelly,skirt,180,2',
'John,skirt,150,3'
]
Then you could consider the following approach, that is generic enough to adapt to different headers, and multiple data rows with repeated keys (person names).
// here, you define a function to transform each row of your data,
// like parsing numeric attributes and calculating the totals
function transform(row) {
row.cost = Number.parseInt(row.cost)
row.amount = Number.parseInt(row.amount)
row.total = row.cost * row.amount
return row
}
// The following logic is generic, and can be used
// to map and aggregate any kind of csv with headers
hdrs = data.shift().split(',').slice(1)
rows = data.map(r => r.split(',')).reduce((acc, [n, ...kvs]) =>
({ ...acc, [n]: [...acc[n] || [], transform(Object.fromEntries(kvs.map((v, i) => [hdrs[i], v])))] }), {})
Output:
{
John: [
{ item: "shoes", cost: 200, amount: 2, total: 400 },
{ item: "skirt", cost: 150, amount: 3, total: 450 }],
Adam: [
{ item: "pants", cost: 60, amount: 1, total: 60 }],
Kelly: [
{ item: "skirt", cost: 180, amount: 2, total: 360 }]
}
I am having difficulties formatting some data. Currently, I receive data in the following structure.
[
{
"q1":"5",
"q2":[
"13",
"12",
],
"q3":"test",
}
]
I essentially need to modify this or even create a new object, that takes the following structure.
[
{
id: 1, //q1
answers: [
{
answer: '5',
},
],
},
{
id: 2, //q2
answers: [
{
answer: '13',
},
{
answer: '12',
},
],
},
{
id: 3, //q3
answers: [
{
answer: 'test',
},
],
},
];
So the id in the above would be obtained by remove the q and getting the number in the first data object. It would then have an answers array that would have an object for each answer.
I have been attempting this but have gotten lost. I don't know if I should use loops, mapping, filters etc. To be honest, the furthest I have got so far is obtaining the keys
var modified = data.map(function(item) {
return Object.keys(item)
})
I have created a JSFiddle where I have been attempting to do this.
Is there any way I can achieve the data I am after?
Many thanks
Please use map function.
const data = {
"q1":"5",
"q2":[
"13",
"12",
],
"q3":"test",
};
const result = Object.keys(data).map(key => {
let item = {id: key.substring(1), answers: []};
if(typeof data[key] === "string")
item.answers.push({answer: data[key]});
else
item.answers = data[key].map(val => ({answer: val}));
return item;
});
console.log(result)
const inputData = [
{
"q1":"5",
"q2":[
"13",
"12",
],
"q3":"test",
}
]
function answerMapper(objVal, id){
return Array.isArray(objVal)
?
{ id, answers: objVal.map(answer => ({ answer }))}
:
{ id, answers: [{answer: objVal }] }
}
function formatObject(obj){
return Object.keys(obj).map((k, i) => answerMapper(obj[k], i+1));
}
const result = inputData.map(obj => formatObject(obj));
// remove flatMap if your inputData has more than one entry
console.log(result.flatMap(x => x));
map over the first element of the data with Object.entries, grab the key and value, create a new answers array and return a new object.
const data = [{
"q1": "5",
"q2": [
"13",
"12",
],
"q3": "test",
}];
const out = Object.entries(data[0]).map(obj => {
const [ key, value ] = obj;
const id = Number(key[1]);
// If the the value is an array
// return a new array of mapped data
// Otherwise return an array containing
// one object
const answers = Array.isArray(value)
? value.map(el => ({ answer: el }))
: [{ answer: value }];
// Return the new object
return { id, answers };
});
console.log(out);
lets create a pure function which accepts the object in the array like so
const processObject = obj => Object.keys(obj).map(id => {
const answer = obj[id];
const answers = Array.isArray(answer) ? answer : [answer]
const answerObjectArray = answers.map(ans => ({
answer: ans
}));
return {
id: +id.substring(1),
answers: answerObjectArray
}
});
const dataArray = [{
"q1": "5",
"q2": [
"13",
"12",
],
"q3": "test",
}];
const output = processObject(dataArray[0]);
console.log(output);
I have an api that return me data in following format:
[
{
"_id": 1567362600000,
"KIDate": "2019-09-02",
"KITools": [
{
"data": 1,
"tool": "A"
},
{
"data": 2,
"tool": "B"
}
]
},
{
"_id": 1567519839316,
"KIDate": "2019-09-01",
"KITools": [
{
"data": 2,
"tool": "A"
},
{
"data": 1,
"tool": "C"
}
]
},
{
"_id": 1567519839317,
"KIDate": "2019-08-31",
"KITools": [
{
"data": 0,
"tool": "C"
}
]
},
]
I want to transform this data to get the following arrays:
Result 1 - [“2019-09-02”,”2019-09-01”,”2019-08-31”]
Result 2 - [ {name: ‘A’, data:[1, 2, 0] }, { name: 'B', data: [2, 0, 0] }, { name: 'C', data: [0, 1, 0]}]
Currently I am able to achieve this by using loops and per-defining variables with the tool name like following and looping the api data to push into this variable.
var result2 = [{
name: 'A',
data: []
}, {
name: 'B',
data: []
}, {
name: 'C',
data: []
}];
But this is not the expected behavior, the tool names can change and I have to figure that out dynamically based on the data returned by the api.
What is the best way to achieve this without looping like crazy.
You could use reduce method to get the result with array of dates and object of values for each tool.
const data = [{"_id":1567362600000,"KIDate":"2019-09-02","KITools":[{"data":1,"tool":"A"},{"data":2,"tool":"B"}]},{"_id":1567519839316,"KIDate":"2019-09-01","KITools":[{"data":2,"tool":"A"},{"data":1,"tool":"C"}]},{"_id":1567519839317,"KIDate":"2019-08-31","KITools":[{"data":0,"tool":"C"}]}]
const result = data.reduce((r, {KIDate, KITools}, i) => {
r.dates.push(KIDate);
KITools.forEach(({data: dt, tool}) => {
if(!r.values[tool]) r.values[tool] = Array(data.length).fill(0);
r.values[tool][i] = dt
})
return r;
}, {dates: [], values: {}})
console.log(result)
You can use reduce and forEach with Set and Map
Initialize accumulator as object with dates and data key, dates is a Set and data is Map
For every element add the KIDate to dates key,
Loop over KITools, check if that particular too exists in data Map if it exists update it's value by adding current values to id, if not set it's value as per current values
let data = [{"_id": 1567362600000,"KIDate": "2019-09-02","KITools": [{"data": 1,"tool": "A"},{"data": 2,"tool": "B"}]},{"_id": 1567519839316,"KIDate": "2019-09-01","KITools": [{"data": 2,"tool": "A"},{"data": 1,"tool": "C"}]},{"_id": 1567519839317,"KIDate": "2019-08-31","KITools": [{"data": 0,"tool": "C"}]},]
let final = data.reduce((op,{KIDate,KITools})=>{
op.dates.add(KIDate)
KITools.forEach(({data,tool})=>{
if(op.data.has(data)){
op.data.get(data).data.push(tool)
} else{
op.data.set(data, {name: data, data:[tool]})
}
})
return op
},{dates:new Set(),data: new Map()})
console.log([...final.dates.values()])
console.log([...final.data.values()])
The result1 array can be obtained via a direct .map(). To build the result2 array will require additional work - one approach would be to do so via .reduce() as detailed below:
const data=[{"_id":1567362600000,"KIDate":"2019-09-02","KITools":[{"data":1,"tool":"A"},{"data":2,"tool":"B"}]},{"_id":1567519839316,"KIDate":"2019-09-01","KITools":[{"data":2,"tool":"A"},{"data":1,"tool":"C"}]},{"_id":1567519839317,"KIDate":"2019-08-31","KITools":[{"data":0,"tool":"C"}]}];
const result1 = data.map(item => item.KIDate);
const result2 = data.reduce((result, item) => {
item.KITools.forEach(kitool => {
/* For current item, search for matching tool on name/tool fields */
let foundTool = result.find(i => i.name === kitool.tool);
if (foundTool) {
/* Add data to data sub array if match found */
foundTool.data.push(kitool.data);
} else {
/* Add new tool if no match found and init name and data array */
result.push({
name: kitool.tool,
data: [kitool.data]
});
}
});
return result;
}, []).map((item, i, arr) => {
/* Second phase of processing here to pad the data arrays with 0 values
if needed */
for (let i = item.data.length; i < arr.length; i++) {
item.data.push(0);
}
return item;
});
console.log('result1:', result1);
console.log('result2:', result2);
I've been playing around with Lodash and not getting close to a solution that doesn't involve a lot of extra looping and overhead.
data: [
{
name: "FirstResult", values: [
{
value: { NameCount: 1, OtherCount: 1 },
date: 2019-05-15T07:00:00+0000
},
{
value: { NameCount: 1 },
date: 2019-05-16T07:00:00+0000
}
]
},
{
name: "SecondResult",
values: [
{
value: { NameCount: 1 },
date: 2019-05-15T07:00:00+0000
},
{
value: { BuyCount: 2, SellCount: 1 },
date: 2019-05-16T07:00:00+0000
}
]
}
]
I'd like to flatten this and have it combined and aggregated by using the date as the key returning some configuration like:
[
{ date: 2019-05-15T07:00:00+0000, values: { NameCount: 2, OtherCount: 1 } },
{ date: 2019-05-16T07:00:00+0000, values: { NameCount: 1, BuyCount: 2, SellCount: 1 } }
]
Or even just a flat object array is fine like:
[
{ date: 2019-05-15T07:00:00+0000, NameCount: 2, OtherCount: 1 },
{ date: 2019-05-16T07:00:00+0000, NameCount: 1, BuyCount: 2, SellCount: 1 }
]
Does anyone have any ideas on how to do this with either a Lodash or Vanilla solution?
You can use a lodash's chain to flatten, group by the date, and then map and merge each group to a single object:
const fn = data => _(data)
.flatMap('values') // flatten to array of objects
.groupBy(o => o.date.toISOString()) // group by the iso representation
.map(group => { // map the groups by merging, and converting to require format
const { date, value } = _.mergeWith({}, ...group, (objValue, srcValue) =>
_.isNumber(objValue) ? objValue + srcValue : undefined // combine numeric values
)
return {
date,
...value,
}
})
.value()
const data = [{"name":"FirstResult","values":[{"value":{"NameCount":1,"OtherCount":1},"date": new Date("2019-05-15T07:00:00.000Z")},{"value":{"NameCount":1},"date": new Date("2019-05-16T07:00:00.000Z")}]},{"name":"SecondResult","values":[{"value":{"NameCount":1},"date":new Date("2019-05-15T07:00:00.000Z")},{"value":{"BuyCount":2,"SellCount":1},"date": new Date("2019-05-16T07:00:00.000Z")}]}]
const result = fn(data)
console.log(result)
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.11/lodash.js"></script>
Or you can use _.flow() to generate the function (I'm using lodash/fp here):
const { flow, flatMap, groupBy, map, mergeAllWith, cond, isNumber, add } = _
const fn = flow(
flatMap('values'), // flatten to array of objects
groupBy(o => o.date.toISOString()), // group by the iso representation
map(mergeAllWith(cond([[isNumber, add]]))), // combine numeric values
map(({ date, value }) => ({ date, ...value })) // format the objects
)
const data = [{"name":"FirstResult","values":[{"value":{"NameCount":1,"OtherCount":1},"date": new Date("2019-05-15T07:00:00.000Z")},{"value":{"NameCount":1},"date": new Date("2019-05-16T07:00:00.000Z")}]},{"name":"SecondResult","values":[{"value":{"NameCount":1},"date":new Date("2019-05-15T07:00:00.000Z")},{"value":{"BuyCount":2,"SellCount":1},"date": new Date("2019-05-16T07:00:00.000Z")}]}]
const result = fn(data)
console.log(result)
<script src='https://cdn.jsdelivr.net/g/lodash#4(lodash.min.js+lodash.fp.min.js)'></script>
Here is pure ES6 solution based on Array.reduce and Array.forEach for the object keys:
const data = [{"name":"FirstResult","values":[{"value":{"NameCount":1,"OtherCount":1},"date": new Date("2019-05-15T07:00:00.000Z")},{"value":{"NameCount":1},"date": new Date("2019-05-16T07:00:00.000Z")}]},{"name":"SecondResult","values":[{"value":{"NameCount":1},"date":new Date("2019-05-15T07:00:00.000Z")},{"value":{"BuyCount":2,"SellCount":1},"date": new Date("2019-05-16T07:00:00.000Z")}]}]
let result = data.reduce((r, { values }) => {
values.forEach(({ value, date }) => {
let keys = Object.keys(value), d = date.toISOString()
r[d] = r[d] || Object.assign({}, ...keys.map(x => ({ date: d, [x]: 0 })))
keys.forEach(k => r[d][k] = (r[d][k] || 0) + value[k])
})
return r
}, {})
console.log(Object.values(result))
The main idea is to get the keys of the value object white iterating and compose an object with them while grouping by the date. Then last forEach just sums each result object value.