Develop Reference

NODEJS - Prevent duplicate keys from being added into an object array

I have a function I am trying to use to not add duplicates (Later on will combine)
function arrayCombine(arrayOfValues, arrayOfValues2) {
for (var arrName in arrayOfValues2) {
if (arrayOfValues.indexOf(arrName)==-1) arrayOfValues.push([arrName, arrayOfValues2[arrName]]);
}
return arrayOfValues;
}
The arrays are lets say:
arrayOfValues
[
[ 'test', 11 ],
[ 'test2', 13 ],
[ 'test3', 16 ],
]
arrayOfValues2
[
[ 'test4', 12 ],
[ 'test2', 25 ],
]
When I try to combine these, it does NOT remove the duplicate test2 here. It pushes it anyways.
This does not occur if the number does not exist so I assume when I'm checking for INDEXOF, there has to be a way to check for only the named value and not the numbered value too. What I mean is:
function arrayCombine(arrayOfValues, arrayOfValues2) {
for (var arrName in arrayOfValues2) {
if (arrayOfValues.indexOf(arrName)==-1) arrayOfValues.push(arrName);
}
return arrayOfValues;
}
Did work originally.
How can I have it only 'check' the name? In the future I will also combine but for now I just want to make sure no duplicate names get added.

Since objects only allow unique keys it may be simpler to use one to collate your data from both arrays. By concatenating the arrays, and then reducing over them to create an object, you can add/combine each nested arrays values as you see fit. Then, to get an array back from the function use Object.values on the object.
const arr1=[["test",11],["test2",13],["test3",16]],arr2=[["test4",12],["test2",25]];
// Accepts the arrays
function merge(arr1, arr2) {
// Concatentate the arrays, and reduce over that array
const obj = arr1.concat(arr2).reduce((acc, c) => {
// Destructure the "key" and "value" from the
// nested array
const [ key, value ] = c;
// If the "key" doesn't exist on the object
// create it and assign an array to it, setting
// the second element to zero
acc[key] ??= [ key, 0 ];
// Increment that element with the value
acc[key][1] += value;
// Return the accumulator for the next iteration
return acc;
}, {});
// Finally return only the values of the object
// which will be an array of arrays
return Object.values(obj);
}
console.log(merge(arr1, arr2));
Additional documentation
Logical nullish assignment
Destructuring assignment

Merge Javascript Objects WITH Same Key AND Include Duplicate Values WHERE Key Matches

I am trying to merge some JSON data sets BY key value WHILE including duplicate values WHERE the key matches.
I have tried this quite a bit now but can't seem to produce the object that I need.
Object 1
[
{"userId":"1",
"email":"email1#gmail.com"
},
{"userId":"2",
"email":"email2#gmail.com"
}
]
Object 2
[
{"id":"1abc",
"listingId":"4def",
"userId":"2"
},
{"id":"2abc",
"listingId":"2def",
"userId":"1"
},
{"id":"3abc",
"listingId":"3def",
"userId":"2"
}
]
I need to merge these objects in a way that looks like this:
Desired Output
[
{"id":"1abc",
"listingId":"4def",
"userId":"2",
"email":"email2#gmail.com"
},
{"id":"2abc",
"listingId":"2def",
"userId":"1",
"email":"email1#gmail.com"
},
{"id":"3abc",
"listingId":"3def",
"userId":"2",
"email":"email2#gmail.com"
}
]
Problems I am Experiencing
I am able to merge the data sets successfully using a function that looks like this:
function merge(a, b, key) {
function x(a) {
a.forEach(function (b) {
if (!(b[key] in obj)) {
obj[b[key]] = obj[b[key]] || {};
array.push(obj[b[key]]);
}
Object.keys(b).forEach(function (k) {
obj[b[key]][k] = b[k];
});
});
}
var array = [],
obj = {};
x(a);
x(b);
return array;
}
https://stackoverflow.com/a/35094948/1951144
But it produces results that look like this:
[
{"id":"1abc",
"listingId":"4def",
"userId":"2",
"email":"email2#gmail.com"
},
{"id":"2abc",
"listingId":"2def",
"userId":"1",
"email":"email1#gmail.com"
}
]
Is there a way to use the above function WHILE keeping AND including the duplicate values where my keys match?

For each element in arr2, create a new element containing the props of the item from arr2, and the email of the corresponding entry in arr1.
let arr1 = [
{"userId":"1",
"email":"email1#gmail.com"
},
{"userId":"2",
"email":"email2#gmail.com"
}
];
let arr2 = [
{"id":"1abc",
"listingId":"4def",
"userId":"2"
},
{"id":"2abc",
"listingId":"2def",
"userId":"1"
},
{"id":"3abc",
"listingId":"3def",
"userId":"2"
}
];
let output = arr2.map(a2 => ({...a2, email: arr1.find(a1 => a1.userId === a2.userId)?.email}));
console.log(output);

This solution works even if the key isn't known yet. .flatMap() both arrays and pass in the desired key (in example it's "userId"). Use Object.entries() on each object so they will be an array of pairs.
[{A1: A1v}, {A2: A2v},...]
// into
[[A1, A1v], [A2, A2v],...]
.flatMap() the second array and on each iteration .flatMap() the first array. Then compare the given key ("userID") with the key of each object from the second array ("a") AND the value of that key and the value of the key of the object in the first array.
a === key && av === bv
If both criteria are meet then merge those objects and return it, otherwise...
? {...objA, ...objB}
return an empty array, which ultimately results to nothing since .flatMap() flattens one level of arrays.
: []
const arrA=[{userId:"1",email:"email1#gmail.com"},{userId:"2",email:"email2#gmail.com"}];const arrB=[{id:"1abc",listingId:"4def",userId:"2"},{id:"2abc",listingId:"2def",userId:"1"},{id:"3abc",listingId:"3def",userId:"2"}];
function masterKey(primary, key, secondary) {
let result = secondary.flatMap(objB => Object.entries(objB).flatMap(([b, bv]) =>
primary.flatMap(objA => Object.entries(objA).flatMap(([a, av]) =>
a === key && av === bv ? {...objA, ...objB} : []))));
return result;
}
console.log(masterKey(arrA, "userId", arrB));

what is the best way to convert the values in this data set from strings to int

This is my dataset:
data_p = [
[ {"key":"Device_Model","value":"test_model"},
{"key":">20MB/30","value":"11"},
{"key":">200MB/30","value":"33"},
{"key":">2048MB/30","value":"10"},
{"key":">5120MB/30","value":"55"},
{"key":">10240MB/30","value":"10"}
],
[{"key":"Device_Model","value":"0P6B670"},
{"key":">20MB/30","value":"9"},
{"key":">200MB/30","value":"8"},
{"key":">2048MB/30","value":"2"},
{"key":">5120MB/30","value":"23"},
{"key":">10240MB/30","value":"23"}],
[{"key":"Device_Model","value":"0P6B680"},
{"key":">20MB/30","value":"1"},
{"key":">200MB/30","value":"23"},
{"key":">2048MB/30","value":"23"},
{"key":">5120MB/30","value":"23"},
{"key":">10240MB/30","value":"23"}],
[{"key":"Device_Model","value":"0P6B810"},
{"key":">20MB/30","value":"5"},
{"key":">200MB/30","value":"4"},
{"key":">2048MB/30","value":"1"},
{"key":">5120MB/30","value":"1"},
{"key":">10240MB/30","value":"1"}],
[ {"key":"Device_Model","value":"0P6B900"},
{"key":">20MB/30","value":"4"},
{"key":">200MB/30","value":"4"},
{"key":">2048MB/30","value":"1"},
{"key":">5120MB/30","value":"23"},
{"key":">10240MB/30","value":"23"}]]
This is the way I have done, but I am wondering is reduce, map or filter a better way to do it. I mean should I be using a for loop inside a forEach function? I also don't like the i<6 as this is very static what if there was more than 6 in the dataset?
data_p.forEach(function(d){for(i=1; i<6; i++){console.log(d[i].value); d[i].value=+d[i].value}})
This will give me:
data_p = [
[ {"key":"Device_Model","value":"test_model"},
{"key":">20MB/30","value":11},
{"key":">200MB/30","value":33},
{"key":">2048MB/30","value":10},
{"key":">5120MB/30","value":55},
{"key":">10240MB/30","value":10}
],
...
So the question is is there a better way to achieve this?

Iterate all elements. Check if value is numeric (isNaN) then update the property
var data_p = [
[ {"key":"Device_Model","value":"test_model"},
{"key":">20MB/30","value":"11"},
{"key":">200MB/30","value":"33"},
{"key":">2048MB/30","value":"10"},
{"key":">5120MB/30","value":"55"},
{"key":">10240MB/30","value":"10"}
],
[{"key":"Device_Model","value":"0P6B670"},
{"key":">20MB/30","value":"9"},
{"key":">200MB/30","value":"8"},
{"key":">2048MB/30","value":"2"},
{"key":">5120MB/30","value":"23"},
{"key":">10240MB/30","value":"23"}],
[{"key":"Device_Model","value":"0P6B680"},
{"key":">20MB/30","value":"1"},
{"key":">200MB/30","value":"23"},
{"key":">2048MB/30","value":"23"},
{"key":">5120MB/30","value":"23"},
{"key":">10240MB/30","value":"23"}],
[{"key":"Device_Model","value":"0P6B810"},
{"key":">20MB/30","value":"5"},
{"key":">200MB/30","value":"4"},
{"key":">2048MB/30","value":"1"},
{"key":">5120MB/30","value":"1"},
{"key":">10240MB/30","value":"1"}],
[ {"key":"Device_Model","value":"0P6B900"},
{"key":">20MB/30","value":"4"},
{"key":">200MB/30","value":"4"},
{"key":">2048MB/30","value":"1"},
{"key":">5120MB/30","value":"23"},
{"key":">10240MB/30","value":"23"}]]
data_p.forEach(list =>
list.forEach(pair =>
!isNaN(pair.value) && (pair.value = Number(pair.value))
)
);
console.log(data_p);

Rather than using i<6, why not d.length?
However, I'm not sure what is being accomplished or why. The end result seems to be that with your code values are converted from strings to numbers, but it is not obvious why.

The are many ways of doing this.
E.G. Update data_p in place using array.forEach and array.reduce
data_p.forEach( function(a) {
a.reduce( function( ignore, obj) {
obj.value = Number(obj.value);
});
});
This modifies uses reduce to iterate over all array elements except the first. It can be expanded to check the value converted to number is a number:
data_p.forEach( function(a) {
a.reduce( function( ignore, obj) {
var temp = Number(obj.value);
if( !isNaN(temp) {
obj.value = temp;
}
});
});
E.G. Using array.map to create a new array and content arrays
var newArray = data_p.map( a => a.map( (o,i)=>(
{ key: o.key,
value: o.key=="Device_Model" ? o.value : +o.value}
)));
Unlike the first example this method does not mutate data_p (or objects within it) and assumes data_p contains valid entries.

Find Duplicate Array within Array

Given an array of arrays, what would be the efficient way of identifying the duplicate item?
var array = [
[
11.31866455078125,
44.53836644772605
],
[ // <-- Here's the duplicate
11.31866455078125,
44.53836644772605
],
[
11.371536254882812,
44.53836644772605
],
[
11.371536254882812,
44.50140292110874
]
]
I've been working on this with lodash as an accepted dependency, and I get how to just return the "unique" list using _.uniqWith and _.isEqual:
_.uniqWith(array,_.isEqual)
With would give the "unique" version of the list:
[
[ 11.31866455078125, 44.53836644772605 ],
[ 11.371536254882812, 44.53836644772605 ],
[ 11.371536254882812, 44.50140292110874 ]
]
But rather than just reporting the unique elements, I need just the element that is duplicated, and ideally the index of the first occurrence.
Is this actually covered in the lodash library by some combination of methods that I'm missing? Or am I just going to have to live with writing loops to compare elements.
Probably just overtired on this, so fresh eyes on the problem would be welcome.
Trying not to rewrite functions if there are library methods that suit, so I basically am stuck with:
Returning just the duplicate or at least the comparison difference with the "unique list".
Basically identifying the "index of" an array within an array. Though I suppose that can be a filter reduction with _.isEqual once the duplicate item is identified.
Trying also to avoid creating an object Hash/Map and counting the occurrences of keys here as well, or at least not as a separate object, and as something that can be done functionally "in-line".

Lodash gives a lot of useful functions to achieve finding the first duplicate index.
Using the _.findIndex() and _.isEqual() the following code will find the first duplicate index:
var duplicateIndex = _.findIndex(array, function(value, index, collection) {
var equal = _.isEqual.bind(undefined, value);
return _.findIndex(collection.slice(0, index), equal) !== -1;
});
or a bit faster but more verbose:
var duplicateIndex = _.findIndex(array, function(value, index, collection) {
var equal = _.isEqual.bind(undefined, value);
return _.findIndex(collection, function(val, ind) {
return ind < index && equal(val);
}) !== -1;
});
Notice that if no duplicate exists, -1 will be returned.
In a few words the algorithm iterates through array and looks back if the current element does not exist already. If it does, just return the current iteration index.
Please check the working demo.

Here's an approach that uses uniqWith(), and difference():
_.indexOf(array, _.head(_.difference(array, _.uniqWith(array, _.isEqual))));
The basic idea is:
Use uniqWith() to remove the duplicates from array.
Use difference() to compare array against the duplicate-free version. This gets us an array of the duplicates.
Use head() to get the first item of the array. This is the duplicate that we're interested in.
Use indexOf() to find the index of the duplicate, in this case it's 1.
However, if you need the index of the original, and not it's duplicate, we have to make some adjustments:
var duplicate = _.head(_.difference(array, _.uniqWith(array, _.isEqual)));
_.findIndex(array, _.unary(_.partial(_.isEqual, duplicate)));
We're still using uniqWith(), and difference() to find the duplicate. But now, we're using findIndex() to get the index. The reason is that we need to use isEqual() to find the first position of the duplicate, not the second. We construct the predicate using partial() and unary(). The result this time is 0.

You can just use plain javascript to do that, it's not that hard, here is my implementation
for (let i = 0; i < array.length; i++) {
for (let j = i + 1; j < array.length; j++) {
// quick elimination by comparing sub-array lengths
if (array[i].length !== array[j].length) {
continue;
}
// look for dupes
var dupe = true;
for (var k = 0; k < array[i].length; k++) {
if (array[i][k] !== array[j][k]) {
dupe = false;
break;
}
}
// if a dupe then print
if (dupe) {
console.debug("%d is a dupe", j);
}
}
}
The nice part about this implementation is that it will print you multiple times that an array at an index is a dupe for multiple dupes, you can use that fact to count your dupes in each index!
This is actually a very efficient way to do this because the inner for loop (j) always runs from the next position of the outer loop (i). so you half your check count.
And here is a plunk

I don't know how to do this other than to just write the algorithm yourself. Both this answer and the other posted ones aren't very efficient but should be fine:
function findIndex(array, startingIndex, value) {
var predicate = _.partial(_.isEqual, value);
var arraySubset = array.slice(startingIndex+1);
var index = arraySubset.findIndex(predicate);
return index === -1 ? index : index+startingIndex+1;
}
function findDuplicates(array) {
return array.map((value, index) => {
return {
value,
index: findIndex(array, index, value)
};
}).filter(info => info.index !== -1);
}
findDuplicates([1, 2, 3, 4, 1, [ 3 ], [ 4 ], [ 3 ] ]);
// [ { value: 1, index: 4 }, { value: [ 3 ], index: 7 } ] // [ { value: 1, index: 4 }, { value: [ 3 ], index: 7 } ]
This basically creates a map of the array, calling .findIndex() on the remainder of the array, noting down the index of any duplicates, returning information about every item that has a duplicate and what the index of the duplicate is.
One nice thing about this is that it will work for triplicates or any amount of occurrences of a value.

I believe constructing a LUT is one of the most efficient ways when it comes to making comparisons. The following method constructs a LUT by utilizing Array.prototype.reduce() and eventually mutates the original array by removing not only one but all duplicate elements regardless of how many there are.
var arr = [
[
11.31866455078125,
44.53836644772605
],
[
11.31866455078125,
44.53836644772605
],
[
11.371536254882812,
44.53836644772605
],
[
11.371536254882812,
44.50140292110874
]
];
arr.reduce((p,c,i)=> { var prop = c[0]+"" + c[1]+"";
p[prop] === void 0 ? p[prop] = i : p.dups.push(i);
return p;
},{dups:[]}).dups.reverse().forEach( i => arr.splice(i,1))
document.write('<pre>' + JSON.stringify(arr, 0, 2) + '</pre>');
However if you would like to have a new array by keeping the original then obviously it would be much faster procedure.

How to remove elements from an array using values stored in another array

I have two arrays
var a = ['tayyar', '14march' ];
and
b = [{
"feedsource": "tayyar",
"hash": "46cc3d067df1ea7877140c67b60e9a7a"
}, {
"feedsource": "elmarada",
"hash": "a9fb75f2aa4771945ec597ecf9ae49ea"
}, {
"feedsource": "14march",
"hash": "fce7a6a87b53358c4be47507b0dc327b"
}, {
"feedsource": "tayyar",
"hash": "b85d2a9c22ac4831477de15ba24b4ac5"
}]
I want to remove objects from b whose feedsources are not defined in a.
So far I have tried
b.forEach(function(e) {
var indexVal = b.indexOf(e);
if(a.indexOf(e.feedsource) ==-1){
console.log(e.feedsource);
b.splice(indexVal,1);
}
});
But it doesnt seems to work and I still see the elements that shouldn't be there as well as elements that should be there getting removed. What am I doing wrong in my function ?

A better solution would be to not alter the original array, but instead creating a new, filtered array. Luckily, JavaScript provides .filter() to help you with that!
var c = b.filter(function(el) {
// Only keep those where the following is true
return a.indexOf(el.feedsource) > 0;
});

Create a filtered array with lodash filter():
_.filter(b, function(item) {
return _.includes(a, item.feedsource);
});
Mutate the existing array with lodash remove():
_.remove(b, function(item) {
return _.includes(a, item.feedsource) === false;
});