Is there an equivalent to python's zip() in javascript? [duplicate] - javascript

Is there a javascript equivalent of Python's zip function? That is, given multiple arrays of equal lengths create an array of pairs.
For instance, if I have three arrays that look like this:
var array1 = [1, 2, 3];
var array2 = ['a','b','c'];
var array3 = [4, 5, 6];
The output array should be:
var outputArray = [[1,'a',4], [2,'b',5], [3,'c',6]]

2016 update:
Here's a snazzier Ecmascript 6 version:
zip= rows=>rows[0].map((_,c)=>rows.map(row=>row[c]))
Illustration equiv. to Python{zip(*args)}:
> zip([['row0col0', 'row0col1', 'row0col2'],
['row1col0', 'row1col1', 'row1col2']]);
[["row0col0","row1col0"],
["row0col1","row1col1"],
["row0col2","row1col2"]]
(and FizzyTea points out that ES6 has variadic argument syntax, so the following function definition will act like python, but see below for disclaimer... this will not be its own inverse so zip(zip(x)) will not equal x; though as Matt Kramer points out zip(...zip(...x))==x (like in regular python zip(*zip(*x))==x))
Alternative definition equiv. to Python{zip}:
> zip = (...rows) => [...rows[0]].map((_,c) => rows.map(row => row[c]))
> zip( ['row0col0', 'row0col1', 'row0col2'] ,
['row1col0', 'row1col1', 'row1col2'] );
// note zip(row0,row1), not zip(matrix)
same answer as above
(Do note that the ... syntax may have performance issues at this time, and possibly in the future, so if you use the second answer with variadic arguments, you may want to perf test it. That said it's been quite a while since it's been in the standard.)
Make sure to note the addendum if you wish to use this on strings (perhaps there's a better way to do it now with es6 iterables).
Here's a oneliner:
function zip(arrays) {
return arrays[0].map(function(_,i){
return arrays.map(function(array){return array[i]})
});
}
// > zip([[1,2],[11,22],[111,222]])
// [[1,11,111],[2,22,222]]]
// If you believe the following is a valid return value:
// > zip([])
// []
// then you can special-case it, or just do
// return arrays.length==0 ? [] : arrays[0].map(...)
The above assumes that the arrays are of equal size, as they should be. It also assumes you pass in a single list of lists argument, unlike Python's version where the argument list is variadic. If you want all of these "features", see below. It takes just about 2 extra lines of code.
The following will mimic Python's zip behavior on edge cases where the arrays are not of equal size, silently pretending the longer parts of arrays don't exist:
function zip() {
var args = [].slice.call(arguments);
var shortest = args.length==0 ? [] : args.reduce(function(a,b){
return a.length<b.length ? a : b
});
return shortest.map(function(_,i){
return args.map(function(array){return array[i]})
});
}
// > zip([1,2],[11,22],[111,222,333])
// [[1,11,111],[2,22,222]]]
// > zip()
// []
This will mimic Python's itertools.zip_longest behavior, inserting undefined where arrays are not defined:
function zip() {
var args = [].slice.call(arguments);
var longest = args.reduce(function(a,b){
return a.length>b.length ? a : b
}, []);
return longest.map(function(_,i){
return args.map(function(array){return array[i]})
});
}
// > zip([1,2],[11,22],[111,222,333])
// [[1,11,111],[2,22,222],[null,null,333]]
// > zip()
// []
If you use these last two version (variadic aka. multiple-argument versions), then zip is no longer its own inverse. To mimic the zip(*[...]) idiom from Python, you will need to do zip.apply(this, [...]) when you want to invert the zip function or if you want to similarly have a variable number of lists as input.
addendum:
To make this handle any iterable (e.g. in Python you can use zip on strings, ranges, map objects, etc.), you could define the following:
function iterView(iterable) {
// returns an array equivalent to the iterable
}
However if you write zip in the following way, even that won't be necessary:
function zip(arrays) {
return Array.apply(null,Array(arrays[0].length)).map(function(_,i){
return arrays.map(function(array){return array[i]})
});
}
Demo:
> JSON.stringify( zip(['abcde',[1,2,3,4,5]]) )
[["a",1],["b",2],["c",3],["d",4],["e",5]]
(Or you could use a range(...) Python-style function if you've written one already. Eventually you will be able to use ECMAScript array comprehensions or generators.)

Check out the library Underscore.
Underscore provides over 100 functions that support both your favorite workaday functional helpers: map, filter, invoke ā€” as well as more specialized goodies: function binding, javascript templating, creating quick indexes, deep equality testing, and so on.
ā€“ Say the people who made it
I recently started using it specifically for the zip() function and it has left a great first impression. I am using jQuery and CoffeeScript, and it just goes perfectly with them. Underscore picks up right where they leave off and so far it hasn't let me down. Oh by the way, it's only 3kb minified.
Check it out:
_.zip(['moe', 'larry', 'curly'], [30, 40, 50], [true, false, false]);
// returns [["moe", 30, true], ["larry", 40, false], ["curly", 50, false]]

Modern ES6 example with a generator:
function *zip (...iterables){
let iterators = iterables.map(i => i[Symbol.iterator]() )
while (true) {
let results = iterators.map(iter => iter.next() )
if (results.some(res => res.done) ) return
else yield results.map(res => res.value )
}
}
First, we get a list of iterables as iterators. This usually happens transparently, but here we do it explicitly, as we yield step-by-step until one of them is exhausted. We check if any of results (using the .some() method) in the given array is exhausted, and if so, we break the while loop.

In addition to ninjagecko's excellent and comprehensive answer, all it takes to zip two JS-arrays into a "tuple-mimic" is:
//Arrays: aIn, aOut
Array.prototype.map.call( aIn, function(e,i){return [e, aOut[i]];})
Explanation:
Since Javascript doesn't have a tuples type, functions for tuples, lists and sets wasn't a high priority in the language specification.
Otherwise, similar behavior is accessible in a straightforward manner via Array map in JS >1.6. (map is actually often implemented by JS engine makers in many >JS 1.4 engines, despite not specified).
The major difference to Python's zip, izip,... results from map's functional style, since map requires a function-argument. Additionally it is a function of the Array-instance. One may use Array.prototype.map instead, if an extra declaration for the input is an issue.
Example:
_tarrin = [0..constructor, function(){}, false, undefined, '', 100, 123.324,
2343243243242343242354365476453654625345345, 'sdf23423dsfsdf',
'sdf2324.234dfs','234,234fsf','100,100','100.100']
_parseInt = function(i){return parseInt(i);}
_tarrout = _tarrin.map(_parseInt)
_tarrin.map(function(e,i,a){return [e, _tarrout[i]]})
Result:
//'('+_tarrin.map(function(e,i,a){return [e, _tarrout[i]]}).join('),\n(')+')'
>>
(function Number() { [native code] },NaN),
(function (){},NaN),
(false,NaN),
(,NaN),
(,NaN),
(100,100),
(123.324,123),
(2.3432432432423434e+42,2),
(sdf23423dsfsdf,NaN),
(sdf2324.234dfs,NaN),
(234,234fsf,234),
(100,100,100),
(100.100,100)
Related Performance:
Using map over for-loops:
See: What is the most efficient way of merging [1,2] and [7,8] into [[1,7], [2,8]]
Note: the base types such as false and undefined do not posess a prototypal object-hierarchy and thus do not expose a toString function. Hence these are shown as empty in the output.
As parseInt's second argument is the base/number radix, to which to convert the number to, and since map passes the index as the second argument to its argument-function, a wrapper function is used.

Along other Python-like functions, pythonic offers a zip function, with the extra benefit of returning a lazy evaluated Iterator, similar to the behaviour of its Python counterpart:
import {zip, zipLongest} from 'pythonic';
const arr1 = ['a', 'b'];
const arr2 = ['c', 'd', 'e'];
for (const [first, second] of zip(arr1, arr2))
console.log(`first: ${first}, second: ${second}`);
// first: a, second: c
// first: b, second: d
for (const [first, second] of zipLongest(arr1, arr2))
console.log(`first: ${first}, second: ${second}`);
// first: a, second: c
// first: b, second: d
// first: undefined, second: e
// unzip
const [arrayFirst, arraySecond] = [...zip(...zip(arr1, arr2))];
Disclosure I'm author and maintainer of Pythonic

Python has two function to zip sequences: zip and itertools.zip_longest. An implementation in Javascript for the same functionality is this:
Implementation of Python`s zip on JS/ES6
const zip = (...arrays) => {
const length = Math.min(...arrays.map(arr => arr.length));
return Array.from({ length }, (value, index) => arrays.map((array => array[index])));
};
Results in:
console.log(zip(
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[11, 221]
));
[ [ 1, 667, 111, 11 ] ]
console.log(zip(
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111, 212, 323, 433, '1111']
));
[ [ 1, 667, 111 ], [ 2, false, 212 ], [ 3, -378, 323 ], [ 'a',
'337', 433 ] ]
console.log(zip(
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[]
));
[]
Implementation of Python`s zip_longest on JS/ES6
(https://docs.python.org/3.5/library/itertools.html?highlight=zip_longest#itertools.zip_longest)
const zipLongest = (placeholder = undefined, ...arrays) => {
const length = Math.max(...arrays.map(arr => arr.length));
return Array.from(
{ length }, (value, index) => arrays.map(
array => array.length - 1 >= index ? array[index] : placeholder
)
);
};
Results:
console.log(zipLongest(
undefined,
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[]
));
[ [ 1, 667, 111, undefined ], [ 2, false, undefined, undefined ],
[ 3, -378, undefined, undefined ], [ 'a', '337', undefined,
undefined ] ]
console.log(zipLongest(
null,
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[]
));
[ [ 1, 667, 111, null ], [ 2, false, null, null ], [ 3, -378,
null, null ], [ 'a', '337', null, null ] ]
console.log(zipLongest(
'Is None',
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[]
));
[ [ 1, 667, 111, 'Is None' ], [ 2, false, 'Is None', 'Is None' ],
[ 3, -378, 'Is None', 'Is None' ], [ 'a', '337', 'Is None', 'Is
None' ] ]

You can make utility function by using ES6.
console.json = obj => console.log(JSON.stringify(obj));
const zip = (arr, ...arrs) =>
arr.map((val, i) => arrs.reduce((a, arr) => [...a, arr[i]], [val]));
// Example
const array1 = [1, 2, 3];
const array2 = ['a','b','c'];
const array3 = [4, 5, 6];
console.json(zip(array1, array2)); // [[1,"a"],[2,"b"],[3,"c"]]
console.json(zip(array1, array2, array3)); // [[1,"a",4],[2,"b",5],[3,"c",6]]
However, in above solution length of the first array defines the length of the output array.
Here is the solution in which you have more control over it. It's bit complex but worth it.
function _zip(func, args) {
const iterators = args.map(arr => arr[Symbol.iterator]());
let iterateInstances = iterators.map((i) => i.next());
ret = []
while(iterateInstances[func](it => !it.done)) {
ret.push(iterateInstances.map(it => it.value));
iterateInstances = iterators.map((i) => i.next());
}
return ret;
}
const array1 = [1, 2, 3];
const array2 = ['a','b','c'];
const array3 = [4, 5, 6];
const zipShort = (...args) => _zip('every', args);
const zipLong = (...args) => _zip('some', args);
console.log(zipShort(array1, array2, array3)) // [[1, 'a', 4], [2, 'b', 5], [3, 'c', 6]]
console.log(zipLong([1,2,3], [4,5,6, 7]))
// [
// [ 1, 4 ],
// [ 2, 5 ],
// [ 3, 6 ],
// [ undefined, 7 ]]

1. Npm Module: zip-array
I found an npm module that can be used as a javascript version of python zip:
zip-array - A javascript equivalent of Python's zip function. Merges together the values of each of the arrays.
https://www.npmjs.com/package/zip-array
2. tf.data.zip() in Tensorflow.js
Another alternate choice is for Tensorflow.js users: if you need a zip function in python to work with tensorflow datasets in Javascript, you can use tf.data.zip() in Tensorflow.js.
tf.data.zip() in Tensorflow.js documented at here

Original answer (see update below)
I modified flm's nifty answer to take an arbitrary number of arrays:
function* zip(arrays, i = 0) {
while (i<Math.min(...arrays.map(({length})=>length))) {
yield arrays.map((arr, j) => arr[j < arrays.length - 1 ? i : i++])
}
}
Updated answer
As pointed out by Tom Pohl this function can't deal with arrays with falsy values in. Here is an updated/improved version that can deal with any types and also unequal length arrays:
function* zip(arrays, i = 0) {
while (i<Math.min(...arrays.map(arr=>arr.length))) {
yield arrays.map((arr, j) => arr[j < arrays.length - 1 ? i : i++])
}
}
const arr1 = [false,0,1,2]
const arr2 = [100,null,99,98,97]
const arr3 = [7,8,undefined,"monkey","banana"]
console.log(...zip([arr1,arr2,arr3]))

Not built-in to Javascript itself. Some of the common Javascript frameworks (such as Prototype) provide an implementation, or you can write your own.

Like #Brandon, I recommend Underscore's zip function. However, it acts like zip_longest, appending undefined values as needed to return something the length of the longest input.
I used the mixin method to extend underscore with a zipShortest, which acts like Python's zip, based off of the library's own source for zip.
You can add the following to your common JS code and then call it as if it were part of underscore: _.zipShortest([1,2,3], ['a']) returns [[1, 'a']], for example.
// Underscore library addition - zip like python does, dominated by the shortest list
// The default injects undefineds to match the length of the longest list.
_.mixin({
zipShortest : function() {
var args = Array.Prototype.slice.call(arguments);
var length = _.min(_.pluck(args, 'length')); // changed max to min
var results = new Array(length);
for (var i = 0; i < length; i++) {
results[i] = _.pluck(args, "" + i);
}
return results;
}});

A variation of the lazy generator solution:
function* iter(it) {
yield* it;
}
function* zip(...its) {
its = its.map(iter);
while (true) {
let rs = its.map(it => it.next());
if (rs.some(r => r.done))
return;
yield rs.map(r => r.value);
}
}
for (let r of zip([1,2,3], [4,5,6,7], [8,9,0,11,22]))
console.log(r.join())
// the only change for "longest" is some -> every
function* zipLongest(...its) {
its = its.map(iter);
while (true) {
let rs = its.map(it => it.next());
if (rs.every(r => r.done))
return;
yield rs.map(r => r.value);
}
}
for (let r of zipLongest([1,2,3], [4,5,6,7], [8,9,0,11,22]))
console.log(r.join())
And this is the python's classic "n-group" idiom zip(*[iter(a)]*n):
triples = [...zip(...Array(3).fill(iter(a)))]

ES2020 shortest variant:
function * zip(arr1, arr2, i = 0) {
while(arr1[i] || arr2[i]) yield [arr1[i], arr2[i++]].filter(x => !!x);
}
[ ...zip(arr1, arr2) ] // result

You could reduce the array of arrays and map new array by taking the result of the index of the inner array.
var array1 = [1, 2, 3],
array2 = ['a','b','c'],
array3 = [4, 5, 6],
array = [array1, array2, array3],
transposed = array.reduce((r, a) => a.map((v, i) => (r[i] || []).concat(v)), []);
console.log(transposed);
Fun with spread.
const
transpose = (r, a) => a.map((v, i) => [...(r[i] || []), v]),
array1 = [1, 2, 3],
array2 = ['a','b','c'],
array3 = [4, 5, 6],
transposed = [array1, array2, array3].reduce(transpose, []);
console.log(transposed);

I took a run at this in pure JS wondering how the plugins posted above got the job done. Here's my result. I'll preface this by saying that I have no idea how stable this will be in IE and the like. It's just a quick mockup.
init();
function init() {
var one = [0, 1, 2, 3];
var two = [4, 5, 6, 7];
var three = [8, 9, 10, 11, 12];
var four = zip(one, two, one);
//returns array
//four = zip(one, two, three);
//returns false since three.length !== two.length
console.log(four);
}
function zip() {
for (var i = 0; i < arguments.length; i++) {
if (!arguments[i].length || !arguments.toString()) {
return false;
}
if (i >= 1) {
if (arguments[i].length !== arguments[i - 1].length) {
return false;
}
}
}
var zipped = [];
for (var j = 0; j < arguments[0].length; j++) {
var toBeZipped = [];
for (var k = 0; k < arguments.length; k++) {
toBeZipped.push(arguments[k][j]);
}
zipped.push(toBeZipped);
}
return zipped;
}
It's not bulletproof, but it's still interesting.

A generator approach to pythons zip function.
function* zip(...arrs){
for(let i = 0; i < arrs[0].length; i++){
a = arrs.map(e=>e[i])
if(a.indexOf(undefined) == -1 ){yield a }else{return undefined;}
}
}
// use as multiple iterators
for( let [a,b,c] of zip([1, 2, 3, 4], ['a', 'b', 'c', 'd'], ['hi', 'hello', 'howdy', 'how are you']) )
console.log(a,b,c)
// creating new array with the combined arrays
let outputArr = []
for( let arr of zip([1, 2, 3, 4], ['a', 'b', 'c', 'd'], ['hi', 'hello', 'howdy', 'how are you']) )
outputArr.push(arr)

I have created a simple function to do so with a option to provide an zipper function
function zip(zipper, ...arrays) {
if (zipper instanceof Array) {
arrays.unshift(zipper)
zipper = (...elements) => elements
}
const length = Math.min(...arrays.map(array => array.length))
const zipped = []
for (let i = 0; i < length; i++) {
zipped.push(zipper(...arrays.map(array => array[i])))
}
return zipped
}
https://gist.github.com/AmrIKhudair/4b740149c29c492859e00f451832975b

I'm not a javascript guy but I feel like many of these answers are trying to find the cutest and most clever solution using Array.map which is fine, but for someone like me that doesn't use javascript every day here are some alternatives that might possibly be a bit more readable.
Maybe a way to avoid some cute and clever code would be:
function zip(a,b){
// pre-allocate an array to hold the results
rval=Array(Math.max(a.length, b.length));
for(i=0; i<rval.length; i++){
rval[i]=[a[i],b[i]]
}
return rval
}
If you like generators:
function* _zip(a,b){
len = Math.max(a.length, b.length) // handle different sized arrays
for(i=0; i<len; i++) { yield [a[i],b[i]] }
}
Or if you really want to use Array.map:
function map(a,b){
x = a.length > b.length ? a : b // call map on the biggest array
return x.map((_,i)=>[a[i],b[i]])
}
As I said, I'm not an everyday javascript guy so, these aren't going to be the most elegant solutions but they are readable to me.

Below is a fast and efficient way of doing this, using iter-ops library, operator zip:
const {pipe, zip} = require('iter-ops');
const i = pipe(array1, zip(array2, array3));
console.log(...i); //=> [ 1, 'a', 4 ] [ 2, 'b', 5 ] [ 3, 'c', 6 ]
The library processes all inputs as iterables, so they are iterated over just once. And it can handle, in the same way, all types of iterable objects - Iterable, AsyncIterable, Iterator, AsyncIterator.
P.S. I'm the author of iter-ops.

The Mochikit library provides this and many other Python-like functions. developer of Mochikit is also a Python fan, so it has the general style of Python, and also the wraps the async calls in a twisted-like framework.

There is no equivalent function. If you have only a few arrays you should use a for loop to get an index and then use the index to access the arrays:
var array1 = [1, 2, 3];
var array2 = ['a','b','c'];
for (let i = 0; i < Math.min(array1.length, array2.length); i++) {
doStuff(array1[i], array2[i]);
}
You can have an inner loop over the arrays if you have more.

Here is my solution
let zip = (a, b) => (a.length < b.length
? a.map((e, i) => [e, b[i]])
: b.map((e, i) => [a[i], e]))

This shaves a line off Ddi's iterator-based answer:
function* zip(...toZip) {
const iterators = toZip.map((arg) => arg[Symbol.iterator]());
const next = () => toZip = iterators.map((iter) => iter.next());
while (next().every((item) => !item.done)) {
yield toZip.map((item) => item.value);
}
}

If you are fine with ES6:
const zip = (arr,...arrs) =>(
arr.map(
(v,i) => arrs.reduce((a,arr)=>[...a, arr[i]], [v])))

Related

how can unique value multiple array in jquery? [duplicate]

I have an array of numbers that I need to make sure are unique. I found the code snippet below on the internet and it works great until the array has a zero in it. I found this other script here on Stack Overflow that looks almost exactly like it, but it doesn't fail.
So for the sake of helping me learn, can someone help me determine where the prototype script is going wrong?
Array.prototype.getUnique = function() {
var o = {}, a = [], i, e;
for (i = 0; e = this[i]; i++) {o[e] = 1};
for (e in o) {a.push (e)};
return a;
}
More answers from duplicate question:
Remove duplicate values from JS array
Similar question:
Get all non-unique values (i.e.: duplicate/more than one occurrence) in an array
With JavaScript 1.6 / ECMAScript 5 you can use the native filter method of an Array in the following way to get an array with unique values:
function onlyUnique(value, index, array) {
return self.indexOf(value) === index;
}
// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);
console.log(unique); // ['a', 1, 2, '1']
The native method filter will loop through the array and leave only those entries that pass the given callback function onlyUnique.
onlyUnique checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.
This solution works without any extra library like jQuery or prototype.js.
It works for arrays with mixed value types too.
For old Browsers (<ie9), that do not support the native methods filter and indexOf you can find work arounds in the MDN documentation for filter and indexOf.
If you want to keep the last occurrence of a value, simply replace indexOf with lastIndexOf.
With ES6 this can be shorten to:
// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((value, index, array) => array.indexOf(value) === index);
console.log(unique); // unique is ['a', 1, 2, '1']
Thanks to Camilo Martin for hint in comment.
ES6 has a native object Set to store unique values. To get an array with unique values you could now do this:
var myArray = ['a', 1, 'a', 2, '1'];
let unique = [...new Set(myArray)];
console.log(unique); // unique is ['a', 1, 2, '1']
The constructor of Set takes an iterable object, like an Array, and the spread operator ... transform the set back into an Array. Thanks to Lukas Liese for hint in comment.
Updated answer for ES6/ES2015: Using the Set and the spread operator (thanks le-m), the single line solution is:
let uniqueItems = [...new Set(items)]
Which returns
[4, 5, 6, 3, 2, 23, 1]
I split all answers to 4 possible solutions:
Use object { } to prevent duplicates
Use helper array [ ]
Use filter + indexOf
Bonus! ES6 Sets method.
Here's sample codes found in answers:
Use object { } to prevent duplicates
function uniqueArray1( ar ) {
var j = {};
ar.forEach( function(v) {
j[v+ '::' + typeof v] = v;
});
return Object.keys(j).map(function(v){
return j[v];
});
}
Use helper array [ ]
function uniqueArray2(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Use filter + indexOf
function uniqueArray3(a) {
function onlyUnique(value, index, self) {
return self.indexOf(value) === index;
}
// usage
var unique = a.filter( onlyUnique ); // returns ['a', 1, 2, '1']
return unique;
}
Use ES6 [...new Set(a)]
function uniqueArray4(a) {
return [...new Set(a)];
}
And I wondered which one is faster. I've made sample Google Sheet to test functions. Note: ECMA 6 is not avaliable in Google Sheets, so I can't test it.
Here's the result of tests:
I expected to see that code using object { } will win because it uses hash. So I'm glad that tests showed the best results for this algorithm in Chrome and IE. Thanks to #rab for the code.
Update 2020
Google Script enabled ES6 Engine. Now I tested the last code with Sets and it appeared faster than the object method.
You can also use underscore.js.
console.log(_.uniq([1, 2, 1, 3, 1, 4]));
<script src="http://underscorejs.org/underscore-min.js"></script>
which will return:
[1, 2, 3, 4]
One Liner, Pure JavaScript
With ES6 syntax
list = list.filter((x, i, a) => a.indexOf(x) == i)
x --> item in array
i --> index of item
a --> array reference, (in this case "list")
With ES5 syntax
list = list.filter(function (x, i, a) {
return a.indexOf(x) == i;
});
Browser Compatibility: IE9+
Remove duplicates using Set.
Array with duplicates
const withDuplicates = [2, 2, 5, 5, 1, 1, 2, 2, 3, 3];
Get a new array without duplicates by using Set
const withoutDuplicates = Array.from(new Set(withDuplicates));
A shorter version
const withoutDuplicates = [...new Set(withDuplicates)];
Result: [2, 5, 1, 3]
Many of the answers here may not be useful to beginners. If de-duping an array is difficult, will they really know about the prototype chain, or even jQuery?
In modern browsers, a clean and simple solution is to store data in a Set, which is designed to be a list of unique values.
const cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
const uniqueCars = Array.from(new Set(cars));
console.log(uniqueCars);
The Array.from is useful to convert the Set back to an Array so that you have easy access to all of the awesome methods (features) that arrays have. There are also other ways of doing the same thing. But you may not need Array.from at all, as Sets have plenty of useful features like forEach.
If you need to support old Internet Explorer, and thus cannot use Set, then a simple technique is to copy items over to a new array while checking beforehand if they are already in the new array.
// Create a list of cars, with duplicates.
var cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
// Create a list of unique cars, to put a car in if we haven't already.
var uniqueCars = [];
// Go through each car, one at a time.
cars.forEach(function (car) {
// The code within the following block runs only if the
// current car does NOT exist in the uniqueCars list
// - a.k.a. prevent duplicates
if (uniqueCars.indexOf(car) === -1) {
// Since we now know we haven't seen this car before,
// copy it to the end of the uniqueCars list.
uniqueCars.push(car);
}
});
To make this instantly reusable, let's put it in a function.
function deduplicate(data) {
if (data.length > 0) {
var result = [];
data.forEach(function (elem) {
if (result.indexOf(elem) === -1) {
result.push(elem);
}
});
return result;
}
}
So to get rid of the duplicates, we would now do this.
var uniqueCars = deduplicate(cars);
The deduplicate(cars) part becomes the thing we named result when the function completes.
Just pass it the name of any array you like.
Using ES6 new Set
var array = [3,7,5,3,2,5,2,7];
var unique_array = [...new Set(array)];
console.log(unique_array); // output = [3,7,5,2]
Using For Loop
var array = [3,7,5,3,2,5,2,7];
for(var i=0;i<array.length;i++) {
for(var j=i+1;j<array.length;j++) {
if(array[i]===array[j]) {
array.splice(j,1);
}
}
}
console.log(array); // output = [3,7,5,2]
I have since found a nice method that uses jQuery
arr = $.grep(arr, function(v, k){
return $.inArray(v ,arr) === k;
});
Note: This code was pulled from Paul Irish's duck punching post - I forgot to give credit :P
Magic
a.filter(e=>!(t[e]=e in t))
O(n) performance - we assume your array is in a and t={}. Explanation here (+Jeppe impr.)
let unique = (a,t={}) => a.filter(e=>!(t[e]=e in t));
// "stand-alone" version working with global t:
// a1.filter((t={},e=>!(t[e]=e in t)));
// Test data
let a1 = [5,6,0,4,9,2,3,5,0,3,4,1,5,4,9];
let a2 = [[2, 17], [2, 17], [2, 17], [1, 12], [5, 9], [1, 12], [6, 2], [1, 12]];
let a3 = ['Mike', 'Adam','Matt', 'Nancy', 'Adam', 'Jenny', 'Nancy', 'Carl'];
// Results
console.log(JSON.stringify( unique(a1) ))
console.log(JSON.stringify( unique(a2) ))
console.log(JSON.stringify( unique(a3) ))
The simplest, and fastest (in Chrome) way of doing this:
Array.prototype.unique = function() {
var a = [];
for (var i=0, l=this.length; i<l; i++)
if (a.indexOf(this[i]) === -1)
a.push(this[i]);
return a;
}
Simply goes through every item in the array, tests if that item is already in the list, and if it's not, pushes to the array that gets returned.
According to JSBench, this function is the fastest of the ones I could find anywhere - feel free to add your own though.
The non-prototype version:
function uniques(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Sorting
When also needing to sort the array, the following is the fastest:
Array.prototype.sortUnique = function() {
this.sort();
var last_i;
for (var i=0;i<this.length;i++)
if ((last_i = this.lastIndexOf(this[i])) !== i)
this.splice(i+1, last_i-i);
return this;
}
or non-prototype:
function sortUnique(arr) {
arr.sort();
var last_i;
for (var i=0;i<arr.length;i++)
if ((last_i = arr.lastIndexOf(arr[i])) !== i)
arr.splice(i+1, last_i-i);
return arr;
}
This is also faster than the above method in most non-Chrome browsers.
We can do this using ES6 sets:
var duplicatesArray = [1, 2, 3, 4, 5, 1, 1, 1, 2, 3, 4];
var uniqueArray = [...new Set(duplicatesArray)];
console.log(uniqueArray); // [1,2,3,4,5]
["Defects", "Total", "Days", "City", "Defects"].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
[0,1,2,0,3,2,1,5].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
After looking into all the 90+ answers here, I saw there is room for one more:
Array.includes has a very handy second-parameter: "fromIndex", so by using it, every iteration of the filter callback method will search the array, starting from [current index] + 1 which guarantees not to include currently filtered item in the lookup and also saves time.
Note - this solution does not retain the order, as it removed duplicated items from left to right, but it wins the Set trick if the Array is a collection of Objects.
// šŸš© šŸš© šŸš©
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
list.filter((v,i) => !list.includes(v,i+1))
)
// [0,1,3,"b",4,5,2,"a"]
Explanation:
For example, lets assume the filter function is currently iterating at index 2) and the value at that index happens to be 2. The section of the array that is then scanned for duplicates (includes method) is everything after index 2 (i+1):
šŸ‘‡ šŸ‘‡
[0, 1, 2, 2 ,3 ,'a', 'b', 4, 5, 2, 'a']
šŸ‘† |---------------------------|
And since the currently filtered item's value 2 is included in the rest of the array, it will be filtered out, because of the leading exclamation mark which negates the filter rule.
If order is important, use this method:
// šŸš© šŸš© šŸš©
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
// Initialize with empty array and fill with non-duplicates
list.reduce((acc, v) => (!acc.includes(v) && acc.push(v), acc), [])
)
// [0,1,2,3,"a","b",4,5]
This has been answered a lot, but it didn't address my particular need.
Many answers are like this:
a.filter((item, pos, self) => self.indexOf(item) === pos);
But this doesn't work for arrays of complex objects.
Say we have an array like this:
const a = [
{ age: 4, name: 'fluffy' },
{ age: 5, name: 'spot' },
{ age: 2, name: 'fluffy' },
{ age: 3, name: 'toby' },
];
If we want the objects with unique names, we should use array.prototype.findIndex instead of array.prototype.indexOf:
a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);
This prototype getUnique is not totally correct, because if i have a Array like: ["1",1,2,3,4,1,"foo"] it will return ["1","2","3","4"] and "1" is string and 1 is a integer; they are different.
Here is a correct solution:
Array.prototype.unique = function(a){
return function(){ return this.filter(a) }
}(function(a,b,c){ return c.indexOf(a,b+1) < 0 });
using:
var foo;
foo = ["1",1,2,3,4,1,"foo"];
foo.unique();
The above will produce ["1",2,3,4,1,"foo"].
You can simlply use the built-in functions Array.prototype.filter() and Array.prototype.indexOf()
array.filter((x, y) => array.indexOf(x) == y)
var arr = [1, 2, 3, 3, 4, 5, 5, 5, 6, 7, 8, 9, 6, 9];
var newarr = arr.filter((x, y) => arr.indexOf(x) == y);
console.log(newarr);
[...new Set(duplicates)]
This is the simplest one and referenced from MDN Web Docs.
const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]
Array.prototype.getUnique = function() {
var o = {}, a = []
for (var i = 0; i < this.length; i++) o[this[i]] = 1
for (var e in o) a.push(e)
return a
}
Without extending Array.prototype (it is said to be a bad practice) or using jquery/underscore, you can simply filter the array.
By keeping last occurrence:
function arrayLastUnique(array) {
return array.filter(function (a, b, c) {
// keeps last occurrence
return c.indexOf(a, b + 1) < 0;
});
},
or first occurrence:
function arrayFirstUnique(array) {
return array.filter(function (a, b, c) {
// keeps first occurrence
return c.indexOf(a) === b;
});
},
Well, it's only javascript ECMAScript 5+, which means only IE9+, but it's nice for a development in native HTML/JS (Windows Store App, Firefox OS, Sencha, Phonegap, Titanium, ...).
That's because 0 is a falsy value in JavaScript.
this[i] will be falsy if the value of the array is 0 or any other falsy value.
Now using sets you can remove duplicates and convert them back to the array.
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
console.log([...new Set(names)])
Another solution is to use sort & filter
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);
If you're using Prototype framework there is no need to do 'for' loops, you can use http://prototypejs.org/doc/latest/language/Array/prototype/uniq/ like this:
var a = Array.uniq();
Which will produce a duplicate array with no duplicates. I came across your question searching a method to count distinct array records so after uniq() I used size() and there was my simple result.
p.s. Sorry if i mistyped something
edit: if you want to escape undefined records you may want to add compact() before, like this:
var a = Array.compact().uniq();
I had a slightly different problem where I needed to remove objects with duplicate id properties from an array. this worked.
let objArr = [{
id: '123'
}, {
id: '123'
}, {
id: '456'
}];
objArr = objArr.reduce((acc, cur) => [
...acc.filter((obj) => obj.id !== cur.id), cur
], []);
console.log(objArr);
If you're okay with extra dependencies, or you already have one of the libraries in your codebase, you can remove duplicates from an array in place using LoDash (or Underscore).
Usage
If you don't have it in your codebase already, install it using npm:
npm install lodash
Then use it as follows:
import _ from 'lodash';
let idArray = _.uniq ([
1,
2,
3,
3,
3
]);
console.dir(idArray);
Out:
[ 1, 2, 3 ]
I'm not sure why Gabriel Silveira wrote the function that way but a simpler form that works for me just as well and without the minification is:
Array.prototype.unique = function() {
return this.filter(function(value, index, array) {
return array.indexOf(value, index + 1) < 0;
});
};
or in CoffeeScript:
Array.prototype.unique = ->
this.filter( (value, index, array) ->
array.indexOf(value, index + 1) < 0
)
Finding unique Array values in simple method
function arrUnique(a){
var t = [];
for(var x = 0; x < a.length; x++){
if(t.indexOf(a[x]) == -1)t.push(a[x]);
}
return t;
}
arrUnique([1,4,2,7,1,5,9,2,4,7,2]) // [1, 4, 2, 7, 5, 9]
It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:
Array.prototype.getUnique = function(){
var u = {}, a = [];
for (var i = 0, l = this.length; i < l; ++i) {
if (u.hasOwnProperty(this[i])) {
continue;
}
a.push(this[i]);
u[this[i]] = 1;
}
return a;
}
If you do have a mixed-type array, you can serialize the hash key:
Array.prototype.getUnique = function() {
var hash = {}, result = [], key;
for ( var i = 0, l = this.length; i < l; ++i ) {
key = JSON.stringify(this[i]);
if ( !hash.hasOwnProperty(key) ) {
hash[key] = true;
result.push(this[i]);
}
}
return result;
}
strange this hasn't been suggested before.. to remove duplicates by object key (id below) in an array you can do something like this:
const uniqArray = array.filter((obj, idx, arr) => (
arr.findIndex((o) => o.id === obj.id) === idx
))
For an object-based array with some unique id's, I have a simple solution through which you can sort in linear complexity
function getUniqueArr(arr){
const mapObj = {};
arr.forEach(a => {
mapObj[a.id] = a
})
return Object.values(mapObj);
}

What is a JS alternative to Python's zip() function? [duplicate]

Is there a javascript equivalent of Python's zip function? That is, given multiple arrays of equal lengths create an array of pairs.
For instance, if I have three arrays that look like this:
var array1 = [1, 2, 3];
var array2 = ['a','b','c'];
var array3 = [4, 5, 6];
The output array should be:
var outputArray = [[1,'a',4], [2,'b',5], [3,'c',6]]
2016 update:
Here's a snazzier Ecmascript 6 version:
zip= rows=>rows[0].map((_,c)=>rows.map(row=>row[c]))
Illustration equiv. to Python{zip(*args)}:
> zip([['row0col0', 'row0col1', 'row0col2'],
['row1col0', 'row1col1', 'row1col2']]);
[["row0col0","row1col0"],
["row0col1","row1col1"],
["row0col2","row1col2"]]
(and FizzyTea points out that ES6 has variadic argument syntax, so the following function definition will act like python, but see below for disclaimer... this will not be its own inverse so zip(zip(x)) will not equal x; though as Matt Kramer points out zip(...zip(...x))==x (like in regular python zip(*zip(*x))==x))
Alternative definition equiv. to Python{zip}:
> zip = (...rows) => [...rows[0]].map((_,c) => rows.map(row => row[c]))
> zip( ['row0col0', 'row0col1', 'row0col2'] ,
['row1col0', 'row1col1', 'row1col2'] );
// note zip(row0,row1), not zip(matrix)
same answer as above
(Do note that the ... syntax may have performance issues at this time, and possibly in the future, so if you use the second answer with variadic arguments, you may want to perf test it. That said it's been quite a while since it's been in the standard.)
Make sure to note the addendum if you wish to use this on strings (perhaps there's a better way to do it now with es6 iterables).
Here's a oneliner:
function zip(arrays) {
return arrays[0].map(function(_,i){
return arrays.map(function(array){return array[i]})
});
}
// > zip([[1,2],[11,22],[111,222]])
// [[1,11,111],[2,22,222]]]
// If you believe the following is a valid return value:
// > zip([])
// []
// then you can special-case it, or just do
// return arrays.length==0 ? [] : arrays[0].map(...)
The above assumes that the arrays are of equal size, as they should be. It also assumes you pass in a single list of lists argument, unlike Python's version where the argument list is variadic. If you want all of these "features", see below. It takes just about 2 extra lines of code.
The following will mimic Python's zip behavior on edge cases where the arrays are not of equal size, silently pretending the longer parts of arrays don't exist:
function zip() {
var args = [].slice.call(arguments);
var shortest = args.length==0 ? [] : args.reduce(function(a,b){
return a.length<b.length ? a : b
});
return shortest.map(function(_,i){
return args.map(function(array){return array[i]})
});
}
// > zip([1,2],[11,22],[111,222,333])
// [[1,11,111],[2,22,222]]]
// > zip()
// []
This will mimic Python's itertools.zip_longest behavior, inserting undefined where arrays are not defined:
function zip() {
var args = [].slice.call(arguments);
var longest = args.reduce(function(a,b){
return a.length>b.length ? a : b
}, []);
return longest.map(function(_,i){
return args.map(function(array){return array[i]})
});
}
// > zip([1,2],[11,22],[111,222,333])
// [[1,11,111],[2,22,222],[null,null,333]]
// > zip()
// []
If you use these last two version (variadic aka. multiple-argument versions), then zip is no longer its own inverse. To mimic the zip(*[...]) idiom from Python, you will need to do zip.apply(this, [...]) when you want to invert the zip function or if you want to similarly have a variable number of lists as input.
addendum:
To make this handle any iterable (e.g. in Python you can use zip on strings, ranges, map objects, etc.), you could define the following:
function iterView(iterable) {
// returns an array equivalent to the iterable
}
However if you write zip in the following way, even that won't be necessary:
function zip(arrays) {
return Array.apply(null,Array(arrays[0].length)).map(function(_,i){
return arrays.map(function(array){return array[i]})
});
}
Demo:
> JSON.stringify( zip(['abcde',[1,2,3,4,5]]) )
[["a",1],["b",2],["c",3],["d",4],["e",5]]
(Or you could use a range(...) Python-style function if you've written one already. Eventually you will be able to use ECMAScript array comprehensions or generators.)
Check out the library Underscore.
Underscore provides over 100 functions that support both your favorite workaday functional helpers: map, filter, invoke ā€” as well as more specialized goodies: function binding, javascript templating, creating quick indexes, deep equality testing, and so on.
ā€“ Say the people who made it
I recently started using it specifically for the zip() function and it has left a great first impression. I am using jQuery and CoffeeScript, and it just goes perfectly with them. Underscore picks up right where they leave off and so far it hasn't let me down. Oh by the way, it's only 3kb minified.
Check it out:
_.zip(['moe', 'larry', 'curly'], [30, 40, 50], [true, false, false]);
// returns [["moe", 30, true], ["larry", 40, false], ["curly", 50, false]]
Modern ES6 example with a generator:
function *zip (...iterables){
let iterators = iterables.map(i => i[Symbol.iterator]() )
while (true) {
let results = iterators.map(iter => iter.next() )
if (results.some(res => res.done) ) return
else yield results.map(res => res.value )
}
}
First, we get a list of iterables as iterators. This usually happens transparently, but here we do it explicitly, as we yield step-by-step until one of them is exhausted. We check if any of results (using the .some() method) in the given array is exhausted, and if so, we break the while loop.
In addition to ninjagecko's excellent and comprehensive answer, all it takes to zip two JS-arrays into a "tuple-mimic" is:
//Arrays: aIn, aOut
Array.prototype.map.call( aIn, function(e,i){return [e, aOut[i]];})
Explanation:
Since Javascript doesn't have a tuples type, functions for tuples, lists and sets wasn't a high priority in the language specification.
Otherwise, similar behavior is accessible in a straightforward manner via Array map in JS >1.6. (map is actually often implemented by JS engine makers in many >JS 1.4 engines, despite not specified).
The major difference to Python's zip, izip,... results from map's functional style, since map requires a function-argument. Additionally it is a function of the Array-instance. One may use Array.prototype.map instead, if an extra declaration for the input is an issue.
Example:
_tarrin = [0..constructor, function(){}, false, undefined, '', 100, 123.324,
2343243243242343242354365476453654625345345, 'sdf23423dsfsdf',
'sdf2324.234dfs','234,234fsf','100,100','100.100']
_parseInt = function(i){return parseInt(i);}
_tarrout = _tarrin.map(_parseInt)
_tarrin.map(function(e,i,a){return [e, _tarrout[i]]})
Result:
//'('+_tarrin.map(function(e,i,a){return [e, _tarrout[i]]}).join('),\n(')+')'
>>
(function Number() { [native code] },NaN),
(function (){},NaN),
(false,NaN),
(,NaN),
(,NaN),
(100,100),
(123.324,123),
(2.3432432432423434e+42,2),
(sdf23423dsfsdf,NaN),
(sdf2324.234dfs,NaN),
(234,234fsf,234),
(100,100,100),
(100.100,100)
Related Performance:
Using map over for-loops:
See: What is the most efficient way of merging [1,2] and [7,8] into [[1,7], [2,8]]
Note: the base types such as false and undefined do not posess a prototypal object-hierarchy and thus do not expose a toString function. Hence these are shown as empty in the output.
As parseInt's second argument is the base/number radix, to which to convert the number to, and since map passes the index as the second argument to its argument-function, a wrapper function is used.
Along other Python-like functions, pythonic offers a zip function, with the extra benefit of returning a lazy evaluated Iterator, similar to the behaviour of its Python counterpart:
import {zip, zipLongest} from 'pythonic';
const arr1 = ['a', 'b'];
const arr2 = ['c', 'd', 'e'];
for (const [first, second] of zip(arr1, arr2))
console.log(`first: ${first}, second: ${second}`);
// first: a, second: c
// first: b, second: d
for (const [first, second] of zipLongest(arr1, arr2))
console.log(`first: ${first}, second: ${second}`);
// first: a, second: c
// first: b, second: d
// first: undefined, second: e
// unzip
const [arrayFirst, arraySecond] = [...zip(...zip(arr1, arr2))];
Disclosure I'm author and maintainer of Pythonic
Python has two function to zip sequences: zip and itertools.zip_longest. An implementation in Javascript for the same functionality is this:
Implementation of Python`s zip on JS/ES6
const zip = (...arrays) => {
const length = Math.min(...arrays.map(arr => arr.length));
return Array.from({ length }, (value, index) => arrays.map((array => array[index])));
};
Results in:
console.log(zip(
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[11, 221]
));
[ [ 1, 667, 111, 11 ] ]
console.log(zip(
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111, 212, 323, 433, '1111']
));
[ [ 1, 667, 111 ], [ 2, false, 212 ], [ 3, -378, 323 ], [ 'a',
'337', 433 ] ]
console.log(zip(
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[]
));
[]
Implementation of Python`s zip_longest on JS/ES6
(https://docs.python.org/3.5/library/itertools.html?highlight=zip_longest#itertools.zip_longest)
const zipLongest = (placeholder = undefined, ...arrays) => {
const length = Math.max(...arrays.map(arr => arr.length));
return Array.from(
{ length }, (value, index) => arrays.map(
array => array.length - 1 >= index ? array[index] : placeholder
)
);
};
Results:
console.log(zipLongest(
undefined,
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[]
));
[ [ 1, 667, 111, undefined ], [ 2, false, undefined, undefined ],
[ 3, -378, undefined, undefined ], [ 'a', '337', undefined,
undefined ] ]
console.log(zipLongest(
null,
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[]
));
[ [ 1, 667, 111, null ], [ 2, false, null, null ], [ 3, -378,
null, null ], [ 'a', '337', null, null ] ]
console.log(zipLongest(
'Is None',
[1, 2, 3, 'a'],
[667, false, -378, '337'],
[111],
[]
));
[ [ 1, 667, 111, 'Is None' ], [ 2, false, 'Is None', 'Is None' ],
[ 3, -378, 'Is None', 'Is None' ], [ 'a', '337', 'Is None', 'Is
None' ] ]
You can make utility function by using ES6.
console.json = obj => console.log(JSON.stringify(obj));
const zip = (arr, ...arrs) =>
arr.map((val, i) => arrs.reduce((a, arr) => [...a, arr[i]], [val]));
// Example
const array1 = [1, 2, 3];
const array2 = ['a','b','c'];
const array3 = [4, 5, 6];
console.json(zip(array1, array2)); // [[1,"a"],[2,"b"],[3,"c"]]
console.json(zip(array1, array2, array3)); // [[1,"a",4],[2,"b",5],[3,"c",6]]
However, in above solution length of the first array defines the length of the output array.
Here is the solution in which you have more control over it. It's bit complex but worth it.
function _zip(func, args) {
const iterators = args.map(arr => arr[Symbol.iterator]());
let iterateInstances = iterators.map((i) => i.next());
ret = []
while(iterateInstances[func](it => !it.done)) {
ret.push(iterateInstances.map(it => it.value));
iterateInstances = iterators.map((i) => i.next());
}
return ret;
}
const array1 = [1, 2, 3];
const array2 = ['a','b','c'];
const array3 = [4, 5, 6];
const zipShort = (...args) => _zip('every', args);
const zipLong = (...args) => _zip('some', args);
console.log(zipShort(array1, array2, array3)) // [[1, 'a', 4], [2, 'b', 5], [3, 'c', 6]]
console.log(zipLong([1,2,3], [4,5,6, 7]))
// [
// [ 1, 4 ],
// [ 2, 5 ],
// [ 3, 6 ],
// [ undefined, 7 ]]
1. Npm Module: zip-array
I found an npm module that can be used as a javascript version of python zip:
zip-array - A javascript equivalent of Python's zip function. Merges together the values of each of the arrays.
https://www.npmjs.com/package/zip-array
2. tf.data.zip() in Tensorflow.js
Another alternate choice is for Tensorflow.js users: if you need a zip function in python to work with tensorflow datasets in Javascript, you can use tf.data.zip() in Tensorflow.js.
tf.data.zip() in Tensorflow.js documented at here
Original answer (see update below)
I modified flm's nifty answer to take an arbitrary number of arrays:
function* zip(arrays, i = 0) {
while (i<Math.min(...arrays.map(({length})=>length))) {
yield arrays.map((arr, j) => arr[j < arrays.length - 1 ? i : i++])
}
}
Updated answer
As pointed out by Tom Pohl this function can't deal with arrays with falsy values in. Here is an updated/improved version that can deal with any types and also unequal length arrays:
function* zip(arrays, i = 0) {
while (i<Math.min(...arrays.map(arr=>arr.length))) {
yield arrays.map((arr, j) => arr[j < arrays.length - 1 ? i : i++])
}
}
const arr1 = [false,0,1,2]
const arr2 = [100,null,99,98,97]
const arr3 = [7,8,undefined,"monkey","banana"]
console.log(...zip([arr1,arr2,arr3]))
Not built-in to Javascript itself. Some of the common Javascript frameworks (such as Prototype) provide an implementation, or you can write your own.
Like #Brandon, I recommend Underscore's zip function. However, it acts like zip_longest, appending undefined values as needed to return something the length of the longest input.
I used the mixin method to extend underscore with a zipShortest, which acts like Python's zip, based off of the library's own source for zip.
You can add the following to your common JS code and then call it as if it were part of underscore: _.zipShortest([1,2,3], ['a']) returns [[1, 'a']], for example.
// Underscore library addition - zip like python does, dominated by the shortest list
// The default injects undefineds to match the length of the longest list.
_.mixin({
zipShortest : function() {
var args = Array.Prototype.slice.call(arguments);
var length = _.min(_.pluck(args, 'length')); // changed max to min
var results = new Array(length);
for (var i = 0; i < length; i++) {
results[i] = _.pluck(args, "" + i);
}
return results;
}});
A variation of the lazy generator solution:
function* iter(it) {
yield* it;
}
function* zip(...its) {
its = its.map(iter);
while (true) {
let rs = its.map(it => it.next());
if (rs.some(r => r.done))
return;
yield rs.map(r => r.value);
}
}
for (let r of zip([1,2,3], [4,5,6,7], [8,9,0,11,22]))
console.log(r.join())
// the only change for "longest" is some -> every
function* zipLongest(...its) {
its = its.map(iter);
while (true) {
let rs = its.map(it => it.next());
if (rs.every(r => r.done))
return;
yield rs.map(r => r.value);
}
}
for (let r of zipLongest([1,2,3], [4,5,6,7], [8,9,0,11,22]))
console.log(r.join())
And this is the python's classic "n-group" idiom zip(*[iter(a)]*n):
triples = [...zip(...Array(3).fill(iter(a)))]
ES2020 shortest variant:
function * zip(arr1, arr2, i = 0) {
while(arr1[i] || arr2[i]) yield [arr1[i], arr2[i++]].filter(x => !!x);
}
[ ...zip(arr1, arr2) ] // result
You could reduce the array of arrays and map new array by taking the result of the index of the inner array.
var array1 = [1, 2, 3],
array2 = ['a','b','c'],
array3 = [4, 5, 6],
array = [array1, array2, array3],
transposed = array.reduce((r, a) => a.map((v, i) => (r[i] || []).concat(v)), []);
console.log(transposed);
Fun with spread.
const
transpose = (r, a) => a.map((v, i) => [...(r[i] || []), v]),
array1 = [1, 2, 3],
array2 = ['a','b','c'],
array3 = [4, 5, 6],
transposed = [array1, array2, array3].reduce(transpose, []);
console.log(transposed);
I took a run at this in pure JS wondering how the plugins posted above got the job done. Here's my result. I'll preface this by saying that I have no idea how stable this will be in IE and the like. It's just a quick mockup.
init();
function init() {
var one = [0, 1, 2, 3];
var two = [4, 5, 6, 7];
var three = [8, 9, 10, 11, 12];
var four = zip(one, two, one);
//returns array
//four = zip(one, two, three);
//returns false since three.length !== two.length
console.log(four);
}
function zip() {
for (var i = 0; i < arguments.length; i++) {
if (!arguments[i].length || !arguments.toString()) {
return false;
}
if (i >= 1) {
if (arguments[i].length !== arguments[i - 1].length) {
return false;
}
}
}
var zipped = [];
for (var j = 0; j < arguments[0].length; j++) {
var toBeZipped = [];
for (var k = 0; k < arguments.length; k++) {
toBeZipped.push(arguments[k][j]);
}
zipped.push(toBeZipped);
}
return zipped;
}
It's not bulletproof, but it's still interesting.
A generator approach to pythons zip function.
function* zip(...arrs){
for(let i = 0; i < arrs[0].length; i++){
a = arrs.map(e=>e[i])
if(a.indexOf(undefined) == -1 ){yield a }else{return undefined;}
}
}
// use as multiple iterators
for( let [a,b,c] of zip([1, 2, 3, 4], ['a', 'b', 'c', 'd'], ['hi', 'hello', 'howdy', 'how are you']) )
console.log(a,b,c)
// creating new array with the combined arrays
let outputArr = []
for( let arr of zip([1, 2, 3, 4], ['a', 'b', 'c', 'd'], ['hi', 'hello', 'howdy', 'how are you']) )
outputArr.push(arr)
I have created a simple function to do so with a option to provide an zipper function
function zip(zipper, ...arrays) {
if (zipper instanceof Array) {
arrays.unshift(zipper)
zipper = (...elements) => elements
}
const length = Math.min(...arrays.map(array => array.length))
const zipped = []
for (let i = 0; i < length; i++) {
zipped.push(zipper(...arrays.map(array => array[i])))
}
return zipped
}
https://gist.github.com/AmrIKhudair/4b740149c29c492859e00f451832975b
I'm not a javascript guy but I feel like many of these answers are trying to find the cutest and most clever solution using Array.map which is fine, but for someone like me that doesn't use javascript every day here are some alternatives that might possibly be a bit more readable.
Maybe a way to avoid some cute and clever code would be:
function zip(a,b){
// pre-allocate an array to hold the results
rval=Array(Math.max(a.length, b.length));
for(i=0; i<rval.length; i++){
rval[i]=[a[i],b[i]]
}
return rval
}
If you like generators:
function* _zip(a,b){
len = Math.max(a.length, b.length) // handle different sized arrays
for(i=0; i<len; i++) { yield [a[i],b[i]] }
}
Or if you really want to use Array.map:
function map(a,b){
x = a.length > b.length ? a : b // call map on the biggest array
return x.map((_,i)=>[a[i],b[i]])
}
As I said, I'm not an everyday javascript guy so, these aren't going to be the most elegant solutions but they are readable to me.
Below is a fast and efficient way of doing this, using iter-ops library, operator zip:
const {pipe, zip} = require('iter-ops');
const i = pipe(array1, zip(array2, array3));
console.log(...i); //=> [ 1, 'a', 4 ] [ 2, 'b', 5 ] [ 3, 'c', 6 ]
The library processes all inputs as iterables, so they are iterated over just once. And it can handle, in the same way, all types of iterable objects - Iterable, AsyncIterable, Iterator, AsyncIterator.
P.S. I'm the author of iter-ops.
The Mochikit library provides this and many other Python-like functions. developer of Mochikit is also a Python fan, so it has the general style of Python, and also the wraps the async calls in a twisted-like framework.
There is no equivalent function. If you have only a few arrays you should use a for loop to get an index and then use the index to access the arrays:
var array1 = [1, 2, 3];
var array2 = ['a','b','c'];
for (let i = 0; i < Math.min(array1.length, array2.length); i++) {
doStuff(array1[i], array2[i]);
}
You can have an inner loop over the arrays if you have more.
Here is my solution
let zip = (a, b) => (a.length < b.length
? a.map((e, i) => [e, b[i]])
: b.map((e, i) => [a[i], e]))
This shaves a line off Ddi's iterator-based answer:
function* zip(...toZip) {
const iterators = toZip.map((arg) => arg[Symbol.iterator]());
const next = () => toZip = iterators.map((iter) => iter.next());
while (next().every((item) => !item.done)) {
yield toZip.map((item) => item.value);
}
}
If you are fine with ES6:
const zip = (arr,...arrs) =>(
arr.map(
(v,i) => arrs.reduce((a,arr)=>[...a, arr[i]], [v])))

Filter out duplicates from .map function [duplicate]

I have an array of numbers that I need to make sure are unique. I found the code snippet below on the internet and it works great until the array has a zero in it. I found this other script here on Stack Overflow that looks almost exactly like it, but it doesn't fail.
So for the sake of helping me learn, can someone help me determine where the prototype script is going wrong?
Array.prototype.getUnique = function() {
var o = {}, a = [], i, e;
for (i = 0; e = this[i]; i++) {o[e] = 1};
for (e in o) {a.push (e)};
return a;
}
More answers from duplicate question:
Remove duplicate values from JS array
Similar question:
Get all non-unique values (i.e.: duplicate/more than one occurrence) in an array
With JavaScript 1.6 / ECMAScript 5 you can use the native filter method of an Array in the following way to get an array with unique values:
function onlyUnique(value, index, array) {
return self.indexOf(value) === index;
}
// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);
console.log(unique); // ['a', 1, 2, '1']
The native method filter will loop through the array and leave only those entries that pass the given callback function onlyUnique.
onlyUnique checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.
This solution works without any extra library like jQuery or prototype.js.
It works for arrays with mixed value types too.
For old Browsers (<ie9), that do not support the native methods filter and indexOf you can find work arounds in the MDN documentation for filter and indexOf.
If you want to keep the last occurrence of a value, simply replace indexOf with lastIndexOf.
With ES6 this can be shorten to:
// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((value, index, array) => array.indexOf(value) === index);
console.log(unique); // unique is ['a', 1, 2, '1']
Thanks to Camilo Martin for hint in comment.
ES6 has a native object Set to store unique values. To get an array with unique values you could now do this:
var myArray = ['a', 1, 'a', 2, '1'];
let unique = [...new Set(myArray)];
console.log(unique); // unique is ['a', 1, 2, '1']
The constructor of Set takes an iterable object, like an Array, and the spread operator ... transform the set back into an Array. Thanks to Lukas Liese for hint in comment.
Updated answer for ES6/ES2015: Using the Set and the spread operator (thanks le-m), the single line solution is:
let uniqueItems = [...new Set(items)]
Which returns
[4, 5, 6, 3, 2, 23, 1]
I split all answers to 4 possible solutions:
Use object { } to prevent duplicates
Use helper array [ ]
Use filter + indexOf
Bonus! ES6 Sets method.
Here's sample codes found in answers:
Use object { } to prevent duplicates
function uniqueArray1( ar ) {
var j = {};
ar.forEach( function(v) {
j[v+ '::' + typeof v] = v;
});
return Object.keys(j).map(function(v){
return j[v];
});
}
Use helper array [ ]
function uniqueArray2(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Use filter + indexOf
function uniqueArray3(a) {
function onlyUnique(value, index, self) {
return self.indexOf(value) === index;
}
// usage
var unique = a.filter( onlyUnique ); // returns ['a', 1, 2, '1']
return unique;
}
Use ES6 [...new Set(a)]
function uniqueArray4(a) {
return [...new Set(a)];
}
And I wondered which one is faster. I've made sample Google Sheet to test functions. Note: ECMA 6 is not avaliable in Google Sheets, so I can't test it.
Here's the result of tests:
I expected to see that code using object { } will win because it uses hash. So I'm glad that tests showed the best results for this algorithm in Chrome and IE. Thanks to #rab for the code.
Update 2020
Google Script enabled ES6 Engine. Now I tested the last code with Sets and it appeared faster than the object method.
You can also use underscore.js.
console.log(_.uniq([1, 2, 1, 3, 1, 4]));
<script src="http://underscorejs.org/underscore-min.js"></script>
which will return:
[1, 2, 3, 4]
One Liner, Pure JavaScript
With ES6 syntax
list = list.filter((x, i, a) => a.indexOf(x) == i)
x --> item in array
i --> index of item
a --> array reference, (in this case "list")
With ES5 syntax
list = list.filter(function (x, i, a) {
return a.indexOf(x) == i;
});
Browser Compatibility: IE9+
Remove duplicates using Set.
Array with duplicates
const withDuplicates = [2, 2, 5, 5, 1, 1, 2, 2, 3, 3];
Get a new array without duplicates by using Set
const withoutDuplicates = Array.from(new Set(withDuplicates));
A shorter version
const withoutDuplicates = [...new Set(withDuplicates)];
Result: [2, 5, 1, 3]
Many of the answers here may not be useful to beginners. If de-duping an array is difficult, will they really know about the prototype chain, or even jQuery?
In modern browsers, a clean and simple solution is to store data in a Set, which is designed to be a list of unique values.
const cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
const uniqueCars = Array.from(new Set(cars));
console.log(uniqueCars);
The Array.from is useful to convert the Set back to an Array so that you have easy access to all of the awesome methods (features) that arrays have. There are also other ways of doing the same thing. But you may not need Array.from at all, as Sets have plenty of useful features like forEach.
If you need to support old Internet Explorer, and thus cannot use Set, then a simple technique is to copy items over to a new array while checking beforehand if they are already in the new array.
// Create a list of cars, with duplicates.
var cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
// Create a list of unique cars, to put a car in if we haven't already.
var uniqueCars = [];
// Go through each car, one at a time.
cars.forEach(function (car) {
// The code within the following block runs only if the
// current car does NOT exist in the uniqueCars list
// - a.k.a. prevent duplicates
if (uniqueCars.indexOf(car) === -1) {
// Since we now know we haven't seen this car before,
// copy it to the end of the uniqueCars list.
uniqueCars.push(car);
}
});
To make this instantly reusable, let's put it in a function.
function deduplicate(data) {
if (data.length > 0) {
var result = [];
data.forEach(function (elem) {
if (result.indexOf(elem) === -1) {
result.push(elem);
}
});
return result;
}
}
So to get rid of the duplicates, we would now do this.
var uniqueCars = deduplicate(cars);
The deduplicate(cars) part becomes the thing we named result when the function completes.
Just pass it the name of any array you like.
Using ES6 new Set
var array = [3,7,5,3,2,5,2,7];
var unique_array = [...new Set(array)];
console.log(unique_array); // output = [3,7,5,2]
Using For Loop
var array = [3,7,5,3,2,5,2,7];
for(var i=0;i<array.length;i++) {
for(var j=i+1;j<array.length;j++) {
if(array[i]===array[j]) {
array.splice(j,1);
}
}
}
console.log(array); // output = [3,7,5,2]
I have since found a nice method that uses jQuery
arr = $.grep(arr, function(v, k){
return $.inArray(v ,arr) === k;
});
Note: This code was pulled from Paul Irish's duck punching post - I forgot to give credit :P
Magic
a.filter(e=>!(t[e]=e in t))
O(n) performance - we assume your array is in a and t={}. Explanation here (+Jeppe impr.)
let unique = (a,t={}) => a.filter(e=>!(t[e]=e in t));
// "stand-alone" version working with global t:
// a1.filter((t={},e=>!(t[e]=e in t)));
// Test data
let a1 = [5,6,0,4,9,2,3,5,0,3,4,1,5,4,9];
let a2 = [[2, 17], [2, 17], [2, 17], [1, 12], [5, 9], [1, 12], [6, 2], [1, 12]];
let a3 = ['Mike', 'Adam','Matt', 'Nancy', 'Adam', 'Jenny', 'Nancy', 'Carl'];
// Results
console.log(JSON.stringify( unique(a1) ))
console.log(JSON.stringify( unique(a2) ))
console.log(JSON.stringify( unique(a3) ))
The simplest, and fastest (in Chrome) way of doing this:
Array.prototype.unique = function() {
var a = [];
for (var i=0, l=this.length; i<l; i++)
if (a.indexOf(this[i]) === -1)
a.push(this[i]);
return a;
}
Simply goes through every item in the array, tests if that item is already in the list, and if it's not, pushes to the array that gets returned.
According to JSBench, this function is the fastest of the ones I could find anywhere - feel free to add your own though.
The non-prototype version:
function uniques(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Sorting
When also needing to sort the array, the following is the fastest:
Array.prototype.sortUnique = function() {
this.sort();
var last_i;
for (var i=0;i<this.length;i++)
if ((last_i = this.lastIndexOf(this[i])) !== i)
this.splice(i+1, last_i-i);
return this;
}
or non-prototype:
function sortUnique(arr) {
arr.sort();
var last_i;
for (var i=0;i<arr.length;i++)
if ((last_i = arr.lastIndexOf(arr[i])) !== i)
arr.splice(i+1, last_i-i);
return arr;
}
This is also faster than the above method in most non-Chrome browsers.
We can do this using ES6 sets:
var duplicatesArray = [1, 2, 3, 4, 5, 1, 1, 1, 2, 3, 4];
var uniqueArray = [...new Set(duplicatesArray)];
console.log(uniqueArray); // [1,2,3,4,5]
["Defects", "Total", "Days", "City", "Defects"].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
[0,1,2,0,3,2,1,5].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
After looking into all the 90+ answers here, I saw there is room for one more:
Array.includes has a very handy second-parameter: "fromIndex", so by using it, every iteration of the filter callback method will search the array, starting from [current index] + 1 which guarantees not to include currently filtered item in the lookup and also saves time.
Note - this solution does not retain the order, as it removed duplicated items from left to right, but it wins the Set trick if the Array is a collection of Objects.
// šŸš© šŸš© šŸš©
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
list.filter((v,i) => !list.includes(v,i+1))
)
// [0,1,3,"b",4,5,2,"a"]
Explanation:
For example, lets assume the filter function is currently iterating at index 2) and the value at that index happens to be 2. The section of the array that is then scanned for duplicates (includes method) is everything after index 2 (i+1):
šŸ‘‡ šŸ‘‡
[0, 1, 2, 2 ,3 ,'a', 'b', 4, 5, 2, 'a']
šŸ‘† |---------------------------|
And since the currently filtered item's value 2 is included in the rest of the array, it will be filtered out, because of the leading exclamation mark which negates the filter rule.
If order is important, use this method:
// šŸš© šŸš© šŸš©
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
// Initialize with empty array and fill with non-duplicates
list.reduce((acc, v) => (!acc.includes(v) && acc.push(v), acc), [])
)
// [0,1,2,3,"a","b",4,5]
This has been answered a lot, but it didn't address my particular need.
Many answers are like this:
a.filter((item, pos, self) => self.indexOf(item) === pos);
But this doesn't work for arrays of complex objects.
Say we have an array like this:
const a = [
{ age: 4, name: 'fluffy' },
{ age: 5, name: 'spot' },
{ age: 2, name: 'fluffy' },
{ age: 3, name: 'toby' },
];
If we want the objects with unique names, we should use array.prototype.findIndex instead of array.prototype.indexOf:
a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);
This prototype getUnique is not totally correct, because if i have a Array like: ["1",1,2,3,4,1,"foo"] it will return ["1","2","3","4"] and "1" is string and 1 is a integer; they are different.
Here is a correct solution:
Array.prototype.unique = function(a){
return function(){ return this.filter(a) }
}(function(a,b,c){ return c.indexOf(a,b+1) < 0 });
using:
var foo;
foo = ["1",1,2,3,4,1,"foo"];
foo.unique();
The above will produce ["1",2,3,4,1,"foo"].
You can simlply use the built-in functions Array.prototype.filter() and Array.prototype.indexOf()
array.filter((x, y) => array.indexOf(x) == y)
var arr = [1, 2, 3, 3, 4, 5, 5, 5, 6, 7, 8, 9, 6, 9];
var newarr = arr.filter((x, y) => arr.indexOf(x) == y);
console.log(newarr);
[...new Set(duplicates)]
This is the simplest one and referenced from MDN Web Docs.
const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]
Array.prototype.getUnique = function() {
var o = {}, a = []
for (var i = 0; i < this.length; i++) o[this[i]] = 1
for (var e in o) a.push(e)
return a
}
Without extending Array.prototype (it is said to be a bad practice) or using jquery/underscore, you can simply filter the array.
By keeping last occurrence:
function arrayLastUnique(array) {
return array.filter(function (a, b, c) {
// keeps last occurrence
return c.indexOf(a, b + 1) < 0;
});
},
or first occurrence:
function arrayFirstUnique(array) {
return array.filter(function (a, b, c) {
// keeps first occurrence
return c.indexOf(a) === b;
});
},
Well, it's only javascript ECMAScript 5+, which means only IE9+, but it's nice for a development in native HTML/JS (Windows Store App, Firefox OS, Sencha, Phonegap, Titanium, ...).
That's because 0 is a falsy value in JavaScript.
this[i] will be falsy if the value of the array is 0 or any other falsy value.
Now using sets you can remove duplicates and convert them back to the array.
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
console.log([...new Set(names)])
Another solution is to use sort & filter
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);
If you're using Prototype framework there is no need to do 'for' loops, you can use http://prototypejs.org/doc/latest/language/Array/prototype/uniq/ like this:
var a = Array.uniq();
Which will produce a duplicate array with no duplicates. I came across your question searching a method to count distinct array records so after uniq() I used size() and there was my simple result.
p.s. Sorry if i mistyped something
edit: if you want to escape undefined records you may want to add compact() before, like this:
var a = Array.compact().uniq();
I had a slightly different problem where I needed to remove objects with duplicate id properties from an array. this worked.
let objArr = [{
id: '123'
}, {
id: '123'
}, {
id: '456'
}];
objArr = objArr.reduce((acc, cur) => [
...acc.filter((obj) => obj.id !== cur.id), cur
], []);
console.log(objArr);
If you're okay with extra dependencies, or you already have one of the libraries in your codebase, you can remove duplicates from an array in place using LoDash (or Underscore).
Usage
If you don't have it in your codebase already, install it using npm:
npm install lodash
Then use it as follows:
import _ from 'lodash';
let idArray = _.uniq ([
1,
2,
3,
3,
3
]);
console.dir(idArray);
Out:
[ 1, 2, 3 ]
I'm not sure why Gabriel Silveira wrote the function that way but a simpler form that works for me just as well and without the minification is:
Array.prototype.unique = function() {
return this.filter(function(value, index, array) {
return array.indexOf(value, index + 1) < 0;
});
};
or in CoffeeScript:
Array.prototype.unique = ->
this.filter( (value, index, array) ->
array.indexOf(value, index + 1) < 0
)
Finding unique Array values in simple method
function arrUnique(a){
var t = [];
for(var x = 0; x < a.length; x++){
if(t.indexOf(a[x]) == -1)t.push(a[x]);
}
return t;
}
arrUnique([1,4,2,7,1,5,9,2,4,7,2]) // [1, 4, 2, 7, 5, 9]
It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:
Array.prototype.getUnique = function(){
var u = {}, a = [];
for (var i = 0, l = this.length; i < l; ++i) {
if (u.hasOwnProperty(this[i])) {
continue;
}
a.push(this[i]);
u[this[i]] = 1;
}
return a;
}
If you do have a mixed-type array, you can serialize the hash key:
Array.prototype.getUnique = function() {
var hash = {}, result = [], key;
for ( var i = 0, l = this.length; i < l; ++i ) {
key = JSON.stringify(this[i]);
if ( !hash.hasOwnProperty(key) ) {
hash[key] = true;
result.push(this[i]);
}
}
return result;
}
strange this hasn't been suggested before.. to remove duplicates by object key (id below) in an array you can do something like this:
const uniqArray = array.filter((obj, idx, arr) => (
arr.findIndex((o) => o.id === obj.id) === idx
))
For an object-based array with some unique id's, I have a simple solution through which you can sort in linear complexity
function getUniqueArr(arr){
const mapObj = {};
arr.forEach(a => {
mapObj[a.id] = a
})
return Object.values(mapObj);
}

unique values of attribute ā€œzā€ [duplicate]

I have an array of numbers that I need to make sure are unique. I found the code snippet below on the internet and it works great until the array has a zero in it. I found this other script here on Stack Overflow that looks almost exactly like it, but it doesn't fail.
So for the sake of helping me learn, can someone help me determine where the prototype script is going wrong?
Array.prototype.getUnique = function() {
var o = {}, a = [], i, e;
for (i = 0; e = this[i]; i++) {o[e] = 1};
for (e in o) {a.push (e)};
return a;
}
More answers from duplicate question:
Remove duplicate values from JS array
Similar question:
Get all non-unique values (i.e.: duplicate/more than one occurrence) in an array
With JavaScript 1.6 / ECMAScript 5 you can use the native filter method of an Array in the following way to get an array with unique values:
function onlyUnique(value, index, array) {
return self.indexOf(value) === index;
}
// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);
console.log(unique); // ['a', 1, 2, '1']
The native method filter will loop through the array and leave only those entries that pass the given callback function onlyUnique.
onlyUnique checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.
This solution works without any extra library like jQuery or prototype.js.
It works for arrays with mixed value types too.
For old Browsers (<ie9), that do not support the native methods filter and indexOf you can find work arounds in the MDN documentation for filter and indexOf.
If you want to keep the last occurrence of a value, simply replace indexOf with lastIndexOf.
With ES6 this can be shorten to:
// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((value, index, array) => array.indexOf(value) === index);
console.log(unique); // unique is ['a', 1, 2, '1']
Thanks to Camilo Martin for hint in comment.
ES6 has a native object Set to store unique values. To get an array with unique values you could now do this:
var myArray = ['a', 1, 'a', 2, '1'];
let unique = [...new Set(myArray)];
console.log(unique); // unique is ['a', 1, 2, '1']
The constructor of Set takes an iterable object, like an Array, and the spread operator ... transform the set back into an Array. Thanks to Lukas Liese for hint in comment.
Updated answer for ES6/ES2015: Using the Set and the spread operator (thanks le-m), the single line solution is:
let uniqueItems = [...new Set(items)]
Which returns
[4, 5, 6, 3, 2, 23, 1]
I split all answers to 4 possible solutions:
Use object { } to prevent duplicates
Use helper array [ ]
Use filter + indexOf
Bonus! ES6 Sets method.
Here's sample codes found in answers:
Use object { } to prevent duplicates
function uniqueArray1( ar ) {
var j = {};
ar.forEach( function(v) {
j[v+ '::' + typeof v] = v;
});
return Object.keys(j).map(function(v){
return j[v];
});
}
Use helper array [ ]
function uniqueArray2(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Use filter + indexOf
function uniqueArray3(a) {
function onlyUnique(value, index, self) {
return self.indexOf(value) === index;
}
// usage
var unique = a.filter( onlyUnique ); // returns ['a', 1, 2, '1']
return unique;
}
Use ES6 [...new Set(a)]
function uniqueArray4(a) {
return [...new Set(a)];
}
And I wondered which one is faster. I've made sample Google Sheet to test functions. Note: ECMA 6 is not avaliable in Google Sheets, so I can't test it.
Here's the result of tests:
I expected to see that code using object { } will win because it uses hash. So I'm glad that tests showed the best results for this algorithm in Chrome and IE. Thanks to #rab for the code.
Update 2020
Google Script enabled ES6 Engine. Now I tested the last code with Sets and it appeared faster than the object method.
You can also use underscore.js.
console.log(_.uniq([1, 2, 1, 3, 1, 4]));
<script src="http://underscorejs.org/underscore-min.js"></script>
which will return:
[1, 2, 3, 4]
One Liner, Pure JavaScript
With ES6 syntax
list = list.filter((x, i, a) => a.indexOf(x) == i)
x --> item in array
i --> index of item
a --> array reference, (in this case "list")
With ES5 syntax
list = list.filter(function (x, i, a) {
return a.indexOf(x) == i;
});
Browser Compatibility: IE9+
Remove duplicates using Set.
Array with duplicates
const withDuplicates = [2, 2, 5, 5, 1, 1, 2, 2, 3, 3];
Get a new array without duplicates by using Set
const withoutDuplicates = Array.from(new Set(withDuplicates));
A shorter version
const withoutDuplicates = [...new Set(withDuplicates)];
Result: [2, 5, 1, 3]
Many of the answers here may not be useful to beginners. If de-duping an array is difficult, will they really know about the prototype chain, or even jQuery?
In modern browsers, a clean and simple solution is to store data in a Set, which is designed to be a list of unique values.
const cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
const uniqueCars = Array.from(new Set(cars));
console.log(uniqueCars);
The Array.from is useful to convert the Set back to an Array so that you have easy access to all of the awesome methods (features) that arrays have. There are also other ways of doing the same thing. But you may not need Array.from at all, as Sets have plenty of useful features like forEach.
If you need to support old Internet Explorer, and thus cannot use Set, then a simple technique is to copy items over to a new array while checking beforehand if they are already in the new array.
// Create a list of cars, with duplicates.
var cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
// Create a list of unique cars, to put a car in if we haven't already.
var uniqueCars = [];
// Go through each car, one at a time.
cars.forEach(function (car) {
// The code within the following block runs only if the
// current car does NOT exist in the uniqueCars list
// - a.k.a. prevent duplicates
if (uniqueCars.indexOf(car) === -1) {
// Since we now know we haven't seen this car before,
// copy it to the end of the uniqueCars list.
uniqueCars.push(car);
}
});
To make this instantly reusable, let's put it in a function.
function deduplicate(data) {
if (data.length > 0) {
var result = [];
data.forEach(function (elem) {
if (result.indexOf(elem) === -1) {
result.push(elem);
}
});
return result;
}
}
So to get rid of the duplicates, we would now do this.
var uniqueCars = deduplicate(cars);
The deduplicate(cars) part becomes the thing we named result when the function completes.
Just pass it the name of any array you like.
Using ES6 new Set
var array = [3,7,5,3,2,5,2,7];
var unique_array = [...new Set(array)];
console.log(unique_array); // output = [3,7,5,2]
Using For Loop
var array = [3,7,5,3,2,5,2,7];
for(var i=0;i<array.length;i++) {
for(var j=i+1;j<array.length;j++) {
if(array[i]===array[j]) {
array.splice(j,1);
}
}
}
console.log(array); // output = [3,7,5,2]
I have since found a nice method that uses jQuery
arr = $.grep(arr, function(v, k){
return $.inArray(v ,arr) === k;
});
Note: This code was pulled from Paul Irish's duck punching post - I forgot to give credit :P
Magic
a.filter(e=>!(t[e]=e in t))
O(n) performance - we assume your array is in a and t={}. Explanation here (+Jeppe impr.)
let unique = (a,t={}) => a.filter(e=>!(t[e]=e in t));
// "stand-alone" version working with global t:
// a1.filter((t={},e=>!(t[e]=e in t)));
// Test data
let a1 = [5,6,0,4,9,2,3,5,0,3,4,1,5,4,9];
let a2 = [[2, 17], [2, 17], [2, 17], [1, 12], [5, 9], [1, 12], [6, 2], [1, 12]];
let a3 = ['Mike', 'Adam','Matt', 'Nancy', 'Adam', 'Jenny', 'Nancy', 'Carl'];
// Results
console.log(JSON.stringify( unique(a1) ))
console.log(JSON.stringify( unique(a2) ))
console.log(JSON.stringify( unique(a3) ))
The simplest, and fastest (in Chrome) way of doing this:
Array.prototype.unique = function() {
var a = [];
for (var i=0, l=this.length; i<l; i++)
if (a.indexOf(this[i]) === -1)
a.push(this[i]);
return a;
}
Simply goes through every item in the array, tests if that item is already in the list, and if it's not, pushes to the array that gets returned.
According to JSBench, this function is the fastest of the ones I could find anywhere - feel free to add your own though.
The non-prototype version:
function uniques(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Sorting
When also needing to sort the array, the following is the fastest:
Array.prototype.sortUnique = function() {
this.sort();
var last_i;
for (var i=0;i<this.length;i++)
if ((last_i = this.lastIndexOf(this[i])) !== i)
this.splice(i+1, last_i-i);
return this;
}
or non-prototype:
function sortUnique(arr) {
arr.sort();
var last_i;
for (var i=0;i<arr.length;i++)
if ((last_i = arr.lastIndexOf(arr[i])) !== i)
arr.splice(i+1, last_i-i);
return arr;
}
This is also faster than the above method in most non-Chrome browsers.
We can do this using ES6 sets:
var duplicatesArray = [1, 2, 3, 4, 5, 1, 1, 1, 2, 3, 4];
var uniqueArray = [...new Set(duplicatesArray)];
console.log(uniqueArray); // [1,2,3,4,5]
["Defects", "Total", "Days", "City", "Defects"].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
[0,1,2,0,3,2,1,5].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
After looking into all the 90+ answers here, I saw there is room for one more:
Array.includes has a very handy second-parameter: "fromIndex", so by using it, every iteration of the filter callback method will search the array, starting from [current index] + 1 which guarantees not to include currently filtered item in the lookup and also saves time.
Note - this solution does not retain the order, as it removed duplicated items from left to right, but it wins the Set trick if the Array is a collection of Objects.
// šŸš© šŸš© šŸš©
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
list.filter((v,i) => !list.includes(v,i+1))
)
// [0,1,3,"b",4,5,2,"a"]
Explanation:
For example, lets assume the filter function is currently iterating at index 2) and the value at that index happens to be 2. The section of the array that is then scanned for duplicates (includes method) is everything after index 2 (i+1):
šŸ‘‡ šŸ‘‡
[0, 1, 2, 2 ,3 ,'a', 'b', 4, 5, 2, 'a']
šŸ‘† |---------------------------|
And since the currently filtered item's value 2 is included in the rest of the array, it will be filtered out, because of the leading exclamation mark which negates the filter rule.
If order is important, use this method:
// šŸš© šŸš© šŸš©
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
// Initialize with empty array and fill with non-duplicates
list.reduce((acc, v) => (!acc.includes(v) && acc.push(v), acc), [])
)
// [0,1,2,3,"a","b",4,5]
This has been answered a lot, but it didn't address my particular need.
Many answers are like this:
a.filter((item, pos, self) => self.indexOf(item) === pos);
But this doesn't work for arrays of complex objects.
Say we have an array like this:
const a = [
{ age: 4, name: 'fluffy' },
{ age: 5, name: 'spot' },
{ age: 2, name: 'fluffy' },
{ age: 3, name: 'toby' },
];
If we want the objects with unique names, we should use array.prototype.findIndex instead of array.prototype.indexOf:
a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);
This prototype getUnique is not totally correct, because if i have a Array like: ["1",1,2,3,4,1,"foo"] it will return ["1","2","3","4"] and "1" is string and 1 is a integer; they are different.
Here is a correct solution:
Array.prototype.unique = function(a){
return function(){ return this.filter(a) }
}(function(a,b,c){ return c.indexOf(a,b+1) < 0 });
using:
var foo;
foo = ["1",1,2,3,4,1,"foo"];
foo.unique();
The above will produce ["1",2,3,4,1,"foo"].
You can simlply use the built-in functions Array.prototype.filter() and Array.prototype.indexOf()
array.filter((x, y) => array.indexOf(x) == y)
var arr = [1, 2, 3, 3, 4, 5, 5, 5, 6, 7, 8, 9, 6, 9];
var newarr = arr.filter((x, y) => arr.indexOf(x) == y);
console.log(newarr);
[...new Set(duplicates)]
This is the simplest one and referenced from MDN Web Docs.
const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]
Array.prototype.getUnique = function() {
var o = {}, a = []
for (var i = 0; i < this.length; i++) o[this[i]] = 1
for (var e in o) a.push(e)
return a
}
Without extending Array.prototype (it is said to be a bad practice) or using jquery/underscore, you can simply filter the array.
By keeping last occurrence:
function arrayLastUnique(array) {
return array.filter(function (a, b, c) {
// keeps last occurrence
return c.indexOf(a, b + 1) < 0;
});
},
or first occurrence:
function arrayFirstUnique(array) {
return array.filter(function (a, b, c) {
// keeps first occurrence
return c.indexOf(a) === b;
});
},
Well, it's only javascript ECMAScript 5+, which means only IE9+, but it's nice for a development in native HTML/JS (Windows Store App, Firefox OS, Sencha, Phonegap, Titanium, ...).
That's because 0 is a falsy value in JavaScript.
this[i] will be falsy if the value of the array is 0 or any other falsy value.
Now using sets you can remove duplicates and convert them back to the array.
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
console.log([...new Set(names)])
Another solution is to use sort & filter
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);
If you're using Prototype framework there is no need to do 'for' loops, you can use http://prototypejs.org/doc/latest/language/Array/prototype/uniq/ like this:
var a = Array.uniq();
Which will produce a duplicate array with no duplicates. I came across your question searching a method to count distinct array records so after uniq() I used size() and there was my simple result.
p.s. Sorry if i mistyped something
edit: if you want to escape undefined records you may want to add compact() before, like this:
var a = Array.compact().uniq();
I had a slightly different problem where I needed to remove objects with duplicate id properties from an array. this worked.
let objArr = [{
id: '123'
}, {
id: '123'
}, {
id: '456'
}];
objArr = objArr.reduce((acc, cur) => [
...acc.filter((obj) => obj.id !== cur.id), cur
], []);
console.log(objArr);
If you're okay with extra dependencies, or you already have one of the libraries in your codebase, you can remove duplicates from an array in place using LoDash (or Underscore).
Usage
If you don't have it in your codebase already, install it using npm:
npm install lodash
Then use it as follows:
import _ from 'lodash';
let idArray = _.uniq ([
1,
2,
3,
3,
3
]);
console.dir(idArray);
Out:
[ 1, 2, 3 ]
I'm not sure why Gabriel Silveira wrote the function that way but a simpler form that works for me just as well and without the minification is:
Array.prototype.unique = function() {
return this.filter(function(value, index, array) {
return array.indexOf(value, index + 1) < 0;
});
};
or in CoffeeScript:
Array.prototype.unique = ->
this.filter( (value, index, array) ->
array.indexOf(value, index + 1) < 0
)
Finding unique Array values in simple method
function arrUnique(a){
var t = [];
for(var x = 0; x < a.length; x++){
if(t.indexOf(a[x]) == -1)t.push(a[x]);
}
return t;
}
arrUnique([1,4,2,7,1,5,9,2,4,7,2]) // [1, 4, 2, 7, 5, 9]
It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:
Array.prototype.getUnique = function(){
var u = {}, a = [];
for (var i = 0, l = this.length; i < l; ++i) {
if (u.hasOwnProperty(this[i])) {
continue;
}
a.push(this[i]);
u[this[i]] = 1;
}
return a;
}
If you do have a mixed-type array, you can serialize the hash key:
Array.prototype.getUnique = function() {
var hash = {}, result = [], key;
for ( var i = 0, l = this.length; i < l; ++i ) {
key = JSON.stringify(this[i]);
if ( !hash.hasOwnProperty(key) ) {
hash[key] = true;
result.push(this[i]);
}
}
return result;
}
strange this hasn't been suggested before.. to remove duplicates by object key (id below) in an array you can do something like this:
const uniqArray = array.filter((obj, idx, arr) => (
arr.findIndex((o) => o.id === obj.id) === idx
))
For an object-based array with some unique id's, I have a simple solution through which you can sort in linear complexity
function getUniqueArr(arr){
const mapObj = {};
arr.forEach(a => {
mapObj[a.id] = a
})
return Object.values(mapObj);
}

Javascript Find unique letters [duplicate]

I have an array of numbers that I need to make sure are unique. I found the code snippet below on the internet and it works great until the array has a zero in it. I found this other script here on Stack Overflow that looks almost exactly like it, but it doesn't fail.
So for the sake of helping me learn, can someone help me determine where the prototype script is going wrong?
Array.prototype.getUnique = function() {
var o = {}, a = [], i, e;
for (i = 0; e = this[i]; i++) {o[e] = 1};
for (e in o) {a.push (e)};
return a;
}
More answers from duplicate question:
Remove duplicate values from JS array
Similar question:
Get all non-unique values (i.e.: duplicate/more than one occurrence) in an array
With JavaScript 1.6 / ECMAScript 5 you can use the native filter method of an Array in the following way to get an array with unique values:
function onlyUnique(value, index, array) {
return self.indexOf(value) === index;
}
// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);
console.log(unique); // ['a', 1, 2, '1']
The native method filter will loop through the array and leave only those entries that pass the given callback function onlyUnique.
onlyUnique checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.
This solution works without any extra library like jQuery or prototype.js.
It works for arrays with mixed value types too.
For old Browsers (<ie9), that do not support the native methods filter and indexOf you can find work arounds in the MDN documentation for filter and indexOf.
If you want to keep the last occurrence of a value, simply replace indexOf with lastIndexOf.
With ES6 this can be shorten to:
// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((value, index, array) => array.indexOf(value) === index);
console.log(unique); // unique is ['a', 1, 2, '1']
Thanks to Camilo Martin for hint in comment.
ES6 has a native object Set to store unique values. To get an array with unique values you could now do this:
var myArray = ['a', 1, 'a', 2, '1'];
let unique = [...new Set(myArray)];
console.log(unique); // unique is ['a', 1, 2, '1']
The constructor of Set takes an iterable object, like an Array, and the spread operator ... transform the set back into an Array. Thanks to Lukas Liese for hint in comment.
Updated answer for ES6/ES2015: Using the Set and the spread operator (thanks le-m), the single line solution is:
let uniqueItems = [...new Set(items)]
Which returns
[4, 5, 6, 3, 2, 23, 1]
I split all answers to 4 possible solutions:
Use object { } to prevent duplicates
Use helper array [ ]
Use filter + indexOf
Bonus! ES6 Sets method.
Here's sample codes found in answers:
Use object { } to prevent duplicates
function uniqueArray1( ar ) {
var j = {};
ar.forEach( function(v) {
j[v+ '::' + typeof v] = v;
});
return Object.keys(j).map(function(v){
return j[v];
});
}
Use helper array [ ]
function uniqueArray2(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Use filter + indexOf
function uniqueArray3(a) {
function onlyUnique(value, index, self) {
return self.indexOf(value) === index;
}
// usage
var unique = a.filter( onlyUnique ); // returns ['a', 1, 2, '1']
return unique;
}
Use ES6 [...new Set(a)]
function uniqueArray4(a) {
return [...new Set(a)];
}
And I wondered which one is faster. I've made sample Google Sheet to test functions. Note: ECMA 6 is not avaliable in Google Sheets, so I can't test it.
Here's the result of tests:
I expected to see that code using object { } will win because it uses hash. So I'm glad that tests showed the best results for this algorithm in Chrome and IE. Thanks to #rab for the code.
Update 2020
Google Script enabled ES6 Engine. Now I tested the last code with Sets and it appeared faster than the object method.
You can also use underscore.js.
console.log(_.uniq([1, 2, 1, 3, 1, 4]));
<script src="http://underscorejs.org/underscore-min.js"></script>
which will return:
[1, 2, 3, 4]
One Liner, Pure JavaScript
With ES6 syntax
list = list.filter((x, i, a) => a.indexOf(x) == i)
x --> item in array
i --> index of item
a --> array reference, (in this case "list")
With ES5 syntax
list = list.filter(function (x, i, a) {
return a.indexOf(x) == i;
});
Browser Compatibility: IE9+
Remove duplicates using Set.
Array with duplicates
const withDuplicates = [2, 2, 5, 5, 1, 1, 2, 2, 3, 3];
Get a new array without duplicates by using Set
const withoutDuplicates = Array.from(new Set(withDuplicates));
A shorter version
const withoutDuplicates = [...new Set(withDuplicates)];
Result: [2, 5, 1, 3]
Many of the answers here may not be useful to beginners. If de-duping an array is difficult, will they really know about the prototype chain, or even jQuery?
In modern browsers, a clean and simple solution is to store data in a Set, which is designed to be a list of unique values.
const cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
const uniqueCars = Array.from(new Set(cars));
console.log(uniqueCars);
The Array.from is useful to convert the Set back to an Array so that you have easy access to all of the awesome methods (features) that arrays have. There are also other ways of doing the same thing. But you may not need Array.from at all, as Sets have plenty of useful features like forEach.
If you need to support old Internet Explorer, and thus cannot use Set, then a simple technique is to copy items over to a new array while checking beforehand if they are already in the new array.
// Create a list of cars, with duplicates.
var cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
// Create a list of unique cars, to put a car in if we haven't already.
var uniqueCars = [];
// Go through each car, one at a time.
cars.forEach(function (car) {
// The code within the following block runs only if the
// current car does NOT exist in the uniqueCars list
// - a.k.a. prevent duplicates
if (uniqueCars.indexOf(car) === -1) {
// Since we now know we haven't seen this car before,
// copy it to the end of the uniqueCars list.
uniqueCars.push(car);
}
});
To make this instantly reusable, let's put it in a function.
function deduplicate(data) {
if (data.length > 0) {
var result = [];
data.forEach(function (elem) {
if (result.indexOf(elem) === -1) {
result.push(elem);
}
});
return result;
}
}
So to get rid of the duplicates, we would now do this.
var uniqueCars = deduplicate(cars);
The deduplicate(cars) part becomes the thing we named result when the function completes.
Just pass it the name of any array you like.
Using ES6 new Set
var array = [3,7,5,3,2,5,2,7];
var unique_array = [...new Set(array)];
console.log(unique_array); // output = [3,7,5,2]
Using For Loop
var array = [3,7,5,3,2,5,2,7];
for(var i=0;i<array.length;i++) {
for(var j=i+1;j<array.length;j++) {
if(array[i]===array[j]) {
array.splice(j,1);
}
}
}
console.log(array); // output = [3,7,5,2]
I have since found a nice method that uses jQuery
arr = $.grep(arr, function(v, k){
return $.inArray(v ,arr) === k;
});
Note: This code was pulled from Paul Irish's duck punching post - I forgot to give credit :P
Magic
a.filter(e=>!(t[e]=e in t))
O(n) performance - we assume your array is in a and t={}. Explanation here (+Jeppe impr.)
let unique = (a,t={}) => a.filter(e=>!(t[e]=e in t));
// "stand-alone" version working with global t:
// a1.filter((t={},e=>!(t[e]=e in t)));
// Test data
let a1 = [5,6,0,4,9,2,3,5,0,3,4,1,5,4,9];
let a2 = [[2, 17], [2, 17], [2, 17], [1, 12], [5, 9], [1, 12], [6, 2], [1, 12]];
let a3 = ['Mike', 'Adam','Matt', 'Nancy', 'Adam', 'Jenny', 'Nancy', 'Carl'];
// Results
console.log(JSON.stringify( unique(a1) ))
console.log(JSON.stringify( unique(a2) ))
console.log(JSON.stringify( unique(a3) ))
The simplest, and fastest (in Chrome) way of doing this:
Array.prototype.unique = function() {
var a = [];
for (var i=0, l=this.length; i<l; i++)
if (a.indexOf(this[i]) === -1)
a.push(this[i]);
return a;
}
Simply goes through every item in the array, tests if that item is already in the list, and if it's not, pushes to the array that gets returned.
According to JSBench, this function is the fastest of the ones I could find anywhere - feel free to add your own though.
The non-prototype version:
function uniques(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Sorting
When also needing to sort the array, the following is the fastest:
Array.prototype.sortUnique = function() {
this.sort();
var last_i;
for (var i=0;i<this.length;i++)
if ((last_i = this.lastIndexOf(this[i])) !== i)
this.splice(i+1, last_i-i);
return this;
}
or non-prototype:
function sortUnique(arr) {
arr.sort();
var last_i;
for (var i=0;i<arr.length;i++)
if ((last_i = arr.lastIndexOf(arr[i])) !== i)
arr.splice(i+1, last_i-i);
return arr;
}
This is also faster than the above method in most non-Chrome browsers.
We can do this using ES6 sets:
var duplicatesArray = [1, 2, 3, 4, 5, 1, 1, 1, 2, 3, 4];
var uniqueArray = [...new Set(duplicatesArray)];
console.log(uniqueArray); // [1,2,3,4,5]
["Defects", "Total", "Days", "City", "Defects"].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
[0,1,2,0,3,2,1,5].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
After looking into all the 90+ answers here, I saw there is room for one more:
Array.includes has a very handy second-parameter: "fromIndex", so by using it, every iteration of the filter callback method will search the array, starting from [current index] + 1 which guarantees not to include currently filtered item in the lookup and also saves time.
Note - this solution does not retain the order, as it removed duplicated items from left to right, but it wins the Set trick if the Array is a collection of Objects.
// šŸš© šŸš© šŸš©
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
list.filter((v,i) => !list.includes(v,i+1))
)
// [0,1,3,"b",4,5,2,"a"]
Explanation:
For example, lets assume the filter function is currently iterating at index 2) and the value at that index happens to be 2. The section of the array that is then scanned for duplicates (includes method) is everything after index 2 (i+1):
šŸ‘‡ šŸ‘‡
[0, 1, 2, 2 ,3 ,'a', 'b', 4, 5, 2, 'a']
šŸ‘† |---------------------------|
And since the currently filtered item's value 2 is included in the rest of the array, it will be filtered out, because of the leading exclamation mark which negates the filter rule.
If order is important, use this method:
// šŸš© šŸš© šŸš©
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
// Initialize with empty array and fill with non-duplicates
list.reduce((acc, v) => (!acc.includes(v) && acc.push(v), acc), [])
)
// [0,1,2,3,"a","b",4,5]
This has been answered a lot, but it didn't address my particular need.
Many answers are like this:
a.filter((item, pos, self) => self.indexOf(item) === pos);
But this doesn't work for arrays of complex objects.
Say we have an array like this:
const a = [
{ age: 4, name: 'fluffy' },
{ age: 5, name: 'spot' },
{ age: 2, name: 'fluffy' },
{ age: 3, name: 'toby' },
];
If we want the objects with unique names, we should use array.prototype.findIndex instead of array.prototype.indexOf:
a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);
This prototype getUnique is not totally correct, because if i have a Array like: ["1",1,2,3,4,1,"foo"] it will return ["1","2","3","4"] and "1" is string and 1 is a integer; they are different.
Here is a correct solution:
Array.prototype.unique = function(a){
return function(){ return this.filter(a) }
}(function(a,b,c){ return c.indexOf(a,b+1) < 0 });
using:
var foo;
foo = ["1",1,2,3,4,1,"foo"];
foo.unique();
The above will produce ["1",2,3,4,1,"foo"].
You can simlply use the built-in functions Array.prototype.filter() and Array.prototype.indexOf()
array.filter((x, y) => array.indexOf(x) == y)
var arr = [1, 2, 3, 3, 4, 5, 5, 5, 6, 7, 8, 9, 6, 9];
var newarr = arr.filter((x, y) => arr.indexOf(x) == y);
console.log(newarr);
[...new Set(duplicates)]
This is the simplest one and referenced from MDN Web Docs.
const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]
Array.prototype.getUnique = function() {
var o = {}, a = []
for (var i = 0; i < this.length; i++) o[this[i]] = 1
for (var e in o) a.push(e)
return a
}
Without extending Array.prototype (it is said to be a bad practice) or using jquery/underscore, you can simply filter the array.
By keeping last occurrence:
function arrayLastUnique(array) {
return array.filter(function (a, b, c) {
// keeps last occurrence
return c.indexOf(a, b + 1) < 0;
});
},
or first occurrence:
function arrayFirstUnique(array) {
return array.filter(function (a, b, c) {
// keeps first occurrence
return c.indexOf(a) === b;
});
},
Well, it's only javascript ECMAScript 5+, which means only IE9+, but it's nice for a development in native HTML/JS (Windows Store App, Firefox OS, Sencha, Phonegap, Titanium, ...).
That's because 0 is a falsy value in JavaScript.
this[i] will be falsy if the value of the array is 0 or any other falsy value.
Now using sets you can remove duplicates and convert them back to the array.
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
console.log([...new Set(names)])
Another solution is to use sort & filter
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);
If you're using Prototype framework there is no need to do 'for' loops, you can use http://prototypejs.org/doc/latest/language/Array/prototype/uniq/ like this:
var a = Array.uniq();
Which will produce a duplicate array with no duplicates. I came across your question searching a method to count distinct array records so after uniq() I used size() and there was my simple result.
p.s. Sorry if i mistyped something
edit: if you want to escape undefined records you may want to add compact() before, like this:
var a = Array.compact().uniq();
I had a slightly different problem where I needed to remove objects with duplicate id properties from an array. this worked.
let objArr = [{
id: '123'
}, {
id: '123'
}, {
id: '456'
}];
objArr = objArr.reduce((acc, cur) => [
...acc.filter((obj) => obj.id !== cur.id), cur
], []);
console.log(objArr);
If you're okay with extra dependencies, or you already have one of the libraries in your codebase, you can remove duplicates from an array in place using LoDash (or Underscore).
Usage
If you don't have it in your codebase already, install it using npm:
npm install lodash
Then use it as follows:
import _ from 'lodash';
let idArray = _.uniq ([
1,
2,
3,
3,
3
]);
console.dir(idArray);
Out:
[ 1, 2, 3 ]
I'm not sure why Gabriel Silveira wrote the function that way but a simpler form that works for me just as well and without the minification is:
Array.prototype.unique = function() {
return this.filter(function(value, index, array) {
return array.indexOf(value, index + 1) < 0;
});
};
or in CoffeeScript:
Array.prototype.unique = ->
this.filter( (value, index, array) ->
array.indexOf(value, index + 1) < 0
)
Finding unique Array values in simple method
function arrUnique(a){
var t = [];
for(var x = 0; x < a.length; x++){
if(t.indexOf(a[x]) == -1)t.push(a[x]);
}
return t;
}
arrUnique([1,4,2,7,1,5,9,2,4,7,2]) // [1, 4, 2, 7, 5, 9]
It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:
Array.prototype.getUnique = function(){
var u = {}, a = [];
for (var i = 0, l = this.length; i < l; ++i) {
if (u.hasOwnProperty(this[i])) {
continue;
}
a.push(this[i]);
u[this[i]] = 1;
}
return a;
}
If you do have a mixed-type array, you can serialize the hash key:
Array.prototype.getUnique = function() {
var hash = {}, result = [], key;
for ( var i = 0, l = this.length; i < l; ++i ) {
key = JSON.stringify(this[i]);
if ( !hash.hasOwnProperty(key) ) {
hash[key] = true;
result.push(this[i]);
}
}
return result;
}
strange this hasn't been suggested before.. to remove duplicates by object key (id below) in an array you can do something like this:
const uniqArray = array.filter((obj, idx, arr) => (
arr.findIndex((o) => o.id === obj.id) === idx
))
For an object-based array with some unique id's, I have a simple solution through which you can sort in linear complexity
function getUniqueArr(arr){
const mapObj = {};
arr.forEach(a => {
mapObj[a.id] = a
})
return Object.values(mapObj);
}

Categories

Resources