Related
I have an array of numbers that I need to make sure are unique. I found the code snippet below on the internet and it works great until the array has a zero in it. I found this other script here on Stack Overflow that looks almost exactly like it, but it doesn't fail.
So for the sake of helping me learn, can someone help me determine where the prototype script is going wrong?
Array.prototype.getUnique = function() {
var o = {}, a = [], i, e;
for (i = 0; e = this[i]; i++) {o[e] = 1};
for (e in o) {a.push (e)};
return a;
}
More answers from duplicate question:
Remove duplicate values from JS array
Similar question:
Get all non-unique values (i.e.: duplicate/more than one occurrence) in an array
With JavaScript 1.6 / ECMAScript 5 you can use the native filter method of an Array in the following way to get an array with unique values:
function onlyUnique(value, index, array) {
return self.indexOf(value) === index;
}
// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);
console.log(unique); // ['a', 1, 2, '1']
The native method filter will loop through the array and leave only those entries that pass the given callback function onlyUnique.
onlyUnique checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.
This solution works without any extra library like jQuery or prototype.js.
It works for arrays with mixed value types too.
For old Browsers (<ie9), that do not support the native methods filter and indexOf you can find work arounds in the MDN documentation for filter and indexOf.
If you want to keep the last occurrence of a value, simply replace indexOf with lastIndexOf.
With ES6 this can be shorten to:
// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((value, index, array) => array.indexOf(value) === index);
console.log(unique); // unique is ['a', 1, 2, '1']
Thanks to Camilo Martin for hint in comment.
ES6 has a native object Set to store unique values. To get an array with unique values you could now do this:
var myArray = ['a', 1, 'a', 2, '1'];
let unique = [...new Set(myArray)];
console.log(unique); // unique is ['a', 1, 2, '1']
The constructor of Set takes an iterable object, like an Array, and the spread operator ... transform the set back into an Array. Thanks to Lukas Liese for hint in comment.
Updated answer for ES6/ES2015: Using the Set and the spread operator (thanks le-m), the single line solution is:
let uniqueItems = [...new Set(items)]
Which returns
[4, 5, 6, 3, 2, 23, 1]
I split all answers to 4 possible solutions:
Use object { } to prevent duplicates
Use helper array [ ]
Use filter + indexOf
Bonus! ES6 Sets method.
Here's sample codes found in answers:
Use object { } to prevent duplicates
function uniqueArray1( ar ) {
var j = {};
ar.forEach( function(v) {
j[v+ '::' + typeof v] = v;
});
return Object.keys(j).map(function(v){
return j[v];
});
}
Use helper array [ ]
function uniqueArray2(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Use filter + indexOf
function uniqueArray3(a) {
function onlyUnique(value, index, self) {
return self.indexOf(value) === index;
}
// usage
var unique = a.filter( onlyUnique ); // returns ['a', 1, 2, '1']
return unique;
}
Use ES6 [...new Set(a)]
function uniqueArray4(a) {
return [...new Set(a)];
}
And I wondered which one is faster. I've made sample Google Sheet to test functions. Note: ECMA 6 is not avaliable in Google Sheets, so I can't test it.
Here's the result of tests:
I expected to see that code using object { } will win because it uses hash. So I'm glad that tests showed the best results for this algorithm in Chrome and IE. Thanks to #rab for the code.
Update 2020
Google Script enabled ES6 Engine. Now I tested the last code with Sets and it appeared faster than the object method.
You can also use underscore.js.
console.log(_.uniq([1, 2, 1, 3, 1, 4]));
<script src="http://underscorejs.org/underscore-min.js"></script>
which will return:
[1, 2, 3, 4]
One Liner, Pure JavaScript
With ES6 syntax
list = list.filter((x, i, a) => a.indexOf(x) == i)
x --> item in array
i --> index of item
a --> array reference, (in this case "list")
With ES5 syntax
list = list.filter(function (x, i, a) {
return a.indexOf(x) == i;
});
Browser Compatibility: IE9+
Remove duplicates using Set.
Array with duplicates
const withDuplicates = [2, 2, 5, 5, 1, 1, 2, 2, 3, 3];
Get a new array without duplicates by using Set
const withoutDuplicates = Array.from(new Set(withDuplicates));
A shorter version
const withoutDuplicates = [...new Set(withDuplicates)];
Result: [2, 5, 1, 3]
Many of the answers here may not be useful to beginners. If de-duping an array is difficult, will they really know about the prototype chain, or even jQuery?
In modern browsers, a clean and simple solution is to store data in a Set, which is designed to be a list of unique values.
const cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
const uniqueCars = Array.from(new Set(cars));
console.log(uniqueCars);
The Array.from is useful to convert the Set back to an Array so that you have easy access to all of the awesome methods (features) that arrays have. There are also other ways of doing the same thing. But you may not need Array.from at all, as Sets have plenty of useful features like forEach.
If you need to support old Internet Explorer, and thus cannot use Set, then a simple technique is to copy items over to a new array while checking beforehand if they are already in the new array.
// Create a list of cars, with duplicates.
var cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
// Create a list of unique cars, to put a car in if we haven't already.
var uniqueCars = [];
// Go through each car, one at a time.
cars.forEach(function (car) {
// The code within the following block runs only if the
// current car does NOT exist in the uniqueCars list
// - a.k.a. prevent duplicates
if (uniqueCars.indexOf(car) === -1) {
// Since we now know we haven't seen this car before,
// copy it to the end of the uniqueCars list.
uniqueCars.push(car);
}
});
To make this instantly reusable, let's put it in a function.
function deduplicate(data) {
if (data.length > 0) {
var result = [];
data.forEach(function (elem) {
if (result.indexOf(elem) === -1) {
result.push(elem);
}
});
return result;
}
}
So to get rid of the duplicates, we would now do this.
var uniqueCars = deduplicate(cars);
The deduplicate(cars) part becomes the thing we named result when the function completes.
Just pass it the name of any array you like.
Using ES6 new Set
var array = [3,7,5,3,2,5,2,7];
var unique_array = [...new Set(array)];
console.log(unique_array); // output = [3,7,5,2]
Using For Loop
var array = [3,7,5,3,2,5,2,7];
for(var i=0;i<array.length;i++) {
for(var j=i+1;j<array.length;j++) {
if(array[i]===array[j]) {
array.splice(j,1);
}
}
}
console.log(array); // output = [3,7,5,2]
I have since found a nice method that uses jQuery
arr = $.grep(arr, function(v, k){
return $.inArray(v ,arr) === k;
});
Note: This code was pulled from Paul Irish's duck punching post - I forgot to give credit :P
Magic
a.filter(e=>!(t[e]=e in t))
O(n) performance - we assume your array is in a and t={}. Explanation here (+Jeppe impr.)
let unique = (a,t={}) => a.filter(e=>!(t[e]=e in t));
// "stand-alone" version working with global t:
// a1.filter((t={},e=>!(t[e]=e in t)));
// Test data
let a1 = [5,6,0,4,9,2,3,5,0,3,4,1,5,4,9];
let a2 = [[2, 17], [2, 17], [2, 17], [1, 12], [5, 9], [1, 12], [6, 2], [1, 12]];
let a3 = ['Mike', 'Adam','Matt', 'Nancy', 'Adam', 'Jenny', 'Nancy', 'Carl'];
// Results
console.log(JSON.stringify( unique(a1) ))
console.log(JSON.stringify( unique(a2) ))
console.log(JSON.stringify( unique(a3) ))
The simplest, and fastest (in Chrome) way of doing this:
Array.prototype.unique = function() {
var a = [];
for (var i=0, l=this.length; i<l; i++)
if (a.indexOf(this[i]) === -1)
a.push(this[i]);
return a;
}
Simply goes through every item in the array, tests if that item is already in the list, and if it's not, pushes to the array that gets returned.
According to JSBench, this function is the fastest of the ones I could find anywhere - feel free to add your own though.
The non-prototype version:
function uniques(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Sorting
When also needing to sort the array, the following is the fastest:
Array.prototype.sortUnique = function() {
this.sort();
var last_i;
for (var i=0;i<this.length;i++)
if ((last_i = this.lastIndexOf(this[i])) !== i)
this.splice(i+1, last_i-i);
return this;
}
or non-prototype:
function sortUnique(arr) {
arr.sort();
var last_i;
for (var i=0;i<arr.length;i++)
if ((last_i = arr.lastIndexOf(arr[i])) !== i)
arr.splice(i+1, last_i-i);
return arr;
}
This is also faster than the above method in most non-Chrome browsers.
We can do this using ES6 sets:
var duplicatesArray = [1, 2, 3, 4, 5, 1, 1, 1, 2, 3, 4];
var uniqueArray = [...new Set(duplicatesArray)];
console.log(uniqueArray); // [1,2,3,4,5]
["Defects", "Total", "Days", "City", "Defects"].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
[0,1,2,0,3,2,1,5].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
After looking into all the 90+ answers here, I saw there is room for one more:
Array.includes has a very handy second-parameter: "fromIndex", so by using it, every iteration of the filter callback method will search the array, starting from [current index] + 1 which guarantees not to include currently filtered item in the lookup and also saves time.
Note - this solution does not retain the order, as it removed duplicated items from left to right, but it wins the Set trick if the Array is a collection of Objects.
// 🚩 🚩 🚩
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
list.filter((v,i) => !list.includes(v,i+1))
)
// [0,1,3,"b",4,5,2,"a"]
Explanation:
For example, lets assume the filter function is currently iterating at index 2) and the value at that index happens to be 2. The section of the array that is then scanned for duplicates (includes method) is everything after index 2 (i+1):
👇 👇
[0, 1, 2, 2 ,3 ,'a', 'b', 4, 5, 2, 'a']
👆 |---------------------------|
And since the currently filtered item's value 2 is included in the rest of the array, it will be filtered out, because of the leading exclamation mark which negates the filter rule.
If order is important, use this method:
// 🚩 🚩 🚩
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
// Initialize with empty array and fill with non-duplicates
list.reduce((acc, v) => (!acc.includes(v) && acc.push(v), acc), [])
)
// [0,1,2,3,"a","b",4,5]
This has been answered a lot, but it didn't address my particular need.
Many answers are like this:
a.filter((item, pos, self) => self.indexOf(item) === pos);
But this doesn't work for arrays of complex objects.
Say we have an array like this:
const a = [
{ age: 4, name: 'fluffy' },
{ age: 5, name: 'spot' },
{ age: 2, name: 'fluffy' },
{ age: 3, name: 'toby' },
];
If we want the objects with unique names, we should use array.prototype.findIndex instead of array.prototype.indexOf:
a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);
This prototype getUnique is not totally correct, because if i have a Array like: ["1",1,2,3,4,1,"foo"] it will return ["1","2","3","4"] and "1" is string and 1 is a integer; they are different.
Here is a correct solution:
Array.prototype.unique = function(a){
return function(){ return this.filter(a) }
}(function(a,b,c){ return c.indexOf(a,b+1) < 0 });
using:
var foo;
foo = ["1",1,2,3,4,1,"foo"];
foo.unique();
The above will produce ["1",2,3,4,1,"foo"].
You can simlply use the built-in functions Array.prototype.filter() and Array.prototype.indexOf()
array.filter((x, y) => array.indexOf(x) == y)
var arr = [1, 2, 3, 3, 4, 5, 5, 5, 6, 7, 8, 9, 6, 9];
var newarr = arr.filter((x, y) => arr.indexOf(x) == y);
console.log(newarr);
[...new Set(duplicates)]
This is the simplest one and referenced from MDN Web Docs.
const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]
Array.prototype.getUnique = function() {
var o = {}, a = []
for (var i = 0; i < this.length; i++) o[this[i]] = 1
for (var e in o) a.push(e)
return a
}
Without extending Array.prototype (it is said to be a bad practice) or using jquery/underscore, you can simply filter the array.
By keeping last occurrence:
function arrayLastUnique(array) {
return array.filter(function (a, b, c) {
// keeps last occurrence
return c.indexOf(a, b + 1) < 0;
});
},
or first occurrence:
function arrayFirstUnique(array) {
return array.filter(function (a, b, c) {
// keeps first occurrence
return c.indexOf(a) === b;
});
},
Well, it's only javascript ECMAScript 5+, which means only IE9+, but it's nice for a development in native HTML/JS (Windows Store App, Firefox OS, Sencha, Phonegap, Titanium, ...).
That's because 0 is a falsy value in JavaScript.
this[i] will be falsy if the value of the array is 0 or any other falsy value.
Now using sets you can remove duplicates and convert them back to the array.
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
console.log([...new Set(names)])
Another solution is to use sort & filter
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);
If you're using Prototype framework there is no need to do 'for' loops, you can use http://prototypejs.org/doc/latest/language/Array/prototype/uniq/ like this:
var a = Array.uniq();
Which will produce a duplicate array with no duplicates. I came across your question searching a method to count distinct array records so after uniq() I used size() and there was my simple result.
p.s. Sorry if i mistyped something
edit: if you want to escape undefined records you may want to add compact() before, like this:
var a = Array.compact().uniq();
I had a slightly different problem where I needed to remove objects with duplicate id properties from an array. this worked.
let objArr = [{
id: '123'
}, {
id: '123'
}, {
id: '456'
}];
objArr = objArr.reduce((acc, cur) => [
...acc.filter((obj) => obj.id !== cur.id), cur
], []);
console.log(objArr);
If you're okay with extra dependencies, or you already have one of the libraries in your codebase, you can remove duplicates from an array in place using LoDash (or Underscore).
Usage
If you don't have it in your codebase already, install it using npm:
npm install lodash
Then use it as follows:
import _ from 'lodash';
let idArray = _.uniq ([
1,
2,
3,
3,
3
]);
console.dir(idArray);
Out:
[ 1, 2, 3 ]
I'm not sure why Gabriel Silveira wrote the function that way but a simpler form that works for me just as well and without the minification is:
Array.prototype.unique = function() {
return this.filter(function(value, index, array) {
return array.indexOf(value, index + 1) < 0;
});
};
or in CoffeeScript:
Array.prototype.unique = ->
this.filter( (value, index, array) ->
array.indexOf(value, index + 1) < 0
)
Finding unique Array values in simple method
function arrUnique(a){
var t = [];
for(var x = 0; x < a.length; x++){
if(t.indexOf(a[x]) == -1)t.push(a[x]);
}
return t;
}
arrUnique([1,4,2,7,1,5,9,2,4,7,2]) // [1, 4, 2, 7, 5, 9]
It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:
Array.prototype.getUnique = function(){
var u = {}, a = [];
for (var i = 0, l = this.length; i < l; ++i) {
if (u.hasOwnProperty(this[i])) {
continue;
}
a.push(this[i]);
u[this[i]] = 1;
}
return a;
}
If you do have a mixed-type array, you can serialize the hash key:
Array.prototype.getUnique = function() {
var hash = {}, result = [], key;
for ( var i = 0, l = this.length; i < l; ++i ) {
key = JSON.stringify(this[i]);
if ( !hash.hasOwnProperty(key) ) {
hash[key] = true;
result.push(this[i]);
}
}
return result;
}
strange this hasn't been suggested before.. to remove duplicates by object key (id below) in an array you can do something like this:
const uniqArray = array.filter((obj, idx, arr) => (
arr.findIndex((o) => o.id === obj.id) === idx
))
For an object-based array with some unique id's, I have a simple solution through which you can sort in linear complexity
function getUniqueArr(arr){
const mapObj = {};
arr.forEach(a => {
mapObj[a.id] = a
})
return Object.values(mapObj);
}
I have an array of numbers that I need to make sure are unique. I found the code snippet below on the internet and it works great until the array has a zero in it. I found this other script here on Stack Overflow that looks almost exactly like it, but it doesn't fail.
So for the sake of helping me learn, can someone help me determine where the prototype script is going wrong?
Array.prototype.getUnique = function() {
var o = {}, a = [], i, e;
for (i = 0; e = this[i]; i++) {o[e] = 1};
for (e in o) {a.push (e)};
return a;
}
More answers from duplicate question:
Remove duplicate values from JS array
Similar question:
Get all non-unique values (i.e.: duplicate/more than one occurrence) in an array
With JavaScript 1.6 / ECMAScript 5 you can use the native filter method of an Array in the following way to get an array with unique values:
function onlyUnique(value, index, array) {
return self.indexOf(value) === index;
}
// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);
console.log(unique); // ['a', 1, 2, '1']
The native method filter will loop through the array and leave only those entries that pass the given callback function onlyUnique.
onlyUnique checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.
This solution works without any extra library like jQuery or prototype.js.
It works for arrays with mixed value types too.
For old Browsers (<ie9), that do not support the native methods filter and indexOf you can find work arounds in the MDN documentation for filter and indexOf.
If you want to keep the last occurrence of a value, simply replace indexOf with lastIndexOf.
With ES6 this can be shorten to:
// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((value, index, array) => array.indexOf(value) === index);
console.log(unique); // unique is ['a', 1, 2, '1']
Thanks to Camilo Martin for hint in comment.
ES6 has a native object Set to store unique values. To get an array with unique values you could now do this:
var myArray = ['a', 1, 'a', 2, '1'];
let unique = [...new Set(myArray)];
console.log(unique); // unique is ['a', 1, 2, '1']
The constructor of Set takes an iterable object, like an Array, and the spread operator ... transform the set back into an Array. Thanks to Lukas Liese for hint in comment.
Updated answer for ES6/ES2015: Using the Set and the spread operator (thanks le-m), the single line solution is:
let uniqueItems = [...new Set(items)]
Which returns
[4, 5, 6, 3, 2, 23, 1]
I split all answers to 4 possible solutions:
Use object { } to prevent duplicates
Use helper array [ ]
Use filter + indexOf
Bonus! ES6 Sets method.
Here's sample codes found in answers:
Use object { } to prevent duplicates
function uniqueArray1( ar ) {
var j = {};
ar.forEach( function(v) {
j[v+ '::' + typeof v] = v;
});
return Object.keys(j).map(function(v){
return j[v];
});
}
Use helper array [ ]
function uniqueArray2(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Use filter + indexOf
function uniqueArray3(a) {
function onlyUnique(value, index, self) {
return self.indexOf(value) === index;
}
// usage
var unique = a.filter( onlyUnique ); // returns ['a', 1, 2, '1']
return unique;
}
Use ES6 [...new Set(a)]
function uniqueArray4(a) {
return [...new Set(a)];
}
And I wondered which one is faster. I've made sample Google Sheet to test functions. Note: ECMA 6 is not avaliable in Google Sheets, so I can't test it.
Here's the result of tests:
I expected to see that code using object { } will win because it uses hash. So I'm glad that tests showed the best results for this algorithm in Chrome and IE. Thanks to #rab for the code.
Update 2020
Google Script enabled ES6 Engine. Now I tested the last code with Sets and it appeared faster than the object method.
You can also use underscore.js.
console.log(_.uniq([1, 2, 1, 3, 1, 4]));
<script src="http://underscorejs.org/underscore-min.js"></script>
which will return:
[1, 2, 3, 4]
One Liner, Pure JavaScript
With ES6 syntax
list = list.filter((x, i, a) => a.indexOf(x) == i)
x --> item in array
i --> index of item
a --> array reference, (in this case "list")
With ES5 syntax
list = list.filter(function (x, i, a) {
return a.indexOf(x) == i;
});
Browser Compatibility: IE9+
Remove duplicates using Set.
Array with duplicates
const withDuplicates = [2, 2, 5, 5, 1, 1, 2, 2, 3, 3];
Get a new array without duplicates by using Set
const withoutDuplicates = Array.from(new Set(withDuplicates));
A shorter version
const withoutDuplicates = [...new Set(withDuplicates)];
Result: [2, 5, 1, 3]
Many of the answers here may not be useful to beginners. If de-duping an array is difficult, will they really know about the prototype chain, or even jQuery?
In modern browsers, a clean and simple solution is to store data in a Set, which is designed to be a list of unique values.
const cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
const uniqueCars = Array.from(new Set(cars));
console.log(uniqueCars);
The Array.from is useful to convert the Set back to an Array so that you have easy access to all of the awesome methods (features) that arrays have. There are also other ways of doing the same thing. But you may not need Array.from at all, as Sets have plenty of useful features like forEach.
If you need to support old Internet Explorer, and thus cannot use Set, then a simple technique is to copy items over to a new array while checking beforehand if they are already in the new array.
// Create a list of cars, with duplicates.
var cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
// Create a list of unique cars, to put a car in if we haven't already.
var uniqueCars = [];
// Go through each car, one at a time.
cars.forEach(function (car) {
// The code within the following block runs only if the
// current car does NOT exist in the uniqueCars list
// - a.k.a. prevent duplicates
if (uniqueCars.indexOf(car) === -1) {
// Since we now know we haven't seen this car before,
// copy it to the end of the uniqueCars list.
uniqueCars.push(car);
}
});
To make this instantly reusable, let's put it in a function.
function deduplicate(data) {
if (data.length > 0) {
var result = [];
data.forEach(function (elem) {
if (result.indexOf(elem) === -1) {
result.push(elem);
}
});
return result;
}
}
So to get rid of the duplicates, we would now do this.
var uniqueCars = deduplicate(cars);
The deduplicate(cars) part becomes the thing we named result when the function completes.
Just pass it the name of any array you like.
Using ES6 new Set
var array = [3,7,5,3,2,5,2,7];
var unique_array = [...new Set(array)];
console.log(unique_array); // output = [3,7,5,2]
Using For Loop
var array = [3,7,5,3,2,5,2,7];
for(var i=0;i<array.length;i++) {
for(var j=i+1;j<array.length;j++) {
if(array[i]===array[j]) {
array.splice(j,1);
}
}
}
console.log(array); // output = [3,7,5,2]
I have since found a nice method that uses jQuery
arr = $.grep(arr, function(v, k){
return $.inArray(v ,arr) === k;
});
Note: This code was pulled from Paul Irish's duck punching post - I forgot to give credit :P
Magic
a.filter(e=>!(t[e]=e in t))
O(n) performance - we assume your array is in a and t={}. Explanation here (+Jeppe impr.)
let unique = (a,t={}) => a.filter(e=>!(t[e]=e in t));
// "stand-alone" version working with global t:
// a1.filter((t={},e=>!(t[e]=e in t)));
// Test data
let a1 = [5,6,0,4,9,2,3,5,0,3,4,1,5,4,9];
let a2 = [[2, 17], [2, 17], [2, 17], [1, 12], [5, 9], [1, 12], [6, 2], [1, 12]];
let a3 = ['Mike', 'Adam','Matt', 'Nancy', 'Adam', 'Jenny', 'Nancy', 'Carl'];
// Results
console.log(JSON.stringify( unique(a1) ))
console.log(JSON.stringify( unique(a2) ))
console.log(JSON.stringify( unique(a3) ))
The simplest, and fastest (in Chrome) way of doing this:
Array.prototype.unique = function() {
var a = [];
for (var i=0, l=this.length; i<l; i++)
if (a.indexOf(this[i]) === -1)
a.push(this[i]);
return a;
}
Simply goes through every item in the array, tests if that item is already in the list, and if it's not, pushes to the array that gets returned.
According to JSBench, this function is the fastest of the ones I could find anywhere - feel free to add your own though.
The non-prototype version:
function uniques(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
Sorting
When also needing to sort the array, the following is the fastest:
Array.prototype.sortUnique = function() {
this.sort();
var last_i;
for (var i=0;i<this.length;i++)
if ((last_i = this.lastIndexOf(this[i])) !== i)
this.splice(i+1, last_i-i);
return this;
}
or non-prototype:
function sortUnique(arr) {
arr.sort();
var last_i;
for (var i=0;i<arr.length;i++)
if ((last_i = arr.lastIndexOf(arr[i])) !== i)
arr.splice(i+1, last_i-i);
return arr;
}
This is also faster than the above method in most non-Chrome browsers.
We can do this using ES6 sets:
var duplicatesArray = [1, 2, 3, 4, 5, 1, 1, 1, 2, 3, 4];
var uniqueArray = [...new Set(duplicatesArray)];
console.log(uniqueArray); // [1,2,3,4,5]
["Defects", "Total", "Days", "City", "Defects"].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
[0,1,2,0,3,2,1,5].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
After looking into all the 90+ answers here, I saw there is room for one more:
Array.includes has a very handy second-parameter: "fromIndex", so by using it, every iteration of the filter callback method will search the array, starting from [current index] + 1 which guarantees not to include currently filtered item in the lookup and also saves time.
Note - this solution does not retain the order, as it removed duplicated items from left to right, but it wins the Set trick if the Array is a collection of Objects.
// 🚩 🚩 🚩
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
list.filter((v,i) => !list.includes(v,i+1))
)
// [0,1,3,"b",4,5,2,"a"]
Explanation:
For example, lets assume the filter function is currently iterating at index 2) and the value at that index happens to be 2. The section of the array that is then scanned for duplicates (includes method) is everything after index 2 (i+1):
👇 👇
[0, 1, 2, 2 ,3 ,'a', 'b', 4, 5, 2, 'a']
👆 |---------------------------|
And since the currently filtered item's value 2 is included in the rest of the array, it will be filtered out, because of the leading exclamation mark which negates the filter rule.
If order is important, use this method:
// 🚩 🚩 🚩
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
// Initialize with empty array and fill with non-duplicates
list.reduce((acc, v) => (!acc.includes(v) && acc.push(v), acc), [])
)
// [0,1,2,3,"a","b",4,5]
This has been answered a lot, but it didn't address my particular need.
Many answers are like this:
a.filter((item, pos, self) => self.indexOf(item) === pos);
But this doesn't work for arrays of complex objects.
Say we have an array like this:
const a = [
{ age: 4, name: 'fluffy' },
{ age: 5, name: 'spot' },
{ age: 2, name: 'fluffy' },
{ age: 3, name: 'toby' },
];
If we want the objects with unique names, we should use array.prototype.findIndex instead of array.prototype.indexOf:
a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);
This prototype getUnique is not totally correct, because if i have a Array like: ["1",1,2,3,4,1,"foo"] it will return ["1","2","3","4"] and "1" is string and 1 is a integer; they are different.
Here is a correct solution:
Array.prototype.unique = function(a){
return function(){ return this.filter(a) }
}(function(a,b,c){ return c.indexOf(a,b+1) < 0 });
using:
var foo;
foo = ["1",1,2,3,4,1,"foo"];
foo.unique();
The above will produce ["1",2,3,4,1,"foo"].
You can simlply use the built-in functions Array.prototype.filter() and Array.prototype.indexOf()
array.filter((x, y) => array.indexOf(x) == y)
var arr = [1, 2, 3, 3, 4, 5, 5, 5, 6, 7, 8, 9, 6, 9];
var newarr = arr.filter((x, y) => arr.indexOf(x) == y);
console.log(newarr);
[...new Set(duplicates)]
This is the simplest one and referenced from MDN Web Docs.
const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]
Array.prototype.getUnique = function() {
var o = {}, a = []
for (var i = 0; i < this.length; i++) o[this[i]] = 1
for (var e in o) a.push(e)
return a
}
Without extending Array.prototype (it is said to be a bad practice) or using jquery/underscore, you can simply filter the array.
By keeping last occurrence:
function arrayLastUnique(array) {
return array.filter(function (a, b, c) {
// keeps last occurrence
return c.indexOf(a, b + 1) < 0;
});
},
or first occurrence:
function arrayFirstUnique(array) {
return array.filter(function (a, b, c) {
// keeps first occurrence
return c.indexOf(a) === b;
});
},
Well, it's only javascript ECMAScript 5+, which means only IE9+, but it's nice for a development in native HTML/JS (Windows Store App, Firefox OS, Sencha, Phonegap, Titanium, ...).
That's because 0 is a falsy value in JavaScript.
this[i] will be falsy if the value of the array is 0 or any other falsy value.
Now using sets you can remove duplicates and convert them back to the array.
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
console.log([...new Set(names)])
Another solution is to use sort & filter
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);
If you're using Prototype framework there is no need to do 'for' loops, you can use http://prototypejs.org/doc/latest/language/Array/prototype/uniq/ like this:
var a = Array.uniq();
Which will produce a duplicate array with no duplicates. I came across your question searching a method to count distinct array records so after uniq() I used size() and there was my simple result.
p.s. Sorry if i mistyped something
edit: if you want to escape undefined records you may want to add compact() before, like this:
var a = Array.compact().uniq();
I had a slightly different problem where I needed to remove objects with duplicate id properties from an array. this worked.
let objArr = [{
id: '123'
}, {
id: '123'
}, {
id: '456'
}];
objArr = objArr.reduce((acc, cur) => [
...acc.filter((obj) => obj.id !== cur.id), cur
], []);
console.log(objArr);
If you're okay with extra dependencies, or you already have one of the libraries in your codebase, you can remove duplicates from an array in place using LoDash (or Underscore).
Usage
If you don't have it in your codebase already, install it using npm:
npm install lodash
Then use it as follows:
import _ from 'lodash';
let idArray = _.uniq ([
1,
2,
3,
3,
3
]);
console.dir(idArray);
Out:
[ 1, 2, 3 ]
I'm not sure why Gabriel Silveira wrote the function that way but a simpler form that works for me just as well and without the minification is:
Array.prototype.unique = function() {
return this.filter(function(value, index, array) {
return array.indexOf(value, index + 1) < 0;
});
};
or in CoffeeScript:
Array.prototype.unique = ->
this.filter( (value, index, array) ->
array.indexOf(value, index + 1) < 0
)
Finding unique Array values in simple method
function arrUnique(a){
var t = [];
for(var x = 0; x < a.length; x++){
if(t.indexOf(a[x]) == -1)t.push(a[x]);
}
return t;
}
arrUnique([1,4,2,7,1,5,9,2,4,7,2]) // [1, 4, 2, 7, 5, 9]
It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:
Array.prototype.getUnique = function(){
var u = {}, a = [];
for (var i = 0, l = this.length; i < l; ++i) {
if (u.hasOwnProperty(this[i])) {
continue;
}
a.push(this[i]);
u[this[i]] = 1;
}
return a;
}
If you do have a mixed-type array, you can serialize the hash key:
Array.prototype.getUnique = function() {
var hash = {}, result = [], key;
for ( var i = 0, l = this.length; i < l; ++i ) {
key = JSON.stringify(this[i]);
if ( !hash.hasOwnProperty(key) ) {
hash[key] = true;
result.push(this[i]);
}
}
return result;
}
strange this hasn't been suggested before.. to remove duplicates by object key (id below) in an array you can do something like this:
const uniqArray = array.filter((obj, idx, arr) => (
arr.findIndex((o) => o.id === obj.id) === idx
))
For an object-based array with some unique id's, I have a simple solution through which you can sort in linear complexity
function getUniqueArr(arr){
const mapObj = {};
arr.forEach(a => {
mapObj[a.id] = a
})
return Object.values(mapObj);
}
Let's suppose I wanted a sort function that returns a sorted copy of the inputted array. I naively tried this
function sort(arr) {
return arr.sort();
}
and I tested it with this, which shows that my sort method is mutating the array.
var a = [2,3,7,5,3,7,1,3,4];
sort(a);
alert(a); //alerts "1,2,3,3,3,4,5,7,7"
I also tried this approach
function sort(arr) {
return Array.prototype.sort(arr);
}
but it doesn't work at all.
Is there a straightforward way around this, preferably a way that doesn't require hand-rolling my own sorting algorithm or copying every element of the array into a new one?
You need to copy the array before you sort it. One way with es6:
const sorted = [...arr].sort();
The spread-syntax as array literal (copied from mdn):
var arr = [1, 2, 3];
var arr2 = [...arr]; // like arr.slice()
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator
Just copy the array. There are many ways to do that:
function sort(arr) {
return arr.concat().sort();
}
// Or:
return Array.prototype.slice.call(arr).sort(); // For array-like objects
Try the following
function sortCopy(arr) {
return arr.slice(0).sort();
}
The slice(0) expression creates a copy of the array starting at element 0.
You can use slice with no arguments to copy an array:
var foo,
bar;
foo = [3,1,2];
bar = foo.slice().sort();
You can also do this
d = [20, 30, 10]
e = Array.from(d)
e.sort()
This way d will not get mutated.
function sorted(arr) {
temp = Array.from(arr)
return temp.sort()
}
//Use it like this
x = [20, 10, 100]
console.log(sorted(x))
Update - Array.prototype.toSorted() proposal
The Array.prototype.toSorted(compareFn) -> Array is a new method which was proposed to be added to the Array.prototype and is currently in stage 3 (Soon to be available).
This method will keep the target Array untouched and returns a copy of it with the change performed instead.
Anyone who wants to do a deep copy (e.g. if your array contains objects) can use:
let arrCopy = JSON.parse(JSON.stringify(arr))
Then you can sort arrCopy without changing arr.
arrCopy.sort((obj1, obj2) => obj1.id > obj2.id)
Please note: this can be slow for very large arrays.
Try this to sort the numbers. This does not mutate the original array.
function sort(arr) {
return arr.slice(0).sort((a,b) => a-b);
}
There's a new tc39 proposal, which adds a toSorted method to Array that returns a copy of the array and doesn't modify the original.
For example:
const sequence = [3, 2, 1];
sequence.toSorted(); // => [1, 2, 3]
sequence; // => [3, 2, 1]
As it's currently in stage 3, it will likely be implemented in browser engines soon, but in the meantime a polyfill is available here or in core-js.
I think that my answer is a bit too late but if someone come across this issue again the solution may be useful.
I can propose yet another approach with a native function which returns a sorted array.
This code still mutates the original object but instead of native behaviour this implementation returns a sorted array.
// Remember that it is not recommended to extend build-in prototypes
// or even worse override native functions.
// You can create a seperate function if you like
// You can specify any name instead of "sorted" (Python-like)
// Check for existence of the method in prototype
if (typeof Array.prototype.sorted == "undefined") {
// If it does not exist you provide your own method
Array.prototype.sorted = function () {
Array.prototype.sort.apply(this, arguments);
return this;
};
}
This way of solving the problem was ideal in my situation.
You can also extend the existing Array functionality. This allows chaining different array functions together.
Array.prototype.sorted = function (compareFn) {
const shallowCopy = this.slice();
shallowCopy.sort(compareFn);
return shallowCopy;
}
[1, 2, 3, 4, 5, 6]
.filter(x => x % 2 == 0)
.sorted((l, r) => r - l)
.map(x => x * 2)
// -> [12, 8, 4]
Same in typescript:
// extensions.ts
Array.prototype.sorted = function (compareFn?: ((a: any, b: any) => number) | undefined) {
const shallowCopy = this.slice();
shallowCopy.sort(compareFn);
return shallowCopy;
}
declare global {
interface Array<T> {
sorted(compareFn?: (a: T, b: T) => number): Array<T>;
}
}
export {}
// index.ts
import 'extensions.ts';
[1, 2, 3, 4, 5, 6]
.filter(x => x % 2 == 0)
.sorted((l, r) => r - l)
.map(x => x * 2)
// -> [12, 8, 4]
It's always tricky to think Array.concat thing. Often, I just want to use mutable Array.push because I simply add extra-data on the immutable data. So, I usually do:
array[array.length] = newData;
I've asked a question related got some answers here: How to store data of a functional chain
const L = (a) => {
const m = a => (m.list ? m.list : m.list = [])
.push(a) && m;
//use `concat` and refactor needed instead of `push`
//that is not immutable
return m(a); // Object construction!
};
console.log(L);
console.log(L(2));
console.log(L(1)(2)(3))
some outputs:
{ [Function: m] list: [ 2 ] }
{ [Function: m] list: [ 1, 2, 3 ] }
I feel that push should be replaced with using concat, but still, push makes the code elegant simply because we don't want to prepare another object here.
Basically, I want to do:
arr1 = arr1.concat(arr2);
but, is there any way to write
arr1[arr1.length] = arr2;
which ends up with a nested array, and does not work.
You could assign a new array with a default array for not given m.list.
const L = (a) => {
const m = a => (m.list = (m.list || []).concat(a), m);
return m(a);
};
console.log(L.list);
console.log(L(2).list);
console.log(L(1)(2)(3).list);
You can use multiple parameters in Array.push so:
var a = [];
a.push(3, 4) // a is now [3, 4]
Combined with the ES6 spread syntax:
var a = [1, 2];
var b = [3, 4]
a.push(...b); // a is now [1, 2, 3, 4]
arr1[arr1.length] represents a single value, the value at index arr1.length.
Imagine this array
[ 1 , 2 , 3 , 4 ] // arr of length 4
^0 ^1 ^2 ^3 // indexes
If we say arr1[arr1.length] = someThing
We ask javascript to put something right here, and only here, at index 4:
[ 1 , 2 , 3 , 4 , ] // arr of length 4
^0 ^1 ^2 ^3 ^4 // add someThing in index 4
So, if we want to add something strictly with arr1[arr1.length], then we need to keep doing that for each index. for each meaning any kind of loop. E.G:
// Not recommended to use
var arr1 = [1,2,3];
var arr2 = [3,4,5];
while (arr2.length){
arr1[arr1.length] = arr2.shift();
}
console.log(arr1); // [1,2,3,3,4,5]
console.log(arr2); // []
But, as you can see, this method, or any similar one, even if optimized, is not the right approach. You need a concatenation.
Since you mention a functional one, which returns the resulting array, you can simply replace the initial array and make use of spread operator:
var arr1 = [1,2,3];
var arr2 = [3,4,5];
console.log(arr1 = [...arr1,...arr2]); // [1,2,3,3,4,5]
Let's suppose I wanted a sort function that returns a sorted copy of the inputted array. I naively tried this
function sort(arr) {
return arr.sort();
}
and I tested it with this, which shows that my sort method is mutating the array.
var a = [2,3,7,5,3,7,1,3,4];
sort(a);
alert(a); //alerts "1,2,3,3,3,4,5,7,7"
I also tried this approach
function sort(arr) {
return Array.prototype.sort(arr);
}
but it doesn't work at all.
Is there a straightforward way around this, preferably a way that doesn't require hand-rolling my own sorting algorithm or copying every element of the array into a new one?
You need to copy the array before you sort it. One way with es6:
const sorted = [...arr].sort();
The spread-syntax as array literal (copied from mdn):
var arr = [1, 2, 3];
var arr2 = [...arr]; // like arr.slice()
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator
Just copy the array. There are many ways to do that:
function sort(arr) {
return arr.concat().sort();
}
// Or:
return Array.prototype.slice.call(arr).sort(); // For array-like objects
Try the following
function sortCopy(arr) {
return arr.slice(0).sort();
}
The slice(0) expression creates a copy of the array starting at element 0.
You can use slice with no arguments to copy an array:
var foo,
bar;
foo = [3,1,2];
bar = foo.slice().sort();
You can also do this
d = [20, 30, 10]
e = Array.from(d)
e.sort()
This way d will not get mutated.
function sorted(arr) {
temp = Array.from(arr)
return temp.sort()
}
//Use it like this
x = [20, 10, 100]
console.log(sorted(x))
Update - Array.prototype.toSorted() proposal
The Array.prototype.toSorted(compareFn) -> Array is a new method which was proposed to be added to the Array.prototype and is currently in stage 3 (Soon to be available).
This method will keep the target Array untouched and returns a copy of it with the change performed instead.
Anyone who wants to do a deep copy (e.g. if your array contains objects) can use:
let arrCopy = JSON.parse(JSON.stringify(arr))
Then you can sort arrCopy without changing arr.
arrCopy.sort((obj1, obj2) => obj1.id > obj2.id)
Please note: this can be slow for very large arrays.
Try this to sort the numbers. This does not mutate the original array.
function sort(arr) {
return arr.slice(0).sort((a,b) => a-b);
}
There's a new tc39 proposal, which adds a toSorted method to Array that returns a copy of the array and doesn't modify the original.
For example:
const sequence = [3, 2, 1];
sequence.toSorted(); // => [1, 2, 3]
sequence; // => [3, 2, 1]
As it's currently in stage 3, it will likely be implemented in browser engines soon, but in the meantime a polyfill is available here or in core-js.
I think that my answer is a bit too late but if someone come across this issue again the solution may be useful.
I can propose yet another approach with a native function which returns a sorted array.
This code still mutates the original object but instead of native behaviour this implementation returns a sorted array.
// Remember that it is not recommended to extend build-in prototypes
// or even worse override native functions.
// You can create a seperate function if you like
// You can specify any name instead of "sorted" (Python-like)
// Check for existence of the method in prototype
if (typeof Array.prototype.sorted == "undefined") {
// If it does not exist you provide your own method
Array.prototype.sorted = function () {
Array.prototype.sort.apply(this, arguments);
return this;
};
}
This way of solving the problem was ideal in my situation.
You can also extend the existing Array functionality. This allows chaining different array functions together.
Array.prototype.sorted = function (compareFn) {
const shallowCopy = this.slice();
shallowCopy.sort(compareFn);
return shallowCopy;
}
[1, 2, 3, 4, 5, 6]
.filter(x => x % 2 == 0)
.sorted((l, r) => r - l)
.map(x => x * 2)
// -> [12, 8, 4]
Same in typescript:
// extensions.ts
Array.prototype.sorted = function (compareFn?: ((a: any, b: any) => number) | undefined) {
const shallowCopy = this.slice();
shallowCopy.sort(compareFn);
return shallowCopy;
}
declare global {
interface Array<T> {
sorted(compareFn?: (a: T, b: T) => number): Array<T>;
}
}
export {}
// index.ts
import 'extensions.ts';
[1, 2, 3, 4, 5, 6]
.filter(x => x % 2 == 0)
.sorted((l, r) => r - l)
.map(x => x * 2)
// -> [12, 8, 4]