I have a JavaScript array of numbers. My array is defined like this:
var customerIds = [];
I have a function that is responsible for inserting and removing ids to/from this array. Basically, my function looks like this:
function addOrRemove(shouldAdd, customerId) {
if (shouldAdd) {
if (customerIds.contains(customerId) === false) {
customerIds.push(customerId);
}
} else {
customerIds.remove(customerId);
}
}
This function is basically pseudocode. A JavaScript array does not have a contains or remove function. My question is, is there any elegant way of tackling this problem? The best I can come up with is always looping through the array myself and tracking the index of the first item found.
Thank you for any insights you can provide.
The contains can be achieved with Array.prototype.indexOf, like this
if (customerIds.indexOf(customerId) === -1) {
indexOf function returns -1, if it couldn't find the parameter in the array, otherwise the first index of the match. So, if the result is -1, it means that customerIds doesn't contain customerId.
The remove can be achieved with Array.prototype.indexOf and Array.prototype.splice, like this
var index = customerIds.indexOf(customerId);
if (index !== -1) {
customerIds.splice(index, 1);
}
Similarly, indexOf function returns -1, if it couldn't find the parameter in the array, otherwise the first index of the match. So, if the result is -1, we skip deleteing, otherwise splice 1 element starting from the position index.
You can extend the Array method like below after that you are free to use 'contains' and 'remove'
if (!Array.contains)
Array.prototype.contains = function(a) {
for (var i in this) {
if (this[i] == a) return true;
}
return false
}
if (!Array.remove)
Array.prototype.remove = function(a) {
for (var i in this) {
if (this[i] == a) {
this.splice(i, 1);
}
}
}
Use indexOf and splice
function addOrRemove(shouldAdd, customerId) {
if (shouldAdd) {
if (customerIds.indexOf(customerId) == -1) {
customerIds.push(customerId);
}
} else {
var index = customerIds.indexOf(customerId)
customerIds.splice(index, 1);
}
}
You could definitely use the splice and indexOf as stated by #thefourtheye, yet I would like to provide another approach.
Instead of using an array you could use an object.
var customerIds = {};
//This could also be stated as: var customerIds = new Object(); this is just shorthand
function addOrRemove(shouldAdd, customerId)
{
if(shouldAd)
{
if(!customerIds[customerId])
{
customerIds[customerId] = new Object();
customerIds[customerId].enabled = true;
}
}
else
{
if(customerIds[customerId])
{
customerIds[customerId].enabled = false;
}
}
}
You now can query against the customerIds object for a specific customerId
if(customerIds[customerId].enabled)
Using this method not only provides you with the capability of attaching multiple attributes to a given customerId, but also allows you to keep records of all customerIds after disabling (removing).
Unfortunately, in order to truely remove the customerId, you would need to loop through the object and append each property of the object to a new object except for the one you do not want. The function would look like this:
function removeId(customerId)
{
var n_customerIds = new Object();
for(var key in customerIds)
{
if(key != customerId)
{
n_customerIds[key] = customerIds[key];
}
}
customerIds = n_customerIds;
}
In no way am I stating that this would be the proper approach for your implementation, but I am just providing another method of achieving your goal. There are many equivalent ways to solve your dilemma, and it is solely decided by you which method will best suit your projects functionality. I have personally used this method in many projects, as well as I have used the methods posted by others in many other projects. Each method has their pros and cons.
If you do wish to use this method, I would only suggest doing so if you are not collecting many customerIds and do want a lot of customerData per each customerId, or, if you are collecting many customerIds and do not want a lot of customerData per each customerId. If you store a lot of customerData for a lot of customerIds, you will consume a very large amount of memory.
Related
I have an array of ~11,000 JavaScript dictionaries, each representing 1 row in an Excel file.
I want to loop through this array and parse each element into a new datastructure. For example, I might have a function that will count for {"foo": true} or something.
As I have multiple of these functions, my question is would it be better to loop through this array for each function, or have one single loop with functions that parse each element and store it in a global variable?
Ex. I'm currently doing one single loop, and parsing each element into a global variable
const arr = [...]; // array of ~11,000 dictionaries
// example parsing function
let count = 0;
function countFoos(el) {
if (el["foo"] === true) count++;
}
let count2 = 0;
function countBars(el) {
if (el["bar"] === false) count2++;
}
arr.forEach(el => {
countFoos(el);
countBars(el);
});
But would it be better to do it this way?
class Parse {
constructor(arr) {
this.arr = arr;
this.count = 0;
this.count2 = 0;
}
countFoos() {
this.arr.forEach((el) => {
if (el["foo"] === true) this.count++;
});
}
countBars() {
this.arr.forEach((el) => {
if (el["bar"] === false) this.count2++;
});
}
}
const arr = [...]; // array of ~11,000 dictionaries
let x = Parse();
x.countFoos();
x.countBars();
EDIT: I should've clarified early, the examples shown above are just very simplified examples of the production code. Approximately 20 'parsing functions' are being run on for each element, with each of its corresponding global variables being large dictionaries or arrays.
You should generally do just one iteration that calls both functions.
Iterating takes time, so doing two iterations will double the time taken to perform the iterations. How significant this is to the entire application depends on how much work is done in the body of the iteration. If the bodies are very expensive, the iteration time might fall into the noise. But if it's really simple, as in your examples of a simple test and variable increment, the iteration time will probably be significant.
If you are worried about performance, the first method is better as it only involves one iteration over the entire array while the second approach requires two.
If think using classes is more readable, you could simply put write that as one method in the class.
class Parse {
constructor(arr) {
this.arr = arr;
this.count = 0;
this.count2 = 0;
}
count() {
this.arr.forEach((el) => {
countFoos(el), countBars(el);
});
}
countFoos(el){
if(el.foo === true) this.count1++;
}
countBars() {
if(el.bar === false) this.count2++;
}
}
I would approach this by using the Array.prototype.reduce function, which would only require a single pass over the given array. I also would not use a class here as it would not really make sense, but you can if you really want!
function count(arr) {
return arr.reduce(([fooCount, barCount], next) => {
if (next.foo === true) {
fooCount = fooCount + 1
}
if (next.bar === false) {
barCount = barCount + 1
}
return [fooCount, barCount]
}, [0, 0]);
}
const [fooCount, barCount] = count(...);
You can also use generators to accomplish this, which is even better because it doesn't require that you to iterate the entire set of words in the dictionary, but it's a little more unwieldy to use.
This is actually easier to use than other examples that require if statements, because you could quite easily run a battery of functions over each result and add it to the accumulator.
Just remember though that you don't want to optimize before you prove something is a problem. Iterating 22000 objects is obviously more than iterating 11000, but it is still going to be quite fast!
Restricting the number of loops is your best option as it requires less overhead.
Also here is an idea, using the foreach to do the processing with if statements and using a single counter object to hold all of the values so they mean something and can be easily referenced later on.
const arr = [
{"foo" : true,"bar" : false},{"bar" : true,"foo" : false}
];
let counters = {};
function inc(field) {
if (counters[field] == undefined) {
counters[field] = 0;
}
counters[field]++;
}
arr.forEach(el => {
if (el["foo"] === true) {
inc("foo");
}
if (el["bar"] === true) {
inc("bar");
}
});
console.log(counters);
I dont want to push duplicate values into selectedOwners, so in below code user is selecting owner if owner already existed in selectedOwners array i dont want to push , How can i check that to avoid duplicate values in an array ?
ctrl.js
var selectedOwners = [];
$scope.addProcessOwner = function(dataItem){
var selectedOwner = {
fullName: dataItem.fullName,
workerKey: dataItem.workerKey
}
if(selectedOwners.indexOf(selectedOwner) !== -1) {
selectedOwners.push(selectedOwner);
}
console.log('WORKER DATA',selectedOwners);
}
You can use Array.prototype.some method
The some() method tests whether some element in the array passes the test implemented by the provided function.
var isExists = function(e) {
if (e.fullName == selectedOwner.fullName
&& e.workerKey == selectedOwner.workerKey) {
return true;
}
}
if (!selectedOwners.some(isExists)) {
selectedOwners.push(selectedOwner);
}
The use of Array.indexOf is obvious for simple types like strings and numbers.
However, when you are looking for an object, you have to pass the exact same object. A different object with all the same properties and values will still not work. Think of the array as containing pointers to the objects and you must look for the same pointer.
Instead you will need to write your own method to compare the owners for equality and loop through the array doing this check.
Try wrapping your "if" logic in a for-loop .
Example
//Following code loops through array and check for already existing value
for(var i = 0; i < selectedOwners.length; i++){
if(selectedOwners.indexOf(selectedOwner) !== -1) {
selectedOwners.push(selectedOwner);
}
}
I have a data:
data: function() {
return {
conversations:
[
]
}
}
I'm getting my data from response object: response.data.conversation
Is there a way to check this.conversations already contains response.data.conversation?
To build on your answer, if you're already using underscore or lodash you can use its _.any()/_.some() function:
var exists = _.any(this.conversations, function(conversation) {
return _.isEqual(conversation, response.data.conversation);
})
You can also use Array.prototype.some to do the same kind of thing:
var exists = this.conversations.some(function(conversation) {
return _.isEqual(conversation, response.data.conversation);
})
The benefits of these over your solution is that they'll return as soon as they find a match (instead of iterating through the whole array), though you could easily update your code to break out of the loop early.
Also, while _.isEqual() is cool, you might be able to get away with some simple property comparisons (if your objects are flat enough or, even better, you have a key that uniquely identifies a conversation) to determine if two objects are equivalent:
var exists = this.conversations.some(function(conversation) {
return conversation.id === response.data.conversation.id;
})
I figured it out:
Used underscore.js.
Iterate trought all objects in array and compare them with _.isEqual(a,b)
function
var count=0;
for(var i=0; i<this.conversations.length; i++ ) {
if(_.isEqual(this.conversations[i], response.data.conversation)) {
count++;
}
}
Then check the value of count variable:
if (count == 0) {
//Add element to array
this.conversations.push(response.data.conversation);
} else {
console.warn('exists');
}
I am trying to figure out if all of the elements in an array are keys in the object.
var obj = { name: 'Computer', cost: '$1,000' };
var myArray = [ 'name', 'cost', 'bio' ]; //another example would be var myArray = [];
for(var x = 0; x < myArray.length; x++){
if (myArray[x] in obj)
{
return true;
}
}
How can I check if all of the elements in an array are keys in the object?
Do it the other way around. If you find someone in the array who is NOT in the object then you return false. If you reach the end of the loop then you return true because all the keys were in the object.
Depending on what you want, this might do the trick:
function hasKeys(obj, keys) {
for (var i=0; i != keys.length; ++i) {
if (!(keys[i] in obj))
return false;
}
return true;
};
One subtlety you need to ask yourself: do you want to know if the object has the keys directly (i.e. not somewhere in its prototype stack?) If so, then replace keys[i] in obj with obj.hasOwnProperty(keys[i])
function hasKeys(obj, keys) {
return keys.every(Object.prototype.hasOwnProperty.bind(obj));
}
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/every states, "The every method executes the provided callback function once for each element present in the array until it finds one where callback returns a falsy value (a value that becomes false when converted to a Boolean). If such an element is found, the every method immediately returns false. Otherwise, if callback returned a true value for all elements, every will return true. callback is invoked only for indexes of the array which have assigned values; it is not invoked for indexes which have been deleted or which have never been assigned values" (emphasis mine).
Array.some() makes for a clean solution.
// object in question
var obj = { ... };
// keys that need to be present in the object
var keys = [ ... ];
// iterate through the whitelist until we find a key that doesn't exist in the object. If all exist, that means Array.some() is false.
var valid = !keys.some(function(key) {
return !obj.hasOwnProperty(key);
});
An alternative solution would be using a similar concept, but with Array.every(). It is to note that this will generally be slower because it always has to touch every element in the whitelist.
// iterate through the whitelist, making sure the object has each key.
var valid = keys.every(obj.hasOwnProperty);
This problem can be expressed in terms of set inclusion: does the set of property keys completely include the array of required keys? So we can write it as
includes(Object.keys(obj), arr)
So now we just need to write includes.
function includes(arr1, arr2) {
return arr2.every(function(key) {
return contains(arr1, key);
}
}
For contains, we could use Underscore's _.contains, or just write it ourselves:
function contains(arr, val) {
return arr.indexOf(val) !== -1;
}
If we are interested in conciseness at the possible expense of readability, we could shorten our definition of includes to use Function#bind instead of the anonymous function:
function includes(arr1, arr2) {
return arr2.every(contains.bind(0, arr1));
}
Now we have functions we can use for other things, instead of mixing up the two different aspects of the problem--the keys of an object, and set inclusion. If we really want to write an all-in-one function, it becomes the somewhat more readable:
function hasMany(obj, arr) {
return arr.every(_.contains.bind(0, Object.keys(obj));
}
If we want more readability, like we were writing a novel:
function object_has_required_keys(object, required_keys) {
var object_keys = Object.keys(object);
function key_is_present(key) {
return object_keys.indexOf(key) !== -1;
}
return required_keys.every(key_is_present);
}
Underscore's _.intersection
If we're lazy (or smart), we could use Underscore's _.intersection to implement includes:
function includes(arr1, arr2) {
return _.intersection(arr1, arr2).length === arr2.length;
}
The idea is to take the intersection, and if the first array includes the second entirely, then the intersection will contain all the elements of the second array, which we can check by comparing their lengths.
Using ES6 sets
Thinking ahead to ES6, we could implement include using its sets, which ought to be faster:
function includes(arr1, arr2) {
var set = new Set(arr1);
return arr2.every(Set.prototype.has.bind(set));
}
TL;DR version: I want to avoid adding duplicate Javascript objects to an array of similar objects, some of which might be really big. What's the best approach?
I have an application where I'm loading large amounts of JSON data into a Javascript data structure. While it's a bit more complex than this, assume that I'm loading JSON into an array of Javascript objects from a server through a series of AJAX requests, something like:
var myObjects = [];
function processObject(o) {
myObjects.push(o);
}
for (var x=0; x<1000; x++) {
$.getJSON('/new_object.json', processObject);
}
To complicate matters, the JSON:
is in an unknown schema
is of arbitrary length (probably not enormous, but could be in the 100-200 kb range)
might contain duplicates across different requests
My initial thought is to have an additional object to store a hash of each object (via JSON.stringify?) and check against it on each load, like this:
var myHashMap = {};
function processObject(o) {
var hash = JSON.stringify(o);
// is it in the hashmap?
if (!(myHashMap[hash])) {
myObjects.push(o);
// set the hashmap key for future checks
myHashMap[hash] = true;
}
// else ignore this object
}
but I'm worried about having property names in myHashMap that might be 200 kb in length. So my questions are:
Is there a better approach for this problem than the hashmap idea?
If not, is there a better way to make a hash function for a JSON object of arbitrary length and schema than JSON.stringify?
What are the possible issues with super-long property names in an object?
I'd suggest you create an MD5 hash of the JSON.stringify(o) and store that in your hashmap with a reference to your stored object as the data for the hash. And to make sure that there are no object key order differences in the JSON.stringify(), you have to create a copy of the object that orders the keys.
Then, when each new object comes in, you check it against the hash map. If you find a match in the hash map, then you compare the incoming object with the actual object that you've stored to see if they are truly duplicates (since there can be MD5 hash collisions). That way, you have a manageable hash table (with only MD5 hashes in it).
Here's code to create a canonical string representation of an object (including nested objects or objects within arrays) that handles object keys that might be in a different order if you just called JSON.stringify().
// Code to do a canonical JSON.stringify() that puts object properties
// in a consistent order
// Does not allow circular references (child containing reference to parent)
JSON.stringifyCanonical = function(obj) {
// compatible with either browser or node.js
var Set = typeof window === "object" ? window.Set : global.Set;
// poor man's Set polyfill
if (typeof Set !== "function") {
Set = function(s) {
if (s) {
this.data = s.data.slice();
} else {
this.data = [];
}
};
Set.prototype = {
add: function(item) {
this.data.push(item);
},
has: function(item) {
return this.data.indexOf(item) !== -1;
}
};
}
function orderKeys(obj, parents) {
if (typeof obj !== "object") {
throw new Error("orderKeys() expects object type");
}
var set = new Set(parents);
if (set.has(obj)) {
throw new Error("circular object in stringifyCanonical()");
}
set.add(obj);
var tempObj, item, i;
if (Array.isArray(obj)) {
// no need to re-order an array
// but need to check it for embedded objects that need to be ordered
tempObj = [];
for (i = 0; i < obj.length; i++) {
item = obj[i];
if (typeof item === "object") {
tempObj[i] = orderKeys(item, set);
} else {
tempObj[i] = item;
}
}
} else {
tempObj = {};
// get keys, sort them and build new object
Object.keys(obj).sort().forEach(function(item) {
if (typeof obj[item] === "object") {
tempObj[item] = orderKeys(obj[item], set);
} else {
tempObj[item] = obj[item];
}
});
}
return tempObj;
}
return JSON.stringify(orderKeys(obj));
}
And, the algorithm
var myHashMap = {};
function processObject(o) {
var stringifiedCandidate = JSON.stringifyCanonical(o);
var hash = CreateMD5(stringifiedCandidate);
var list = [], found = false;
// is it in the hashmap?
if (!myHashMap[hash] {
// not in the hash table, so it's a unique object
myObjects.push(o);
list.push(myObjects.length - 1); // put a reference to the object with this hash value in the list
myHashMap[hash] = list; // store the list in the hash table for future comparisons
} else {
// the hash does exist in the hash table, check for an exact object match to see if it's really a duplicate
list = myHashMap[hash]; // get the list of other object indexes with this hash value
// loop through the list
for (var i = 0; i < list.length; i++) {
if (stringifiedCandidate === JSON.stringifyCanonical(myObjects[list[i]])) {
found = true; // found an exact object match
break;
}
}
// if not found, it's not an exact duplicate, even though there was a hash match
if (!found) {
myObjects.push(o);
myHashMap[hash].push(myObjects.length - 1);
}
}
}
Test case for jsonStringifyCanonical() is here: https://jsfiddle.net/jfriend00/zfrtpqcL/
Maybe. For example if You know what kind object goes by You could write better indexing and searching system than JS objects' keys. But You could only do that with JavaScript and object keys are written in C...
Must Your hashing be lossless or not? If can than try to lose compression (MD5). I guessing You will lose some speed and gain some memory. By the way, do JSON.stringify(o) guarantees same key ordering. Because {foo: 1, bar: 2} and {bar: 2, foo: 1} is equal as objects, but not as strings.
Cost memory
One possible optimization:
Instead of using getJSON use $.get and pass "text" as dataType param. Than You can use result as Your hash and convert to object afterwards.
Actually by writing last sentence I though about another solution:
Collect all results with $.get into array
Sort it with buildin (c speed) Array.sort
Now You can easily spot and remove duplicates with one for
Again different JSON strings can make same JavaScript object.