I currently have a problem with the performance of a JavaScript operation.
I want to clean up an object array (aEmployees) for those objects whose ID is already in the array aInvited or which are not in the aInvitationAllowed array:
var iLength = aEmployees.length - 1;
for (var i = iLength; i >= 0; --i) {
if (aInvited.indexOf(aEmployees[i].id) !== -1 || aInvitationAllowed.indexOf(aEmployees[i].id) === -1) {
aEmployees.splice(i, 1);
}
}
You can imagine that I have Employee objects (aEmployees). I also have a list of Employee IDs that have already been invited to an event (aInvited) and a list of Employee IDs that are allowed to be invited to the event (aInvitationAllowed). I want to have all Employees as a result that I can still invite to the event.
The problem is the line with the indexOf queries. I have to assume that the arrays have very many entries (~ 100,000 entries). Then it may be that taking the loop takes a while.
So my question to you: How can I make the loop faster? Are there faster alternatives for indexOf?
Thanks for your tips!
Consider using a Set to compare instead. Set.has is an O(1) operation, whereas indexOf is an O(n) operation, reducing the computational complexity overall to O(n) instead of O(n^2):
const aInvitedSet = new Set(aInvited);
const aAllowedSet = new Set(aAllowed);
const iLength = aEmployees.length - 1;
for (let i = iLength; i >= 0; --i) {
const { id } = aEmployees[i];
if (aInvitedSet.has(id) || !aAllowedSet.has(id)) {
aEmployees.splice(i, 1);
}
}
Also, splice is slow-ish. Unless you have to mutate the exising array, you might consider .pushing to a new array instead (which looks to be faster than Array.prototype.filter):
const aInvitedSet = new Set(aInvited);
const aAllowedSet = new Set(aAllowed);
const newArr = [];
const iLength = aEmployees.length;
for (let i = 0; i < iLength; i++) {
const employee = aEmployees[i];
const { id } = employee;
if (!aInvitedSet.has(id) && aAllowedSet.has(id)) {
newArr.push(employee);
}
}
Related
my current problem is I have two mongodb collections. My code in essence takes each name from table 1 which is 250000~ items then it compares against the list of names from table 2 (3000~) to try and match.
now I'm just brute forcing this by going through every item in the 250000 list and looking at every item in the 3000 list to see if it matches
the problem is that this takes about 2 hours to run through
Is there a faster way to do matching? I've put the stripped down version of my code below so you can see it's a simple loop in a loop.
Thanks
const SCCM = await ProgramDev.find({})
const Cat = await Software.find({})
for (var i = 0, len = SCCM.length; i < len; i++) {
for (var n = 0, Catlen = Cat.length; n < Catlen; n++) {
var program = SCCM[i]['program name']
var software = Cat[n]['Application']
var sat = Cat[n].Status
if (SCCM[i].Status == undefined) {
if (program == software) {
SCCM[i].Status = sat
break
}
}
}
if (SCCM[i].Status == undefined) {
SCCM[i].Status = "Un-authorised"
SCCM[i].matchType = "None"
}
}
If you have already extracted each name from collection1 and from collection2 into arrays this should work faster
array1 = ['john','smith','vova','putin', 'eug', 'hilly', 'tom', 'smith','vova','putin' ]
array2 = ['vova', 'dr.cacis']
const newArray = array1.filter(value => array2.includes(value));
console.log(newArray);
Output matches
[ 'vova', 'vova' ]
You can also use Array.prototype.includes
or Array.prototupe.indexOf
You can try to use map if you don't have memory restriction and put the values of Cat in the map like:
var catMap = new Map();
for (var n = 0, Catlen = Cat.length; n < Catlen; n++) {
catMap.set(Cat[n]['Application'] , Cat[n].Status);
}
And run the second loop on SCCM like:
for (var i = 0, len = SCCM.length; i < len; i++) {
if (SCCM[i].Status == undefined && catMap.has(SCCM[i]['program name'])){
SCCM[i].Status = catMap.get(SCCM[i]['program name']);
}else if(SCCM[i].Status == undefined){
SCCM[i].Status = "Un-authorised"
SCCM[i].matchType = "None"
}
}
So, the first loop will take O(n) and second loop will take O(m), total will be O(m+n) time complexity.
The space complexity will be O(n) because of map.
Overall, program will take less time to run.
JS noob here.
I'm trying to add unique values to an array based on a condition, however each time I'm adding an item to the array it will be nested. How can I add items to an array without nesting them?
var Settings = sp.getSheetByName('Settings');
var email_list = Settings.getRange("D2:D").getValues().filter(String);
var portfolio_list = Settings.getRange("E2:E").getValues().filter(String);
var i;
var distribution_list = [];
for (i = 0; i < email_list.length +1; i++) {
if ((portfolio_list[i] == 'someValue') && (distribution_list.indexOf(email_list[i]) == -1)) {
distribution_list.push(email_list[i]);
}
}
What I've noticed that .push() adds the elements as a nested array. This way I cannot check it with indexOf() because it never returns -1 when it comes to duplicates.
How could I add an item(s) to an array only if it is not present already in array?
Use Array#flat to flatten the 2D array
Use Set to store/get unique values.
Snippet:
const email_list = Settings.getRange(
'D2:D'
) /* Better to limit range ("D2:D"+Settings.getLastRow())*/
.getValues()
.flat() /*Flatten the array*/
.filter(String);
const portfolio_list = Settings.getRange('E2:E')
.getValues()
.flat()
.filter(String);
const distribution_set = new Set();
for (let i = 0; i < email_list.length /*+1 will throw error*/; i++) {
if (
portfolio_list[i] == 'someValue' &&
!distribution_set.has(
email_list[i]
) /*Added, but not needed as set can only contain unique values*/
) {
distribution_set.add(email_list[i]);
}
}
/*Instead of a for-loop, you can also do this:
const dSet = new Set(portfolio_list.filter(e => e === 'someValue'));*/
const distribution_list = [...distribution_set]; /*back to array*/
Is there a quicker or more efficient way to check that two arrays contain the same values in javascript?
Here is what I'm currently doing to check this. It works, but is lengthy.
var arraysAreDifferent = false;
for (var i = 0; i < array1.length; i++) {
if (!array2.includes(array1[i])) {
arraysAreDifferent = true;
}
}
for (var i = 0; i < array2.length; i++) {
if (!array1.includes(array2[i])) {
arraysAreDifferent = true;
}
}
To reduce computational complexity from O(n ^ 2) to O(n), use Sets instead - Set.has is O(1), but Array.includes is O(n).
Rather than a regular for loop's verbose manual iteration, use .every to check if every item in an array passes a test. Also check that both Set's sizes are the same - if that's done, then if one of the arrays is iterated over, there's no need to iterate over the other (other than for the construction of its Set):
const arr1Set = new Set(array1);
const arr2Set = new Set(array2);
const arraysAreDifferent = (
arr1Set.size === arr2Set.size &&
array1.every(item => arr2Set.has(item))
);
function same(arr1, arr2){
//----if you want to check by length as well
// if(arr1.length != arr2.length){
// return false;
// }
// creating an object with key => arr1 value and value => number of time that value repeat;
let frequencyCounter1 = {};
let frequencyCounter2 = {};
for(let val of arr1){
frequencyCounter1[val] = (frequencyCounter1[val] || 0) + 1;
}
for(let val of arr2){
frequencyCounter2[val] = (frequencyCounter2[val] || 0) + 1;
}
for(let key in frequencyCounter1){
//check if the key is present in arr2 or not
if(!(key in frequencyCounter2)) return false;
//check the number of times the value repetiton is same or not;
if(frequencyCounter2[key]!==frequencyCounter1[key]) return false;
}
return true;
}
So I've been doing this ProductSearchPage using React and it has a bunch of filter values that I need to set to filter my product list and show results.
Up until now, I've been handling my product list as an array (even though I'm fetching it as an object, I'm converting it to an array) and I've been using lots of map, forEach and A LOT of filter loops over those arrays over and over again.
I'll get a productList, I'll filter based on category
I'll take the new filteredList and filter based on priceFilters
I'll take the new filteredList and filter based on ratingFilter
And so on for brandFilter, featuresFilters, etc.
I began to think that I might be creating a black hole of iterations and that might hurt my performance at some point. I'm doing client side searching and filtering. We're talking about 2k products maximum.
So I wondered if it would be faster to iterate and filter over an object instead of an array. I would be deleting properties and creating new objects along the way.
So I did this snippet to test:
And for my surprise the results were a lot in favor of the array loops.
Looping object with for...in: 0.31ms
Looping array with forEach: 0.08ms
Looping array with filter: 0.10ms
Looping array with map: 0.09ms
QUESTION
Is this enough evidence that looping through arrays is faster than looping through objects and I should stick to the forEach, map and filter methods?
NOTE: This is really simplified. In my real case, each product is an object with some properties (some of them are nested properties). So my options are to keep the list as an array of object (like I've been doing so far) or I could keep a big object allProducts with each product as a property of that object. Could this change the results?
const myObject = {};
const myArray = []
for (let i=0; i<=2000; i++) {
myObject['prop'+i] = i;
}
for (let k=0; k<=2000; k++) {
myArray[k] = k;
}
const t0 = window.performance.now();
for (const key in myObject) {
if (myObject[key] % 37 === 0) {
//console.log(myObject[key] + ' is a multiple of 37');
}
}
const t1 = window.performance.now();
console.log('Looping object with for...in: ' + (t1 - t0).toFixed(2) + 'ms');
const t2 = window.performance.now();
myArray.forEach((item) => {
if (item % 37 === 0) {
//console.log(item + ' is a multiple of 37');
}
});
const t3 = window.performance.now();
console.log('Looping array with forEach: ' + (t3 - t2).toFixed(2) + 'ms');
const t4 = window.performance.now();
const newArray = myArray.filter((item) => item % 37 === 0);
const t5 = window.performance.now();
console.log('Looping array with filter: ' + (t5 - t4).toFixed(2) + 'ms');
const t6 = window.performance.now();
const newArray2 = myArray.map((item) => item*2);
const t7 = window.performance.now();
console.log('Looping array with map: ' + (t7 - t6).toFixed(2) + 'ms');
I would be deleting properties and creating new objects along the way.
These operations will likely take orders of magnitude longer than the time it takes to just perform the loop.
Unless of course the way you loop also affects the way you create or delete objects/properties, but I assume we're considering a loop that otherwise does identical instructions.
In the vast majority of cases it's a tiny part of the performance budget (like 1 millionth), and the wrong place to start if you want to optimize a complex application. Just run some profiling tools, get an overview of where the application is spending time, and focus on the slowest parts.
Is this enough evidence that looping through arrays is faster than looping through objects and I should stick to the forEach, map and filter methods?
No, because it's a single simplified example. It doesn't tell anything about how big of a chunk of the performance budget it represents. It's probably also different depending on which JS runtime is used. All you can derive from it is that with 2000 iterations it takes at worst 0.31 ms.
I expanded the example a bit by adding a very small amount of extra work inside the loop. This can then be multiplied to see how fast it starts being more significant than just the loop. See the iteration function below. It internally runs identically for both cases.
If the complexity is set to 0 (run extra work 0 times), it performs just like the results posted in the question. Array is 2 to 4 times faster.
However just running this work once, the difference is almost gone (~0.7ms vs ~0.8ms for me). From 2 times and upwards sometimes array wins, sometimes object, but never by a big margin.
So the difference becomes insignificant once you do pretty much anything at all inside the loop.
const myObject = {};
const myArray = []
const iterations = 2000;
for (let i = 0; i < iterations; i++) {
myObject['prop' + i] = i;
myArray[i] = i;
}
let total = 0;
function iteration(a, complexity) {
const x = {};
for (let i = 0; i < complexity; i++) {
// Do some simple instructions
const rand = Math.random();
x[`${a}~${i}`] = rand;
total += rand;
}
return x;
}
function loopObject(complexity) {
const results = [];
for (const key in myObject) {
results.push(iteration(myObject[key], complexity));
}
return results;
}
function loopArray(complexity) {
const results = [];
myArray.forEach((item) => {
results.push(iteration(item, complexity))
});
return results;
}
const samples = 10;
const decimals = 6;
function test(complexity) {
console.log(`COMPLEXITY ${complexity} (${samples} samples)`)
let arrayTimes = [];
let objectTimes = [];
for (let i = 0; i < samples; i++) {
const tA = performance.now();
const resultArray = loopArray(complexity);
arrayTimes.push(performance.now() - tA);
const tO = performance.now();
const resultObject = loopObject(complexity);
objectTimes.push(performance.now() - tO);
}
const arraySum = arrayTimes.reduce((p, c) => p + c, 0);
const objectSum = objectTimes.reduce((p, c) => p + c, 0);
const arrayWins = arraySum < objectSum;
console.log(
`ARRAY ${arrayWins ? ' (winner)' : ''}
avg: ${(arraySum / samples).toFixed(decimals)} min: ${Math.min(...arrayTimes).toFixed(decimals)} max: ${Math.max(...arrayTimes).toFixed(decimals)}`);
console.log(
`OBJECT ${!arrayWins ? ' (winner)' : ''}
avg: ${(objectSum / samples).toFixed(decimals)} min: ${Math.min(...objectTimes).toFixed(decimals)} max: ${Math.max(...objectTimes).toFixed(decimals)}`);
}
const complexities = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 50, 100];
complexities.forEach(test);
var player1=["t1", "t9", "t7", "t8", "t2"];
var player2=["t5", "t3", "t4", "t6"];
var winmoves=[[t1,t2,t3],[t4,t5,t6],[t7,t8,t9],[t1,t4,t7],[t2,t5,t8],[t3,t6,t9],[t1,t5,t9],[t3,t5,t7]];
else if (moves>=3 && moves<=9) {
moves+=1;
if (turn==1) {
var i=0;
turn=2;
x.innerHTML="<img src='x.png'/>";
player1.push(x.id);
while(i<=moves){
if (winmoves[i] in player1) {
alert("player1 wins");
document.getElementById("player1").innerHTML=1;
}
i+=1;
}
}
i have 1d array player1 and 2d array winmoves in javascript and i want to check that w[0]'s all values are present in p and so on for w[1],w[2],etc.
if condition with (winmoves[i] in player1) is not working.
i don't know if i am writing this write.
help me guys i am stuck here how can i do so.
It is not working even after i have made these changes.
else if (moves>=3 && moves<9) {
moves+=1;
if (turn==1) {
var i=0;
turn=2;
x.innerHTML="<img src='x.png'/>";
player1.push(x.id);
while(i<=moves){
mapped1 = winmoves.map(a1 => a1.every(e1 => player1.includes(e1)));
if (mapped1[i]) {
alert("player1 wins");
document.getElementById("player1").innerHTML=1;
}
i+=1;
}
}
else if (turn==2) {
turn=1;
var i=0;
x.innerHTML="<img src='o.png'/>";
turn=1;
player2.push(x.id);
while(i<=moves)
{
mapped2 = winmoves.map(a => a.every(e => player2.includes(e)));
if (mapped2[i]) {
alert("player2 wins");
document.getElementById("player2").innerHTML=1;
}
i+=1;
}
}
}
I would simply do this with a simple invention of Array.prototype.intersect() and the rest is such a straightforward single liner.
Array.prototype.intersect = function(a) {
return this.filter(e => a.includes(e));
};
var player1 = ["t1","t2","t3","t5","t7"],
winmoves = [["t1","t2","t3"],["t4","t5","t6"],["t7","t8","t9"],["t1","t4","t7"],["t2","t5","t8"],["t3","t6","t9"],["t1","t5","t9"],["t3","t5","t7"]];
filtered = winmoves.filter(a => a.intersect(player1).length == a.length);
mapped = winmoves.map(a => a.intersect(player1).length == a.length);
console.log(filtered);
console.log(mapped);
OK we have a generic Array method which finds the intersection of two arrays. (inbetween the array it's called upon and the one provided as argument) It's basically a filter checking each item of first array to see whether it is included in the second array. So we filter out the items exist in both arrays.
To obtain the filtered array we utilize Array.prototype.filter() again. This time our first array is winmoves which includes arrays that we will check for each the intersection with player1 array. If the intersection length is equal to winmove's item's length that means all elements of winmove's item is existing in the player1 array. So we return that array item to the filtered.
Specific to your case without using an intersect method you can utilize Array.prototype.every() as follows;
var player1 = ["t1","t2","t3","t5","t7"],
winmoves = [["t1","t2","t3"],["t4","t5","t6"],["t7","t8","t9"],["t1","t4","t7"],["t2","t5","t8"],["t3","t6","t9"],["t1","t5","t9"],["t3","t5","t7"]];
filtered = winmoves.filter(a => a.every(e => player1.includes(e)));
mapped = winmoves.map(a => a.every(e => player1.includes(e)));
console.log(filtered);
console.log(mapped);
What you could do is use nested for loops
for(var j = 0, j < elemInP, j++){
int flag = 0;
for(var x = 0, x < elemInWx, x++){
for(var y = 0, y < elemInWy, y++){
if(p[j] == w[x][y]){
flag = 1;
/*There is no need to run this loop once the value has been found*/
break;
}
}
if(flag){
/*If we have found the value no need to keep looking*/
break;
}
}
if(!flag){
print p[j] is not in w;
}
}
This is just a general idea of one way to compare the two arrays. The actual syntax of the code will need to be edited to work in JavaScript as this is just basic pseudocode. Flag is just a variable that holds if the value was found or not and is reset for each new value. For efficiency, you could have a break/return after setting flag = 1, but this is