my current problem is I have two mongodb collections. My code in essence takes each name from table 1 which is 250000~ items then it compares against the list of names from table 2 (3000~) to try and match.
now I'm just brute forcing this by going through every item in the 250000 list and looking at every item in the 3000 list to see if it matches
the problem is that this takes about 2 hours to run through
Is there a faster way to do matching? I've put the stripped down version of my code below so you can see it's a simple loop in a loop.
Thanks
const SCCM = await ProgramDev.find({})
const Cat = await Software.find({})
for (var i = 0, len = SCCM.length; i < len; i++) {
for (var n = 0, Catlen = Cat.length; n < Catlen; n++) {
var program = SCCM[i]['program name']
var software = Cat[n]['Application']
var sat = Cat[n].Status
if (SCCM[i].Status == undefined) {
if (program == software) {
SCCM[i].Status = sat
break
}
}
}
if (SCCM[i].Status == undefined) {
SCCM[i].Status = "Un-authorised"
SCCM[i].matchType = "None"
}
}
If you have already extracted each name from collection1 and from collection2 into arrays this should work faster
array1 = ['john','smith','vova','putin', 'eug', 'hilly', 'tom', 'smith','vova','putin' ]
array2 = ['vova', 'dr.cacis']
const newArray = array1.filter(value => array2.includes(value));
console.log(newArray);
Output matches
[ 'vova', 'vova' ]
You can also use Array.prototype.includes
or Array.prototupe.indexOf
You can try to use map if you don't have memory restriction and put the values of Cat in the map like:
var catMap = new Map();
for (var n = 0, Catlen = Cat.length; n < Catlen; n++) {
catMap.set(Cat[n]['Application'] , Cat[n].Status);
}
And run the second loop on SCCM like:
for (var i = 0, len = SCCM.length; i < len; i++) {
if (SCCM[i].Status == undefined && catMap.has(SCCM[i]['program name'])){
SCCM[i].Status = catMap.get(SCCM[i]['program name']);
}else if(SCCM[i].Status == undefined){
SCCM[i].Status = "Un-authorised"
SCCM[i].matchType = "None"
}
}
So, the first loop will take O(n) and second loop will take O(m), total will be O(m+n) time complexity.
The space complexity will be O(n) because of map.
Overall, program will take less time to run.
Related
I currently have a problem with the performance of a JavaScript operation.
I want to clean up an object array (aEmployees) for those objects whose ID is already in the array aInvited or which are not in the aInvitationAllowed array:
var iLength = aEmployees.length - 1;
for (var i = iLength; i >= 0; --i) {
if (aInvited.indexOf(aEmployees[i].id) !== -1 || aInvitationAllowed.indexOf(aEmployees[i].id) === -1) {
aEmployees.splice(i, 1);
}
}
You can imagine that I have Employee objects (aEmployees). I also have a list of Employee IDs that have already been invited to an event (aInvited) and a list of Employee IDs that are allowed to be invited to the event (aInvitationAllowed). I want to have all Employees as a result that I can still invite to the event.
The problem is the line with the indexOf queries. I have to assume that the arrays have very many entries (~ 100,000 entries). Then it may be that taking the loop takes a while.
So my question to you: How can I make the loop faster? Are there faster alternatives for indexOf?
Thanks for your tips!
Consider using a Set to compare instead. Set.has is an O(1) operation, whereas indexOf is an O(n) operation, reducing the computational complexity overall to O(n) instead of O(n^2):
const aInvitedSet = new Set(aInvited);
const aAllowedSet = new Set(aAllowed);
const iLength = aEmployees.length - 1;
for (let i = iLength; i >= 0; --i) {
const { id } = aEmployees[i];
if (aInvitedSet.has(id) || !aAllowedSet.has(id)) {
aEmployees.splice(i, 1);
}
}
Also, splice is slow-ish. Unless you have to mutate the exising array, you might consider .pushing to a new array instead (which looks to be faster than Array.prototype.filter):
const aInvitedSet = new Set(aInvited);
const aAllowedSet = new Set(aAllowed);
const newArr = [];
const iLength = aEmployees.length;
for (let i = 0; i < iLength; i++) {
const employee = aEmployees[i];
const { id } = employee;
if (!aInvitedSet.has(id) && aAllowedSet.has(id)) {
newArr.push(employee);
}
}
I have an strng array.
e.g:
StrArray = ["hook", "hour", "javascript", "nemo", "case"];
I know that if I use StrArray.Sort() it will sorted alphabetical. But I would to like to sort by using this alphabet = "jngmclqskrzfvbwpxdht"
I've searched but I only find people using HashTable to solve it.
Is it possible to do in JS?
let order = 'jngmclqskrzfvbwpxdht'
let StrArray = ["hook", "javascript", "hour", "nemo", "case"];
StrArray.sort(function(a,b) {
let firstAlphaOfParam1 = a.charAt(0);
let firstAlphaOfParam2 = b.charAt(0);
return order.indexOf(firstAlphaOfParam1) - order.indexOf(firstAlphaOfParam2);
})
console.log(StrArray);
The solution is only taking into consideration of sorting by only the first alphabet of element in StrArray. Basically we take the first alphabet and find the index in your jngmclqskrzfvbwpxdht and compare them
Your sort vocabulary doesn't contain all the letters in you words, so it's not completely clear how to proceed with words like 'hour' and 'hook' as there's no 'o' in the sort list. You can just ignore them and treat anything that isn't in the list as equal in the sort order. You also should test against similar bases like "hook" and "hooks"
For example:
let StrArray = ["hook", "javascript", "nemo", "hours", "case", "hour", "houn"];
const sort_order = "jngmclqskrzfvbwpxdht"
StrArray.sort((a, b) => {
let i = 0;
while (i < a.length && i < b.length ){
let a_i = sort_order.indexOf(a[i]),
b_i = sort_order.indexOf(b[i]);
if (a_i === b_i ) {
i++
continue
}
return a_i - b_i
}
// one is a substring of the other, sort by length
return a.length - b.length
})
console.log(StrArray)
I can leave a small contribution, this code can sort using the first letter.
var arr1 = ["hook", "javascript", "nemo", "case"];
var myAbc = 'jngmclqskrzfvbwpxdht';
var final_array = [];
for (i = 0; i < myAbc.length; i++) {
for (j = 0; j < arr1.length; j++) {
if (arr1[j].charAt(0) == myAbc.charAt(i)) {
final_array.push(arr1[j]);
}
}
};
console.log(final_array);
I have array:
let arr = ["logerror", "log:today", "log:1"]
I am looking for function how to get longest substring from this items.
Result:
log
Another example:
let arr = ["dog+ěě+", "dog15qwqqq", "dogggggg"]
Result:
dog
Sure, I can write some algorithm, but is there any simple way?
How? Thanks
If you can phrase your question succinctly, you can often find what to search for. In this case, it looks like:
"Find the longest common substring from within an array of strings"
A quick google reveals an algorithm for finding the largest common substring between two strings:
https://en.wikibooks.org/wiki/Algorithm_Implementation/Strings/Longest_common_substring
I don't want to copy the code as written there, as unsure of the copyright, but you could take the implementation and take something that will work with your array.
I would note that for large arrays, this may turn out to be a lengthy operation...
I used a simple approach:
It sorts the array using sort() method.
Then, the most important step is to look just at the first and last items.
function commonSubsequence(array){
let sortedArray = array.sort();
let first = sortedArray[0];
let last = sortedArray.pop();
let length = first.length;
let index = 0;
while(index<length && first[index] === last[index])
index++;
return first.substring(0, index);
}
console.log(commonSubsequence(["logerror", "log:today", "log:1"]));
console.log(commonSubsequence(["dog+ěě+", "dog15qwqqq", "dogggggg"]));
Here is my suggestion
function subStrArr(arr) {
let chars = arr[0].split(""), sub = "";
for (let i=0;i<chars.length;i++) {
for (let j=1;j<arr.length;j++) {
if (arr[j].indexOf(chars[i])==-1) return sub;
}
sub+=chars[i];
}
}
let arr1 = ["logerror", "log:today", "log:1"];
let arr2 = ["dog+ěě+", "dog15qwqqq", "dogggggg"];
console.log(subStrArr(arr1))
console.log(subStrArr(arr2))
After some looking around I went for the string-algorithms npm package, which did the job nicely for me.
From the docs:
import { longestCommonSubstring } from 'string-algorithms';
const strings = [
'12apple',
'3apple4',
'apple56'
];
console.log(longestCommonSubstring(strings));
produces the output apple.
without DP approach
var lcs = function (n, m) {
let lcs = 0 //to store longest common substring
let s1 = n.length
let s2 = m.length
for(let i = 0;i < s1;i++){
for(let j = 0; j< s2;j++){
let track = 0
//if letter are same, do while to check next letter
if(n[i] == m[j]){
while(i + track < s1 && j + track < s2 && n[i + track] == m[j + track]){
track += 1 // to track
if (lcs < track) {
lcs += 1
}
}
}
}
}
return lcs;
};
var m = "abcdxyz"
var n = "xyzabcd" // 4
// var m = "dadef"
// var n = "adwce"//2
// var m = "acdghr";
// var n = "bgh"; //2
// var m = "A"
// var n = "A" //1
console.log(lcs(m, n));
Ok, this might be easy for some genius out there but I'm struggling...
This is for a project I'm working on with a slider, I want an array the slider can use for snap points/increments... I'm probably going about this in a mental way but its all good practice! Please help.
var frootVals = [1,2,3,4,5];
var frootInc = [];
for (i=0; i<=frootVals.length; i++) {
if (i == 0){
frootInc.push(frootVals[i]);
}
else{
frootInc.push(frootInc[i-1] += frootVals[i])
}
};
What I'm trying to do is create the new array so that its values are totals of the array elements in frootVals.
The result I'm looking for would be this:
fruitInc = [1,3,6,10,15]
For a different take, I like the functional approach:
var frootVals = [1,2,3,4,5];
var frootInc = [];
var acc = 0;
frootVals.forEach(function(i) {
acc = acc + i;
frootInc.push(acc);
});
var frootVals = [1,2,3,4,5]
, frootInc = [];
// while i < length, <= will give us NaN for last iteration
for ( i = 0; i < frootVals.length; i++) {
if (i == 0) {
frootInc.push(frootVals[i]);
} else {
// rather than frootIne[ i-1 ] += ,
// we will just add both integers and push the value
frootInc.push( frootInc[ i-1 ] + frootVals[ i ] )
}
};
There were a few things wrong with your code check out the commenting in my code example. Hope it helps,
This will do:
var frootVals = [1,2,3,4,5];
var frootInc = [];
for (i=0; i < frootVals.length; i++) { // inferior to the length of the array to avoid iterating 6 times
if (i == 0) {
frootInc.push(frootVals[i]);
}
else {
frootInc.push(frootInc[i-1] + frootVals[i]) // we add the value, we don't reassign values
}
};
alert(JSON.stringify(frootInc));
jsfiddle here: http://jsfiddle.net/f01yceo4/
change your code to:
var frootVals = [1,2,3,4,5];
var frootInc = [frootvals[0]]; //array with first item of 'frootVals' array
for (i=1; i<frootVals.length; i++) {
frootInc.push(frootInc[i-1] + frootVals[i]); //remove '='
}
Here's a very simple pure functional approach (no vars, side-effects, or closures needed):
[1,2,3,4,5].map(function(a){return this[0]+=a;}, [0]);
// == [1, 3, 6, 10, 15]
if you name and un-sandwich the function, you can use it over and over again, unlike a hard-coded var name, property name, or for-loop...
I have a month array in javascript, for example:
2012/09/01,2012/10/01,2012/11/01,2012/12/01,2013/01/01,2013/02/01,2013/03/01,
2012/09/01,2012/10/01,2012/11/01,2012/12/01,2013/01/01,2013/02/01,2013/03/01,2013/04/01,
2012/09/01,2012/10/01,2012/11/01,2012/12/01
What I wanna separate the Array is that:
if (monthArray[i] > monthArray[i + 1])
// slice the Array.
So, for the above example, I should get 3 new Arrays. They are:
Array1: 2012/09/01,2012/10/01,2012/11/01,2012/12/01,2013/01/01,2013/02/01,2013/03/01
Array2: 2012/09/01,2012/10/01,2012/11/01,2012/12/01,2013/01/01,2013/02/01,2013/03/01,2013/04/01
Array3:2012/09/01,2012/10/01,2012/11/01,2012/12/01
I know it is easy to do it if we know specific length, my question is, how to do it if we dynamically get a month Array(it may be divided into n groups). How to do that? Thanks!
I don't know of any better way than to iterate over the array to build your slices:
var arr = ['2012/09/01','2012/10/01','2012/11/01','2012/12/01','2013/01/01','2013/02/01','2013/03/01','2012/09/01','2012/10/01','2012/11/01','2012/12/01','2013/01/01','2013/02/01','2013/03/01','2013/04/01','2012/09/01','2012/10/01','2012/11/01','2012/12/01'];
var slices = [];
var start = 0;
for (var i=0; i<arr.length; i++) {
if (check(arr, i)) {
slices.push(arr.slice(start, i+1));
start = i+1;
}
}
function check(array, index) {
if (index+1 === array.length) return true;
return Date.parse(array[index]) > Date.parse(array[index+1]);
}
This solution has the advantage that it doesn't build the slices one element at a time, instead it builds them one slice at a time.
Assuming you want an array-of-arrays as a result, you can do this with .reduce:
var partitions = dateList.reduce(function(rv, month) {
var partition = rv[rv.length - 1], prevMonth = partition[partition.length - 1];
if (!prevMonth || prevMonth < month)
partition.push(month);
else
rv.push([month]);
return rv;
}, [ [] ]);
Starting from a list of partitions with one (empty) partition, this just checks the last month in the last partition to see if it's smaller than the current month under examination. If so (or if we're on the very first one), we add the month onto the partition. If not, then a new partition is started, containing just the current month.
So assuming you want to end up with an array of arrays, then just do it with a for loop...
var result = []; //this will contain multiple arrays once finished
var currentArray = [];
for (var i = 0; i < monthArray.length; i++) {
currentArray.push(monthArray[i]);
if (i < monthArray.length - 1 && monthArray[i] > monthArray[i + 1]) {
result.push(currentArray);
currentArray = [];
}
}
result.push(currentArray);
//result[0] is Array1
//result[1] is Array2
//result[2] is Array3
Here is a working example