Develop Reference

Default Javascript objects are very slow when large?

I am doing a modified version of collecting word co-occurrences, so I wrote my own javascript, and I am tracking the occurrences in three objects. However, once the objects get large (~8 million, 3 million, and 172000) a function that took 5 seconds per 100000 sentences now takes minutes to do one sentence with 30 words (30 tokens). I am nowhere near my RAM cap (I have 12 more GBs of RAM it could be using, and the program is only using 2.2GB). Using Node.js v17.3.1.
Why does my function take so long when the objects get bigger (even though the sentences remain the same length)? Should I be using a different object besides Javascript's default object, or is there a way improve the speed of access and setting these objects when they are so big?
Code:
let posCounts = {};
let negCounts = {};
// the number of times each word occurs
let wordCounts = {};
let tokens = // some function that gets tokens;
for (let k = 0; k < tokens.length; k++) {
// count word occurences
if (tokens[k] in wordCounts) {
wordCounts[tokens[k]] += 1;
} else {
wordCounts[tokens[k]] = 1;
}
for(let tok = k + 1; tok < tokens.length; tok++) {
if (tok == k) {
// avoid word to self cooccurrence
// should no longer be possible
continue;
} else {
// check which form of the cooccurence exists already in either count
actual_tok = (tokens[k] + "-" + tokens[tok]);
if(actual_tok in posCounts || actual_tok in negCounts) {
// no-op
} else {
actual_tok = (tokens[tok] + "-" + tokens[k]);
}
// condition set before this block of code
if(condition) {
if (actual_tok in posCounts) {
posCounts[actual_tok] += 1;
} else {
posCounts[actual_tok] = 1;
}
} else {
if (actual_tok in negCounts) {
negCounts[actual_tok] += 1;
} else {
negCounts[actual_tok] = 1;
}
}
}
}
}
Update: I've tried increasing the heap size via node train_matrices.js --max-old-space-size=12288 and node train_matrices.js --max_old_space_size=12288 (underline instead of dash), and that didn't work either.

Probably not the main issue in your code, but you can reduce the number of lookups by changing this structure from this:
if (tokens[k] in wordCounts) {
wordCounts[tokens[k]] += 1;
} else {
wordCounts[tokens[k]] = 1;
}
to this:
let token = tokens[k];
let cnt = wordCounts[token] || 0;
wordCounts[token] = cnt + 1;
And, as I said in a comment, I've read that a Map object with .get() and .set() is better suited when there are lots of dynamically created keys whereas plain objects are better suited when you have lots of objects with all the same keys (as the JS compiler can sometimes make a C-like struct for it), but this can't be done when you're regularly adding new keys.

The answer was to both use the increase memory flag node <YOUR_FILE_NAME>.js --max-old-space-size=12288 and change to using a Map instead of an object - thanks to #jfriend00 and #Norman Breau for the suggestions. That said, maps have a max capacity of 2^24 items or 1 GB, so I ended up using a modified version of the BigMap from this stackoverflow (modified to limit the total number of items still - ended up running completely out of RAM).
Modified code (you can replace BigMap with Map if you want):
let posCounts = new BigMap();
let negCounts = new BigMap();
let wordCounts = new BigMap();
let actual_tok;
tokens = // some code
// mark every cooccurrence
for (let k = 0; k < tokens.length; k++) {
// count word occurences
if (wordCounts.has(tokens[k])) {
wordCounts.set(tokens[k], wordCounts.get(tokens[k]) + 1);
} else {
wordCounts.set(tokens[k], 1);
}
for(let tok = k + 1; tok < tokens.length; tok++) {
if (tok == k) {
// avoid word to self cooccurrence
// should no longer be possible
continue;
} else {
// check which form of the cooccurence exists already in either count
actual_tok = (tokens[k] + "-" + tokens[tok]);
if(posCounts.has(actual_tok) || negCounts.has(actual_tok)) {
// no-op
} else {
actual_tok = (tokens[tok] + "-" + tokens[k]);
}
if(condition) {
if (posCounts.has(actual_tok)) {
posCounts.set(actual_tok, posCounts.get(actual_tok) + 1);
} else {
posCounts.set(actual_tok, 1);
}
} else {
if (negCounts.has(actual_tok)) {
negCounts.set(actual_tok, negCounts.get(actual_tok) + 1);
} else {
negCounts.set(actual_tok, 1);
}
}
}
}
}
}

Can't create an opportune return for my function

I have to create a function that checks if a word is palindrome or not. My reasoning is to break down the word letter by letter, normal and reverse, and then compare the two results to determine if a word is a palindrome or not. With an if - else I give the user an alert that tells perfectly the result. Now, I've learned that most of the functions must have a return with a variable that contains that desired result.
In this case I really can't have this, I think it can work perfectly this way. I tried with
var palindromeResult = (leftToRightWord == rightToLeftWord) ? 'true':'false';
console.log(palindromeResult);
return palindromeResult;
but it works only for the developer if he reads the console.log in the console, but it's a bad solution for me... Have you got any better idea than this? Below my full function code
function isPalindrome(wordToCheck) {
for(var i = 0; i < wordToCheck.length; i++) {
var leftToRightWord = wordToCheck[i];
console.log('Left ' + leftToRightWord);
}
for(var j = wordToCheck.length - 1; j >= 0; j--) {
var rightToLeftWord = wordToCheck[j];
console.log('Right ' + rightToLeftWord);
}
if ( leftToRightWord === rightToLeftWord) {
alert('La parola è palindroma');
} else {
alert('La parola non è palindroma');
}
Edit: At the end I changed a lot my code for a better legibility.
//Data
var userWord;
//I ask a word to the user
do {
userWord = prompt("Dimmi una parola");
} while (userWord.length === 0)
//Here the result of the function is saved and it can be reused
var functionResult = isPalindrome(userWord);
console.log (functionResult);
//Function to know if the word inserted is a palindrome or not
//the cycle with rightToLeft reverse the word so it can be compared to the normal word
//A pop-up will give the solution so it can be seen clearly on your screen
//The function result will ben saved outside the function in var functionResult so it can be seen with a console.log or reused for whatever use
function isPalindrome(wordToCheck) {
var rightToLeftWord = '';
for(var j = wordToCheck.length - 1; j >= 0; j--) {
rightToLeftWord = rightToLeftWord + wordToCheck[j];
}
console.log(rightToLeftWord);
var palindromeResult = wordToCheck == rightToLeftWord;
alert(palindromeResult);
return palindromeResult;
}
Now the cycle with for doesn't have problems anymore as you've pointed out to me and it correctly recognize if a word is a palindrome or not.

See the snippet. The function returns the answer and then you can alert it, assign it to a variable, echo it on the page and so on. I'm not pretty sure about your algorithm since it is telling me that 'abracadabra' is palindrome but it is not. RTL the sequence of the letters is wrong!
function isPalindrome(wordToCheck) {
for (var i = 0; i < wordToCheck.length; i++) {
var leftToRightWord = wordToCheck[i];
console.log('Left ' + leftToRightWord);
}
for (var j = wordToCheck.length - 1; j >= 0; j--) {
var rightToLeftWord = wordToCheck[j];
console.log('Right ' + rightToLeftWord);
}
if (leftToRightWord === rightToLeftWord) {
return 'La parola è palindroma';
} else {
return 'La parola non è palindroma';
}
}
alert(isPalindrome('abracadabra'));

How to fix this "if" statement

I don't really know the correct format to this if statement. I want it to count the frequency each word in my txt file was used.
function countWords(array, word, index) {
var count = 0;
var value = " "
for (var i = 0; i < array.length; i++) {
if (array[i] == 0 && value == word)
count++;
}
}
if (getUserSelectionForm.problem.value == "pay") {
countWords(working2DArray, "pay", 0)
if (getUserSelectionForm.problem.value == "staffing") {
countWords(working2DArray, "staffing", 0)
if (getUserSelectionForm.problem.value == "hours") {
countWords(working2DArray, "hours", 0)
if (getUserSelectionForm.problem.value == "management") {
countWords(working2DArray, "management", 0)
console.log(countWords)
document.getElementById('section2').innerHTML = "The selected word appears " + countWords + " times in the array."
}

Try not to use multiple IF statements and use a switch statement instead. Makes code much clearer and cleaner.
E.g.
switch(expression) {
case x:
// code block
break;
case y:
// code block
break;
default:
// code block
}
See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/switch
So:
var p = getUserSelectionForm.problem.value;
switch (p) {
case 'pay':
countWords(working2DArray, "pay", 0);
break;
case 'staffing':
countWords(working2DArray, "staffing", 0);
}

You are making three mistakes in code:
You are missing some of the closing curly braces } of you if blocks.
You do not return anything from function. You should return count.
countWords is a function you don't need to display that. You need to display its result.
You can make your code much simpler. You don't need these if statements at all because you are passing the same value of getUserSelectionForm.problem.value to the function so directly pass it.
function countWords(array, word, index) {
var count = 0;
var value= " "
for(var i = 0; i < array.length; i++){
if(array[i] == 0 && value == word)
count++;
}
return count;
}
let word = getUserSelectionForm.problem.value
document.getElementById('section2').innerHTML = `The selected word appears ${countWords(working2DArray,word,0)} times in array`
If you want to only check for few items then use create a array of "pay","staffing"... and use includes() on it
let word = getUserSelectionForm.problem.value
if(["pay","staffing","hours","management"].includes(word)){
document.getElementById('section2').innerHTML = `The selected word appears ${countWords(working2DArray,word,0)} times in array`
}

In my understanding , you want to trigger the function whenever problem.value ==='pay'||'staffing'||'hours'||'management' ,here is clearer version for your reference:
var problemValue = getUserSelectionForm.problem.value;
var isProblemValueMatch = ["pay", "staffing" ,"hours", "management"].includes(problemValue);
if (isProblemValueMatch ) {
var countWords = working2DArray.filter(function(v) {
return v === problemValue;
}).length;
console.log(countWords)
document.getElementById('section2').innerHTML = "The selected word appears " + countWords + " times in the array."
}

Count how many times a character happens in a string and store it in an array JavaScript

I've been trying to figure out how to count how many times a character happens in a string and store it in another variable that will hold the character and the number of times it occurs in the string.
For example:
var greeting = "Hello World";
[H] occurs [1] time.
[e] occurs [1] time.
[l] occurs [3] times.
[o] occurs [2] times.
[W] occurs [1] time.
[r] occurs [1] time.
[d] occurs [1] time.
I am a JS Beginner and I tried as much as I can following guides and tutorials but this exercise seems to be out of my league. I would appreciate some help as to how would you guys go on about solving this problem.
Thanks!

You basically want to create a mapped set of characters to it's count in the string. Storing this stuff in an array might be weird as you'd need 2 Dimentional arrays. Instead store it in an hash object.
var greeting = "Hello world!";
var hash = {};
for(var i = 0; i < greeting.length; i++){
if(hash[greeting[i]] === undefined){
hash[greeting[i]] = 1;
} else {
hash[greeting[i]] += 1;
}
}
// printing the stuff in hash.
for(var x in hash){
if(hash.hasOwnProperty(x)){
console.log(x, hash[x]);
}
}
Anyway if you need this stuff in array, you can put so:
var arr = [];
var i = 0;
for(var x in hash){
if(hash.hasOwnProperty(x)){
arr[i++] = [x, hash[x]];
}
}
for(var i = 0; i< arr.length; i++){
console.log(arr[i]);
}
But I wouldn't recommend it. You can see redundancy for yourself.

Try this:
var result = {};
Array.prototype.map.call('Hello world!', function(x) {
if (typeof result[x] == 'undefined') {
result[x] = 1;
} else {
result[x] += 1;
}
});
console.log(result);
var result = {};
Array.prototype.map.call('Hello world!', function(x) {
if (typeof result[x] == 'undefined') {
result[x] = 1;
} else {
result[x] += 1;
}
});
console.log(result);

Javascript var name counting variable

I have the following bit of code:
function finalCheck(theForm) {
var z = 0;
for(var i=0;i<15;i++){
var _i = theForm.elements[i].value;
if(_i == ""){
theForm.elements[i].style.background = '#FFD6D6';
z = 1;
}
}
if(z == 1){
alert("Please correct the fields highlighted in red");
return false;
} else {
return true;
}
}
What I was attempting to do was set the name of var _i to var _ and then the index that the counting variable was currently on. For example, _1 _2 _3 etc. Any way to do this?

So I under stand what you are trying to do, and your solution is to use eval
eval("var _" + i + "= theForm.elements[i].value;");
Anyways your code doesn't need any dynamic generation of variable names as var _i is a local variable.
I hope you are asking this question for learning purpose.