Develop Reference

how to work with a large array in javascript [duplicate]

This question already has answers here:
Best way to iterate over an array without blocking the UI
(4 answers)
Closed 6 years ago.
In my application I have a very big array (arround 60k records). Using a for loop I am doing some operations on it as shown below.
var allPoints = [];
for (var i = 0, cLength = this._clusterData.length; i < cLength; i+=1) {
if (allPoints.indexOf(this._clusterData[i].attributes.PropertyAddress) == -1) {
allPoints.push(this._clusterData[i].attributes.PropertyAddress);
this._DistClusterData.push(this._clusterData[i])
}
}
When I run this loop the browser hangs as it is very big & in Firefox is shows popup saying "A script on this page may be busy, or it may have stopped responding. You can stop the script now, or you can continue to see if the script will complete". What can I do so that browser do not hang?

You need to return control back to the browser in order to keep it responsive. That means you need to use setTimeout to end your current processing and schedule it for resumption sometime later. E.g.:
function processData(i) {
var data = clusterData[i];
...
if (i < clusterData.length) {
setTimeout(processData, 0, i + 1);
}
}
processData(0);
This would be the simplest thing to do from where you currently are.
Alternatively, if it fits what you want to do, Web Workers would be a great solution, since they actually shunt the work into a separate thread.
Having said this, what you're currently doing is extremely inefficient. You push values into an array, and consequently keep checking the ever longer array over and over for the values it contains. You should be using object keys for the purpose of de-duplication instead:
var allPoints = {};
// for (...) ...
if (!allPoints[address]) { // you can even omit this entirely
allPoints[address] = true;
}
// later:
allPoints = allPoints.keys();

First of all, avoid the multiple this._clusterData[i] calls. Extract it to a variable like so:
var allPoints = [];
var current;
for (var i = 0, cLength = this._clusterData.length; i < cLength; i+=1) {
current = this._clusterData[i];
if (allPoints.indexOf(current.attributes.PropertyAddress) == -1) {
allPoints.push(current.attributes.PropertyAddress);
this._DistClusterData.push(current)
}
}
This should boost your performance quite a bit :-)

As others already pointed out, you can do this asynchronously, so the browser remains responsive.
It should be noted however that the indexOf operation you do can become very costly. It would be better if you would create a Map keyed by the PropertyAddress value. That will take care of the duplicates.
(function (clusterData, batchSize, done) {
var myMap = new Map();
var i = 0;
(function nextBatch() {
for (data of clusterData.slice(i, i+batchSize)) {
myMap.set(data.attributes.PropertyAddress, data);
}
i += batchSize;
if (i < clusterData.length) {
setTimeout(nextBatch, 0);
} else {
done(myMap);
}
})();
})(this._clusterData, 1000, function (result) {
// All done
this._DistClusterData = result;
// continue here with other stuff you want to do with it.
}.bind(this));

Try considering adding to the array asynchronously with a list, for a set of 1000 records at a time, or for what provides the best performance. This should free up your application while a set of items is added to a list.
Here is some additional information: async and await while adding elements to List<T>

How to add multiples of a number and overwrite previous entry using js timing?

I am trying to create a function that continuously adds the same number to itself. Or simply displays multiples of one number every so many seconds with the setInterval method.
For now, let's just say I want to display multiples of ten.
I know how to get a regular while loop to simply display multiples of ten in a row, but what I want to do here is continually replace the previous text every time the function is called. I am trying to create a game and this is going to be the experience calculator. So it needs to display the total experience earned over the given time.
I was trying something along the lines of this:
window.setInterval(
function writeExp ()
{
var j;
while (rounded > 0)
{
var i = w*10;
var j = j + i;
}
document.getElementByID("exp").innerHTML=j;
}, experience)
This seems logical enough to me, but obviously something is wrong, as it does not work. I have tried googling various things like how to sum numbers in javascript or continuously sum, among others, but it is somewhat difficult to word to get it more centered to my needs. So this is the best way to get my questions answered.

There are lot of undefineds in your code, but general example would be
var interval = 1000,
result = 0,
elem = document.getElementByID("exp");
window.setInterval(function () {
result *= 10;
elem.innerHTML = result;
}, interval);
or without global variables
var interval = 1000,
multiply = function (elem) {
var result = 0;
return function () {
result *= 10;
elem.innerHTML = result;
}
};
window.setInterval(multiply(document.getElementByID("exp")), interval);

Using shift() and push() to loop array values vs. using a counter variable, what is the best approach?

I'm looping through a set of images within an array of animation frames. There are 7 images, and looping from 1-7 completes the animation. I need this animation to loop indefinitely, but I was wondering which of these is the best approach:
Loop by modifying array
/* Pull image from start of array. */
var image = frames.shift();
/* Process image. */
...
/* Add image back to end of array. */
frames.push(image );
Loop using counter variable
/* Pull image by counter offset. */
var image = frames[counter];
/* Process image. */
...
/* Increment or reset counter value. */
counter + 1 === frames.length ? counter = 0 : counter = counter + 1;
Is there a reason I'd chose one over the other? Alternatively, is there a better approach to this?

Modifying the array is going to be more expensive than simply using a variable to keep track of your position in the array. The better way to do this, if you're looping indefinitely, seems to just be to use a while loop (rather than using a for loop where you reset the counter inside):
var i = 0;
while (true) {
doSomething to array[i];
i = (i+1) % array.length;
}
However if your goal really is having an animation proceed indefinitely every time a given interval elapses, a loop isn't ideal at all. Use setInterval instead.
var frames = ...; //your images
var i = 0;
function animate() {
do something to frames[i];
i = (i+1) % array.length;
}
setInterval(animate, time_between_runs);
where time_between_runs is how much time should elapse before the function is called again.

Alternatively a circular linked list also can be used I think. To turn an array of objects into a circular linked list:
frames.forEach(function(elem, index) {
elem.next = frames[index + 1] || frames[0];
});
And now you can do something like this:
setInterval(function() {
frame = frame.next;
....
}, delay);

One possibility is to ditch the array and use a linked list.
Make each frame an object that points to the next object. The last one then points to the first. Then all you need to do is reference the next object.
var curr = first; // reference to first frame object
setInterval(function() {
// process image
curr.image.doSomething();
// proceed to next
curr = curr.next;
}, 1000);
No counters to mess with this way.
Setting up the linked list is usually pretty simple, and can likely be done with just a little modification to the current code that's setting up the Array.
var first = new Frame(); // This is your entry point
var current = first; // This holds the current frame during setup
for (var i = 0; i < totalFrames; i++) {
current.next = new Frame(); // Make the current reference a new frame
current = current.next; // Make the new frame current
}
current.next = first; // circular reference back to the first.
function Frame() {
// set up this frame
}
Then first is your starting point.
Or the linking could be done within the Frame constructor.
var first = new Frame(null);
var current = first;
for (var i = 0; i < totalFrames; i++) {
current = new Frame(current);
}
current.next = first;
function Frame(currFrame) {
// link this frame to the given one
if (currFrame)
currFrame.next = this;
// set up the rest of this frame
}

Displaying a Random div on Click

I am using the Colorbox Lightbox script to call a hidden div on a page. It works great but there is a catch. I have 15 hidden divs. When a link is clicked I would like a new lightbox to show each time without repeating until all have been shown. I do not know how to do this.
Here is my code to call the lightbox:
$(".colorbox").colorbox({
inline:true
});
Here is the HTML of the hidden divs
<div class="hide">
<div id="lightbox1">
<!-- Content -->
</div>
<div id="lightbox2">
<!-- Content -->
</div>
<!-- etc -->
</div>
How would I call each div at random until all have been shown then start over?
Also is there a way that once divs 1 - 15 have been shown to then show one last div (id="last-div") before restarting?
Note: All divs would be called on a click and I am using jQuery 1.8.2.
I do not know where to start, I have seen scripts using Math.random() but I do not understand enough to make that work.
UPDATE
I have tried Ian's answer but the lightbox is not showing (but I can see in the console log that the script is working)
Originally he has this in his script:
$(selector).show();
which I changed to this:
$(selector).colorbox({inline:true });
What do I need to do to call the lightbox?
Note: No errors are thrown.

So my idea was similar to Eric's, but I wanted to make it work "completely". So instead of storing references to all the divs in an array, I just decided to store an array of ints representing each div. The way I eventually select them with jQuery is "#lightbox + i", so if you don't have this exact structure (where the divs have an id like "lightbox" and an int - from 1 to the last count), then you can use .eq() or nth-child. It won't be the exact same results, but it will have the same random effect, just done in a different way. I found a function that "randomizes" an array - I'm guessing like what Eric's Shuffle does. But here's where I got it from - How to randomize (shuffle) a JavaScript array? . I had to modify it to return a new array instead of modify the one passed to the function. Also, I kept everything in the document.ready scope, instead of the global scope, so things are passed/returned a lot. It worked fine before when I had all and randomed declared globally and didn't pass them around, I just thought this would be "better" since they weren't global.
Here's the fiddle:
http://jsfiddle.net/6qYCL/1/
And here's the Javascript:
$(document).ready(function () {
var all,
randomed;
all = generateAll();
randomed = generateRandomed(all);
$("#generator").on("click", function (evt) {
evt.preventDefault();
randomed = doNext(all, randomed);
});
});
function generateAll() {
// Generates the array of "all" divs to work on
var a = [];
var divs = $(".hide > div.lightbox");
for (var i = 1; i <= divs.length; i++) {
a.push(i);
}
console.log("List of divs available to toggle: " + a);
return a;
}
function generateRandomed(all) {
// Randomizes the original array
randomed = fisherYates(all);
console.log("Setting randomized array: " + randomed);
return randomed;
}
function doNext(all, randomed) {
$(".lightbox, #last-div").hide();
if (randomed.length < 1) {
console.log("All lightboxes toggled, showing last, then starting over");
$("#last-div").show();
randomed = generateRandomed(all);
} else {
var next = randomed.shift();
var selector = "#lightbox" + next;
console.log("Showing " + selector);
$(selector).show();
console.log("What's left: " + randomed);
}
return randomed;
}
// Randomizes an array and returns the new one (doesn't modify original)
function fisherYates ( myArray ) {
var return_arr = myArray.slice(0);
var i = return_arr.length;
if ( i == 0 ) return false;
while ( --i ) {
var j = Math.floor( Math.random() * ( i + 1 ) );
var tempi = return_arr[i];
var tempj = return_arr[j];
return_arr[i] = tempj;
return_arr[j] = tempi;
}
return return_arr;
}
It accounts for getting to the end of the list and display #new-div like you mentioned, then starting the process over. If you look in your browser's console, you can "watch" what's happening during initialization and when clicking the link.
I think this is close to what you were looking for. I'm not sure which is a better solution - storing references to the elements or just an array of ints to loop through and eventually find. I know there are many variations on how to do this - when/how to store the counting stuff, when/how to randomize the array or retrieve a random value (and how to keep track of which has been used), where to store all references, and plenty more. I hope this at least helps!

Create an array of all of them, then shuffle that array, then pull the next one each time you get a click. When you run out, you can repopulate the array if necessary.
Something like this (using the Shuffle method from this source):
Fiddle
var array = $(".hide div").toArray(); // Add all divs within the hide div to an array
var randomArray = Shuffle(array); // Shuffle the array
$("a").click(function() {
if (randomArray.length > 0)
alert(randomArray.shift().innerHTML); // Show contents of div, as an example
else
alert("None left!");
return false;
});

The solution below works by passing an array of elements to a function. As each div is displayed it is taken out of the array. Then from the divs left in the array the next one is picked at random.
// call this function on page load
function begin( ) {
var arr = $(".hide div").toArray();
// further elements can be added to arr i.e. last-div
showDivs( arr.length, arr );
}
// begin displaying divs
function showDivs( numberOfDivs, divArray ) {
var i, lastDiv;
function nextDiv( ) {
// depending on number of slides left get random number
i = randomInt( numberOfDivs );
if( lastDiv ) { $(lastDiv).hide() };
$( divArray[ i ] ).fadeIn( 3000 );
// now that this div has been displayed
// remove from array and cache
lastDiv = divArray.splice( i, 1 );
numberOfDivs--;
// no more divs to display
if( numberOfDivs == 0 ) { return };
setTimeout( nextDiv, 4000);
}
setTimeout( nextDiv, 1000);
}
// calculate next random index
function randomInt( divsLeft ) {
var i = Math.random() * divsLeft;
return Math.round( i );
}
Fiddle here

turn array into function, making a javascript loader with jquery

how can we make a function from arrays like
$.loader({
js: [
['1.js','3.js','2.js'],
['4.js'],
['5.js']
]
});
into something that does
$.when(
$.getScript("1.js"),
$.getScript('3.js'),
$.getScript('2.js'))
.then(
function(){
$.getScript('4.js').then(function(){
$.getScript('5.js');
})
});
heres what currently im working on
loader: function(arg){
var js = arg;
var phase = 0;
for (var i = 0; i < o.length; i++) {
console.log(o[i]);
$.each(o[i], function(key,src) {
//append javascript
});
};
}
the problem is i dont know where to start.
heres what i have in mind
i divide them into phases.
set phase 0, it will run the fist array but how can i do it ? $.getScript(src) for each src dosent stack. or chain. maybe put it one var sting ? like string + = $.getScript(src). so it will be like $.getScript("1.js").getScript("3.js").getScript("2.js") until there is arrays of string which is 3 in this example. then do something like append $.when( infront of the string then at the back of the string we put .then(function(){ until it finish then somehow closed the string.
ok i dont know ( and here goes the post at stackoverflow )

function lScripts(startAt){
for (var i = startAt; i < js.length; i++) {
if (js[i].constructor == Array){
for (var j = 0; j < js[i].length; j++) {
if(j==js[i].length){
$.getScript('js[i][j]',function(){
lScripts(startAt+1);
});
}else{
$.getScript('js[i][j]');
}
}
}else{
$.getScript('js[i]',function(){
lScripts(startAt+1);
});
}
}
}
lScripts(0);

Just to be clear, are you talking about writing a solution yourself that loads up a bunch of scripts? If so, dont do it yourself. Stand on someone else's shoulders.
RequireJS already does what you're asking.
http://requirejs.org/
It even has documentation for use with jQuery.
http://requirejs.org/docs/jquery.html
Someone else already did the hard work and wrote and tested a good product. Save yourself time and stress.

I hope this is what you mean: http://jsfiddle.net/pimvdb/hdN3Q/1/.
(function($) {
function load(files, index) {
var arr = [];
// fill arr with getScript requests (fake here)
for(var i = 0; i < files[index].length; i++) {
arr.push(console.log(files[index][i]));
}
// call $.when with arr as array of arguments (using apply)
$.when.apply(null, arr).then(function() {
if(index < files.length - 1) {
// if we have to move to the next chunk,
// load that next one
setTimeout(function() {
// timeout is to show they get processed in chunks
load(files, index + 1);
}, 1000);
}
});
}
$.loader = function(obj) {
load(obj.js, 0);
}
})(jQuery);
$.loader({
js: [
['1.js','3.js','2.js'],
['4.js'],
['5.js']
]
});
.apply essentially does the same thing as calling a function. However, because the amount of arguments to $.when differs depending on the input, you can't just call $.when(...) because you don't have a fixed number of arguments. The way to call a function with a variable amount of arguments is using .apply. It works like this:
someFunction(1, 2, 3);
is equal to:
someFunction.apply(null, [1, 2, 3]);
(The null refers to the execution context which is out of scope here.) The great thing about this is that you can build an array of any size. So you can call the function with any variable amount of arguments this way.
In the load function, arr gets filled with getScript functions, and it works the same way:
var arr = [getScript('file1'), getScript('file2')];
$.when.apply(null, arr);
is equal to:
$.when(getScript('file1'), getScript('file2'));
In load, we fill arr with the files of a chunk, e.g. in your case the first chunk is 1.js, 3.js and 2.js. This is done using a for loop but the array you will end up with you can just pass to .apply.
When all files are loaded, .then is called. The function we pass there should load the next chunk, but only if there is a next chunk. If there are 3 chunks, it should go to the next one when index is 0 or 1. When chunk 2 has finished, there is no next chunk so it should not continue to the next one as there isn't a next one.
So we need this if clause:
if(index < files.length - 1) {
Say files.length is 3. Then this if conditional will only pass when index is 0 or 1, which is what we want.