Node.js loops and JSON building - javascript

Respected ppl ....
This is my node.js code ...
https://gist.github.com/SkyKOG/99d47dbe5a2cec97426b
Im trying to parse the data of our exam results ...example ...
http://www.vtualerts.com/results/get_res.php?usn=1MV09IS002&sem=7
Im getting the results ...and i am traversing back for previous seems too ...
All works but traversing back happens in random ... prolly something wrong with the loops ...
json.results = [];
var output = '';
var k = response.query.results.body.div.div[0].table[1].tr.length;
for (var j = 1; j < k; j++) {
for (var i = 0; i <= 5; i++) {
var result_obj = {};
result_obj.subjects = [];
for (key in response.query.results.body.div.div[0].table[1].tr[j].td[i]) {
if (typeof response.query.results.body.div.div[0].table[1].tr[j].td[i].em === "undefined") {
continue;
}
var subject_obj = {};
output += "Subject : " + response.query.results.body.div.div[0].table[1].tr[j].td[i].em + " " + "\n";
var subtext = response.query.results.body.div.div[0].table[1].tr[j].td[i].em + " " + "\n";
subject_obj.subjectname = subtext.replace(/[(].*[)]/, "").trim();
result_obj.subjects.push(subject_obj);
console.log(subject_obj);
break;
}
console.log(result_obj.subjects);
I presume there is something like async concepts which need to implemented correctly to make the reordering of sems in correct order ...
And to get the JSON in this format ...
https://gist.github.com/SkyKOG/3845d6a94cea3b744296
I dont think im pushing the created objects at the right scope ...
Kindly help in this regard .... thank you ...

(I'll answer the ordering part. Suggest making the JSON issue a separate question to fit in with the Q&A format.)
When you make the HTTP request in your code (see the line below) you're bringing a varying delay into the order that responses are executed
new YQL.exec(queryname, function (response) {
You need to track the order of the requests yourself, or use a library to do it for you.
Code it yourself
In order to get around that you need something that keeps track of the original order of the requests. Because of the way closures work you can't just increment a simple counter because it'll be changed in the global scope as your loop progresses. The idiomatic way to solve this is by passing the counter into an immediately executed function (as a value type)
e.g.
var responseData = [];
for ( var i = 0; i < 100; i++ ){
(function(){
...
// http call goes in here somewhere
responseData[i] = data_from_this_response
...
})(i)
}
Use a library
Check out the async.parallel() call in Caolan's excellent library. You pass it an array of functions and it'll return to your callback with an array of the results.
https://github.com/caolan/async/#parallel
You'll need to create a loop that populates the array with curried versions of your function containing the appropriated variables.

Related

Code is getting stuck somewhere in a succession of for-loops and I'm not sure why

EDIT - I changed the code to correctly declare variables below but nothing seems to have changed
I've written code using a for-loop that has to satisfy a number of criteria before executing what's within it. The problem is that, somewhere along the way, the code is getting stuck inside one of the loops, causing the computer to crash.
I've tried breaking the loop but this doesn't seem to help.
function compareKeypoints(varifiedKeypoints) {
outer_loop: for (i = 0; i < varifiedKeypoints.length; i++) {
let initialKeypoint = varifiedKeypoints[i];
for (j = 0; j < varifiedKeypoints.length; j++) {
let comparisonKeypoint = varifiedKeypoints[j];
if (initialKeypoint.part != comparisonKeypoint.part) {
if (Math.abs(comparisonKeypoint.position.x - initialKeypoint.position.x) <= 20
&& Math.abs(comparisonKeypoint.position.y - initialKeypoint.position.y) <= 20) {
if (keypointsCompatible(initialKeypoint.part, comparisonKeypoint.part)) {
console.log("Activating part: " + initialKeypoint.part);
console.log("Activated part: " + comparisonKeypoint.part);
let keypointPair = {
point_1: initialKeypoint.part,
point_2: comparisonKeypoint.part
}
console.log("Pushing parts!");
activeParts.push(keypointPair);
console.log("breaking loop!");
break outer_loop;
console.log("Loop NOT broken!!");
}
}
}
}
}
if (activeParts.length > 0) {
console.log(activeParts);
}
}
function keypointsCompatible(keypoint_1, keypoint_2) {
var outcome = true;
if (activeParts.length > 0) {
compatibility_loop: for (i = 0; i < activeParts.length; i++) {
if (Object.values(activeParts[i]).includes(keypoint_1) && Object.values(activeParts[i]).includes(keypoint_2)) {
console.log(keypoint_1 + " and " + keypoint_2 + " are not compatible because they already exist as " + activeParts[i].point_1 + " and " + activeParts[i].point_2 + " respectively");
outcome = false;
break compatibility_loop;
console.log("Compatibility NOT broken!!");
}
}
}
console.log("Compatibility outcome is " + outcome);
return outcome;
}
The code is suppose to take two values in the same array and compare them. If a number of conditions are met, including if they're a certain distance apart from one another, they will be pushed into a secondary array. If the values already appear in the secondary array, which the keypointCompatible function is suppose to determine, the loop should either continue looking for other candidates or stop before being called again. For some reason, however, the code is getting stuck within the keypointCompatible function when it detects that the values have already appeared in the secondary array and the console will repeatedly print "Compatibility is false" until the browser crashes.
Working Solution
Use let or const instead of var or nothing. Your issue may be related to closures and variables reused between loops. Make sure you use let or const in your loops too. for (let i=0).
When you use let or const, the runtime will create a new instance every time the block or loop iterates. However, using var will reuse the internal allocation.
So what happens with the standard var is the multiple closures or loops each use the same instance of the variable.
Unless you want the var behavior, always use let or const.
Another Solution
Put a newline after the label compatibility_loop
Still Another Solution
The first function is pushing into activeParts. The second function is looping activeParts. This can go on forever, or longer than expected. Pushing into the array could possibly make the loop limit never reached.
Put a log on the length of activeParts in the second function to see if it is growing out of control.
Your code should be OK if varifiedKeypoints.length has reasonable value. And all internal variables are declared properly!
You have two loops (this inner can start at j=i+1 to save time and multiple calculations) with few conditions inside.
function compareKeypoints(varifiedKeypoints) {
outer_loop: for (let i = 0; i < varifiedKeypoints.length; i++) {
let initialKeypoint = varifiedKeypoints[i];
for (let j = i+1; j < varifiedKeypoints.length; j++) {
let comparisonKeypoint = varifiedKeypoints[j];

how to work with a large array in javascript [duplicate]

This question already has answers here:
Best way to iterate over an array without blocking the UI
(4 answers)
Closed 6 years ago.
In my application I have a very big array (arround 60k records). Using a for loop I am doing some operations on it as shown below.
var allPoints = [];
for (var i = 0, cLength = this._clusterData.length; i < cLength; i+=1) {
if (allPoints.indexOf(this._clusterData[i].attributes.PropertyAddress) == -1) {
allPoints.push(this._clusterData[i].attributes.PropertyAddress);
this._DistClusterData.push(this._clusterData[i])
}
}
When I run this loop the browser hangs as it is very big & in Firefox is shows popup saying "A script on this page may be busy, or it may have stopped responding. You can stop the script now, or you can continue to see if the script will complete". What can I do so that browser do not hang?
You need to return control back to the browser in order to keep it responsive. That means you need to use setTimeout to end your current processing and schedule it for resumption sometime later. E.g.:
function processData(i) {
var data = clusterData[i];
...
if (i < clusterData.length) {
setTimeout(processData, 0, i + 1);
}
}
processData(0);
This would be the simplest thing to do from where you currently are.
Alternatively, if it fits what you want to do, Web Workers would be a great solution, since they actually shunt the work into a separate thread.
Having said this, what you're currently doing is extremely inefficient. You push values into an array, and consequently keep checking the ever longer array over and over for the values it contains. You should be using object keys for the purpose of de-duplication instead:
var allPoints = {};
// for (...) ...
if (!allPoints[address]) { // you can even omit this entirely
allPoints[address] = true;
}
// later:
allPoints = allPoints.keys();
First of all, avoid the multiple this._clusterData[i] calls. Extract it to a variable like so:
var allPoints = [];
var current;
for (var i = 0, cLength = this._clusterData.length; i < cLength; i+=1) {
current = this._clusterData[i];
if (allPoints.indexOf(current.attributes.PropertyAddress) == -1) {
allPoints.push(current.attributes.PropertyAddress);
this._DistClusterData.push(current)
}
}
This should boost your performance quite a bit :-)
As others already pointed out, you can do this asynchronously, so the browser remains responsive.
It should be noted however that the indexOf operation you do can become very costly. It would be better if you would create a Map keyed by the PropertyAddress value. That will take care of the duplicates.
(function (clusterData, batchSize, done) {
var myMap = new Map();
var i = 0;
(function nextBatch() {
for (data of clusterData.slice(i, i+batchSize)) {
myMap.set(data.attributes.PropertyAddress, data);
}
i += batchSize;
if (i < clusterData.length) {
setTimeout(nextBatch, 0);
} else {
done(myMap);
}
})();
})(this._clusterData, 1000, function (result) {
// All done
this._DistClusterData = result;
// continue here with other stuff you want to do with it.
}.bind(this));
Try considering adding to the array asynchronously with a list, for a set of 1000 records at a time, or for what provides the best performance. This should free up your application while a set of items is added to a list.
Here is some additional information: async and await while adding elements to List<T>

Speed up javascript run time

Although I have reviewed several of previous posts, my rookie capabilities are blind to a solution for speeding up the execution of the following code. There are hundreds of k, and for each k there are (tens)thousands of i and nearSum() has a loop that evaluates testStr.
This code is slow and timing out Chrome – how do I improve the execution?
Before you ask, the only reason for any of the code is ‘because it is working’. The values for nn are global variables.
Function()…
resArrLen = resultArray[k].length;
for (i=0;i<resArrLen;i++)
{
testStr = resultArray[k][i].toString();
resultArray[k][i] = testStr + "'" + nearSum(testStr);
}//end for i
…
function nearSum(seqFrag)
{
var i=0;
var ninj=0;
var seqFragLen=0;
var calcVal=0;
var nn="";
//sum values
seqFragLen = seqFrag.length;
for (i=0; i+1<seqFragLen; i++)
{
nn = seqFrag.substr(i,2); //gets value
ninj = eval(nn);
calcVal = calcVal.valueOf() + ninj.valueOf();
} //end for i
return calcVal.toFixed(2);
} //end nearSum
For one thing, it appears that you are using 'eval' to convert a string to a number. That's not how that's intended to be used. Use 'Number(nn)' or 'parseInt(nn)' instead.
Otherwise, the code is incomplete and with no example data it's difficult to optimize.

Custom for-loop helper for EmberJS/HandlebarsJS

A small two hours ago I started: Nested HandlebarsJS #each helpers with EmberJS not working
Shortly after I figured an acceptable temporary solution myself, question is still unaswered. My problems didn't stop there though.
I am now trying to make a custom helper which will loop through an array of objects, but exclude the first index - pretty much: for(i = 1; i < length; i++) {}. I've read on websites you have to get the length of your context and pass it to options - considering your function looks like: forLoop(context, options).
However, context is a string rather than an actual object. When you do a .length, you will get the length of the string, rather than the size of the array. When I pass that to options, nothing happens - not too mention browser freezes.
I then first tried to do a getPath before passing it to options, this returns an empty string.
What am I supposed to do instead, I made the for-loop code before for just HandlebarsJS and that worked, but EmberJS doesn't seem to take it, why?
EDIT: I pretty much also followed: http://handlebarsjs.com/block_helpers.html -> Simple Iterators
I solved this myself after trying for a long time.
The HandlebarsJS method (as described on the site) is no longer valid for EmberJS, it's now as follows:
function forLoop(context, options) {
var object = Ember.getPath(options.contexts[0], context);
var startIndex = options.hash.start || 0;
for(i = startIndex; i < object.length; i++) {
options(object[i]);
}
}
Heck, you could even extend the for-loop to include an index-value!
function forLoop(context, options) {
var object = Ember.getPath(options.contexts[0], context);
var startIndex = options.hash.start || 0;
for(i = startIndex; i < object.length; i++) {
object[i].index = i;
options(object[i]);
}
}
This is a working for-loop with variable start index. You use it in your templates like so:
{{#for anArray start=1}}
<p>Item #{{unbound index}}</p>
{{/for}}
Here is how I did it (and it works !!!)
First,
i had in my model a 'preview' property/function, that just return the arrayController in an array :
objectToLoop = Ember.Object.extend({
...
arrayController: [],
preview: function() {
return this.get('arrayController').toArray();
}.property('arrayController.#each'),
...
});
Then, I add a new Handlebars helper :
Handlebars.registerHelper("for", function forLoop(arrayToLoop, options) {
var data = Ember.Handlebars.get(this, arrayToLoop, options.fn);
if (data.length == 0) {
return 'Chargement...';
}
filtered = data.slice(options.hash.start || 0, options.hash.end || data.length);
var ret = "";
for(var i=0; i< filtered.length; i++) {
ret = ret + options.fn(filtered[i]);
}
return ret;
});
And thanks to all this magic, I can then call it in my view :
<script type="text/x-handlebars">
<ul>
{{#bind objectToLoop.preview}}
{{#for this end=4}}
<li>{{{someProperty}}}</li>
{{/for}}
{{/bind}}
</ul>
</script>
And that's it.
I know it is not optimal, so whoever have an idea on how to improve it, PLEASE, make me know :)

JavaScript converting an array to array of functions

Hello I'm working on a problem that requires me to change an set array of numbers into an array that returns the original numbers as a function. So we get a return of a2 instead of a[2].
I dont want the answer I just need a hint. I know i can loop through the array and use .pop() to get the last value of the array, but then I dont know how to convert it to a function from there. any hints?
var numToFun = [1, 2, 3];
var numToFunLength = numToFun.length;
for (var i = 0; i < numToFunLength; i++) {
(function(num){
numToFun.unshift(function() {
return num;
});
}(numToFun.pop()))
}
DEMO
basically it pops out a number from the last, builds a function with that number returned, and put back into the first of the array. after one full cycle, all of them are functions.
here's the catch: how this works, it's up to you to research
why the loop does not look like the straightforward pop-unshift:
for (var i = 0; i < numToFunLength; i++) {
numToFun.unshift(function() { //put into first a function
return numToFun.pop() //that returns a number
});
}
and why i did this: (HINT: performance)
var numToFunLength = numToFun.length;
There's three important steps here:
Extract the number value from the array. Within a loop with an iterator of i, it might look like this:
var num = numArray[i];
This is important, because i will not retain its value that it had when you created the new function - it'll end up with the last value it had, once the for loop is finished. The function itself might look like this:
function() { return num; }
There's no reference to i any more, which is important - to understand better, read about closures. The final step would be to add the new function to the array of functions that you want.
...and you're done!
EDIT: See other's answers for good explanations of how to do this right, I will fix mine also though
As others have pointed out, one of the tricky things in javascript that many struggle with (myself included, obviously) is that scoping variables in javascript is dissimilar to many other languages; scopes are almost purely defined by functions, not the {} blocks of, for example, a for loop, as java/C would be.
So, below you can see (and in other answers here) a scoping function can aid with such a problem.
var numArray = [12, 33, 55];
var funcArray = [];
var numArrLength = numArray.length; // Don't do this in for loop to avoid the check multiple times
for(var j=0; j < numArrLength; j++) {
var scopeMe = function() {
var numToReturn = numArray[j];
console.log('now loading... ' + numToReturn);
var newFunc = function() {
return numToReturn;
};
return newFunc;
}();
funcArray.push(scopeMe);
};
console.log('now me');
console.log(funcArray);
console.log(funcArray[0]());
console.log(funcArray[1]());
console.log(funcArray[2]());
console.log(funcArray[1]()); // To ensure it's repeatable
EDIT my old bad answer below
What you'll want to do is something like
var funcArray = [];
for(...) {
var newFunc = function() {
return numArray.pop();
}
funcArray.push(newFunc);
}
The key here is that functions in javascript can be named variables, and passed around as such :)

Categories

Resources