Develop Reference

Waiting for Recursive Function To Complete

I have a recursive Javascript function which gets the links from one Wikipedia page, follows them, and then gets all of those links (repeating a specified number of times).
It calls itself an unknown number of times to construct an object of a known depth. When it completes, I want to output the object. Currently the object immediately outputs, and is empty, meaning the function obviously isn't waiting for all the recursive calls to complete.
As you can see, I have attempted to use callbacks, but I assume incorrectly. What am I doing wrong, how should I be doing it? I'm going to presume there's a few other things wrong I haven't spotted too; I'm relatively new to Javascript.
$(document).ready(function ()
{
pageLinks[START_PAGE] = {};
//Get initial pages
links = getLinks(START_PAGE, 0, printLinks));
});
function printLinks()
{
console.log(links);
}
function getLinks(currentPage, level, callback)
{
visitedPages.push(currentPage)
var pageLinks = {}
var data = $.getJSON(URL_BEGIN + currentPage + URL_END, function(data)
{
var pages = data.query.pages;
for(var page in pages)
{
pageContentObj = pages[page].revisions[0];
for(var key in pageContentObj) if(pageContentObj[key].length > 100)
{
var pageContent = pageContentObj[key];
//Get links
hyperlinks = getFromBetween.get(pageContent,"[[","]]");
for(var link in hyperlinks)
{
link = hyperlinks[link].split("|")[0]; //Remove friendly name
link = link.replaceAll(" ", "%20");
//Add to pagelist object
prefix = link.split(":")[0];
if(prefix != "Category" && prefix != "File" && prefix != "wikipedia")
if(level < ITERATIONS && !visitedPages.includes(arguments, link))
{
console.log(level + ": " + link)
pageLinks[link] = getLinks(link, level+1, callback); //===Recursive call===
}
}
}
}
});
if(level == 0 && callback) callback();
return pageLinks;
}
Any help is appreciated, thanks in advance.
**EDIT: ** Link: https://github.com/JakeStanger/Wikipedia-Mapper/blob/master/init.js#L53

The recursive call needs to be like this:
var counter = 0;
//the big for loop
counter++;
getLinks(link, level + 1, function(res) {
for (var key in res) { //with an array it would be concat...
pageLinks[key] = res[key];
}
counter--;
if (counter == 0 && callback) callback(pageLinks); //callback if all callbacks called
});
Also remove this weird code:
if(level == 0 && callback) callback();
No you can do:
getLinks(START_PAGE, 0, console.log);

Javascript array shows in console, but i cant access any properties in loops

I really try my damndest not to ask, but i have to at this point before I tear my hair out.
By the time the js interpreter gets to this particular method, I can print it to the console no problem, it is an array of "event" objects. From FireBug I can see it, but when I try to set a loop to do anything with this array its as if it doesn't exist. I am absolutely baffled......
A few things:
I am a newbie, I have tried a for(var index in list) loop, to no avail, I have also tried a regular old for(var i = 0; i < listIn.length; i++), and I also tried to get the size of the local variable by setting var size = listIn.length.
As soon as I try to loop through it I get nothing, but I can access all the objects inside it from the FireBug console no problem. Please help, even just giving me a little hint on where I should be looking would be great.
As for the array itself, I have no problems with getting an array back from PHP in the form of: [{"Event_Id":"9", "Title":"none"}, etc etc ]
Here is my code from my main launcher JavaScript file. I will also post a sample of the JSON data that is returned. I fear that I may be overextending myself by creating a massive object in the first place called content, which is meant to hold properties such as DOM strings, settings, and common methods, but so far everything else is working.
The init() function is called when the body onload is called on the corresponding html page, and during the call to setAllEvents and setEventNavigation I am lost.
And just to add, I am trying to learn JavaScript fundamentals before I ever touch jQuery.
Thanks
var dom, S, M, currentArray, buttonArray, typesArray, topicsArray;
content = {
domElements: {},
settings: {
allContent: {},
urlList: {
allURL: "../PHP/getEventsListView.php",
typesURL: "../PHP/getTypes.php",
topicsURL: "../PHP/getTopics.php"
},
eventObjArray: [],
buttonObjArray: [],
eventTypesArray: [],
eventTopicsArray: []
},
methods: {
allCallBack: function (j) {
S.allContent = JSON.parse(j);
var list = S.allContent;
for (var index in list) {
var event = new Event(list[index]);
S.eventObjArray.push(event);
}
},
topicsCallBack: function(j) {
S.eventTopicsArray = j;
var list = JSON.parse(S.eventTopicsArray);
topicsArray = list;
M.populateTopicsDropDown(list);
},
typesCallBack: function(j) {
S.eventTypesArray = j;
var list = JSON.parse(S.eventTypesArray);
typesArray = list;
M.populateTypesDropDown(list);
},
ajax: function (url, callback) {
getAjax(url, callback);
},
testList: function (listIn) {
// test method
},
setAllEvents: function (listIn) {
// HERE IS THE PROBLEM WITH THIS ARRAY
console.log("shall we?");
for(var index in listIn) {
console.log(listIn[index]);
}
},
getAllEvents: function () {
return currentArray;
},
setAllButtons: function (listIn) {
buttonArray = listIn;
},
getAllButtons: function () {
return buttonArray;
},
setEventNavigation: function(current) {
// SAME ISSUE AS ABOVE
var l = current.length;
//console.log("length " + l);
var counter = 0;
var endIndex = l - 1;
if (current.length < 4) {
switch (l) {
case 2:
var first = current[0];
var second = current[1];
first.setNextEvent(second);
second.setPreviousEvent(first);
break;
case 3:
var first = current[0];
var second = current[1];
var third = current[2];
first.setNextEvent(second);
second.setPreviousEvent(first);
second.setNextEvent(third);
third.setPreviousEvent(second);
break;
default:
break;
}
} else {
// do something
}
},
populateTopicsDropDown: function(listTopics) {
//console.log("inside topics drop");
//console.log(listTopics);
var topicsDropDown = document.getElementById("eventTopicListBox");
for(var index in listTopics) {
var op = document.createElement("option");
op.setAttribute("id", "dd" + index);
op.innerHTML = listTopics[index].Main_Topic;
topicsDropDown.appendChild(op);
}
},
populateTypesDropDown: function(listTypes) {
//console.log("inside types drodown");
//console.log(listTypes);
var typesDropDown = document.getElementById("eventTypeListBox");
for(var index2 in listTypes) {
var op2 = document.createElement("option");
op2.setAttribute("id", "dd2" + index2);
op2.innerHTML = listTypes[index2].Main_Type;
typesDropDown.appendChild(op2);
}
}
},
init: function() {
dom = this.domElements;
S = this.settings;
M = this.methods;
currentArray = S.eventObjArray;
buttonArray = S.buttonObjArray;
topicsArray = S.eventTopicsArray;
typesArray = S.eventTypesArray;
M.ajax(S.urlList.allURL, M.allCallBack);
//var tempList = currentArray;
//console.log("temp array length: " + tempList.length);
M.setAllEvents(currentArray);
M.testList(currentArray);
M.setEventNavigation(currentArray);
//M.setEventNavigation();
M.ajax(S.urlList.topicsURL, M.topicsCallBack);
M.ajax(S.urlList.typesURL, M.typesCallBack);
}
};

The problem you have is that currentArray gets its value asynchronously, which means you are calling setAllEvents too soon. At that moment the allCallBack function has not yet been executed. That happens only after the current running code has completed (until call stack becomes emtpy), and the ajax request triggers the callback.
So you should call setAllEvents and any other code that depends on currentArray only when the Ajax call has completed.
NB: The reason that it works in the console is that by the time you request the value from the console, the ajax call has already returned the response.
Without having looked at the rest of your code, and any other problems that it might have, this solves the issue you have:
init: function() {
dom = this.domElements;
S = this.settings;
M = this.methods;
currentArray = S.eventObjArray;
buttonArray = S.buttonObjArray;
topicsArray = S.eventTopicsArray;
typesArray = S.eventTypesArray;
M.ajax(S.urlList.allURL, function (j) {
// Note that all the rest of the code is moved in this call back
// function, so that it only executes when the Ajax response is
// available:
M.allCallBack(j);
//var tempList = currentArray;
//console.log("temp array length: " + tempList.length);
M.setAllEvents(currentArray);
M.testList(currentArray);
M.setEventNavigation(currentArray);
//M.setEventNavigation();
// Note that you will need to take care with the following asynchronous
// calls as well: their effect is only available when the Ajax
// callback is triggered:
M.ajax(S.urlList.topicsURL, M.topicsCallBack); //
M.ajax(S.urlList.typesURL, M.typesCallBack);
});
}

Can't add new name and value to JSON object dynamically with condition using JavaScript

I can't add new name and value ff. this given condition:
$.each(names, function (i, name) {
$.get('https://www.example.com/path/' + name, function (data) {
var arrNow = CSVToArray(data, ',');
allArr.push(arrNow);
counter++;
if (counter === names.length) {
for (var j = 0; j < allArr.length; j++) {
for (var k = 1; k < allArr[j].length; k++) {
//console.log(allArr[j][k][0] + ': ' + allArr[j][k][1]);
//var f = moment(allArr[j][k][0]).format('lll');
var f = allArr[j][k][0];
json.push({
"datetime": f
});
if (j == 0) {
if (json[k].datetime === allArr[0][k][0]) {
var newAtt = "water_actual";
var newValue = allArr[0][k][1];
json[k][newAtt] = newValue;
}
}
if (j == 1) {
if (json[k].datetime === allArr[1][k][0]) {
var newAtt = "rainfall_actual";
var newValue = allArr[1][k][1];
json[k][newAtt] = newValue;
}
}if (j == 2) {
if (json[k].datetime == allArr[2][k][0]) {
var newAtt = "forecast_water";
var newValue = allArr[2][k][1];
json[k][newAtt] = newValue;
}
}
}
}
};
});
});
I was able to add a new namewater_actual and its value using if statement. If the datetime from the json object matches to the array value(date and time), I'd like to add it with its specific name as stated above. But I can't seem to make it work.
Here's the fiddle.

If I may provide some general feedback: it's probably good practice to simplify your code to the minimum example that reproduces your problem. Not only can that drastically increase your chances of fixing it yourself, it also increases the odds that you'll get help here.
With that in mind, consider the basic structure of what you're trying here:
var someNames = ["foo", "bar"];
var allTheData = [{
"aardvark": true
}];
$.each(someNames, function (i, name) {
$.get('http://example.com/api/' + name, function (data) {
data.aNewProperty = 'wombat';
allTheData.push(data);
});
});
console.log(allTheData);
Here, $.each iterates through everything in someNames and then proceeds immediately to the console.log statement. For all we know, each individual API call ($.get) could take seconds, or minutes. By this time we've already tried to use the contents of allTheData, which may or may not have been modified.
To avoid this sort of thing in legacy JavaScript we can make use of the callback already provided by $.get:
$.get('http://example.com/api/' + name, function (data) {
data.aNewProperty = 'wombat';
console.log(data);
});
Inside the callback, we know for sure that the API request has already completed (although the above assumes that it succeeded, which is a whole other kettle of fish). This would output the result of each API request as the responses arrive, though not necessarily in the order you'd expect!
JavaScript's asynchronous nature tended to lead in the past to a whole lot of callbacks. With the advent of ES6 we have some more options available to us, especially promises.

Node.js http.get in loop only getting last item

I'm looping over a set of URLs trying to get their HTML, but it only works for the last one in the list. The "NOT WRITING YET" console.log fires for everything in the urls array, as expected, but the console.logs after that only fire for the last one. They are all valid URLs and "res.on('error'....." returns nothing.
Any help is greatly appreciated!
for (var z in urls) {
var getURL = urls[z];
var copyURL = 'copies/'+getURL;
if (copyURL.indexOf('/') >= 0) {
var copyURLArr = copyURL.split('/');
} else {
var copyURLArr = [copyURL];
}
var copyFile = copyURLArr.pop();
var runningDirs = '';
for (var i in copyURLArr) {
if (runningDirs.length > 0) runningDirs += '/';
runningDirs += copyURLArr[i];
if (!require('fs').existsSync(runningDirs)) {
exec('mkdir '+runningDirs);
}
}
console.log('NOT WRITING YET: '+urlPrefix+getURL);
require('http').get(urlPrefix+getURL, function(res) {
console.log(urlPrefix+getURL);
res.on('data', function(data) {
console.log(copyURL);
require('fs').createWriteStream(copyURL, {flags:'a+'}).write(data);
});
});
}

In the line console.log(urlPrefix+getURL); the value will always be for the last url because when the callback is called the values of these variables will be from the last time they got assigned which is the last iteration in the loop.
instead make the request in a separate function like
function doRequest(url, copyURL) {
require('http').get(url, function(res) {
console.log(url);
res.on('data', function(data) {
console.log(copyURL);
require('fs').createWriteStream(copyURL, {flags:'a+'}).write(data);
});
});
}
and then call this function with doRequest(urlPrefix+getURL, copyURL) then check if the problem still exists

IE8 long running script error when using DataTables

I have an application that uses the DataTables jQuery library to render content in my target browser IE8. The problem is when I push a big array to be rendered, IE8 sometimes throws up the infamous long running script error.
After profiling the app it seems that the call to __fnAddData in the following code is causing the problem:
if (bUsePassedData) {
for (var i = 0, len = oInit.aaData.length; i < len; i++) {
_fnAddData(oSettings, oInit.aaData[i]);
}
} else if (oSettings.bDeferLoading ||
(oSettings.sAjaxSource === null && oSettings.ajax === null)) {
_fnAddTr(oSettings, $(oSettings.nTBody).children('tr'));
}
I was looking around for solutions and saw Nicholas Zakas' write up here and tons of other solutions that would work if the for loop wasn't inside of an if else if "block". When I tried, on my 1st attempt of many, to wrap it in a setTimeout function it of course didn't work because the 2nd part of the if else if resolves to true.
(oSettings.sAjaxSource === null && oSettings.ajax === null) // true
What is a good solution for this? Thanks in advance.

I think you might split up your function in 3 functions:
Before the if statement.
Processing the oInit.aaData
After the if statement
Here is the code split up in 3 functions:
function beforeIf(){
if (bUsePassedData) {
procesData(oSettings,oInit.aaData.concat());
} else if (oSettings.bDeferLoading ||
(oSettings.sAjaxSource === null && oSettings.ajax === null)) {
_fnAddTr(oSettings, $(oSettings.nTBody).children('tr'));
}
afterIF();
}
function processData(oSettings,arr){
//process in chuncks of 50;
// setTimeout takes a long time in IE
// it'll noticibly slow donw your script when
// only processing one item at the time
var tmp=arr.splice(0,50);
for (var i = 0, len = tmp.length; i < len; i++) {
_fnAddData(oSettings, tmp[i]);
}
if(arr.length!==0){
setTimeout(function(){
processData(oSettings,arr);
},0);
return;
}
afterIf();
}
function afterIf(){
//continue processing
}

Thanks #HMR. You helped to bring me closer to my goal. To solve the problem I worked my code down to this IIFE:
(function processData(oSettings, arr) {
var tmp = arr.splice(0, 50);
tickApp.$orders.dataTable().fnAddData(tmp);
if (arr.length !== 0) {
setTimeout(function () {
processData(oSettings, arr);
}, 0);
}
}(oSettings, oInit.aaData.concat()));
Instead of using the private _fnAddData function I opted for the DataTables public fnAddData (http://datatables.net/ref#fnAddData) function. This way I am able to push 50 rows at a time into the table which is stored in the tickApp.$orders object which I just a reference to my jQuery object that stores the table in memory:
tickApp.$orders = $('#orders');
In another part of my code. They way you had it it was still pushing 1 row at a time instead of the whole 50.
Thanks again.

If you are using ajax to fetch your data, you can override "fnServerData" in your datatables config object. This will allow you to fetch the data to be loaded and then process it however you want.
In my case, I have a generic datatables config object that I use for all my datatables. I override the default fnServerData function with one that passes rows to the datatable in sets of 200 using fnAddData and setTimeout to call the function again until all the data has been processed, finally I call fnDraw to draw the table.
var DEFAULT_CHUNK_SIZE = 200;
function feedDataToDataTableInChunks(startIndex, data, oSettings) {
var chunk = data.slice(startIndex, DEFAULT_CHUNK_SIZE);
oSettings.oInstance.fnAddData(chunk, false);
if((startIndex += DEFAULT_CHUNK_SIZE) < data.length) {
setTimeout(function () {
feedDataToDataTableInChunks(startIndex, data, oSettings);
});
} else {
oSettings.oApi._fnInitComplete(oSettings, data);
oSettings.oInstance.fnDraw();
}
}
var config = {fnServerData: function(){
oSettings.jqXHR = $.getJSON(sSource, aoData)
.done(function (result) {
feedDataToDataTableInChunks(0, result || [], oSettings);
});
}}
I am using datatables version 1.9.4