Develop Reference

How to handle callbacks in JavaScript within recursive functions?

Trying to compare two sub trees of bookmarks in Chrome, I ran into troubles with the asynchronous API call to query the children of a bookmarks folder.
function titleComparator (lhs, rhs) {
return lhs.title < rhs.title ? -1 : lhs.title > rhs.title ? 1 : 0;
}
// Return whether two bookmark trees have equal content
function compare(lhs, rhs) {
// Not equal if one is a bookmark and another is a folder
if (('url' in lhs) != ('url' in rhs))
return false;
// If both are bookmarks, compare url and title
if ('url' in lhs && 'url' in rhs)
return lhs.title == rhs.title && lhs.url == rhs.url;
// If both are folders, compare contents
chrome.bookmarks.getChildren(lhs.id, function (lhsChildren) {
chrome.bookmarks.getChildren(rhs.id, function (rhsChildren) {
if (lhsChildren.length != rhsChildren.length)
return false; // Want to return from compare()
lhsChildren.sort(titleComparator);
rhsChildren.sort(titleComparator);
for (var i = 0; i < lhsChildren.length; i++)
if (!compare(lhsChildren[i], rhsChildren[i])
return false; // Same here
return true; // Same here
});
});
}
How to handle callbacks in JavaScript within recursive functions?

as explained in detail here
you will need to refactor your code.
somehow it seems that it is not the correct way to use recursion within a asynchronous (often delayed) function to search a tree-based or hierarchical data model.
I think this should be the way to do it:
Seperatate the logic into several functions
Use "lazy loading" to avoid duplicate call of getChilden()
Use recursion and define a new nested function callback
Refactor the for-loop to recursion as well
See my untested code to show what I mean:
function compare(lhs, rhs, callback, index, lhsChilds, rhsChilds){
// Not equal if one is a bookmark and another is a folder
if (('url' in lhs) != ('url' in rhs)) {
callback(false);
return;
}
// If both are bookmarks, compare url and title
if ('url' in lhs && 'url' in rhs) {
callback(lhs.title == rhs.title && lhs.url == rhs.url);
return;
}
// If both are folders, check parameters and compare contents
//First, check if the list has already been loaded (code is running inside a recursive step)
if(lhsChilds != undefined && rhsChilds != undefined){
compareTwoChilds(lhs, rhs, callback, index, lhsChilds, rhsChilds);
}
else{
index = 0; //first recursion for this tuple (lhs, rhs)
chrome.bookmarks.getChildren(lhs.id, function (lhsChildren) {
chrome.bookmarks.getChildren(rhs.id, function (rhsChildren) {
compareTwoChilds(lhs, rhs, callback, index, lhsChilds, rhsChilds);
});
});
}
}
function compareTwoChilds(lhs, rhs, callback, index, lhsChilds, rhsChilds){
if (index < lhsChildren.length){ //just for the safety
if (lhsChildren.length != rhsChildren.length) {
callback(false);
return;
}
lhsChildren.sort(titleComparator);
rhsChildren.sort(titleComparator);
//compare using recursion, with an emtpy lists of rhs and lhs children
compare(lhsChildren[index], rhsChildren[index], function(compareResult){
if(!compareResult){
callback(false); //if the result is false, no more search needed
}else{ // use recursion again to loop through the next childs using the already loaded childs
if (++index < lhsChildren.length){
compare(lhsChildren[index], rhsChildren[index], callback, index, lhsChilds, rhsChilds)
}else{
callback(true); // the loop has ended,
}
}
});
}else{
callback(false); //this should never happen, so not the same...
}
}
you can call the compare function like that:
compare(lhs,rhs, function(result){
var compareResult = result;
//carry on with your code here
});
//and not here :-)

return will only ever exit the callee
You're gonna have to provide a callback for the answer to be spat out into asynchronously.
Everywhere you've written a return statement intended to be consumed by the callback closure, you should standardize instead on passing it to your callback.
function compareAsync(lhs, rhs, callback) {
//…
callback(false); return;
//…
callback(lhs.title == rhs.title && lhs.url == rhs.url); return;
//…
callback(false); return; // Want to return from compare()
//…
var finished = 0;
var whetherSuccess = true;
lhsChildren.forEach(function(iterand, index) {
compareAsync(iterand, rhsChildren[index], function(result) {
whetherSuccess &= result;
if (++finished === lhsChildren.length) {
callback(whetherSuccess);
}
});
});
}
So: upon finding out what the lhsChildren are, we kick off a bunch of async functions. They'll each increment the finished counter at some point. They each have the power to demote the overall whetherSuccess to false via &=. The consumer who discovers they're the the final function to get an answer, is the person who will report whetherSuccess to the original callback.

First of all, I found that Chrome has a getSubTree() function as well which makes things considerably easier. So if you just want to get it work, use this instead of asynchronously traversing the tree node by node. However, this remains an interesting problem and thanks to a reddit user, I figured out a working solution to this.
compare() is the main function that recursively calls itself. However, because of the asynchronous call inside, it cannot consume the return values of it's recursive calls.
// Pass a boolean to the callback indicating whether the recursive contents of
// both bookmarks folders are equal.
function compare(lhs, rhs, callback) {
// Compare titles except for the top-level folder
if (lhs.parent_ && lhs.title !== rhs.title) {
compare_failure(callback);
return;
}
// Compare urls if at least one of the sides is a bookmark
if ('url' in lhs || 'url' in rhs) {
if ((lhs.url || null) === (rhs.url || null))
compare_respond(lhs.parent_, callback);
else
compare_failure(callback);
return;
}
// For both sides being folders, we have to take a look at the contents
chrome.bookmarks.getChildren(lhs.id, function (lhs_children) {
chrome.bookmarks.getChildren(rhs.id, function (rhs_children) {
// Compare amount of children
if (lhs_children.length != rhs_children.length) {
compare_failure(callback);
return;
}
// Keep track of how many children already reported back
lhs.all_children = lhs_children.length;
lhs.equal_children = 0;
// Let pairs of children compare each other
lhs_children.sort(bookmark_comparator);
rhs_children.sort(bookmark_comparator);
for (var i = 0; i < lhs_children.length; i++) {
var lhs_child = lhs_children[i];
var rhs_child = rhs_children[i];
// Store parent reference so the deeper function can
// asynchronously respond with the results once finished.
lhs_child.parent_ = lhs;
compare(lhs_child, rhs_child, callback);
}
});
});
};
compare_respond() is the counterpart that is used to propagate results of deeper nodes back up. It's used instead of return in the main function above.
// Report comparison results back to the parent node. The parent node waits
// until it collected the results from all its children. Then it reports to
// its parent in turn. At the root node, the user callback is executed.
function compare_respond(node, callback) {
// Collect child results
node.equal_children++;
// Respond upwards if we got results from all
if (node.equal_children == node.all_children) {
if ('parent_' in node)
compare_respond(node.parent_, callback);
else
callback(true);
}
};
compare_failure() is used to abort the whole thing at any point when we found a pair of unequal nodes. We don't have to report upwards in that case.
// Break out of the recursive function and report failure to the user. It's
// safe against being called multiple times so multiple children can report
// failure and the user will only be notified once.
function compare_failure(callback) {
if ('called' in callback)
return;
callback.called = true;
callback(false);
};
bookmark_comparator() is a small helper that is used to sort arrays of child bookmarks. Sorting is needed to compare the contents of two folders since I don't want to rely on the item order.
// Comparator to sort lists of bookmark nodes first by title and second by
// url. Take into that folders have to url.
function bookmark_comparator(lhs, rhs) {
if (lhs.title != rhs.title)
return lhs.title < rhs.title ? -1 : 1;
if (lhs.url || null != rhs.url || null)
return lhs.url || null < rhs.url || null ? -1 : 1;
return 0;
};

How to filter an array?

Guys, please don't answer me to use a JavaScript library to solve this problem, I'm using VanillaJS.
Suppose I have an array with 10,000 string records, same as following:
var arr = [
'John',
'Foo',
'Boo',
...
'Some',
'Beer'
];
Please note that the array doesn't follow any sort.
Now, I want to find items with oo in the text, what is the best way to do? Should I populate a new array or just pop items that don't match with the criteria?

You can make use of the filter method, which will create a new array with all the elements that passes the condition.
arr.filter(function(x){ return x.indexOf ('oo') > -1});
If you want to use filter method in every browser you could add the polyfill method (see link) in your code.
Another option (slightly faster) with basic javascript would be:
Looping trough the array with a simple for loop and test on your condition.
var filtered = [];
for(var i=0, length=arr.length; i<length; i++){
var current = arr[i];
if(current.indexOf('oo') > -1){
filtered.push(current);
}
}

my approch
forEach function
function forEach(array, action) {
for(var i=0; i<array.length; i++)
action(array[i]);
}
partial function
function asArray(quasiArray, start) {
var result = [];
for(var i = (start || 0); i < quasiArray.length; i++)
result.push(quasiArray[i]);
return result;
}
function partial(func) {
var fixedArgs = asArray(arguments, 1);
return function() {
return func.apply(null, fixedArgs.concat(asArray(arguments)));
};
}
contains method for String obj
if (!String.prototype.contains) {
String.prototype.contains = function (arg) {
return !!~this.indexOf(arg);
};
}
filter function:
function filter(test, array) {
var result = [];
forEach(array, function(element) {
if (test(element))
result.push(element);
});
return result;
}
test function for test array items
function test(key, el) {
return el.contains(key);
}
finally
filter(partial(test, 'oo'), arr);

NO shortcuts ( p.s. you said I want to find items , not filter - hence my answer)
simple loop :
var g=arr.length; //important since you have big array
for( var i=0;i<g;i++)
{
if ( g[i].indexOf('oo')>-1)
{
console.log(g[i]);
}
}
If you want to filter (without polyfill)
var g=arr.length; //important since you have big array
var h=[];
for( var i=0;i<g;i++)
{
if ( g[i].indexOf('oo')>-1)
{
h.push(g[i]);
}
}
//do something with h

A very simple way, use forEach:
var result = [];
arr.forEach(function (value) {
if (value.indexOf('oo') !== -1) {
result.push(value);
}
});
or map:
var result = [];
arr.map(function (value) {
if (value.indexOf('oo') !== -1) {
result.push(value);
}
});

Javascript: DFS traversal of a directed tree with I/O involved

Given a directed tree T with a variable number of children per node, I would like to find a path the size of PATH_SIZE of "good" nodes starting from root.
every node has an isGood() method and a getChildren() method that work as expected.
A simple DFS recursive solutions would look like this: (please correct me if I'm wrong)
function findGoodPath(node, depth){
if(!node.isGood()){
return null;
} else if (depth==PATH_SIZE){
return [node];
}
var children = node.getChildren();
for (var i=0; i<children.length; i++){
var result = findGoodPath(children[i], depth+1);
if (result){
return result.concat([node]);
}
}
return null;
}
Calling findGoodPath(root, 1) should find a result if one exists.
Now for the problem: thegetChildren() method of the node object is actually an async method that does I/O behind the scenes. it returns nothing and expects a single callback argument to handle returned children.
A modified code solution (which is WRONG) can look like this:
function findGoodPath(node, depth){
if(!node.isGood()){
return null;
} else if (depth==PATH_SIZE){
return [node];
}
node.getChildren(function(children){
for (var i=0; i<children.length; i++){
var result = findGoodPath(children[i], depth+1);
if (result){
return result.concat([node]);
}
}
});
}
This solution won't work: all the getChildren methods of a single node's children will be called at once, so it will actually perform a BFS. and worse, the return statements are associated with the anonymous callback function and will execute after the enclosing function has finished running.
It's clear that there is a need for some sort of a flow control mechanism. What is a simple and elegant solution for this problem?
UPDATE: I've accepted Sebastien's answer since it solves this problem with a recursion, which is how I presented the question. I've also posted an answer which uses the async's library whilst loop, this is what I ended up using. Sebastien was kind enough to benchmark these two methods here. (spoiler: performance is identical)

first, I think you have to call findGoodPath(children[i], depth + 1) if you want the depth equals the PATH_SIZE.
then, you do have a problem of closure. With your async call you always concat with a node instance wich is not the one you want.
One way you could do that could be :
node.getChildren((function(_node) {
return function(children){
for (var i=0; i<children.length; i++){
var result = findGoodPath(children[i], depth);
if (result){
return result.concat([_node]);
}
}
});
})(node));
But I think it's just a part of the problem as you're mixing sync function with async function.
The line:
var result = findGoodPath(children[i], depth)
is written as a sync call whereas findGoodPath is an async function, so it has to be written with callbacks too!
Hope it helps
ps: it would help to have a jsfiddle...
UPDATE : just a try. As I cannot test, it's not working, but it's the idea. I can't figure out if you need to create another scope in the second findGoodPath call, just as in the getChildren call
function findGoodPath(node, depth, callback){
if(!node.isGood()){
return callback(null);
} else if (depth==PATH_SIZE){
return callback([node]);
}
node.getChildren((function(_node, _callback) {
return function(children){
var node = _node, callback = _callback;
for (var i=0; i<children.length; i++){
findGoodPath(children[i], depth, function(result) {
if (result){
return callback(result.concat([node]));
}
});
}
});
})(node, callback));
}

I'm not 100% in focus now, but I am almost sure Async.js seriestasks is the right solution for you (If not seriestasks I'm willing to bet there is another control flow in Async.js that will do the trick.

OK, so there are several ways to achieve an async DFS traversal. Since async recursions have a tendency to become somewhat ugly, I've decided to get rid of the recursion.
I first re-implemented the synchronous version of the function using a while loop instead of a recursion:
function findGoodPathLoop(root){
var nodesToVisit = [{data: root, path:[]}];
while (nodesToVisit.length>0){
var currentNode = nodesToVisit.pop();
if (currentNode.data.isGood()){
var newPath = currentNode.path.concat(currentNode.data);
if (newPath.length==PATH_SIZE){
return newPath;
} else {
var childrenNodes = currentNode.data.getChildren().map(function(child){
return {data: child, path: newPath};
});
nodesToVisit = nodesToVisit.concat(childrenNodes);
}
}
}
return null;
}
Note: I saved the entire path for each node, this is not a necessity, you can just save the depth and maintain an array of the current path, though it's a bit messier.
I then used the async library to convert this function to an async one, replacing the standard while() function with async's whilst():
function findGoodPathAsync(root, pathCallback){
var result = null;
var nodesToVisit = [{data: root, path:[]}];
async.whilst(
function(){
return nodesToVisit.length>0 && result==null ;
},
function(next) {
var currentNode = nodesToVisit.pop();
if (currentNode.data.isGood()){
var newPath = currentNode.path.concat(currentNode);
if(newPath.length==PATH_SIZE){
result = newPath;
next();
} else {
currentNode.data.getChildren(function(children){
var childrenNodes = children.map(function(child){
return {data: child, path: newPath};
});
nodesToVisit = nodesToVisit.concat(childrenNodes);
next();
});
}
} else {
next();
}
},
function(err) {
//error first style callback
pathCallback(err, result);
}
);
}
Not a pretty one, but it's readable and it does the job.

javascript jquery function is this somehow wrong?

function rebuildJSONObject(){
$.getJSON('services.json', function(data) {
//stof start
var input = data;
var output = { myservices: [] };
for (var key in input) {
if (input.hasOwnProperty(key)) {
for (var i = 0, hostsinfo = input[key].hostsinfo; i < hostsinfo.length; i++) {
output.myservices.push({
'nametag': key,
'hostidn': hostsinfo[i]['hostidn'],
'details': hostsinfo[i]['details'],
'currstatus': hostsinfo[i]['currstatus'],
'currstatusclass': hostsinfo[i]['currstatusclass']
});
}
}
}
//stof end
return output;
});
}
//setting it for use later in the script
var serviceJSONObject = rebuildJSONObject();
I know the stuff going on in the function is working properly cause if I apply it to a click event it works charming. However I would rather load the JSON object into memory once and work with it client side there after unless saved. My Problem is however anywhere I call "serviceJSONObject" I get an "undefined" error.
So How am I doing this wrong and how would I define a variable like this early in the game so the rest of the script can use said variable.

The issue is that output is returned before the callback function is called. You should be able to save the value to serviceJSONObject by using a closure:
function rebuildJSONObject(serviceJSONObject){
$.getJSON('services.json', function(data) {
//stof start
var input = data;
// Use the serviceJSONObject that is passed into rebuildJSONObject
serviceJSONObject = { myservices: [] };
for (var key in input) {
if (input.hasOwnProperty(key)) {
for (var i = 0, hostsinfo = input[key].hostsinfo; i < hostsinfo.length; i++) {
serviceJSONObject.myservices.push({
'nametag': key,
'hostidn': hostsinfo[i]['hostidn'],
'details': hostsinfo[i]['details'],
'currstatus': hostsinfo[i]['currstatus'],
'currstatusclass': hostsinfo[i]['currstatusclass']
});
}
}
}
//stof end
});
}
//setting it for use later in the script
var serviceJSONObject;
rebuildJSONObject(serviceJSONObject);

Why not add a cache property to a function that will store the result of the initial output (loaded via ajax) and returning the saved state to any consecutive call.
function rebuildJSONObject(callback) {
var self = this;
if (typeof self.cache !== 'undefined') {
if (typeof callback === 'function') {
callback(self.cache);
}
return;
}
$.getJSON('services.json', function(data) {
//stof start
var input = data,
output = { myservices: [] };
for (var key in input) {
if (input.hasOwnProperty(key)) {
for (var i = 0, hostsinfo = input[key].hostsinfo; i < hostsinfo.length; i++) {
output.myservices.push({
'nametag': key,
'hostidn': hostsinfo[i]['hostidn'],
'details': hostsinfo[i]['details'],
'currstatus': hostsinfo[i]['currstatus'],
'currstatusclass': hostsinfo[i]['currstatusclass']
});
}
}
}
//stof end
self.cache = output;
if (typeof callback === 'function') {
callback(self.cache);
}
return;
});
}
EDIT: For the first time you will need to call this function asynchronously and supply a callback function, for example
rebuildJSONObject(function(output) {
/*
* Process your output here
*/
console.log(output);
});
Each consecutive time you can again use it synchronously:
console.log(rebuildJSONObject.cache);

There are a couple of problems with this.
The call to getJSON is asynchronous so you need to be careful you don't try to use the results before the call has returned your results.
The way it is at the moment, the results will not be returned to serviceJSONObject. The return output statement is setting the return for the anonymous function, not the return value for rebuildJSONObject, so the results will just disappear. If you want the results to be available elsewhwere in code you will either need to store them in a global variable or access them inside the callback.

How can I extend Array.prototype.push()?

I'm trying to extend the Array.push method so that using push will trigger a callback method and then perform the normal array function.
I'm not quite sure how to do this, but here's some code I've been playing with unsuccessfully.
arr = [];
arr.push = function(data){
//callback method goes here
this = Array.push(data);
return this.length;
}
arr.push('test');

Since push allows more than one element to be pushed, I use the arguments variable below to let the real push method have all arguments.
This solution only affects the arr variable:
arr.push = function () {
//Do what you want here...
return Array.prototype.push.apply(this, arguments);
}
This solution affects all arrays. I do not recommend that you do that.
Array.prototype.push = (function() {
var original = Array.prototype.push;
return function() {
//Do what you want here.
return original.apply(this, arguments);
};
})();

First you need subclass Array:
ES6 (https://kangax.github.io/compat-table/es6/):
class SortedArray extends Array {
constructor(...args) {
super(...args);
}
push() {
return super.push(arguments);
}
}
ES5 (proto is almost deprecated, but it is the only solution for now):
function SortedArray() {
var arr = [];
arr.push.apply(arr, arguments);
arr.__proto__ = SortedArray.prototype;
return arr;
}
SortedArray.prototype = Object.create(Array.prototype);
SortedArray.prototype.push = function() {
this.arr.push(arguments);
};

Array.prototype.push was introduced in JavaScript 1.2. It is really as simple as this:
Array.prototype.push = function() {
for( var i = 0, l = arguments.length; i < l; i++ ) this[this.length] = arguments[i];
return this.length;
};
You could always add something in the front of that.

You could do it this way:
arr = []
arr.push = function(data) {
alert(data); //callback
return Array.prototype.push.call(this, data);
}
If you're in a situation without call, you could also go for this solution:
arr.push = function(data) {
alert(data); //callback
//While unlikely, someone may be using "psh" to store something important
//So we save it.
var saved = this.psh;
this.psh = Array.prototype.push;
var ret = this.psh(data);
this.psh = saved;
return ret;
}
While I'm telling you how to do it, you might be better served with using a different method that performs the callback and then just calls push on the array rather than overriding push. You may end up with some unexpected side effects. For instance, push appears to be varadic (takes a variable number of arguments, like printf), and using the above would break that.
You'd need to do mess with _Arguments() and _ArgumentsLength() to properly override this function. I highly suggest against this route.
Or you could use "arguments", and that'd work too. I still advise against taking this route though.

There's another, more native method to achieve this: Proxy
const target = [];
const handler = {
set: function(array, index, value) {
// Call callback function here
// The default behavior to store the value
array[index] = value;
// Indicate success
return true;
}
};
const proxyArray = new Proxy(target, handler);

I wanted to call a function after the object has been pushed to the array, so I did the following:
myArray.push = function() {
Array.prototype.push.apply(this, arguments);
myFunction();
return myArray.length;
};
function myFunction() {
for (var i = 0; i < myArray.length; i++) {
//doSomething;
}
}