I have a tree structure that I want to walk using Promises, and I haven't figured out the right code pattern for it.
Assume that we're given node names, and the act of converting a node name to a node is an asynchronous process (e.g. involves a web access). Also assume that each node contains a (possibly empty) list of children names:
function getNodeAsync(node_name) {
// returns a Promise that produces a node
}
function childrenNamesOf(node) {
// returns a list of child node names
}
What I want to end up with a method with this signature:
function walkTree(root_name, visit_fn) {
// call visit_fn(root_node), then call walkTree() on each of the
// childrenNamesOf(root_node), returning a Promise that is fulfilled
// after the root_node and all of its children have been visited.
}
that returns a Promise that's fulfilled after the root node and all of its children have been visited, so it might be called as follows:
walkTree("grandpa", function(node) { console.log("visiting " + node.name); })
.then(function(nodes) { console.log("found " + nodes.length + " nodes.")});
update
I've create a gist that shows my first attempt. My (slightly buggy) implementation for walkTree() is:
function walkTree(node_name, visit_fn) {
return getNodeAsync(node_name)
.then(function(node) {
visit_fn(node);
var child_names = childrenNamesOf(node);
var promises = child_names.map(function(child_name) {
walkTree(child_name, visit_fn);
});
return Promise.all(promises);
});
};
This visits the nodes in the correct order, but the outermost Promise resolves before all the sub-nodes have been visited. See the gist for full details.
And as #MinusFour points out, using this technique to flatten the list of nodes is rather pointless. In fact, I really just want the final promise to fire when all the nodes have been visited, so a more realistic use case is:
walkTree("grandpa", function(node) { console.log("visiting " + node.name); })
.then(function() { console.log("finished walking the tree")});
Well it isn't much of a problem to handle a function call for each node, but gathering the node values is kind of a problem. Kind of hard to walk that tree, your best bet would be to map it to a tree with no eventual values. You could use something like:
function buildTree(root_name) {
var prom = getNodeAsync(root_name);
return Promise.all([prom, prom.then(function(n){
return Promise.all(childrenNamesOf(n).map(child => buildTree(child)))
})]);
}
From there on you do:
var flatTree = buildTree(root_name).then(flatArray);
flatTree.then(nodes => nodes.forEach(visit_fn));
flatTree.then(nodes => whateverYouWantToDoWithNodes);
To flatten the array you could use:
function flatArray(nodes){
if(Array.isArray(nodes) && nodes.length){
return nodes.reduce(function(n, a){
return flatArray(n).concat(flatArray(a));
});
} else {
return Array.isArray(nodes) ? nodes : [nodes];
}
}
To be honest, it's pointless to have the tree walker if you want a list of nodes you are better up flattening it up and then iterating the elements, but you can walk the array tree if you want.
Despite what I said in the O.P, I don't really care about the return values of the final promise, but I do want to wait until all the nodes have been traversed.
The problem with the original attempt was simply a missing return statement in the map() function. (Despite appearances, this is essentially structurally identical to #MinusFour's answer.) Corrected form below:
function walkTree(node_name, visit_fn) {
return getNodeAsync(node_name)
.then(function(node) {
visit_fn(node);
var child_names = childrenNamesOf(node);
var promises = child_names.map(function(child_name) {
return walkTree(child_name, visit_fn);
});
return Promise.all(promises);
});
};
Here are two use cases for walkTree(). The first simply prints the nodes in order then announces when the tree walk is finished:
walkTree("grandpa", function(node) { console.log("visiting " + node.name); })
.then(function() { console.log("finished walking the tree")});
The second creates a flat list of nodes, made available when the tree walk completes:
var nodes = [];
walkTree("grandpa", function(node) { nodes.push(node) })
.then(function() { console.log('found', nodes.length, 'nodes);
console.log('nodes = ', nodes); });
Related
I was writing a function for a Breadth first search alogrithm. I would have come up with the following code
traverseBF() {
const results = []
const queue = [this.root]
while (queue.length) {
let node = queue.shift()
if (node.children) {
queue.push(...node.children)
}
results.push(node)
}
return results
}
However, the solution was written somewhat differently as
traverseBF(fn) {
const queue = [this.root]
while (queue.length) {
let node = queue.shift()
if (node.children) {
queue.push(...node.children)
}
fn(node)
}
}
I cannot explain what fn(node) purpose is or how it returns the correct result. Is this some kind of recursive call? How are the two solutions different?
Instead of returning results like you are, they're allowing the user to pass in a function that gets passed each node as it's traversed.
Try for example:
obj.traverseBF(node => console.log(node))
Or just
obj.traverseBF(console.log)
Should work as well. console.log is given each node to use as it's found.
This is arguably more general purpose. If the tree was large, accumulating all the results in a list may waste memory if the user doesn't need all the results at once.
If the user wanted to accumulate a list though, they could pass in a function that appends to a list that the function closes over:
nodes = [];
obj.traverseBF(node => nodes.push(node))
console.log(nodes)
My code gets a translation from Google translate, pushes the translation into an array, and then passes the translation array to the next code block. Or it doesn't, depending on the syntax. This works:
translate.translate(text, target)
.then(function(results) {
translation = results[0];
translationArray.push(translation);
return Promise.resolve(translationArray);
})
.then(function(translationArray) {
console.log(translationArray);
This code doesn't work:
translate.translate(text, target)
.then(function(results) {
translation = results[0];
return translationArray.push(translation);
})
.then(function(translationArray) {
console.log(translationArray);
translationArray logs as 1. What does 1 mean? Why do I need to explicitly use Promise.resolve?
I tried this, it doesn't help:
const finalPromise = translate.translate(text, target)
.then(function(results) {
translation = results[0];
return translationArray.push(translation);
})
.then(function(translationArray) {
console.log(translationArray);
Array#push returns the new length of the array. If your array is empty then after adding an element the new length is 1. That's where the 1 comes from.
You don't have to use Promise.resolve, you just need return translationArray;.
I'm itching head with concept of promises and async procedures. I have ordered a list and want to call a function with every item, wait until first procedure with the first item is done, proceed to second, third and so on. And only after every item is processed I want continue the main process.
Below is the code that made it well with the main process. So returning Q.all(promises) resulted that first all promises were processed and then main process continued. But problem was, that items (navigation keys) were processed async while I need them in sync:
function processPages(that) {
var navs = [];
Object.keys(that.navigation).map(function(key) {
navs.push({key: key, order: parseInt(that.navigation[key].index)});
});
var promises = navs.sort(function(a, b) {
return a.order - b.order;
})
.map(function(item) {
return that.parsePage(item.key).then(function(page) {
return page.sections.filter(function(section) {
return section.type == 'normal';
})
.map(function(section) {
collectStore(section, page, that);
});
});
});
return Q.all(promises);
}
Below is the code when I modified that items are processed in sync and right order, but now main process will be out of sync:
function processPages(that) {
var navs = [];
Object.keys(that.navigation).map(function(key) {
navs.push({key: key, order: parseInt(that.navigation[key].index)});
});
var promises = navs.sort(function(a, b) {
return a.order - b.order;
})
.reduce(function(previous, item) {
return previous.then(function () {
return that.parsePage(item.key).then(function(page) {
return page.sections.filter(function(section) {
return section.type == 'normal';
})
.map(function(section) {
collectStore(section, page, that);
});
});
});
}, Q());
return Q.all(promises);
}
Does anyone know what is happening here and how to use promises right way in this case?
Additional information
processPages is called from init hook. If promise (Q.all) is not used, then page hook may fire before init hook is totally processed, which I cannot allow either. This is what I refer with the "main process".
module.exports =
{
hooks: {
"init": function() {
var options = this.options.pluginsConfig['regexplace'] || {};
options.substitutes = options.substitutes || {};
// collects text replacement queries from plugin configuration
options.substitutes.forEach(function (option) {
patterns.push({re: new RegExp(option.pattern, option.flags || ''),
sub: option.substitute,
decode: option.decode || false,
store: option.store || null,
unreset: option.unreset || false});
});
this.config.book.options.variables = this.config.book.options.variables || {};
processPages(this);
},
"page": function(page) {
var that = this;
// process all normal sections in page
page.sections.filter(function(section) {
return section.type == 'normal';
})
.map(function(section) {
collectStore(section, page, that, true);
});
return page;
}
}
};
Code is part of the GitBook plugin code.
Take a look at the runnable examples (from Chrome's dev console) from the Understanding JavaScript Promise with Examples, especially the "chaining" example.
Based on your description of "...I have ordered a list and want to call a function with every item, wait until first procedure with the first item is done, proceed to second, third and so on. And only after every item is processed I want continue the main process.",
From algorithm point of view, you should be "chaining" multiple promises together:
create a Promise for every item. When an item is done, call resolve() so that then() will execute (next item in chain).
put the "main process" as the last item in the chain.
Recommend you test/learn promises' execution flow with simple example before applying it in your problem - makes it easier to understand.
Hope this helps! :-)
Solution was as simple as changing:
processPages(this);
to
return processPages(this);
on init hook.
Can someone explain why this doesn't work as I expect it?
I am trying to return only the id of the created or updated items to use in the Promise.each().then(). This is necessary because create returns an object and update an array of objects.
Promise.each(items, function(item){
if(...item doesn\'t exist...)
return Item.create(item)
.then(function (created) {
return created.id;
///HOW DO I GET THIS ID ONLY?
});
else {
return Item.update(item)
.then(function (updated) {
return updated[0].id;
///AND THIS ID?
});
}
}).then(function(result) {
sails.log("THIS SHOULD BE AN ID" + JSON.stringify(result));
});
result is the whole created object, instead of just the id. I want to return an array of just the updated ids. Apparently nesting promises is bad, but I don't know how i could simplify this.
Don't use .each use .map to map a list of items to a list of items:
Promise.map(items, function(item){
if(...item doesn\'t exist...)
return Item.create(item).get("id"); // get is a shortcut to .then(fn(x){ r x.id; })
else {
return Item.update(item).get(0).get(id);
}).then(function(result) {
sails.log("Here is the array of all IDs + JSON.stringify(result));
});
If you want to go over them one by one and wait for them, you can chain a second .map. If you want to set to do them sequentially (much slowe) - .map also takes a concurrency parameter.
As Esailija said - in 3.0 the behavior of each changes to return the results so your original code would have been ineffective but would have actually worked in 3.0 .
In Scala I can take a list of values, map a future-returning function across them, and get back a future that will collect the values of those futures into a list (or fail with the first error). More concretely:
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
def doSomething(i: Int): Future[Int] = future(i + 1) // Not really doing much.
val incremented = Future.traverse(List(1, 2, 3))(doSomething)
In this case the result will just be the incremented list:
scala> incremented.onSuccess { case xs => println(xs) }
List(2, 3, 4)
I could alternatively create a list of futures and then turn them into a future containing the same result:
val incremented = Future.sequence(List(1, 2, 3).map(doSomething))
This will give me the same thing, but it creates an extra intermediate collection and is a little more noisy.
I want to do something like this with promises in Q, and it looks like Q.all is more or less sequence:
function doSomething(i) { return Q.fcall(function () { return i + 1; }); }
Q.all([1, 2, 3].map(doSomething)).then(function (xs) { console.log(xs) })
Is there a way for me to write a more traverse-like version? It's such a basic operation that it seems like there must be a way to do it, but this is my first afternoon with Q and I'm still working through all the overloads of fcall and friends.
Not directly an answer to your question, but AngularJS uses an extremely reduced version of Q ($q), so there you definitely have to implement this behavior yourself.
Here's one approach:
var traverse = function(fn /*values*/) {
var values = _.rest(arguments);
var promises = _.map(values, fn);
return $q.all(promises);
};
Full example: http://plnkr.co/edit/PGp7QbQYMjOknJwSEn8E
Or with separate parameter lists as in Scala:
var traverse = function(/*values*/) {
var values = arguments;
return function(fn) {
var promises = _.map(values, fn);
return $q.all(promises);
}
};
Full example: http://plnkr.co/edit/pWoGGaZobbx61tAmUWr9?p=preview
You can use Promise Chaining + Static values (instead of promises) for methods and do something like:
Q.all([1,2,3])
.then(function(xs) {
return _(xs).map(doSomething)
})
.then(function(xs) {
console.log(xs);
});
If you want a traverse function like that one, you can easily implement it yourself
Hope it helps!