Progressive (or iterative) Callbacks - javascript

I read the other day about a special type of callback that progresses with you as you call it repeatedly. As in it does A the first time it's called and returns, then it does B the next time it's called. Eventually it signals in some way that the final action has been taken.
I can't remember what it was called and I can't find it in my history so I don't know what to search. I need help.

I think what you might be looking for is generator functions. It has been introduced by ES6.
Calling a generator function does not execute its body immediately; an iterator object for the function is returned instead. When the iterator's next() method is called, the generator function's body is executed until the first yield expression, which specifies the value to be returned from the iterator
See MDN documentation.
Example:
function* idMaker() {
var index = 0;
while (index < 3)
yield index++;
}
var gen = idMaker();
console.log(gen.next().value); // 0
console.log(gen.next().value); // 1
console.log(gen.next().value); // 2
console.log(gen.next().value); // undefined

You may be talking about generators:
> function* x() { yield 1; yield 2; yield 3; return 4; }
undefined
> var g = x();
undefined
> g.next();
{ value: 1, done: false }
> g.next();
{ value: 2, done: false }
> g.next();
{ value: 3, done: false }
> g.next();
{ value: 4, done: true }
> g.next();
{ value: undefined, done: true }
See:
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Statements/function%2A
https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Iterators_and_Generators
Generators can be used for asynchronous operations when a generator yields promises instead of values that it wants to return that it itself expects to get injected as a resolved values of those promises in the return value of the yield statement by the wrapper like that from co or Bluebird.coroutine - see:
https://www.npmjs.com/package/co
http://bluebirdjs.com/docs/api/promise.coroutine.html
This was the basis of the new async and await keywords in newer versions of JavaScript:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function

Related

next() behaves differently directly on generator vs variable with generator value

Why is it that I get different results when calling next() directly on a generator, versus on a variable with the same generator assigned as its value?
All code/output below.
Below is the generator, plus variable declaration/assignment:
function* gen() {
yield 1;
yield 2;
yield 3;
};
let genVar = gen();
First code snippet:
let first = genVar.next();
console.log(first);
second = genVar.next();
console.log(second);
Output of first code snippet:
{ value: 1, done: false }
{ value: 2, done: false }
Second code snippet:
let one = gen().next();
console.log(one);
two = gen().next();
console.log(two);
Output of second code snippet:
{ value: 1, done: false }
{ value: 1, done: false }
My best guess at the moment is this has something to do with assignment by value/reference?
Anytime you execute the generator function, you create a brand new generator object that allows to iterate over the steps of the generator function from the start.
So in your second example, you actually create two different iterators.

How can I find out if a javascript iterator terminates early?

Lets say I have a generator:
function* source() {
yield "hello"; yield "world";
}
I create the iterable, iterate with a for-loop, and then break out of the loop before the iterator fully completes (returns done).
function run() {
for (let item of source()) {
console.log(item);
break;
}
}
Question: How can I find out, from the iterable-side, that the iterator terminated early?
There doesn't seem to be any feedback if you try to do this directly in the generator itself:
function* source2() {
try {
let result = yield "hello";
console.log("foo");
} catch (err) {
console.log("bar");
}
}
... neither "foo" nor "bar" is logged.
Edit: See Newer accepted answer. I will keep this as it does/did work, and I was pretty happy at the time I was able to hack a solution. However, as you can see in the accepted answer, the finally solution is so simple now it has been identified.
I noticed that typescript defines Iterator (lib.es2015) as:
interface Iterator<T> {
next(value?: any): IteratorResult<T>;
return?(value?: any): IteratorResult<T>;
throw?(e?: any): IteratorResult<T>;
}
I intercepted these methods and logged calls and it does appear that if an iterator is terminated early --at least via a for-loop-- then the return method is called. It will also be called if the consumer throws an error. If the loop is allowed to fully iterate the iterator return is not called.
Return hack
So, I made a bit of a hack to allow capturing another iterable - so I don't have to re-implement the iterator.
function terminated(iterable, cb) {
return {
[Symbol.iterator]() {
const it = iterable[Symbol.iterator]();
it.return = function (value) {
cb(value);
return { done: true, value: undefined };
}
return it;
}
}
}
function* source() {
yield "hello"; yield "world";
}
function source2(){
return terminated(source(), () => { console.log("foo") });
}
for (let item of source2()) {
console.log(item);
break;
}
and it works!
hello
foo
remove the break and you get:
hello
world
Check after each yield
While typing this answer, I realised a better problem/solution is to find out in the original generator method.
The only way I can see to pass information back to the original iterable is to use next(value). So if we pick some unique value (say Symbol.for("terminated")) to signal the termination, and we alter the above return-hack to call it.next(Symbol.for("terminated")):
function* source() {
let terminated = yield "hello";
if (terminated == Symbol.for("terminated")) {
console.log("FooBar!");
return;
}
yield "world";
}
function terminator(iterable) {
return {
[Symbol.iterator]() {
const it = iterable[Symbol.iterator]();
const $return = it.return;
it.return = function (value) {
it.next(Symbol.for("terminated"));
return $return.call(it)
}
return it;
}
}
}
for (let item of terminator(source())) {
console.log(item);
break;
}
Success!
hello
FooBar!
Chaining Cascades Return
If you chain some extra transform iterators, then the return call cascades through them all:
function* chain(source) {
for (let item of source) { yield item; }
}
for (let item of chain(chain(terminator(source())))) {
console.log(item);
break
}
hello
FooBar!
Package
I've wrapped the above solution as a package. It supports both [Symbol.iterator] and [Symbol.asyncIterator]. The async iterator case is of particular interest to me, especially when some resource needs to be disposed of correctly.
There's a much simpler way to do this: use a finally block.
function *source() {
let i;
try {
for(i = 0; i < 5; i++)
yield i;
}
finally {
if(i !== 5)
console.log(' terminated early');
}
}
console.log('First:')
for(const val of source()) {
console.log(` ${val}`);
}
console.log('Second:')
for(const val of source()) {
console.log(` ${val}`);
if(val > 2)
break;
}
...yields:
First:
0
1
2
3
4
Second:
0
1
2
3
terminated early
I've run into a similar need to figure out when an iterator terminates early. The accepted answer is really clever and probably the best way to solve the problem generically, but I think this solution could also be helpful for other uses cases.
Say, for example, you have an infinite iterable, such as the fibonacci sequence described in MDN's Iterators and Generators docs.
In any sort of loop there needs to be a condition set to break out of the loop early, like in the solution already given. But what if you want to destructure the iterable in order to create an array of values? In that case, you'd want to limit the number of iterations, essentially setting a maximum length on the iterable.
To do this, I wrote a function called limitIterable, which takes as arguments an iterable, an iteration limit, and an optional callback function to be executed in the event that the iterator terminates early. The return value is a Generator object (which is both an iterator and an iterable) created using an Immediately Invoked (Generator) Function Expression.
When the generator is executed, whether in a for..of loop, with destructuring, or by calling the next() method, it will check to see if iterator.next().done === true or iterationCount < iterationLimit. In the case of an infinite iterable like the fibonacci sequence, the latter will always cause the while loop to be exited. However, note that one could also set an iterationLimit that is greater than the length of some finite iterable, and everything will still work.
In either case, once the while loop is exited the most recent result will be checked to see if the iterator is done. If so, the original iterable's return value will be used. If not, the optional callback function is executed and used as a return value.
Note that this code also allows the user to pass in values to next(), which will in turn be passed to the original iterable (see the example using MDN's fibonacci sequence inside the code snippet attached). It also allows for additional calls to next() beyond the set iterationLimit within the callback function.
Run the code snippet to see the results of a few possible use cases! Here's the limitIterable function code by itself:
function limitIterable(iterable, iterationLimit, callback = (itCount, result, it) => undefined) {
// callback will be executed if iterator terminates early
if (!(Symbol.iterator in Object(iterable))) {
throw new Error('First argument must be iterable');
}
if (iterationLimit < 1 || !Number.isInteger(iterationLimit)) {
throw new Error('Second argument must be an integer greater than or equal to 1');
}
if (!(callback instanceof Function)) {
throw new Error('Third argument must be a function');
}
return (function* () {
const iterator = iterable[Symbol.iterator]();
// value passed to the first invocation of next() is always ignored, so no need to pass argument to next() outside of while loop
let result = iterator.next();
let iterationCount = 0;
while (!result.done && iterationCount < iterationLimit) {
const nextArg = yield result.value;
result = iterator.next(nextArg);
iterationCount++;
}
if (result.done) {
// iterator has been fully consumed, so result.value will be the iterator's return value (the value present alongside done: true)
return result.value;
} else {
// iteration was terminated before completion (note that iterator will still accept calls to next() inside the callback function)
return callback(iterationCount, result, iterator);
}
})();
}
function limitIterable(iterable, iterationLimit, callback = (itCount, result, it) => undefined) {
// callback will be executed if iterator terminates early
if (!(Symbol.iterator in Object(iterable))) {
throw new Error('First argument must be iterable');
}
if (iterationLimit < 1 || !Number.isInteger(iterationLimit)) {
throw new Error('Second argument must be an integer greater than or equal to 1');
}
if (!(callback instanceof Function)) {
throw new Error('Third argument must be a function');
}
return (function* () {
const iterator = iterable[Symbol.iterator]();
// value passed to the first invocation of next() is always ignored, so no need to pass argument to next() outside of while loop
let result = iterator.next();
let iterationCount = 0;
while (!result.done && iterationCount < iterationLimit) {
const nextArg = yield result.value;
result = iterator.next(nextArg);
iterationCount++;
}
if (result.done) {
// iterator has been fully consumed, so result.value will be the iterator's return value (the value present alongside done: true)
return result.value;
} else {
// iteration was terminated before completion (note that iterator will still accept calls to next() inside the callback function)
return callback(iterationCount, result, iterator);
}
})();
}
// EXAMPLE USAGE //
// fibonacci function from:
//https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators#Advanced_generators
function* fibonacci() {
let fn1 = 0;
let fn2 = 1;
while (true) {
let current = fn1;
fn1 = fn2;
fn2 = current + fn1;
let reset = yield current;
if (reset) {
fn1 = 0;
fn2 = 1;
}
}
}
console.log('String iterable with 26 characters terminated early after 10 iterations, destructured into an array. Callback reached.');
const itString = limitIterable('abcdefghijklmnopqrstuvwxyz', 10, () => console.log('callback: string terminated early'));
console.log([...itString]);
console.log('Array iterable with length 3 terminates before limit of 4 is reached. Callback not reached.');
const itArray = limitIterable([1,2,3], 4, () => console.log('callback: array terminated early?'));
for (const val of itArray) {
console.log(val);
}
const fib = fibonacci();
const fibLimited = limitIterable(fibonacci(), 9, (itCount) => console.warn(`Iteration terminated early at fibLimited. ${itCount} iterations completed.`));
console.log('Fibonacci sequences are equivalent up to 9 iterations, as shown in MDN docs linked above.');
console.log('Limited fibonacci: 11 calls to next() but limited to 9 iterations; reset on 8th call')
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next(true).value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log('Original (infinite) fibonacci: 11 calls to next(); reset on 8th call')
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next(true).value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);

How does Generator.next() processes its parameter?

The documentation says that "You can also provide a parameter to the next method to send a value to the generator." Where does it sends it to?
For example, take these 3 generators:
function* one() {
while(true) {
var value = yield null;
}
}
var g1 = one(); g1.next();
g1.next(1000); //yields null
function* two() {
var i = 0;
while (true) {
i += yield i;
}
}
var g2 = two(); g2.next();
g2.next(1000) // yields 1000
function* three(){
var index = 0;
while (true)
yield index++;
}
g3 = three();
g3.next();
g3.next(1000); // yields 1
In generators 3 and 1, the argument passed has no effect on next. Why is that? How does generator 2 calculates its return value? Why it is affected by the given argument?
The key to understanding this is knowing how the next function retrieves the argument passed to next(), which is as the return value of the yield operator:
[rv] = yield [expression];
Independently of the value of [expression], yield will assign to rv the value passed to next().
But, here comes the tricky part: yield will only assign the value passed to next() when resuming execution from a previous iteration. As a consequence, on the first iteration, yield does not assign anything to rv.
For example, if I have this generator:
function* gen() {
// On the first iteration, yield does not return anything.
//because it returns something ONLY when execution is resumed
returnedFromYield = yield 'foo';
yield returnedFromYield;
}
returnedFromYield is undefined on the first iteration. When execution is resumed on the second iteration, yield assigns the passed value to the returnedFromYield variable, which is then returned:
g.next(1); // 'foo'
g.next(2); // 2
Let's review another example:
function* gen() {
yield yield yield 5;
}
On the first iteration, (g.next()), yield will return 5, on the second iteration, (g.next(10)) yield is going to pass 10 to the second yield. That is, yield yield yield 5; on the second iteration is equivalent to yield yield 10;, and, on the third iteration, it's equivalent to yield valuePassedToNext.
Here's an annotated example showing the logical flow and scope for the values passed to a generator's next method. The affected area of the next method call is highlighted in the corresponding color.
a: There is no yield, so argument 'one' is ignored. The generator pauses on line 5, returning the value 1.
b: Execution resumes and 'two' replaces yield on line 5, and is assigned to the yieldValue variable. Line 6 is executed. The generator pauses on line 8, returning the value 2.
c: Execution resumes and 'three' replaces yield on line 8, and is assigned to the yieldValue variable. Line 9 is executed. The generator pauses on line 11, returning the value 3.
d: Execution resumes and 'four' replaces yield on line 11, and is assigned to the yieldValue variable. Line 12 is executed. Line 14 is executed. Finally the generator is now done, returning the value undefined.
See the logs on line 28-36

Using return in ES6 generator function

I am having trouble finding out what happens if you use a return statement instead of yield.
function *gen(){
const val = yield someAsyncFn();
assert.equal(val,4);
return val;
}
how does the return behave differently from the yield? I assume the return acts as a normal return statement, but the context of a generator function, does it also call gen.return() as well? Sort of confusing.
Perhaps the above is merely identical to this?
function *gen(){
const val = yield someAsyncFn();
assert.equal(val,4);
yield val;
}
return deliveres a return value for an iterators last iteration (when done equals true).
I've simplified your example a bit, since the async operation doesn't seem to be relevant to the question:
function *gen(){
const val = yield 4;
return val * 2;
}
var it = gen();
var val = it.next(); // { value: 4, done: false }
console.log(val.value); // 4
var res = it.next(val.value); // { value: 8, done: true }
console.log(res.value); // 8
Whereas without a return value, on the last iteration you will return a value of undefined:
function *gen2(){
const val = yield 4;
yield val * 2;
}
var it2 = gen2();
var val2 = it2.next(); // { value: 4, done: false }
console.log(val2.value); // 4
var res2 = it2.next(val2.value); // { value: 8, done: false }
console.log(res2.value); // 8
it2.next(); // { value: undefined, done: true }
Sidenote: As a rule of thumb, there is always one more next call then there are yield statements, which is why there is one more next call in the second example.
Let's say you're using a generator-runner like co, then the value you get after finishing the generator would be the value you return:
co(function* () {
var result = yield Promise.resolve(true);
return result;
}).then(function (value) {
console.log(value); // value equals result
}, function (err) {
console.error(err.stack); // err equals result
});
Important: If you are iterating through an iterator, using a for ... of loop or something like Array.from, the return value is going to be ignored (Since you are doing async operations, this is probably not the case anyway):
function *gen(){
const val = yield 4;
return val * 2;
}
for (let value of gen()) {
console.log(value);
}
// 4
In the end, calling a generator just creates an iterator. Whether the final value that the iterator returns is relevant, depends entirely on how you use it.
In addition to the thorough answer by #nils, there is one additional way to capture the return value of a generator function, namely as the value of yield* (necessarily inside another generator function):
function* arrayGenerator(arr) {
for (const element of arr)
yield element
return arr.length
}
function* elementsFollowedByLength(arr) {
const len = yield* arrayGenerator(arr);
yield len;
}
Note the first generator function which returns a value, after it is done yielding the array elements.
The second generator function, through yield*, causes the first generator function to yield all its values. When the first generator function is done and returns, that return value becomes the value of the yield* expression.

What is the meaning of *[Symbol.iterator] in this context

I found some code online. I've squashed the original code down into this little excerpt, that when ran, will print 1-20 to the console.
var NumbersFromOne = {
*[Symbol.iterator] () {
for (let i = 1;; ++i) yield i;
}
};
var take = function* (numberToTake, iterable) {
let remaining = numberToTake;
for (let value of NumbersFromOne) {
if (remaining-- <= 0) break;
yield value;
}
}
var printToTwenty = take(20, NumbersFromOne)
console.log(...printToTwenty);
Now, I understand that take() is a GeneratorFunction.
When take() is called, it is given an iterator.
The code "...printToTwenty" uses the spread operator to iterate through that function.
I understand that NumbersFromOne is an object.
I've come here looking for an explanation of what this part means:
*[Symbol.iterator] () {}
Declaring generator functions is done like this: function* () {}
So I'm assuming this isn't declaring a generator function.
* also doesn't represent the function name
* also can't be replaced with another operator (/, -, +)
What is the deal with that syntax, and why is the * before [Symbol.iterator]
If placed after, it will not run.
I had considered that *[Symbol.iterator] () is a way to overwrite the existing iterator property, but then wouldn't it say this[Symbol.iterator].
Thanks!
There are a few things that might make this code look complicated:
It uses the object property shorthand notation. What you're seeing here is actually the following:
var NumbersFromOne = {
[Symbol.iterator]: function* () {
for (let i = 1;; ++i) yield i;
}
};
Symbol.iterator creates a custom iterator for your NumbersFromOne object.
So your code basically means that the iterator of NumbersFromOne is defined as a generator. Instead of manually having to define a function which returns a next and other properties:
var NumbersFromOne = {
[Symbol.iterator]: function () {
var i = 1;
return {
next: function() {
return { value: i++, done: false };
}
};
}
};
Returning the generator creates the next function automatically for. This allows you to yield when you need to.
It can then be called as:
const it = NumbersFromOne[Symbol.iterator]();
it.next(); // 1
it.next(); // 2
it.next(); // 3
// ...
Note: Written this way, this iterator never ends! So if you were to call it in a for ... of loop without an end-condition, it would freeze your program.

Categories

Resources