Develop Reference

Using inputs and outputs of a generator function

I was experimenting using genrator functions, and I came up with the following generator. It works, but I don't understand how it is able to use the yield input, process it and use the same yield statement to output the result...
How does the program flow work?
//My caching Generator
function* cache(){
let input, output, cache = {};
while(true){
input = yield output;
//Some code...
if(input in cache){
output = cache[input];
console.log("Old Value: "+output);
}else{
output = f(input);
cache[input] = output;
console.log("New Value: "+output);
}
}
}
//Some expensive function:
function f(x){
return x.split("").reverse().join("");
}
const c = cache();
console.log("Return value: "+ c.next("this is lost").value);
console.log("Return value: "+ c.next("Hello").value);
console.log("Return value: "+ c.next("World").value);
console.log("Return value: "+ c.next("Hello").value);
console.log("Return value: "+ c.next("Stackoverflow").value);

I'll give it a try:
const c = cache();
console.log("Return value: "+ c.next("this is lost").value);
This assigns c with an iterator function for cache (first line) and then runs cache initially until the first yield expression, so basically this code:
let input, output, cache = {};
yield output;
output is undefined, so undefined is returned from the first next() call. Note, however, that the argument ("this is lost") is not passed into this yield. It'd be passed into the yield where the previous execution was halted. As there was no previous run of c.next it is, indeed, lost.
In the next run:
console.log("Return value: "+ c.next("Hello").value);
the following code is executed:
input = "Hello"; // the yield is substituted with the next() argument here!
//Some code...
if(input in cache){ // false
output = cache[input];
console.log("Old Value: "+output);
} else {
output = f(input);
cache[input] = output;
console.log("New Value: "+output);
}
yield output // "olleH"
The function resumes at the exact line where it was halted before and the yield output gets replaced with the next() argument. MDN says:
Calling the next() method with an argument will resume the generator function execution, replacing the yield expression where execution was paused with the argument from next().

Generators + Promises explanation

I've been studying Promises and Generators but I got stuck in the script below:
function getFile(file) {
return new Promise(function (resolve) {
fakeAjax(file, resolve);
});
}
function* getFiles() {
var p1 = getFile('file1');
var p2 = getFile('file2');
var p3 = getFile('file3');
output(yield p1);
output(yield p2);
output(yield p3);
}
function runner(gen) {
var g = gen();
function run(val) {
val || undefined;
var next = g.next(val);
if(!next.done && !next.value !== undefined) {
next.value
.then(function(v) {
run(v);
});
}
}
run();
}
runner(getFiles);
What I'm trying to figure it out is what happens when I get to the first yield on getFiles? Why does this code work, I don't get it.
*EDIT: output is simply a console.log wrappend in a function. The fakeAjax function returns a text from an object based on the 'file' requested.

What I'm trying to figure it out is what happens when I get to the first yield on getFiles? Why does this code work, I don't get it.
yield does three things:
It pause the execution of the generator function
It defines the value the caller of next() will receive in the value property. In this case, that's the promises you made.
It optionally acts as an expression with the value passed into next(). That will be the file name you passed as a argument to next().
yield is like a two-way conduit, both accepting values and passing values.
In your code at the first yield it will return the object with the promise and pause, but it doesn't log anything to the console at this point — yield can pause mid-expression. When you call next() again it will finish the console.log and then move to the next yield. It can be a little confusing because there is usually one more call to next that there are yields. For example in this code `next is called four times and that's why you get the last console.log.
Here's an MCVE that I assume approximates the undefined functions in your example:
function getFile(file) {
return new Promise(resolve => setTimeout(() => resolve(file), 1000))
}
function* getFiles() {
var p1 = getFile('file1');
var p2 = getFile('file2');
var p3 = getFile('file3');
console.log(yield p1); // return promise, then pause, then log value passed to next()
console.log(yield p2);
console.log(yield p3);
}
function runner(gen) {
var g = gen();
function run(val) {
var next = g.next(val);
if(!next.done && !next.value !== undefined) {
next.value
.then(function(v) {
run(v);
});
}
}
run();
}
runner(getFiles);

Progressive (or iterative) Callbacks

I read the other day about a special type of callback that progresses with you as you call it repeatedly. As in it does A the first time it's called and returns, then it does B the next time it's called. Eventually it signals in some way that the final action has been taken.
I can't remember what it was called and I can't find it in my history so I don't know what to search. I need help.

I think what you might be looking for is generator functions. It has been introduced by ES6.
Calling a generator function does not execute its body immediately; an iterator object for the function is returned instead. When the iterator's next() method is called, the generator function's body is executed until the first yield expression, which specifies the value to be returned from the iterator
See MDN documentation.
Example:
function* idMaker() {
var index = 0;
while (index < 3)
yield index++;
}
var gen = idMaker();
console.log(gen.next().value); // 0
console.log(gen.next().value); // 1
console.log(gen.next().value); // 2
console.log(gen.next().value); // undefined

You may be talking about generators:
> function* x() { yield 1; yield 2; yield 3; return 4; }
undefined
> var g = x();
undefined
> g.next();
{ value: 1, done: false }
> g.next();
{ value: 2, done: false }
> g.next();
{ value: 3, done: false }
> g.next();
{ value: 4, done: true }
> g.next();
{ value: undefined, done: true }
See:
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Statements/function%2A
https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Iterators_and_Generators
Generators can be used for asynchronous operations when a generator yields promises instead of values that it wants to return that it itself expects to get injected as a resolved values of those promises in the return value of the yield statement by the wrapper like that from co or Bluebird.coroutine - see:
https://www.npmjs.com/package/co
http://bluebirdjs.com/docs/api/promise.coroutine.html
This was the basis of the new async and await keywords in newer versions of JavaScript:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function

ES6 yield (yield 1)(yield 2)(yield 3)()

function* generatorFunction() {
yield (yield 1)(yield 2)(yield 3)();
}
var iterator = generatorFunction();
// [1, 2, 3]
var iteratedOver = [iterator.next().value, iterator.next().value, iterator.next().value];
I'm not sure how this works.
yield doesn't return a function reference, so what are the parenthetical statements like (yield 2) doing - are they fat arrow anonymous functions without bodies? How are they called using partial application like that?
I'm missing something here, can someone explain?
Update: Tried on three browsers, Chrome 50.0.2661.86, Safari 9.1 (50.0.2661.86), Firefox 44.0.2, all perform without errors.
ESFiddle also executes it without errors.
Commenters report Babel executes without errors as well.
The source of the question is from http://tddbin.com/#?kata=es6/language/generator/send-function, the second kata.

I'm not sure how this works.
Uh, yeah, it shouldn't work. It's only working because of a bug in Babel.
yield doesn't return a function reference, so what are the parenthetical statements like (yield 2) doing - are they fat arrow anonymous functions without bodies? How are they called using partial application like that?
No, it's really just standard function application, no magic. yield could return a function reference, and when it does this might work. When it doesn't, it will throw an exception on the third .next() call.
As an example for a working version:
function* generatorFunction() {
yield (yield 1)(yield 2)(yield 3)();
}
var test = (a) => {
console.log(a);
return (b) => {
console.log(b);
return (c) => {
console.log(c);
return 4;
};
};
};
var iterator = generatorFunction();
iterator.next(); // {value: 1, done: false}
iterator.next(test); // {value: 2, done: false}
iterator.next("a"); // "a" {value: 3, done: false}
iterator.next("b"); // "b" undefined {value: 4, done: false}
iterator.next("d"); // {value: undefined, done: true}
So how does this work? Those nested/chained yield statements should better be written as
function* generatorFunction() {
let fn1 = yield 1;
let a = yield 2;
let fn2 = fn1(a);
let b = yield 3;
let fn3 = fn2(b);
let res = fn3();
let d = yield res;
return undefined;
}
Commenters report Babel executes without errors as well.
That's because of a babel bug. If you check the transpiler output, it actually behaves like
function* generatorFunction() {
let fn1 = yield 1;
let a = yield 2;
let b = yield 3;
// these are no more executed with only 3 `next` calls
let fn2 = fn1(a);
let fn3 = fn2(b);
let res = fn3();
let d = yield res;
return undefined;
}

Generators in KOA

How does work app.use in KOA?
When I set some generator inside app.use, everything works perfect.
How can I do the same elsewhere?
When I just execute generator manual:
var getRelationsList = function *() {
var res = yield db.relations.find({});
console.log({'inside: ': res});
}
console.log({'outside: ': getRelationsList().next()});
getRelationsList().next();
I'm getting just { 'outside: ': { value: [Function], done: false } }
This what I expect is:
{ 'outside: ': { value: {object_with_results}, done: false } }
{ 'inside: ': {object_with_results}
EDIT
I changed my code like that:
var getRelationsList = function *() {
var res = yield db.relations.find({});
console.log({'inside: ': res});
}
console.log({'outside ': co(getRelationsList)});
Now inside console log show's me good results but outside console log shows me just empty object.

Generators are a powerful tool for organizing asynchronous code, but they don't magically wait for asynchronous code to run.
What's Happening
Let's walk through your code so you can see what is happening:
getRelationsList is a generator function, that when called returns a new generator. At this point, no code in your generator function has been called (although if you were passing params they would be set). You then call .next on your generator to start execution of the generator function. It will execute up until it hits the first yield statement and return an object with the yielded value and the completion status of the generator.
It seems you understand most of that so far, but generators do not magically transform the yielded out values. When you yield out db.relations.find({}), you'll get the return value of the find function which I'm assuming is a Promise or some type of thenable:
so your 'outside' value is { value:Promise, done:false }
The reason your inside console.log never ran is that you're actually creating a new generator each time you call getRelationsList(), so when you call getRelationsList().next() again after the outside console.log you're creating a new generator and calling next, so it only executes up to the first yield, just like the call on the previous line.
In order to finish execution you must call next twice on the same instance of your generator: once to execute up to the yield and once to continue execution to the end of the function.
var gen = getRelationsList()
gen.next() // { value:Promise, done:false }
gen.next() // { value:undefined, done:true } (will also console.log inside)
You'll notice, however, if you run this, the inside console.log will be undefined. That's because the value of a yield statement is equal to the value passed to the following .next() call.
For example:
var gen2 = getRelationsList()
gen2.next() // { value:Promise, done:false }
gen2.next(100) // { value:undefined, done:true }
Outputs
{ inside:100 }
Because we passed 100 to the second .next() call and that became the value of the yield db.relations.find({}) statement which was then assigned to res.
Here's a link demoing all of this: http://jsfiddle.net/qj1aszub/2/
The Solution
The creators of koa use a little library called co which basically takes yielded out promises and waits for them to complete before passing the resolved value back into the generator function (using the .next() function) so that you can write your asynchronous code in a synchronous style.
co will return a promise, which will require you to call the .then method on to get the value returned from the generator function.
var co = require('co');
var getRelationsList = function *() {
var res = yield db.relations.find({});
console.log({'inside: ': res});
return res
}
co(getRelationsList).then(function(res) {
console.log({'outside: ': res })
}).catch(function(err){
console.log('something went wrong')
});
co also allows you to yield out other generator functions and wait for their completion, so you don't have to wrap things with co at every level and deal with promises until you're at some sort of 'top level':
co(function *() {
var list = yield getRelationsList()
, processed = yield processRelations(list)
, response = yield request.post('/some/api', { data:processed })
return reponse.data
}).then(function(data) {
console.log('got:', data)
})

Your problem is that you call getRelationsList() function multiple times which is incorrect.
Change your code to following
var g = getRelationsList();
console.log('outside: ', g.next());
g.next(); //You get your console.log('inside: .... here

Generators must be acted upon by outside code.
Under the hood koa use the co library to 'Run' the generator.
Here is how you might achieve what your wanting outside of koa:
var co = require('co');
var getRelationsList = function *() {
var res = yield db.relations.find({});
console.log({'inside: ': res});
}
co(getRelationsList).catch(function(err){});
I did a short screencast on JavaScript generators that should help you understand what's going on:
http://knowthen.com/episode-2-understanding-javascript-generators/
++ EDIT
If your using generators to program in more of an synchronous style (eliminating callbacks), then all your work needs to be done in the generator and you should use a library like co to execute the generator.
Here is a more detailed example of how you would interact with a generator, manually. This should help you understand the results your getting.
function * myGenerator () {
var a = yield 'some value';
return a;
}
var iterator = myGenerator();
// above line just instantiates the generator
console.log(iterator);
// empty object returned
// {}
var res1 = iterator.next();
// calling next() start the generator to either the
// first yield statement or to return.
console.log(res1);
// res1 is an object with 2 attributes
// { value: 'some value', done: false }
// value is whatever value was to the right of the first
// yield statment
// done is an indication that the generator hasn't run
// to completion... ie there is more to do
var toReturn = 'Yield returned: ' + res1.value;
var res2 = iterator.next(toReturn);
// calling next(toReturn) passes the value of
// the variable toReturn as the return of the yield
// so it's returned to the variable a in the generator
console.log(res2);
// res2 is an object with 2 attributes
// { value: 'Yield returned: some value', done: true }
// no further yield statements so the 'value' is whatever
// is returned by the generator.
// since the generator was run to completion
// done is returned as true