I'm familiar with callbacks and i know what synchronicity is, but there is still something I'm missing when it comes to continuation passing style. I've got this simple code example where I can't tell how to transform it to asynchronous code based on callbacks in continuation passing style:
function performOrder() {
var available, placedOrder;
available = checkCountOnServer(order.item, order.count);
if (!available) {
showNotAvailableMessage(order);
return;
}
placedOrder = checkPayment(order.item, order.count, order.paymentMethod);
if (askUserForConfirmationWithDialog(placedOrder)) {
return confirmOrder(placedOrder);
}
}
My approach to this was:
function performOrder() {
checkCountOnServerC(order.item, order.count, showNotAvailableMessage(order));
return checkPaymentC(order.item, order.count, askUserConfirmationWithDialogC(placedOrder, confirmOrder(placedOrder));
}
All Methods appended with C can be assumed to be modified to accept a callback as continuation.
My Questions are: Are both snippets equivalent and confirms the second one to the continuation passing style?
Thank you very much in advance!
Your code would execute checkCountOnServerC and checkPaymentC in parallel regardless of the status of available. Therefore they are not equivalent.
The correct transformation would depend on the style you want to adopt: node.js standard callback style, Promises, successCallback/errorCallback etc. Assuming you want to use node.js standard callback style:
function performOrderC(callback){
checkCountOnServerC(order.item, order.count, function(err, available){
if (! available ) {
showNotAvailableMessage(order));
callback(new Error('Not available'));
return;
}
checkPaymentC(order.item, order.count, function(err, placedOrder){
askUserConfirmationWithDialogC(placedOrder, function(){
callback(null, confirmOrder(placedOrder)); // this is how you "return" results
});
});
});
}
Node.js standard callback style uses the first argument of callbacks to pass errors up the stack. But before node.js this was not universally adopted by asynchronous functions.
If the above appears a bit dense (sometimes called callback-hell) you can always refactor it to not use anonymous functions:
function checkPaymentHandler (callback) {
return function (err, placedOrder){
askUserConfirmationWithDialogC(placedOrder, function(){
callback(null, confirmOrder(placedOrder)); // this is how you "return" results
});
});
}
function checkCountHandler (callback) {
return function (err, available) {
if (! available ) {
showNotAvailableMessage(order));
callback(new Error('Not available'));
return;
}
checkPaymentC(order.item, order.count, checkPaymentHandler(callback));
}
}
function performOrderC(callback){
checkCountOnServerC(order.item, order.count, checkCountHandler(callback));
}
I have trouble understanding how callbacks in JavaScript get their arguments. Or in other words: how to implement a higher-order function so its callback accepts for instance a standard err and data arguments.
Like in this article on asynchronous JavaScript in the example of a callback used in a usual way (and I know it's usual because I see exactly this pattern used in Node.js moongose (i.e. for creating data in db)):
function getData(options, callback) {
$.get("example.php", options, function(response) {
callback(null, JSON.parse(response));
}, function() {
callback(new Error("AJAX request failed!"));
});
}
// usage
getData({name: "John"}, function(err, data) {
if(err) {
console.log("Error! " + err.toString())
} else {
console.log(data);
}
});
how exactly the callback gets arguments err & data based on how getData() function is declared above?
Arguments are passed to a function when that function is called.
function foo(arg) {
console.log(arg);
}
foo("This is the value");
This is still true when it is a callback function.
function foo(arg) {
console.log(arg);
}
function bar(callback) {
callback("This is the value");
}
bar(foo);
And it is still true when the callback function is called by code written by someone else that exists in a library you aren't examining the source code of.
It is possible to determine the order of TWO tasks using callbacks, as shown below.
a(b);
function a(callback) {
// do something
callback();
}
function b() {
// do next
}
See Fiddle
First do a(), then do b().
I would like to concatenate more than two tasks.
As I´m dealing with quite big functions, I´m looking for something like that:
a(b(c));
First do a(), then do b(), then do c().
However I'm not successful with this. See Fiddle
Is there an easy way to do so, maybe without needing Promises?
You're calling b immediately, not passing a callback to a. You'll need to use a function expression:
a(function(aResult) {
b(c);
});
Of course, you can avoid these by returning closures from all your functions:
function a(callback) {
return function(args) {
// do something
if (callback) callback(res);
};
}
function b(callback) {
return function(aResult) {
// do next
if (callback) callback(res);
};
}
function c(callback) {
return function(bResult) {
// do next
if (callback) callback(res);
};
}
which you would call like this:
a(b(c())();
(this is known as pure continuation passing style)
From what I understand, the future style to write async code in JS is to use generators instead of callbacks. At least, or esp. in the V8 / Nodejs community. Is that right? (But that might be debatable and is not my main question here.)
To write async code with generators, I have found a few libraries:
gen-run (What I'm currently using.)
co
task.js
Galaxy
They all look kind of similar and I'm not that sure which of them to use (or if that even matters). (However, that might again be debatable and is also not my main question here -- but I still would be very happy about any advice.)
(I'm anyway only using pure V8 - if that matters. I'm not using Nodejs but I use pure V8 in my custom C++ app. However, I already have a few node-style elements in my code, including my custom require().)
Now I have some function X written in callback-style, which itself calls other async functions with callback arguments, e.g.:
function X(v, callback) {
return Y(onGotY);
function onGotY(err, res) {
if(err) return callback(err);
return Z(onGotZ);
}
function onGotZ(err, res, resExtended) {
if(err) return callback(err);
return callback(null, v + res + resExtended);
}
}
And I want to turn X into a generator, e.g. I guess function* X(v) { ... }. How would that look like?
I went with my very simple own lib which works quite well for my small V8 environment and also makes it easy to debug because the JS callstack stays intact. To make it work with Nodejs or on the web, it would need some modifications, though.
Rationales here:
For debugging, we don't really want async code - we want to have nice understandable call stacks in debuggers, esp. in the node-inspector debugger.
Also note that so far, all async code is completely artificial and we execute everything completely in sync, somewhat emulated via V8 Microtasks.
Callback-style async code is already hard to understand in call stacks.
Generator-style async code looses the call stack information completely in conventional debuggers - that includes the current Chrome Beta Developer Tools V8 debugger used with node-inspector. That is very much the nature of generators (coroutines in general). Later versions of the debugger might handle that, but that is not the case today. We even need some special C++ code to get the info. Example code can be found here:
https://github.com/bjouhier/galaxy-stack/blob/master/src/galaxy-stack.cc
So if we want to have useful debuggers, today, we cannot use generators -- at least not as it is usually done.
We still want to use generator-style async code because it makes the code much more readable, compared to callback-style code.
We introduce a new function async to overcome this. Imagine the following callback-style async code:
function doSthX(a, b, callback) { ... }
function doSthY(c, callback) {
doSthX(c/2, 42, function(err, res) {
if(err) return callback(err);
callback(null, c + res);
})
}
Now, the same code with the async function and generator-style code:
function* doSthX(a, b) { ... }
function* doSthY(c) {
var res = yield async(doSthX(c/2, 42));
return c + res;
}
We can provide two versions of async(iter):
Run the iterator iter on top of the stack. That will keep a sane call stack and everything is run serially.
Yield the iterator down to some lower handler and make it really async.
Note that there are a few notable libraries which can be used for the second approach:
https://github.com/visionmedia/co
http://taskjs.org/
https://github.com/creationix/gen-run
https://github.com/bjouhier/galaxy
For now, we just implement the first approach - to make debugging easier.
If we once want to have both, we can introduce a debug flag to switch via both implementations.
global.async = async;
function async(iter) {
// must be an iterator
assert(iter.next);
var gotValue;
var sendValue;
while(true) {
var next = iter.next(sendValue);
gotValue = next.value;
if(!next.done) {
// We expect gotValue as a value returned from this function `async`.
assert(gotValue.getResult);
var res = gotValue.getResult();
sendValue = res;
}
if(next.done) break;
}
return {
getResult: function() {
return gotValue;
}
};
}
// Like `async`, but wraps a callback-style function.
global.async_call_cb = async_call_cb;
function async_call_cb(f, thisArg /* , ... */) {
assert(f.apply && f.call);
var args = Array.prototype.slice.call(arguments, 2);
return async((function*() {
var gotCalled = false;
var res;
// This expects that the callback is run on top of the stack.
// We will get this if we always use the wrapped enqueueMicrotask().
// If we have to force this somehow else at some point, we could
// call runMicrotasks() here - or some other waiter function,
// to wait for our callback.
args.push(callback);
f.apply(thisArg, args);
function callback(err, _res) {
assert(!gotCalled);
if(err) throw err;
gotCalled = true;
res = _res;
}
assert(gotCalled);
return res;
})());
}
// get the result synchronously from async
global.sync_from_async = sync_from_async;
function sync_from_async(s) {
assert(s.getResult); // see async()
return s.getResult();
}
// creates a node.js callback-style function from async
global.callback_from_async = callback_from_async;
function callback_from_async(s) {
return function(callback) {
var res;
try { res = sync_from_async(s); }
catch(err) {
return callback(err);
}
return callback(null, res);
};
}
global.sync_get = sync_get;
function sync_get(iter) {
return sync_from_async(async(iter));
}
// this is like in gen-run.
// it's supposed to run the main-function which is expected to be a generator.
// f must be a generator
// returns the result.
global.run = run;
function run(f) {
return sync_get(f());
}
I assume that by "turning X into a generator" you mean "turning X into something you can yield", because that's how these libs work.
If you are using co (which I recommend), you need to make your function return thunks, that is a function, which accepts only a callback. Quite simple:
function coX (v) {
return function (cb) {
X(v, cb);
}
}
And then just:
co(function * () {
yield coX('v');
})();
I rewrote this question, because the old version was obviously misleading.
Please read the text and make shure you understood what I'm asking for. If
there is still anything left in the dark I'll modify this question for clarity.
Just inform me.
One of my projects is to port a library from Python to JavaScript.
The Python library is entirely blocking/synchronous when it comes to I/O
and such. This is of course perfectly normal for Python code.
I plan to port the synchronous/blocking methods as they are to JavaScript.
This has several reasons and whether or not it's worth the effort is a
good but different question.
Additionally I wan't to add an asynchronous/non-blocking api.
Think of it like the fs module in node where there are i.e. fs.open and
fs.openSync coexisting.
The library is pure JavaScript and will run in Node and in the Browser.
The question is what a good/the best approach for the development of these two coexisting APIs would be.
I believe its good to have the same thing happening in one place only.
Hence an approach where some parts of the implementation could be shared would be preferable.
Not at any price of course, that's why I'm asking.
I had a proposal for an approach in here, but I'm going to post it as a
possible answer. However, I'm waiting for some serious discussion to happen
before I decide what I accept as an answer.
So far approaches are:
implement both apis separately and definetly use promises for the asynchronous functions.
use something like the obtain api proposal - beeing a more integrated approach
If you're talking I/O in node.js then most I/O methods have a synchronous version.
There is no direct conversion from Asynchronicity To Synchronicity. I can think of two approaches:
Have each asynchronous method run a polling loop waiting for the async task to complete before returning.
Drop the idea of mimicking synchronous code and instead invest in better coding patterns (such as promises)
To illustrate I will assume option 2 is a better choice. The following example uses Q promises (easily installed with npm install q.
The idea behind promises is that although they are asynchronous the return object is a promise for a value as if it was a normal function.
// Normal function
function foo(input) {
return "output";
}
// With promises
function promisedFoo(input) {
// Does stuff asynchronously
return promise;
}
The first function takes an input and returns a result. The second example takes an input and immediately returns a promise which will eventually resolve to a value when the async task finishes. You then manage this promise as follows:
var promised_value = promisedFoo(input);
promised_value.then(function(value) {
// Yeah, we now have a value!
})
.fail(function(reason) {
// Oh nos.. something went wrong. It passed in a reason
});
Using promises you no longer have to worry when something will happen. You can easily chain promises so things happen synchronously without insane nested callbacks or 100 named functions.
It well worth learning about. Remember promises are meant to make async code behave like sync code even though it isn't blocking.
Write lower level API using promises that takes async/sync flag.
Higher level async API returns these promises directly (while also working with async callbacks like it's 1970).
Higher level sync API unwraps the value synchronously from the promise and returns the value or throws the error.
(Examples use bluebird which is orders of magnitude faster and has more features at the cost of file size compared to Q, although that might not be ideal for browsers.)
Low level api that is not exposed:
//lowLevelOp calculates 1+1 and returns the result
//There is a 20% chance of throwing an error
LowLevelClass.prototype.lowLevelOp = function(async, arg1, arg2) {
return new Promise(function(resolve, reject) {
if (Math.random() < 0.2) {
throw new Error("random error");
}
if (!async) resolve(1+1);
else {
//Async
setTimeout(function(){
resolve(1+1);
}, 50);
}
});
};
High level exposed API that works synchronously, using promises or callbacks:
HighLevelClass.prototype.opSync = function(arg1, arg2) {
var inspection =
this.lowLevel.lowLevelOp(false, arg1, arg2).inspect();
if (inspection.isFulfilled()) {
return inspection.value();
}
else {
throw inspection.error();
}
};
HighLevelClass.prototype.opAsync = function(arg1, arg2, callback) {
//returns a promise as well as accepts callback.
return this.lowLevel.lowLevelOp(true, arg1, arg2).nodeify(callback);
};
You can automatically generate the high level api for synchronous methods:
var LowLevelProto = LowLevelClass.prototype;
Object.keys(LowLevelProto).filter(function(v) {
return typeof LowLevelProto[v] === "function";
}).forEach(function(methodName) {
//If perf is at all a concern you really must do this with a
//new Function instead of closure and reflection
var method = function() {
var inspection = this.lowLevel[methodName].apply(this.lowLevel, arguments);
if (inspection.isFulfilled()) {
return inspection.value();
}
else {
throw inspection.error();
}
};
HighLevelClass.prototype[methodName + "Sync" ] = method;
});
I implemented a library that does what I'm asking for ObtainJS.
(Yes, the Library uses Promises BUT not as others proposed in their ansewers here)
Reposting the Readme.md:
ObtainJS
ObtainJS is a micro framework to bring together asynchronous and
synchronous JavaScript code. It helps you to Don't Repeat Yourself
(DRY) if you are developing a library with interfaces for both
blocking/synchronous and non-blocking/asynchronous execution models.
As a USER
of a library that was implemented with ObtainJS you won't have to learn
a lot. Typically a function defined using ObtainJS has as first argument
the switch, that lets you choose the execution path, followed by its normal
arguments:
// readFile has an obtainJS API:
function readFile(obtainAsyncExecutionSwitch, path) { /* ... */ }
execute synchronously
If the obtainSwitch is a falsy value readFile will execute synchronously
and return the result directly.
var asyncExecution = false, result;
try {
result = readFile(asyncExecution, './file-to-read.js');
} catch(error) {
// handle the error
}
// do something with result
execute asynchronously
If the obtainSwitch is a truthy value readFile will execute asynchronously
and always return a Promise.
See Promises at MDN
var asyncExecution = true, promise;
promise = readFile(asyncExecution, './file-to-read.js');
promise.then(
function(result) {
// do something with result
},
function(error){
// handle the error
}
)
// Alternatively, use the returned promise directly:
readFile(asyncExecution, './file-to-read.js')
.then(
function(result) {
// do something with result
},
function(error){
// handle the error
}
)
You can use a callback based api, too. Note that the Promise is returned anyways.
var asyncExecution;
function unifiedCallback(error, result){
if(error)
// handle the error
else
// do something with result
}
asyncExecution = {unified: unifiedCallback}
readfile(asyncExecution, './file-to-read.js');
or with a separate callback and errback
var asyncExecution;
function callback(result) {
// do something with result
}
function errback(error) {
// handle the error
}
var asyncExecution = {callback: callback, errback: errback}
readfile(asyncExecution, './file-to-read.js');
```
As a smart ;-) LIBRARY AUTHOR
who's going to implement a API using with ObtainJS the work is a bit more.
Stay with me.
The behavior above is achieved by defining a twofold dependency tree: one
for the actions of the synchronous execution path and one for the actions
of the asynchronous execution path.
Actions are small functions with dependencies on the results of other
actions. The asynchronous execution path will fallback to synchronous
actions if there is no asynchronous action defined for a dependency.
You wouldn't define an asynchronous action if its synchronous
equivalent is non-blocking. This is where you DRY!
So, what you do, for example, is splitting your synchronous and blocking
method in small function-junks. These junks depend on the results of each
other. Then you define a non-blocking AND asynchronous junk for each
synchronous AND blocking junk. The rest does obtainJS for you. Namely:
creating a switch for synchronous or asynchronous execution
resolving the dependency tree
executing the junks in the right order
providing you with the results via:
return value when using the synchronous path
promises OR callbacks (your choice!) when using the asynchronous path
Here is the readFile function
from above, taken directly from working code at
ufoJS
define(['ufojs/obtainJS/lib/obtain'], function(obtain) {
// obtain.factory creates our final function
var readFile = obtain.factory(
// this is the synchronous dependency definition
{
// this action is NOT in the async tree, the async execution
// path will fall back to this method
uri: ['path', function _path2uri(path) {
return path.split('/').map(encodeURIComponent).join('/')
}]
// synchronous AJAX request
, readFile:['uri', function(path) {
var request = new XMLHttpRequest();
request.open('GET', path, false);
request.send(null);
if(request.status !== 200)
throw _errorFromRequest(request);
return request.responseText;
}]
}
,
// this is the asynchronous dependency definition
{
// aynchronous AJAX request
readFile:['uri', '_callback', function(path, callback) {
var request = new XMLHttpRequest()
, result
, error
;
request.open('GET', path, true);
request.onreadystatechange = function (aEvt) {
if (request.readyState != 4 /*DONE*/)
return;
if (request.status !== 200)
error = _errorFromRequest(request);
else
result = request.responseText
callback(error, result)
}
request.send(null);
}]
}
// this are the "regular" function arguments
, ['path']
// this is the "job", a driver function that receives as first
// argument the obtain api. A method that the name of an action or
// of an argument as input and returns its result
// Note that job is potentially called multiple times during
// asynchronoys execution
, function(obtain, path){ return obtain('readFile'); }
);
})
a skeleton
var myFunction = obtain.factory(
// sync actions
{},
// async actions
{},
// arguments
[],
//job
function(obtain){}
);
action/getter definition
// To define a getter we give it a name provide a definition array.
{
// sync
sum: ['arg1', 'arg2',
// the last item in the definition array is always the action/getter itself.
// it is called when all dependencies are resolved
function(arg1, arg2) {
// function body.
var value = arg1 + arg2
return value
}]
}
// For asynchronous getters you have different options:
{
// async
// the special name "_callback" will inject a callback function
sample1: ['arg1', '_callback', function(arg1, callback) {
// callback(error, result)
}],
// you can order separate callback and errback when using both special
// names "_callback" and "_errback"
sample2: ['arg1', '_callback', '_errback', function(arg1, callback, errback) {
// errback(error)
// callback(result)
}],
// return a promise
sample3: ['arg1', function(arg1) {
var promise = new Promise(/* do what you have to*/);
return promise
}]
}
The items in the definition array before the action are the dependencies
their values are going to be injected into the call to action, when
available.
If the type of an dependency is not a string: It's injected as a value
directly. This way you can effectively do currying.
If the type of the value is a string: It's looked up in the dependency
tree for the current execution path(sync or async).
If its name is defined as an caller-argument (in the third argument of obtain.factory) the value
is taken from the invoking call.
If its name is defined as the name of another action, that action is
executed and its return value is used as a parameter. An action will
executed only once per run, later invocations will return a cached value.
If the execution path is asynchronous obtain will first look for a
asynchronous action definition. If that is not found it falls back
to a synchronous definition.
If you wish to pass a String as value to your getter you must define it as
an instance of obtain.Argument: new obtain.Argument('mystring argument is not a getter')
A more complete example
from ufoLib/glifLib/GlyphSet.js
Note that: obtainJS is aware of the host object and propagates this
correctly to all actions.
/**
* Read the glif from I/O and cache it. Return a reference to the
* cache object: [text, mtime, glifDocument(if alredy build by this.getGLIFDocument)]
*
* Has the obtainJS sync/async api.
*/
GlypSet.prototype._getGLIFcache = obtain.factory(
{ //sync
fileName: ['glyphName', function fileName(glyphName) {
var name = this.contents[glyphName];
if(!(glyphName in this.contents) || this.contents[glyphName] === undefined)
throw new KeyError(glyphName);
return this.contents[glyphName]
}]
, glyphNameInCache: ['glyphName', function(glyphName) {
return glyphName in this._glifCache;
}]
, path: ['fileName', function(fileName) {
return [this.dirName, fileName].join('/');
}]
, mtime: ['path', 'glyphName', function(path, glyphName) {
try {
return this._io.getMtime(false, path);
}
catch(error) {
if(error instanceof IONoEntryError)
error = new KeyError(glyphName, error.stack);
throw error;
}
}]
, text: ['path', 'glyphName', function(path, glyphName) {
try {
return this._io.readFile(false, path);
}
catch(error) {
if(error instanceof IONoEntryError)
error = new KeyError(glyphName, error.stack);
throw error;
}
}]
, refreshedCache: ['glyphName', 'text', 'mtime',
function(glyphName, text, mtime) {
return (this._glifCache[glyphName] = [text, mtime]);
}]
}
//async getters
, {
mtime: ['path', 'glyphName', '_callback',
function(path, glyphName, callback) {
var _callback = function(error, result){
if(error instanceof IONoEntryError)
error = new KeyError(glyphName, error.stack);
callback(error, result)
}
this._io.getMtime({unified: _callback}, path);
}]
, text: ['path', 'glyphName', '_callback',
function(path, glyphName, callback){
var _callback = function(error, result) {
if(error instanceof IONoEntryError)
error = new KeyError(glyphName, error.stack);
callback(error, result)
}
this._io.readFile({unified: _callback}, path);
}
]
}
, ['glyphName']
, function job(obtain, glyphName) {
if(obtain('glyphNameInCache')) {
if(obtain('mtime').getTime() === this._glifCache[glyphName][1].getTime()) {
// cache is fresh
return this._glifCache[glyphName];
}
}
// still here? need read!
// refreshing the cache:
obtain('refreshedCache')
return this._glifCache[glyphName];
}
)