Develop Reference

Continuation-Passing Style And Concurrency

I find lots of blogs mention concurrent/non-blocking/asynchronous programming as a benefit of Continuation-Passing Style (CPS). I cannot figure out why CPS provides concurrency, e.g., people mention Node.js is implemented using CPS though JavaScript is a synchronous language. Would someone comment on my thoughts?
First, my naive understanding of CPS is that wrapping all subsequent code at a point into a function and pass that function explicitly as a parameter. Some blogs name the continuation function as return(), Gabriel Gonzalez calls it a hole, both of which are brilliant explanations.
My confusion mostly comes from a popular blog article Asynchronous programming and continuation-passing style in JavaScript. At the beginning of the article, Dr. Axel Rauschmayer gives two code snippets, a synchronous program and an asynchronous one in CPS (pasted here for easy reading).
The synchronous code:
function loadAvatarImage(id) {
var profile = loadProfile(id);
return loadImage(profile.avatarUrl);
}
The asynchronous code:
function loadAvatarImage(id, callback) {
loadProfile(id, function (profile) {
loadImage(profile.avatarUrl, callback);
});
}
I don't get it why the CPS one is asynchronous. After I read another article By example: Continuation-passing style in JavaScript, I think maybe there is an assumption to the code: the function loadProfile() and loadImage() are asynchronous functions by themselves. Then it is not CPS that makes it asynchronous. In the second article, the author actually shows an implementation of fetch(), which is similar to loadProfile() in the blog earlier. The fetch() function makes an explicit assumption of the underlying concurrent execution model by calling req.onreadystatechange. This leads me to think maybe it is not CPS that provides concurrency.
Assume the underlying functions are asynchronous, then I go into my second question: can we write asynchronous code without CPS? Think of the implementation of the function loadProfile(). If it is asynchronous not because of CPS, why can't we just take the same mechanism to implement loadAvatarImage() asynchronously? Assume loadProfile() uses fork() to create a new thread to send the request and wait for the response while the main thread is executing in a non-blocking manner, we can possibly do the same for loadAvatarImage().
function loadAvatarImage(id, updateDOM) {
function act () {
var profile = loadProfile(id);
var img = loadImage(profile.avatarUrl);
updateDOM (img);
}
fork (act());
}
I give it a callback function updateDOM(). Without updateDOM(), it is not fair to compare it with the CPS version -- the CPS version has extra information about what to do after the image is fetched, i.e., the callback function, but the original synchronous loadAvatarImage() does not.
Interestingly, #DarthFennec pointed out my new loadAvatarImage() is actually CPS: fork() is CPS, act() is CPS (if we explicitly give it updateDOM), and loadAvatarImage() is CPS. The chain makes loadAvatarImage() asynchronous. loadProfile() and loadImage() do not need to be asynchronous or CPS.
If the reasoning up to here is correct, can I get these two conclusions?
Given a set of synchronous APIs, someone coding following CPS will not magically create asynchronous functions.
If the underlying asynchronous/concurrent APIs are provided in the CPS style, like CPS versions of loadProfile(), loadImage(), fetch(), or fork(), then one can only code in CPS style to ensure the asynchronous APIs are used asynchronously, e.g., return loadImage(profile.avatarUrl) will nullify the concurrency of loadImage().

A Brief Overview of Javascript
Javascript's concurrency model is non-parallel and cooperative:
Javascript is non-parallel because it runs in a single thread; it achieves concurrency by interleaving multiple execution threads, rather than by actually running them at the same time.
Javascript is cooperative because the scheduler only switches to a different thread when the current thread asks it to. The alternative would be preemptive scheduling, where the scheduler decides to arbitrarily switch threads whenever it feels like it.
By doing these two things, Javascript avoids many problems that other languages don't. Parallel code, and non-parallel preemptively-scheduled code, cannot make the basic assumption that variables will not suddenly change their values in the middle of execution, since another thread might be working on the same variable at the same time, or the scheduler might decide to interleave another thread absolutely anywhere. This leads to mutual exclusion problems and confusing race condition bugs. Javascript avoids all of this because in a cooperatively-scheduled system, the programmer decides where all the interleaves happen. The main drawback of this is if the programmer decides not to create interleaves for long periods of time, other threads never have a chance to run. In a browser, even actions like polling for user input and drawing updates to the page run in the same single-threaded environment as the Javascript, so a long-running Javascript thread will cause the entire page to become unresponsive.
In the beginning, CPS was most often used in Javascript for the purpose of event-driven UI programming: if you wanted some code to run every time someone pressed a button, you would register your callback function to the button's 'click' event; when the button was clicked, the callback would run. As it turns out, this same approach could be used for other purposes as well. Say you want to wait one minute and then do a thing. The naive approach would be to stall the Javascript thread for sixty seconds, which (as stated above) would cause the page to crash for that duration. However, if the timer was exposed as a UI event, the thread could be suspended by the scheduler instead, allowing other threads to run in the meantime. Then, the timer would cause the callback to execute, in the same way a button press would. The same approach can be used to request a resource from the server, or to wait for the page to load fully, or a number of other things. The idea is that, to keep Javascript as responsive as possible, any built-in function that might take a long time to complete should be part of the event system; in other words, it should use CPS to enable concurrency.
Most languages that support cooperative scheduling (often in the form of coroutines) have special keywords and syntax that must be used to tell the language to interleave. For example, Python has the yield keyword, C# has async and await, etc. When Javascript was first designed, it had no such syntax. It did, however, have support for closures, which is a really easy way to allow CPS. I expect the intention behind this was to support the event-driven UI system, and that it was never intended to become a general-purpose concurrency model (especially once Node.js came along and removed the UI aspect entirely). I don't know for sure, though.
Why does CPS provide concurrency?
To be clear, continuation-passing style is a method that can be used to enable concurrency. Not all CPS code is concurrent. CPS isn't the only way to create concurrent code. CPS is useful for things other than enabling concurrency. Simply put, CPS does not necessarily imply concurrency, and vice versa.
In order to interleave threads, execution must be interrupted in such a way that it can be resumed later. This means the context of the thread must be preserved, and later re-instated. This context isn't generally accessible from inside of a program. Because of this, the only way to support concurrency (short of the language having special syntax for it) is to write the code in such a way that the thread context is encoded as a value. This is what CPS does: the context to be resumed is encoded as a function that can be called. This function being called is equivalent to a thread being resumed. This can happen any time: after an image is loaded, after a timer triggers, after other threads have had a chance to run for a while, or even immediately. Since the context is all encoded into the continuation closure, it doesn't matter, as long as it runs eventually.
To better understand this, we can write a simple scheduler:
var _threadqueue = []
function fork(cb) {
_threadqueue.push(cb)
}
function run(t) {
_threadqueue.push(t)
while (_threadqueue.length > 0) {
var next = _threadqueue.shift()
next()
}
}
An example of this in use:
run(function() {
fork(function() {
console.print("thread 1, first line")
fork(function() {
console.print("thread 1, second line")
})
})
fork(function() {
console.print("thread 2, first line")
fork(function() {
console.print("thread 2, second line")
})
})
})
This should print the following to the console:
thread 1, first line
thread 2, first line
thread 1, second line
thread 2, second line
The results are interleaved. While not particularly useful on its own, this logic is more or less the foundation of something like Javascript's concurrency system.
Can we write asynchronous code without CPS?
Only if you have access to the context through some other means. As previously stated, many languages do this through special keywords or other syntax. Some languages have special builtins: Scheme has the call/cc builtin, which will wrap the current context into a callable function-like object, and pass that object to its argument. Operating systems get concurrency by literally copying around the thread's callstack (the callstack contains all of the needed context to resume the thread).
If you mean in Javascript specifically, then I'm fairly certain it's impossible to reasonably write asynchronous code without CPS. Or it would be, but newer versions of Javascript also come with the async and await keywords, as well as a yield keyword, so using those is becoming an option.
Conclusion: Given a set of synchronous APIs, someone coding following CPS will not magically create asynchronous functions.
Correct. If an API is synchronous, CPS alone will not make that API asynchronous. It may introduce a level of concurrency (as in the example code earlier), but that concurrency can only exist within the thread. Asynchronous loading in Javascript works because the loading itself runs in parallel to the scheduler, so the only way to make a synchronous API asynchronous is to run it in a separate system thread (which can't be done in Javascript). But even if you did do that, it still wouldn't be asynchronous unless you also used CPS.
CPS doesn't cause asynchronicity. However, asynchronicity does require CPS, or some alternative to CPS.
Conclusion: If the underlying asynchronous/concurrent APIs are provided in the CPS style, then one can only code in CPS style
Correct. If the API is loadImage(url, callback) and you run return loadImage(profile.avatarUrl), it will return null immediately and it will not give you the image. Most likely it will throw an error because callback is undefined, since you didn't pass it. Essentially, if the API is CPS and you decide not to use CPS, you're not using the API correctly.
In general though, it is accurate to say that if you write a function that calls a CPS function, your function also needs to be CPS. This is actually a good thing. Remember what I said about the basic assumption that variables will not suddenly change their values in the middle of execution? CPS solves this issue by making it very clear to the programmer where exactly the interleave boundaries are; or rather, where values might arbitrarily change. But if you could hide CPS function calls inside of non-CPS functions, you would no longer be able to tell. This is also the reason the newer Javascript async and await keywords work the way they do: any function that uses await must be marked as async, and any call to an async function must be prefixed with the await keyword (there's more to it than that, but I don't want to get into how promises work just now). Because of this, you can always tell where your interleave boundaries are, because there will always be await keywords there.

What do we call JS async "strands of execution"?

In Java we have "threads", in CPython we have threads (non-concurrent) and "processes".
In JS, when I kick off an async function or method, how do I officially refer to these "strands of executing code"?
I have heard that each such code block executes from start to finish*, meaning that there is never any concurrent** processing in JS. I'm not quite sure whether this is the same situation as with CPython threads. Personally I hesitate to use "thread" for what we have in JS as these "strands" are so different from Java concurrent threads.
* Just to clarify in light of Stephen Cleary's helpful response: I mean "each such synchronous code block". Obviously if an await is encountered control is released ...
** And obviously never any "true parallel" processing. I'm following the widely accepted distinction between "concurrent" (only one thread at any one time, but one "strand of execution" may give way to another) and "parallel" (multiple processes, implementing true parallel processing, often using multiple CPUs or cores or processes). My understanding is that these "strands" in JS are not even concurrent: once one AJAX method or Promise or async method/function starts executing nothing can happen until it's finished (or an await happens)...

In JS, when I kick off an async function or method, how do I officially refer to these "strands of executing code"?
For lack of a better term, I've referred to these as "asynchronous operations".
I have heard that each such code block executes from start to finish
This was true... until await. Now there's a couple of ways to think about it, both of which are correct:
Code blocks execute exclusively unless there's an await.
await splits its function into multiple code blocks that do execute exclusively (each await is a "split point").
meaning that there is never any concurrent processing in JS.
I'd disagree with this statement. JavaScript forces asynchrony (before async/await, it used callbacks or promises), so it does have asynchronous concurrency, though not parallel concurrency.
A good mental model is that JavaScript is inherently single-threaded, and that one thread has a queue of work to do. await does not block that thread; it returns, and when its thenable/Promise completes, it schedules the remainder of that method (the "continuation") to the queue.
So you can have multiple asynchronous methods executing concurrently. This is exactly how Node.js handles multiple simultaneous requests.
My understanding is that these "strands" in JS are not even concurrent: once one AJAX method or Promise or async method/function starts executing nothing can happen until it's finished...
No, this is incorrect. await will return control to the main thread, freeing it up to do other work.
the way things operate in JS means you "never have to worry about concurrency issues, etc."
Well... yes and no. Since JavaScript is inherently single-threaded, there's never any contention over data shared between multiple threads (obviously). I'd say that's what the original writer was thinking about. So I'd say easily north of 90% of thread-synchronization problems do just go away.
However, as you noted, you still need to be careful modifying shared state from multiple asynchronous operations. If they're running concurrently, then they can complete in any order, and your logic has to properly handle that.
Ideally, the best solution is to move to a more functional mindset - that is, get rid of the shared state as much as possible. Asynchronous operations should return their results instead of updating shared state. Then these concurrent asynchronous operations can be composed into a higher-level asynchronous operation using async, await, and Promise.all. If possible, this more functional approach of returning-instead-of-state and function composition will make the code easier to deal with.
But there's still some situations where this isn't easily achievable. It's possible to develop asynchronous equivalents of classical synchronous coordination primitives. I worked up a proof-of-concept AsyncLock, but can't seem to find the code anywhere. Well, it's possible to do this, anyway.

To date 24 people, no more no fewer, have chosen to look at this.
FWIW, and should anyone eccentric take an interest in this question one day, I propose "hogging strands" or just "strands" ... they hog the browser's JS engine, it would appear, and just don't let go, unless they encounter an await.
I've been developing this app where 3 "strands" operate more or less simultaneously, all involving AJAX calls and a database query... but in fact it is highly preferable if the 2nd of the 3 executes and returns before the 3rd.
But because the 3rd query is quite a bit simpler for MySQL than the 2nd the former tends to return from its AJAX "excursion" quite a bit earlier if everything is allowed to operate with unconstrained asynchronicity. This then causes a big delay before the 2nd strand is allowed to do its stuff. To force the code to perform and complete the 2nd "strand" therefore requires quite a bit of coercion, using async and await.
I read one comment on JS asynchronicity somewhere where someone opined that the way things operate in JS means you "never have to worry about concurrency issues, etc.". I don't agree: if you don't in fact know when awaits will unblock this does necessarily mean that concurrency is involved. Perhaps "hogging strands" are easier to deal with than true "concurrency", let alone true "parallelism", however... ?

function declaration and function expression performance difference

I have used JSperf to test a small sample of code.
According to a few articles I came across, both should have similar performance with test2 having a little edge. But here it's the complete opposite. Can someone explain why the huge difference?
Edit : I also understand the differences between both of them. Please don't mark this as a duplicate of this or other questions which talk about the semantic differences and do not answer my question regarding performance.
Thank you.

With the powerful optimizations JavaScript engines are using these days, Micro-benchmarks like this produce somewhat misleading results. For example, I'm guessing what you were trying to measure is function call overhead. But it looks like the way your code is written, you may be (re)defining the function definition and/or symbol lookup once for every 10 times you execute it; I'm guessing that wasn't the intent.
In this alternative test, I've arranged things to avoid repeated definition of the function, and added a few other ways of invoking the functions. This reduces the difference in performance to something I'd consider dominated by experimental noise. While this there may sometimes be apparent differences but I wouldn't consider them statistically significant given the experimental error levels. In other words, it reduces the contest to a virtual tie.
Even in browsers where there's a consistent difference between approaches, caching the function in a local variable seems to minimize the difference between definition and expression.

I also understand the differences between both of them.
Do you also understand these semantic differences?
Notice that jsPerf puts your code inside a tight loop whose execution time is measured. The function declaration requires the creation of a block scope for each iteration, which slows the test down considerably. That is hardly what you were trying to measure.

Processing.js sleep

I want to write a sleep() function in javascript/processing.js. I.e: a function that interrupts the flow of the program for however many seconds.
It is obvious how to do this with "busy waiting", but this tends to slow down the browser and make things unusable
I know "sleep" is not good javascript. I want this function for didactic purposes (help kids understand their code), not for production use.
Since it is meant do be didactic, an explicit callback is too much of a complication. Calling the function should be as simple as in , say, bash or php -- however, we can use the most arcane things, just as long as they remain hidden inside the sleep function (including processing.js tricks)
I am aware of question What is the JavaScript version of sleep()?, but still hope there is a hack to stop processing.js (or perhaps a real javascript solution, however ill-advised it might be)
This function should work outside a draw() loop -- if it works inside as well, that is a bonus
If it is relevant, this function is meant to be used on Khan Academy

It is obvious how to do this with "busy waiting", but this tends to slow down the browser and make things unusable
A sleep() function would also cause this behavior, since JavaScript is single-threaded.
Since it is meant do be didactic, an explicit callback is too much of a complication.
You've pretty much answered your own question: there is no way to do a sleep() function in JavaScript without using a callback or busy waiting.
You might consider using Java mode to show sleep(), but it sounds like busy waiting is the way to go.
And in my humble opinion, even if you could find a hack to cause a sleep, that's probably not a great example for kids, since their code would never do that. They're much more likely to try to render too many objects. If you're trying to demonstrate that doing too much inside of the draw() function will be bad, then why not just have them do too much inside the draw() function? Teach them about for loops and then ask them to see what happens when they draw 100 rectangles, or 1000, or 1,000,000!

Debugging Techniques in JavaScript. Async Callbacks

In an existing Backbone/jQuery/CoffeeScript app I am working on, it appears there is a function (Backbone.Collection.fetch()) called multiple times (sometimes number may vary). I think it might be a timing thing as I am doing alot of nested callbacks (like AJAX etc) and its becoming hard to debug. I should probably try to convert the code to use jQuery deferred but in the mean time, what can I do?
Just tried walking through the code in Chrome, but it appears the code is jumping here and there, maybe its processing different callbacks at the same time?
I am thinking maybe I add a console.log to every function + its arguments, but there must be a better way?

You can add a stack trace to that fetch() function, and see where it's being called from. There are a number of decent stack trace implementations for JS. I've had good success with
Eric Wendelin's version, but there are plenty of others.
With the stack trace, perhaps you can at least see what the most common paths are into that function, and that might help narrow down where to search. It might even make clear the underlying culprit.