I have such a function in my JS script:
function heavyWork(){
for (i=0; i<300; i++){
doSomethingHeavy(i);
}
}
Maybe "doSomethingHeavy" is ok by itself, but repeating it 300 times causes the browser window to be stuck for a non-negligible time. In Chrome it's not that big of a problem because only one Tab is effected; but for Firefox its a complete disaster.
Is there any way to tell the browser/JS to "take it easy" and not block everything between calls to doSomethingHeavy?
You could nest your calls inside a setTimeout call:
for(...) {
setTimeout(function(i) {
return function() { doSomethingHeavy(i); }
}(i), 0);
}
This queues up calls to doSomethingHeavy for immediate execution, but other JavaScript operations can be wedged in between them.
A better solution is to actually have the browser spawn a new non-blocking process via Web Workers, but that's HTML5-specific.
EDIT:
Using setTimeout(fn, 0) actually takes much longer than zero milliseconds -- Firefox, for example, enforces a minimum 4-millisecond wait time. A better approach might be to use setZeroTimeout, which prefers postMessage for instantaneous, interrupt-able function invocation, but use setTimeout as a fallback for older browsers.
You can try wrapping each function call in a setTimeout, with a timeout of 0. This will push the calls to the bottom of the stack, and should let the browser rest between each one.
function heavyWork(){
for (i=0; i<300; i++){
setTimeout(function(){
doSomethingHeavy(i);
}, 0);
}
}
EDIT: I just realized this won't work. The i value will be the same for each loop iteration, you need to make a closure.
function heavyWork(){
for (i=0; i<300; i++){
setTimeout((function(x){
return function(){
doSomethingHeavy(x);
};
})(i), 0);
}
}
You need to use Web Workers
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers
There are a lot of links on web workers if you search around on google
We need to release control to the browser every so often to avoid monopolizing the browser's attention.
One way to release control is to use a setTimeout, which schedules a "callback" to be called at some period of time. For example:
var f1 = function() {
document.body.appendChild(document.createTextNode("Hello"));
setTimeout(f2, 1000);
};
var f2 = function() {
document.body.appendChild(document.createTextNode("World"));
};
Calling f1 here will add the word hello to your document, schedule a pending computation, and then release control to the browser. Eventually, f2 will be called.
Note that it's not enough to sprinkle setTimeout indiscriminately throughout your program as if it were magic pixie dust: you really need to encapsulate the rest of the computation in the callback. Typically, the setTimeout will be the last thing in a function, with the rest of the computation stuffed into the callback.
For your particular case, the code needs to be transformed carefully to something like this:
var heavyWork = function(i, onSuccess) {
if (i < 300) {
var restOfComputation = function() {
return heavyWork(i+1, onSuccess);
}
return doSomethingHeavy(i, restOfComputation);
} else {
onSuccess();
}
};
var restOfComputation = function(i, callback) {
// ... do some work, followed by:
setTimeout(callback, 0);
};
which will release control to the browser on every restOfComputation.
As another concrete example of this, see: How can I queue a series of sound HTML5 <audio> sound clips to play in sequence?
Advanced JavaScript programmers need to know how to do this program transformation or else they hit the problems that you're encountering. You'll find that if you use this technique, you'll have to write your programs in a peculiar style, where each function that can release control takes in a callback function. The technical term for this style is "continuation passing style" or "asynchronous style".
You can make many things:
optimize the loops - if the heavy works has something to do with DOM access see this answer
if the function is working with some kind of raw data use typed arrays MSDN MDN
the method with setTimeout() is called eteration. Very usefull.
the function seems to be very straight forward typicall for non-functional programming languages. JavaScript gains advantage of callbacks SO question.
one new feature is web workers MDN MSDN wikipedia.
the last thing ( maybe ) is to combine all the methods - with the traditional way the function is using only one thread. If you can use the web workers, you can divide the work between several. This should minimize the time needed to finish the task.
I see two ways:
a) You are allowed to use Html5 feature. Then you may consider to use a worker thread.
b) You split this task and queue a message which just do one call at once and iterating as long there is something to do.
There was a person that wrote a specific backgroundtask javascript library to do such heavy work.. you might check it out at this question here:
Execute Background Task In Javascript
Haven't used that for myself, just used the also mentioned thread usage.
function doSomethingHeavy(param){
if (param && param%100==0)
alert(param);
}
(function heavyWork(){
for (var i=0; i<=300; i++){
window.setTimeout(
(function(i){ return function(){doSomethingHeavy(i)}; })(i)
,0);
}
}())
There is a feature called requestIdleCallback (pretty recently adopted by most larger platforms) where you can run a function that will only execute when no other function takes up the event loop, which means for less important heavy work you can execute it safely without ever impacting the main thread (given that the task takes less than 16ms, which is one frame. Otherwise work has to be batched)
I wrote a function to execute a list of actions without impacting main thread. You can also pass a shouldCancel callback to cancel the workflow at any time. It will fallback to setTimeout:
export const idleWork = async (
actions: (() => void)[],
shouldCancel: () => boolean
): Promise<boolean> => {
const actionsCopied = [...actions];
const isRequestIdleCallbackAvailable = "requestIdleCallback" in window;
const promise = new Promise<boolean>((resolve) => {
if (isRequestIdleCallbackAvailable) {
const doWork: IdleRequestCallback = (deadline) => {
while (deadline.timeRemaining() > 0 && actionsCopied.length > 0) {
actionsCopied.shift()?.();
}
if (shouldCancel()) {
resolve(false);
}
if (actionsCopied.length > 0) {
window.requestIdleCallback(doWork, { timeout: 150 });
} else {
resolve(true);
}
};
window.requestIdleCallback(doWork, { timeout: 200 });
} else {
const doWork = () => {
actionsCopied.shift()?.();
if (shouldCancel()) {
resolve(false);
}
if (actionsCopied.length !== 0) {
setTimeout(doWork);
} else {
resolve(true);
}
};
setTimeout(doWork);
}
});
const isSuccessful = await promise;
return isSuccessful;
};
The above will execute a list of functions. The list can be extremely long and expensive, but as long as every individual task is under 16ms it will not impact main thread. Warning because not all browsers supports this yet, but webkit does
Related
This is a post that might come across as quite conceptual, since I first start with a lot of pseudo code. - At the end you'll see the use case for this problem, though a solution would be a "tool I can add to my tool-belt of useful programming techniques".
The problem
Sometimes one might need to create multiple promises, and either do something after all promises have ended. Or one might create multiple promises, based on the results of the previous promises. The analogy can be made to creating an array of values instead of a single value.
There are two basic cases to be considered, where the number of promises is indepedented of the result of said promises, and the case where it is depedent. Simple pseudo code of what "could" be done.
for (let i=0; i<10; i++) {
promise(...)
.then(...)
.catch(...);
}.then(new function(result) {
//All promises finished execute this code now.
})
The basically creates n (10) promises, and the final code would be executed after all promises are done. Of course the syntax isn't working in javascript, but it shows the idea. This problem is relativelly easy, and could be called completely asynchronous.
Now the second problem is like:
while (continueFn()) {
promise(...)
.then(.. potentially changing outcome of continueFn ..)
.catch(.. potentially changing outcome of continueFn ..)
}.then(new function(result) {
//All promises finished execute this code now.
})
This is much more complex, as one can't just start all promises and then wait for them to finish: in the end you'll have to go "promise-by-promise". This second case is what I wish to figure out (if one can do the second case you can also do the first).
The (bad) solution
I do have a working "solution". This is not a good solution as can probably quickly be seen, after the code I'll talk about why I dislike this method. Basically instead of looping it uses recursion - so the "promise" (or a wrapper around a promise which is a promise) calls itself when it's fulfilled, in code:
function promiseFunction(state_obj) {
return new Promise((resolve, reject) => {
//initialize fields here
let InnerFn = (stateObj) => {
if (!stateObj.checkContinue()) {
return resolve(state_obj);
}
ActualPromise(...)
.then(new function(result) {
newState = stateObj.cloneMe(); //we'll have to clone to prevent asynchronous write problems
newState.changeStateBasedOnResult(result);
return InnerFn(newState);
})
.catch(new function(err) {
return reject(err); //forward error handling (must be done manually?)
});
}
InnerFn(initialState); //kickstart
});
}
Important to note is that the stateObj should not change during its lifetime, but it can be really easy. In my real problem (which I'll explain at the end) the stateObj was simply a counter (number), and the if (!stateObj.checkContinue()) was simply if (counter < maxNumber).
Now this solution is really bad; It is ugly, complicated, error prone and finally impossible to scale.
Ugly because the actual business logic is buried in a mess of code. It doesn't show "on the can" that is actually simply doing what the while loop above does.
Complicated because the flow of execution is impossible to follow. First of all recursive code is never "easy" to follow, but more importantly you also have to keep in mind thread safety with the state-object. (Which might also have a reference to another object to, say, store a list of results for later processing).
It's error prone since there is more redundancy than strictly necessary; You'll have to explicitly forward the rejection. Debugging tools such as a stack trace also quickly become really hard to look through.
The scalability is also a problem at some points: this is a recursive function, so at one point it will create a stackoverflow/encounter maximum recursive depth. Normally one could either optimize by tail recursion or, more common, create a virtual stack (on the heap) and transform the function to a loop using the manual stack. In this case, however, one can't change the recursive calls to a loop-with-manual-stack; simply because of how promise syntax works.
The alternative (bad) solution
A colleague suggested an alternative approach to this problem, something that initially looked much less problematic, but I discarded ultimatelly since it was against everything promises are meant to do.
What he suggested was basically looping over the promises as per above. But instead of letting the loop continue there would be a variable "finished" and an inner loop that constantly checks for this variable; so in code it would be like:
function promiseFunction(state_obj) {
return new Promise((resolve, reject) => {
while (stateObj.checkContinue()) {
let finished = false;
let err = false;
let res = null;
actualPromise(...)
.then(new function(result) {
res = result;
finished = true;
})
.catch(new function(err) {
res = err;
err = true;
finished = true;
});
while(!finished) {
sleep(100); //to not burn our cpu
}
if (err) {
return reject(err);
}
stateObj.changeStateBasedOnResult(result);
}
});
}
While this is less complicated, since it's now easy to follow the flow of execution. This has problems of its own: not for the least that it's unclear when this function will end; and it's really bad for performance.
Conclusion
Well this isn't much to conclude yet, I'd really like something as simple as in the first pseudo code above. Maybe another way of looking at things so that one doesn't have the trouble of deeply recursive functions.
So how would you rewrite a promise that is part of a loop?
The real problem used as motivation
Now this problem has roots in a real thing I had to create. While this problem is now solved (by applying the recursive method above), it might be interesting to know what spawned this; The real question however isn't about this specific case, but rather on how to do this in general with any promise.
In a sails app I had to check a database, which had orders with order-ids. I had to find the first N "non existing order-ids". My solution was to get the "first" M products from the database, find the missing numbers within it. Then if the number of missing numbers was less than N get the next batch of M products.
Now to get an item from a database, one uses a promise (or callback), thus the code won't wait for the database data to return. - So I'm basically at the "second problem:"
function GenerateEmptySpots(maxNum) {
return new Promise((resolve, reject) => {
//initialize fields
let InnerFn = (counter, r) => {
if (r > 0) {
return resolve(true);
}
let query = {
orderNr: {'>=': counter, '<': (counter + maxNum)}
};
Order.find({
where: query,
sort: 'orderNr ASC'})
.then(new function(result) {
n = findNumberOfMissingSpotsAndStoreThemInThis();
return InnerFn(newState, r - n);
}.bind(this))
.catch(new function(err) {
return reject(err);
});
}
InnerFn(maxNum); //kickstart
});
}
EDIT:
Small post scriptus: the sleep function in the alternative is just from another library which provided a non-blocking-sleep. (not that it matters).
Also, should've indicated I'm using es2015.
The alternative (bad) solution
…doesn't actually work, as there is no sleep function in JavaScript. (If you have a runtime library which provides a non-blocking-sleep, you could just have used a while loop and non-blocking-wait for the promise inside it using the same style).
The bad solution is ugly, complicated, error prone and finally impossible to scale.
Nope. The recursive approach is indeed the proper way to do this.
Ugly because the actual business logic is buried in a mess of code. And error-prone as you'll have to explicitly forward the rejection.
This is just caused by the Promise constructor antipattern! Avoid it.
Complicated because the flow of execution is impossible to follow. Recursive code is never "easy" to follow
I'll challenge that statement. You just have to get accustomed to it.
You also have to keep in mind thread safety with the state-object.
No. There is no multi-threading and shared memory access in JavaScript, if you worry about concurrency where other stuff affects your state object while the loop runs that will a problem with any approach.
The scalability is also a problem at some points: this is a recursive function, so at one point it will create a stackoverflow
No. It's asynchronous! The callback will run on a new stack, it's not actually called recursively during the function call and does not carry those stack frames around. The asynchronous event loop already provides the trampoline to make this tail-recursive.
The good solution
function promiseFunction(state) {
const initialState = state.cloneMe(); // clone once for this run
// initialize fields here
return (function recurse(localState) {
if (!localState.checkContinue())
return Promise.resolve(localState);
else
return actualPromise(…).then(result =>
recurse(localState.changeStateBasedOnResult(result))
);
}(initialState)); // kickstart
}
The modern solution
You know, async/await is available in every environment that implemented ES6, as all of them also implemented ES8 now!
async function promiseFunction(state) {
const localState = state.cloneMe(); // clone once for this run
// initialize fields here
while (!localState.checkContinue()) {
const result = await actualPromise(…);
localState = localState.changeStateBasedOnResult(result);
}
return localState;
}
Let’s begin with the simple case: You have N promises that all do some work, and you want to do something when all the promises have finished. There’s actually a built-in way to do exactly that: Promise.all. With that, the code will look like this:
let promises = [];
for (let i=0; i<10; i++) {
promises.push(doSomethingAsynchronously());
}
Promise.all(promises).then(arrayOfResults => {
// all promises finished
});
Now, the second call is a situation you encounter all the time when you want to continue doing something asynchronously depending on the previous asynchronous result. A common example (that’s a bit less abstract) would be to simply fetch pages until you hit the end.
With modern JavaScript, there’s luckily a way to write this in a really readable way: Using asynchronous functions and await:
async function readFromAllPages() {
let shouldContinue = true;
let pageId = 0;
let items = [];
while (shouldContinue) {
// fetch the next page
let result = await fetchSinglePage(pageId);
// store items
items.push.apply(items, result.items);
// evaluate whether we want to continue
if (!result.items.length) {
shouldContinue = false;
}
pageId++;
}
return items;
}
readFromAllPages().then(allItems => {
// items have been read from all pages
});
Without async/await, this will look a bit more complicated, since you need to manage all this yourself. But unless you try to make it super generic, it shouldn’t look that bad. For example, the paging one could look like this:
function readFromAllPages() {
let items = [];
function readNextPage(pageId) {
return fetchSinglePage(pageId).then(result => {
items.push.apply(items, result.items);
if (!result.items.length) {
return Promise.resolve(null);
}
return readNextPage(pageId + 1);
});
}
return readNextPage(0).then(() => items);
}
First of all recursive code is never "easy" to follow
I think the code is fine to read. As I’ve said: Unless you try to make it super generic, you can really keep it simple. And naming also helps a lot.
but more importantly you also have to keep in mind thread safety with the state-object
No, JavaScript is single-threaded. You doing things asynchronously but that does not necessarily mean that things are happening at the same time. JavaScript uses an event loop to work off asynchronous processes, where only one code block runs at a single time.
The scalability is also a problem at some points: this is a recursive function, so at one point it will create a stackoverflow/encounter maximum recursive depth.
Also no. This is recursive in the sense that the function references itself. But it will not call itself directly. Instead it will register itself as a callback when an asynchronous process finishes. So the current execution of the function will finish first, then at some point the asynchronous process finishes, and then the callback will eventually run. These are (at least) three separate steps from the event loop, which all run independently from another, so you do no have a problem with recursion depth here.
The crux of the matter seems to be that "the actual business logic is buried in a mess of code".
Yes it is ... in both solutions.
Things can be separated out by :
having an asyncRecursor function that simply knows how to (asynchronously) recurse.
allowing the recursor's caller(s) to specify the business logic (the terminal test to apply, and the work to be performed).
It is also better to allow caller(s) to be responsible for cloning the original object rather than resolver() assuming cloning always to be necessary. The caller really needs to be in charge in this regard.
function asyncRecursor(subject, testFn, workFn) {
// asyncRecursor orchestrates the recursion
if(testFn(subject)) {
return Promise.resolve(workFn(subject)).then(result => asyncRecursor(result, testFn, workFn));
// the `Promise.resolve()` wrapper safeguards against workFn() not being thenable.
} else {
return Promise.resolve(subject);
// the `Promise.resolve()` wrapper safeguards against `testFn(subject)` failing at the first call of asyncRecursor().
}
}
Now you can write your caller as follows :
// example caller
function someBusinessOrientedCallerFn(state_obj) {
// ... preamble ...
return asyncRecursor(
state_obj, // or state_obj.cloneMe() if necessary
(obj) => obj.checkContinue(), // testFn
(obj) => somethingAsync(...).then((result) => { // workFn
obj.changeStateBasedOnResult(result);
return obj; // return `obj` or anything you like providing it makes a valid parameter to be passed to `testFn()` and `workFn()` at next recursion.
});
);
}
You could theoretically incorporate your terminal test inside the workFn but keeping them separate will help enforce the discipline, in writers of the business-logic, to remember to include a test. Otherwise they will consider it optional and sure as you like, they will leave it out!
Sorry, this doesn't use Promises, but sometimes abstractions just get in the way.
This example, which builds from #poke's answer, is short and easy to comprehend.
function readFromAllPages(done=function(){}, pageId=0, res=[]) {
fetchSinglePage(pageId, res => {
if (res.items.length) {
readFromAllPages(done, ++pageId, items.concat(res.items));
} else {
done(items);
}
});
}
readFromAllPages(allItems => {
// items have been read from all pages
});
This has only a single depth of nested functions. In general, you can solve the nested callback problem without resorting to a subsystem that manages things for you.
If we drop the parameter defaults and change the arrow functions, we get code that runs in legacy ES3 browsers.
JavaScript is a single threaded language and therefore it executes one command at a time. Asynchronous programming is being implemented via Web APIs (DOM for event handling, XMLHttpRequest for AJAX calls, WindowTimers for setTimeout) and the Event queue which are managed by the browser. So far, so good! Consider now, the following very simple code:
$('#mybox').hide(17000);
console.log('Previous command has not yet terminated!');
...
Could someone please explain to me the underlying mechanism of the above? Since .hide() has not yet finished (the animation lasts 17 seconds) and JS engine is dealing with it and it is capable of executing one command at a time, in which way does it go to the next line and continues to run the remaining code?
If your answer is that animation creates promises, the question remains the same: How JavaScript is dealing with more than one thing at the same time (executing the animation itself, watching the animation queue in case of promises and proceeding with the code that follows...).
Moreover, I cannot explain how promises in jQuery work if they have to watch their parent Deferred object till it is resolved or rejected that means code execution and at the same time the remaining code is executed. How is that possible in a single threaded approach? I have no problem to understand AJAX calls for I know they are taken away from JS engine...
tl;dr; it would not be possible in a strictly single threaded environment without outside help.
I think I understand your issue. Let's get a few things out of the way:
JavaScript is always synchronous
No asynchronous APIs are defined in the language specification. All the functions like Array.prototype.map or String.fromCharCode always run synchronously*.
Code will always run to completion. Code does not stop running until it is terminated by a return, an implicit return (reaching the end of the code) or a throw (abruptly).
a();
b();
c();
d(); // the order of these functions executed is always a, b, c, d and nothing else will
// happen until all of them finish executing
JavaScript lives inside a platform
The JavaScript language defines a concept called a host environment:
In this way, the existing system is said to provide a host environment of objects and facilities, which completes the capabilities of the scripting language.
The host environment in which JavaScript is run in the browser is called the DOM or document object model. It specifies how your browser window interacts with the JavaScript language. In NodeJS for example the host environment is entirely different.
While all JavaScript objects and functions run synchronously to completion - the host environment may expose functions of its own which are not necessarily defined in JavaScript. They do not have the same restrictions standard JavaScript code has and may define different behaviors - for example the result of document.getElementsByClassName is a live DOM NodeList which has very different behavior from your ordinary JavaScript code:
var els = document.getElementsByClassName("foo");
var n = document.createElement("div");
n.className = "foo";
document.body.appendChild(n);
els.length; // this increased in 1, it keeps track of the elements on the page
// it behaves differently from a JavaScript array for example.
Some of these host functions have to perform I/O operations like schedule timers, perform network requests or perform file access. These APIs like all the other APIs have to run to completion. These APIs are by the host platform - they invoke capabilities your code doesn't have - typically (but not necessarily) they're written in C++ and use threading and operating system facilities for running things concurrently and in parallel. This concurrency can be just background work (like scheduling a timer) or actual parallelism (like WebWorkers - again part of the DOM and not JavaScript).
So, when you invoke actions on the DOM like setTimeout, or applying a class that causes CSS animation it is not bound to the same requirements your code has. It can use threading or operating system async io.
When you do something like:
setTimeout(function() {
console.log("World");
});
console.log("Hello");
What actually happens is:
The host function setTimeout is called with a parameter of type function. It pushes the function into a queue in the host environment.
the console.log("Hello") is executed synchronously.
All other synchronous code is run (note, the setTimeout call was completely synchronous here).
JavaScript finished running - control is transferred to the host environment.
The host environment notices it has something in the timers queue and enough time has passed so it calls its argument (the function) - console.log("World") is executed.
All other code in the function is run synchronously.
Control is yielded back to the host environment (platform).
Something else happens in the host environment (mouse click, AJAX request returning, timer firing). The host environment calls the handler the user passed to these actions.
Again all JavaScript is run synchronously.
And so on and so on...
Your specific case
$('#mybox').hide(17000);
console.log('Previous command has not yet terminated!');
Here the code is run synchronously. The previous command has terminated, but it did not actually do much - instead it scheduled a callback on the platform a(in the .hide(17000) and then executed the console.log since again - all JavaScirpt code runs synchronously always.
That is - hide performs very little work and runs for a few milliseconds and then schedules more work to be done later. It does not run for 17 seconds.
Now the implementation of hide looks something like:
function hide(element, howLong) {
var o = 16 / howLong; // calculate how much opacity to reduce each time
// ask the host environment to call us every 16ms
var t = setInterval(function
// make the element a little more transparent
element.style.opacity = (parseInt(element.style.opacity) || 1) - o;
if(parseInt(element.style.opacity) < o) { // last step
clearInterval(t); // ask the platform to stop calling us
o.style.display = "none"; // mark the element as hidden
}
,16);
}
So basically our code is single threaded - it asks the platform to call it 60 times a second and makes the element a little less visible each time. Everything is always run to completion but except for the first code execution the platform code (the host environment) is calling our code except for vice versa.
So the actual straightforward answer to your question is that the timing of the computation is "taken away" from your code much like in when you make an AJAX request. To answer it directly:
It would not be possible in a single threaded environment without help from outside.
That outside is the enclosing system that uses either threads or operating system asynchronous facilities - our host environment. It could not be done without it in pure standard ECMAScript.
* With the ES2015 inclusion of promises, the language delegates tasks back to the platform (host environment) - but that's an exception.
You have several kind of functions in javascript:
Blocking and non blocking.
Non blocking function will return immediately and the event loop continues execution while it work in background waiting to call the callback function (like Ajax promises).
Animation relies on setInterval and/or setTimeout and these two methods return immediately allowing code to resume. The callback is pushed back into the event loop stack, executed, and the main loop continues.
Hope this'll help.
You can have more information here or here
Event Loop
JavaScript uses what is called an event loop. The event loop is like a while(true) loop.
To simplify it, assume that JavaScript has one gigantic array where it stores all the events. The event loop loops through this event loop, starting from the oldest event to the newest event. That is, JavaScript does something like this:
while (true) {
var event = eventsArray.unshift();
if (event) {
event.process();
}
}
If, during the processing of the event (event.process), a new event is fired (let's call this eventA), the new event is saved in the eventsArray and execution of the current continues. When the current event is done processing, the next event is processed and so on, until we reach eventA.
Coming to your sample code,
$('#mybox').hide(17000);
console.log('Previous command has not yet terminated!');
When the first line is executed, an event listener is created and a timer is started. Say jQuery uses 100ms frames. A timer of 100ms is created, with a callback function. The timer starts running in the background (the implementation of this is internal to the browser), while the control is given back to your script. So, while the timer is running in the background, your script continues to line two. After 100ms, the timer finishes, and fires an event. This event is saved in the eventsArray above, it does not get executed immediately. Once your code is done executing, JavaScript checks the eventsArray and sees that there is one new event, and then executes it.
The event is then run, and your div or whatever element it is moves a few pixels, and a new 100ms timer starts.
Please note that this is a simplification, not the actual working of the whole thing. There are a few complications to the whole thing, like the stack and all. Please see the MDN article here for more info.
Could someone please explain to me the underlying mechanism of the
above? Since .hide() has not yet finished (the animation lasts 17
seconds) and JS engine is dealing with it and it is capable of
executing one command at a time, in which way does it go to the next
line and continues to run the remaining code?
jQuery.fn.hide() internally calls jQuery.fn.animate which calls jQuery.Animation which returns a jQuery deferred.promise() object; see also jQuery.Deferred()
The deferred.promise() method allows an asynchronous function to
prevent other code from interfering with the progress or status of its
internal request.
For description of Promise see Promises/A+ , promises-unwrapping , Basic Javascript promise implementation attempt ; also , What is Node.js?
jQuery.fn.hide:
function (speed, easing, callback) {
return speed == null || typeof speed === "boolean"
? cssFn.apply(this, arguments)
: this.animate(genFx(name, true), speed, easing, callback);
}
jQuery.fn.animate:
function animate(prop, speed, easing, callback) {
var empty = jQuery.isEmptyObject(prop),
optall = jQuery.speed(speed, easing, callback),
doAnimation = function () {
// Operate on a copy of prop so per-property easing won't be lost
var anim = Animation(this, jQuery.extend({},
prop), optall);
// Empty animations, or finishing resolves immediately
if (empty || jQuery._data(this, "finish")) {
anim.stop(true);
}
};
doAnimation.finish = doAnimation;
return empty || optall.queue === false ? this.each(doAnimation) : this.queue(optall.queue, doAnimation);
}
jQuery.Animation:
function Animation(elem, properties, options) {
var result, stopped, index = 0,
length = animationPrefilters.length,
deferred = jQuery.Deferred().always(function () {
// don't match elem in the :animated selector
delete tick.elem;
}),
tick = function () {
if (stopped) {
return false;
}
var currentTime = fxNow || createFxNow(),
remaining = Math.max(0, animation.startTime + animation.duration - currentTime),
// archaic crash bug won't allow us to use 1 - ( 0.5 || 0 ) (#12497)
temp = remaining / animation.duration || 0,
percent = 1 - temp,
index = 0,
length = animation.tweens.length;
for (; index < length; index++) {
animation.tweens[index].run(percent);
}
deferred.notifyWith(elem, [animation, percent, remaining]);
if (percent < 1 && length) {
return remaining;
} else {
deferred.resolveWith(elem, [animation]);
return false;
}
},
animation = deferred.promise({
elem: elem,
props: jQuery.extend({},
properties),
opts: jQuery.extend(true, {
specialEasing: {}
},
options),
originalProperties: properties,
originalOptions: options,
startTime: fxNow || createFxNow(),
duration: options.duration,
tweens: [],
createTween: function (prop, end) {
var tween = jQuery.Tween(elem, animation.opts, prop, end, animation.opts.specialEasing[prop] || animation.opts.easing);
animation.tweens.push(tween);
return tween;
},
stop: function (gotoEnd) {
var index = 0,
// if we are going to the end, we want to run all the tweens
// otherwise we skip this part
length = gotoEnd ? animation.tweens.length : 0;
if (stopped) {
return this;
}
stopped = true;
for (; index < length; index++) {
animation.tweens[index].run(1);
}
// resolve when we played the last frame
// otherwise, reject
if (gotoEnd) {
deferred.resolveWith(elem, [animation, gotoEnd]);
} else {
deferred.rejectWith(elem, [animation, gotoEnd]);
}
return this;
}
}),
props = animation.props;
propFilter(props, animation.opts.specialEasing);
for (; index < length; index++) {
result = animationPrefilters[index].call(animation, elem, props, animation.opts);
if (result) {
return result;
}
}
jQuery.map(props, createTween, animation);
if (jQuery.isFunction(animation.opts.start)) {
animation.opts.start.call(elem, animation);
}
jQuery.fx.timer(
jQuery.extend(tick, {
elem: elem,
anim: animation,
queue: animation.opts.queue
}));
// attach callbacks from options
return animation.progress(animation.opts.progress).done(animation.opts.done, animation.opts.complete).fail(animation.opts.fail).always(animation.opts.always);
}
When .hide() is called , a jQuery.Deferred() is created that processes the animation tasks.
This is the reason console.log() is called.
If include start option of .hide() can review that .hide() begins before console.log() is called on next line, though does not block the user interface from performing asynchronous tasks.
$("#mybox").hide({
duration:17000,
start:function() {
console.log("start function of .hide()");
}
});
console.log("Previous command has not yet terminated!");
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js">
</script>
<div id="mybox">mybox</div>
Native Promise implementation
function init() {
function $(id) {
return document.getElementById(id.slice(1))
}
function hide(duration, start) {
element = this;
var height = parseInt(window.getComputedStyle(element)
.getPropertyValue("height"));
console.log("hide() start, height", height);
var promise = new Promise(function(resolve, reject) {
var fx = height / duration;
var start = null;
function step(timestamp) {
if (!start) start = timestamp;
var progress = timestamp - start;
height = height - fx * 20.5;
element.style.height = height + "px";
console.log(height, progress);
if (progress < duration || height > 0) {
window.requestAnimationFrame(step);
} else {
resolve(element);
}
}
window.requestAnimationFrame(step);
});
return promise.then(function(el) {
console.log("hide() end, height", height);
el.innerHTML = "animation complete";
return el
})
}
hide.call($("#mybox"), 17000);
console.log("Previous command has not yet terminated!");
}
window.addEventListener("load", init)
#mybox {
position: relative;
height:200px;
background: blue;
}
<div id="mybox"></div>
One of my javascript function is processing millions of data and it is called ~1 time every second from a hardware event. Then the web browser is idle in that function processing.
I tried to set a flag for running (or not running) that function:
if (!is_calculating)
is_calculating = true;
else
return;
my_function(); // do heavy stuff
is_calculating = false;
but it's not working, because it is entering into the code and the web browser enter in an idle status until is finishing. When it is returning, the flag is always OK, because it finished the // do heavy stuff
Can I do something for this behavior? I'd like to jump function execution if a flag is set.
The problem is, by default javascript runs in a single thread on browsers, so your code is executing completely before it even begins to process the next call, resulting in is_calculating always being false when the function is called. One workaround you could use (not the cleanest solution in the world), is to divide your monolithic 'heavy stuff' function into a number of smaller functions and have them call each other with setTimeout(nextFunc, 1). Having them call each other that way gives the browser a moment to do what it needs to do, and additionally call your function again if that's what it's doing. This time, because your function is called in the 'middle' of it already being executed, is_calculating is still going to be true, and the call will return at the beginning like you expect it to. Note this solution probably isn't as preferable as the Web Workers solution, but it is simpler.
function sleep(millis) {
var date = new Date()
var curDate = null
do { curDate = new Date() }
while(curDate-date < millis)
}
function reallyLong() {
if(!reallyLong.flag) {
reallyLong.flag = true
} else {
console.log("Not executing")
return
}
sleep(250)
setTimeout(reallyLong2, 1)
function reallyLong2() {
sleep(250)
setTimeout(reallyLong3, 1)
}
function reallyLong3() {
sleep(250)
setTimeout(reallyLong4, 1)
}
function reallyLong4() {
sleep(250)
console.log("executed")
reallyLong.flag = false
}
}
If you define all your consecutive functions inside the primary function, it also allows them all to access the same data simply and easily.
The only catch now is if your function was returning some value, you need to rewrite it to either return a promise (Either of your own design or using a library like Q), or accept a callback as a parameter that the last function in the 'chain' will call with the return value as a parameter.
Note that the sleep function above is a hack, and awful, and terrible, and should never be used.
By default JavaScript execution in browsers is not concurrent. This means, usually there can be only one currently executing piece of code.
You have to use Web Workers API to make you code run concurrently.
Supposed, I have a async function in Node.js, basically something such as:
var addAsync = function (first, second, callback) {
setTimeout(function () {
callback(null, first + second);
}, 1 * 1000);
};
Now of course I can call this function in an asynchronous style:
addAsync(23, 42, function (err, result) {
console.log(result); // => 65
});
What I am wondering about is whether you can make it somehow to call this function synchronously. For that, I'd like to have a wrapper function sync, which basically does the following thing:
var sync = function (fn, params) {
var res,
finished = false;
fn.call(null, params[0], params[1], function (err, result) {
res = result;
finished = true;
});
while (!finished) {}
return res;
};
Then, I'd be able to run addAsync synchronously, by calling it this way:
var sum = sync(addAsync, [23, 42]);
Note: Of course you wouldn't work using params[0] and params[1] in reality, but use the arguments array accordingly, but I wanted to keep things simple in this example.
Now, the problem is, that the above code does not work. It just blocks, as the while loop blocks and does not release the event loop.
My question is: Is it possible in any way to make this sample run as intended?
I have already experimented with setImmediate and process.nextTick and various other things, but non of them helped. Basically, what you'd need was a way to tell Node.js to please pause the current function, continue running the event loop, and getting back at a later point in time.
I know that you can achieve something similar using yield and generator functions, at least in Node.js 0.11.2 and above. But, I'm curious whether it works even without?
Please note that I am fully aware of how to do asynchronous programming in Node.js, of the event loop and all the related stuff. I am also fully aware that writing code like this is a bad idea, especially in Node.js. And I am also fully aware that an 'active wait' is a stupid idea, as well. So please don't give the advice to learn how to do it asynchronously or something like that. I know that.
The reason why I am asking is just out of curiosity and for the wish to learn.
I've recently created simpler abstraction WaitFor to call async functions in sync mode (based on Fibers). It's at an early stage but works. Please try it: https://github.com/luciotato/waitfor
using WaitFor your code will be:
console.log ( wait.for ( addAsync,23,42 ) ) ;
You can call any standard nodejs async function, as if it were a sync function.
wait.for(fn,args...) fulfills the same need as the "sync" function in your example, but inside a Fiber (without blocking node's event loop)
You can use npm fibers (C++ AddOn project) & node-sync
implement a blocking call in C(++) and provide it as a library
Yes I know-you know - BUT EVER-EVER-EVER ;)
Non-Blocking) use a control flow library
Standard propmises and yield (generator functions) will make this straigthforward:
http://blog.alexmaccaw.com/how-yield-will-transform-node
http://jlongster.com/A-Study-on-Solving-Callbacks-with-JavaScript-Generators
In Javascript, I have two versions of a recursive function, one that runs synchronously and one that uses simple scheduling to run asynchronously. Given certain inputs, in both cases the function is expected to have an infinite execution path. I need to develop tests for these functions, specifically a test to check that the asynchronous version does not block the main thread.
I already have tests that check the output callback behavior of these functions in non-returning cases, I am only concerned about testing the blocking behavior. I can limit how long the function runs to some long but finite amount of time for testing purposes as well. I am currently using QUnit but can switch to another testing framework.
How can I test that a non-returning, asynchronous function does not block?
Edit, For Clarification
This would be a bare bones example of the function I am working with:
function a()
{
console.log("invoked");
setTimeout(a, 1000);
}
a();
I am intentionally misusing some threading terms in my description because I felt they most clearly expressed the problem. By not blocking the main thread, I mean that invoking the function does not prevent the scheduling and execution of other logic. I expect the function itself will be executed on the main thread but I consider the function running as long as it is scheduled for execution in the future.
Unit Test are based on single-responsability-principle and isolation (separate the subject under test from it's dependencies).
In this case, you expect your function to run asynchronously but this behaviour is not done by your function, is done by the "setTimeout" function, so I think you must isolate your function from "setTimeout" since it's a dependency you don't want to test, the browser guarantees you it will work.
Then, as we trust "setTimeout" will do the asyncrhonous logic, we can only test our function calls to "setTimeout" and we can do this replacing "window.setTimeout" with another function while we must always restore it after the test is complete.
function replaceSetTimeout() {
var originalSetTimeout = window.setTimeout;
var callCount = 0;
window.setTimeout = function() {
callCount++;
};
window.setTimeout.restore = function() {
window.setTimeout = originalSetTimeout;
};
window.setTimeout.getCallCount = function() {
return callCount;
};
}
replaceSetTimeout();
asyncFunction();
assert(setTimeout.getCallCount() === 1);
setTimeout.restore();
I recommend you to use sinon.js as it provides many tools like spies who are functions than will inform you about how many times and with what arguments where called.
var originalSetTimeout = window.setTimeout;
window.setTimeout = sinon.spy();
asyncFunction();
// check called only once
assert(setTimeout.calledOnce);
// check the first argument was asyncFunction
assert(setTimeout.calledWith(asyncFunction));
Sinon also provides fake timers who does the setTimeout substitution but with so much more features, like the .tick(x) method who will simulate "x" milliseconds but in this case I think it doesn't help you.
Update to answer question edit:
1 - Your function executes infinitely so you cannot test it without interrupting it's execution, so you must overwrite "setTimeout" somewhere.
2 - You want your function to execute recursively allowing other code to be executed between iterations? great! but understand than your function can not do this your function only can call setTimeout or setInterval and hope this function work as expected. You should test what your function does.
3 - You want to test from Javascript (a sandboxed environment) than another Javascript code uses and releases the only one execution thread (the same you are using to test). Do you really think this is an easy test?
4 - but the most important one - I don't like white box because it couples the test with the dependency, if you change your dependency or how it's called in the future you will have to change the test. This problem doesn't exist with DOM function, DOM functions will keep the same interface for years, and for now, you have no other way to do what you want than calling one of those two functions, so I don't think in this case "white box testing" is a bad idea.
I told you this because I had the same problem testing a Promise pattern implementation than had to be always asynchronous, even if the promise is already fulfilled, and I've tested it using test-engine asynchronous-test way (using callbacks and stuff) and it was a mess, test failing randomly, so much slow test execution. Then I asked a TDD expert how can test be so hard and he answered than I was not following Single Responsability Principle since I was trying to test my promise implementation AND the setTimeout behaviour.
If you think about it from a Behaviour Driven Testing perspective then 'Does my function block?' is not a useful question. It will definitely block, a better question might be 'does it return in no more than 50ms'.
You could do this with something like :
test( "speed test", function() {
var start = new Date();
a();
ok(new Date() - start < 50, "Passed!" );
});
The issue with this is that if someone does do something silly that makes your function block indefinitely the test won't fail, it will hang.
Because JavaScript is single threaded there is no way around this. If I come along and change your function to :
function a() {
while(true) {
console.log("invoked")
}
}
The test will hang.
You can make breaking things this way harder by refactoring things a little. There are 2 separate things being done. Your chunk of work and the scheduling. Separate these and you'll end up with something like the following functions :
function a() {
// doWork
var stopRunning = true;
return stopRunning;
}
function doAsync(workFunc, scheduleFunc, timeout) {
if (!workFunc()) {
scheduleFunc(doAsync, [workFunc, scheduleFunc, timeout], timeout);
}
}
function schedule(func, args, timeout) {
setTimeout(function() {func.apply(window, args);}, timeout);
}
Now you're free to test everything in isolation. You can supply a mock workFunc and scheduleFunc to a test for doAsync to verify it behaves as expected and you can test your function a() without worrying about how it is scheduled.
It's still possible for a dunce programmer to put an infinite loop into the function a(), but because they don't have to consider how to run further units of work it should be less likely.
To test or prove an infinitely executing execution path will never block is next to impossible, so you have to split your problem up into parts.
Your path is basically foo(foo(foo(foo(...etc...)))), nevermind that SetTimeout actually removes recursion. So all you have to do is test or prove that your foo does not block (I tell you now that testing will be "a bit" easier than proving, more below)
So, does function foo block?
Talking a bit maths, if you want to know whether f(f(...f(x)...)) always has a value, you actually only have to prove that f(x) always has a value for any x that f can return. It does not matter how many recursions you have, if you can make sure their return values are fine.
What that means for your foo is that you only have to prove that foo does not block for any possible input value. Keep in mind that in this case, all global variables and closures are input values too. This means you have to sanity-check every single value you are using on every call.
To test, of course you will have to replace SetTimeout, but that is trivial, and if you replace it with an empty function (function(){}) it is easy to prove that this function does not block or otherwise alter your execution. You will then
Making things easier
Taking in what I wrote above, this also means that you would have to make sure no global function or variable that you are ever using will ever be changed to a point that your function breaks to a point it breaks. This actually is quite hard, but you can still make things easier for you by making sure you always use the same functions and values and that other functions can not touch them by using closures.
function foo(n, setTimeout)
{
var x = global_var;
// sanity check n here
function f()
{
setTimeout(f, n)
}
return f();
}
This way, you only have to test those values on the first execution. It's nice to be able to assume Math.Pi is actually Pi and not a string value containing "noodles". Really nice.
Do not use global mutable objects
Call those you can not circumvent using setTimeout to ensure they can not block
If you need return values, things will get really tricky, but possible, consider this:
function() {
var x = 0;
setTimeout(function(){x = insecure();}, 1);
}
All you have to do is
Use x next iteration
Sanity check value of x first!
Does SetTimeout block?
Of course this depends on whether setTimeout blocks. This is quite hard to prove, but a bit easier to test. You can't actually prove it since it's implementation is up to the interpreter.
Personally I would assume that setTimeout behaves like an empty function when it's return value is discarded.
Performing this asynchronous testing is actually possible in QUnit but is handled better in another JavaScript testing framework, Jasmine JS. I'll provide examples in both.
In QUnit you need to first call the stop() function to signal that the test is expected to run asynchronously, you should then call setTimeout with a function that includes your expectations as well as a call to the start() function to complete the block. Here's an example:
test( "a test", function() {
stop();
asyncOp();
setTimeout(function() {
equals( asyncOp.result, "someExpectedValue" );
start();
}, 150 );
});
Edit: Apparently there's also a whole asyncTest construct that you can use that simplifies this process. Take a look: http://api.qunitjs.com/asyncTest/
In Jasmine (http://pivotal.github.com/jasmine/), a Behavior Driven Development (BDD) testing framework, there are built-in methods for writing asynchronous tests. Here's an example of an asynchronous test in Jasmine:
describe('Some module', function() {
it('should run asynchronously', function() {
var isDone = false;
runs(function() {
// The first call to runs should trigger some async operation
// that has a side-effect that can be tested for. In this case,
// lets say that the doSomethingAsyncWithCallback function
// does something asynchronously and then calls the passed callback
doSomethingAsyncWithCallback(function() { isDone = true; });
});
waitsFor(function() {
// The call to waits for is a polling function that will get called
// periodically until either a condition is met (the function should return
// a boolean testing for this condition) or the timeout expires.
// The optional text is what error to display if the test fails.
return isDone === true;
}, "Should set isDone to true", 500);
runs(function() {
// The second call to runs should contain any assertions you need to make
// after the async call is complete.
expect(isDone).toBe(true);
});
});
});
Edit: Also, Jasmine has several built-in methods of faking out the setTimeout and setInterval functions of the browser without hosing any other tests in your suite that may depend on that. I would take a look at using those rather than manually overriding the setTimeout/setInterval functions.
Basically, JavaScript is single-threaded, so it will block the main thread. But :
I assume you're using setTimesout to schedule your function, so it won't be noticeable to the user if calls to that function don't take too much time (say, less than 200 or 300ms).
If you're doing DOM manipulation during that function (including Canvas or WebGL), then you're screwed. But if not, you can look into Web Workers, which can spawn separate threads that are guaranteed not to block the UI.
But anyway, JavaScript and the main loop, that's a tricky issue that's been bugging me a lot these past months, so you're not alone!
As soon as your function returns (after having set the timeout for it's next run), javascript will look at the next thing that requires running and run that.
As far as I can tell, the 'main thread' in javascript is just a loop that is responding to events (such as onload for a script tag, which runs the contents of that tag).
Based on the above two conditions, the calling thread is always going to run to completion despite any setTimeouts, and those timeouts will begin after the calling thread has nothing left to run.
The way I tested this was to run the following function right after the call to a()
function looper(name,duration) {
var start = (new Date()).getTime();
var elapsed = 0;
while (elapsed < duration) {
elapsed = (new Date()).getTime() - start;
console.log(name + ": " + elapsed);
}
}
Duration should be set to some period of time longer than the setTimeout duration in a(). The expected output would be the output of 'looper', followed by the output of repeated calls to a().
The next thing to test would be whether other script tags are able to run while a() and its child calls are executing.
You can do this like so:
<script>
a();
</script>
<script>
looper('delay',500); // ie; less than the 1000 timeout in a();
</script>
<script>
console.log('OK');
</script>
You would expect 'OK' to appear in the log despite the fact that a() and its children are still executing. You can also test variations of this, such as window.onload(), etc.
Finally, you'd want to ensure that other timer events work fine as well. Simply delaying 2 calls by half a second and checking that they interleave should show that works fine:
function b()
{
console.log("invoked b")
setTimeout(b, 1000);
}
a();
looper('wait',500);
b();
Should produce output like
invoked
invoked b
invoked
invoked b
invoked
invoked b
Hope that's what you were looking for!
EDIT in case you need some technical details on how to do it in Qunit:
If Qunit can't capture console.log output (i'm not sure), just push those strings into an array or a string and check that after it's run. You could override console.log in the test module() setup and restore it at teardown. I'm not sure how Qunit works but 'this' might have to be removed and globals used to store the old_console_log and test_output
// in the setup
this.old_console_log = console.log;
this.test_output = [];
var self = this;
console.log = function(text) { self.test_output.push(text); }
// in the teardown
console.log = this.old_console_log;
Finally, you can utilize stop() and start() so that Qunit knows to wait for all the events in the test to finish running.
stop();
kickoff_async_test();
setTimeout(function(){
// assertions
start();
},<expected duration of run>);
Based on all the answers, I came up with this solution that works for my case:
testAsync("Doesn't hang", function(){
expect(1);
var ranToLong = false;
var last = new Date();
var sched = setInterval(function(){
var now = new Date();
ranToLong = ranToLong || (now - last) >= 50;
last = now;
}, 0);
// In this case, asyncRecursiveFunction runs for a long time and
// returns a single value in callback
asyncRecursiveFunction(function callback(v){
clearInterval(sched);
var now = new Date();
ranToLong = ranToLong || (now - last) >= 50;
assert.equal(ranToLong, false);
start();
});
});
It tests that 'asyncRecursiveFunction' does not hang while processing by looking at the time between another scheduled function calls.
This is really ugly and not be applicable to every case but it seems to work for me because I can restrict my function to some large set of async recursive calls so it runs for a long but not infinite time. As I mentioned in the question, I am happy proving that such cases do not block.
BTW, the actual code in question is found in gen.js. The main problem was an async reduce generator. It correctly returned a value asynchronously, but in previous versions would stall because of synchronous internal implementation.