Can someone please explain the mechanics behind the asynchronous behavior of javascript, specifically when it comes to a .subscribe(), or provide a link that would help me understand what's really going on under the hood?
Everything appears to be single-threaded, then I run into situations that make it either multi-threaded, or seem multi-threaded, because I still can't shake this feeling I'm being fooled.
Everything seems to run perfectly synchronously, one instruction at a time, until I call .subscribe(). Then, suddenly, something happens - it returns immediately, and somehow the next outside function runs even though the subscription code has not yet. This feels asynchronous to me, how does the browser keep things straight - if not threads, can someone please explain the mechanics in play that is allowing this to occur?
I also notice something happens with debug stepping, I can't step in to the callback, I can only put a breakpoint within it. In what order should I really expect things to occur?
Your Javascript itself runs single threaded unless you're talking about WebWorkers (in the browser) or WorkerThreads (in nodejs).
But, many functions you call or operations you initiate in Javascript can run separate from that single threaded nature of your Javascript. For example, networking is all non-blocking and asynchronous. That means that when you make a call to fetch() in the browser of http.get() in nodejs, you are initiating an http request and then the mechanics of making that request and receiving the response are done independent of your Javascript. Here are some example steps:
You make a call to fetch() in the browser to make an http request to some other host.
The code behind fetch, sends out the http request and then immediately returns.
Your Javascript continues to execute (whatever you had on the lines of code immediately after the fetch() call will execute.
Sometime later, the response from the fetch() call arrives back on the network interface. Some code (internal to the Javascript environment) and independent from your single thread of Javascript sees that there's incoming data to read from the http response. When that code has gathered the http response, it inserts an event into the Javascript event queue.
When the single thread of Javascript has finished executing whatever event it was executing, it checks the event queue to see if there's anything else to do. At some point, it will find the event that signified the completion of the previous fetch() call and there will be a callback function associated with that event. It will call that callback which will resolve the promise associated with the fetch() call which will cause your own Javascript in the .then() handler for that promise to run and your Javascript will be presented the http response you were waiting for.
Because these non-blocking, asynchronous operations (such as networking, timers, disk I/O, etc...) all work in this same manner and can proceed independent of your Javascript execution, they can run in parallel with your Javascript execution and many of them can be in flight at the same time while your Javascript is doing other things. This may sometimes give the appearance of multiple threads and there likely are some native code threads participating in some of this, though there is still only one thread of your Javascript (as long as we're not talking about WebWorkers or WorkerThreads).
The completion of these asynchronous operations is then synchronized with the single thread of Javascript via the event queue. When they finish, they place an event in the event queue and when the single thread of Javascript finishes what it was last doing, it grabs the next event from the event queue and processes it (calling a Javascript callback).
I also notice something happens with debug stepping, I can't step in to the callback, I can only put a breakpoint within it. In what order should I really expect things to occur?
This is because the callback isn't called synchronously. When you step over the function that you passed a callback to, it executes that function and it returns. Your callback is not yet called. So, the debugger did exactly what you asked it to do. It executed that function and that function returned. The callback has not yet executed. Your are correct that if you want to see the asynchronous callback execute, you need to place a breakpoint in it so when it does get called some time in the future, the debugger will then stop there and let you step further. This is a complication of debugging asynchronous things, but you get used to it after awhile.
Everything appears to be single-threaded, then I run into situations that make it either multi-threaded, or seem multi-threaded, because I still can't shake this feeling I'm being fooled.
This is because there can be many asynchronous operations in flight at the same time as they aren't running your Javascript while they are doing their thing. So, there's still only one thread of Javascript, even though there are multiple asynchronous operations in flight. Here's a code snippet:
console.log("starting...");
fetch(host1).then(result1 => { console.log(result1)});
fetch(host2).then(result2 => { console.log(result2)});
fetch(host3).then(result3 => { console.log(result3)});
fetch(host4).then(result4 => { console.log(result4)});
console.log("everything running...");
If you run this, you will see something like this in the console:
starting...
everything running...
result1
result2
result3
result4
Actually, the 4 results may be in any order relative to one another, but they will be after the first two lines of debug output. In this case, you started four separate fetch() operations and they are running independent of your Javascript. In fact, your Javascript is free to do other things while they run. When they finish and when your Javascript isn't doing anything else, your four .then() handlers will each get called at the appropriate time.
When the networking behind each fetch() operation finishes, it will insert an event in the Javascript event queue and if your Javascript isn't doing anything at the time, the insertion of that event will wake it up and cause it to process that completion event. If it was doing something at the time, then when it finishes that particular piece of Javascript and control returns back to the system, it will see there's an event waiting in the event queue and will process it, calling your Javascript callback associated with that event.
Search for articles or videos on the “JavaScript Event Loop”. You’ll find plenty of them. Probably go through a few of them and it will start to make sense (particularly the single-threaded aspect of it)
I found this one with a quick search, and it does a good, very high-level walkthrough.
The JavaScript Event Loop explained
The main impact of JavaScript single threading is that your code is never pre-emptively interrupted to execute code on another thread.
It may seem that some code is being skipped, but close attention to the syntax will show that what is being “skipped” is just a callback function or closure to be called at a time in the future (when it will be placed into the event queue and processed single-threadedly, itself).
Careful, it can be tricky to follow nested callbacks.
Probably also look into Promises and async /await (‘just syntactic convenience on the event loop, but can really help code readability)
It seems most everyone does asynchronous requests with XMLHttpRequest but obviously the fact that there is the ability to do synchronous requests indicates there might be a valid reason to do so. So what might that valid reason be?
Synchronous XHRs are useful for saving user data. If you handle the beforeunload event you can upload data to the server as the user closes the page.
If this were done using the async option, then the page could close before the request completes. Doing this synchronously ensures the request completes or fails in an expected way.
I think they might become more popular as HTML 5 standards progress. If a web application is given access to web workers, I could foresee developers using a dedicated web worker to make synchronous requests for, as Jonathan said, to ensure one request happens before another. With the current situation of one thread, it is a less than ideal design as it blocks until the request is complete.
Update:
The below hinted at - but was unsuccessful in delivering - that with the advent of better asynchronous request handling, there really is no reason to use synchronous requests, unless intending to purposely block the users from doing anything until a request is complete - sounds malicious :)
Although, this may sound bad, there may be times where it's important that a request (or series of requests) occur before a user leaves a page, or before an action is performed - blocking other code execution (e.g., preventing back button) could possibly reduce errors/maintenance for a poorly designed system; that said, I've never seen it in the wild and stress that it should be avoided.
Libraries, like promise, feign synchronicity by chaining processes via callbacks. This suits the majority of development needs where the desire is to have ordered, non-blocking events that enable the browsers to retain responsiveness for the user (good UX).
As stated in the Mozilla docs there are cases where you have to use synchronous requests; however, also listed is a workaround that uses beacon (not available in IE/Safari) for such cases. While this is experimental, if it ever reaches standards-acceptance, it could possibly put a nail in the synchronous-request coffin.
You'd want to perform synchronous calls in any sort of transaction-like processing, or wherever any order of operation is necessary.
For instance, let's say you want to customize an event to log you out after playing a song. If the logout operation occurs first, then the song will never be played. This requires synchronizing the requests.
Another reason would be when working with a WebService, especially when performing math on the server.
Example: Server has a variable with value of 1.
Step (1) Perform Update: add 1 to variable
Step (2) Perform Update: set variable to the power of 3
End Value: variable equals 8
If Step (2) occurs first, then the end value is 2, not 8; thus order of operation matters and synchronization is needed.
There are very few times that a synchronous call may be justified in a common real world example. Perhaps when clicking login and then clicking a portion of the site that requires a user to be logged in.
As others have said, it will tie up your browser, so stay away from it where you can.
Instead of synchronous calls, though, often users want to stop an event that is currently loading and then perform some other operation. In a way this is synchronization, since the first event is quit before the second begins. To do this, use the abort() method on the xml connection object.
I'd say that if you consider blocking the user's browser while the request completes acceptable, then sure use a synchronous request.
If serialization of requests is your aim, then this can be accomplished using async requests, by having the onComplete callback of your previous request fire the next in line.
There are many real world cases where blocking the UI is exactly the desired behaviour.
Take an app with multiple fields and some fields must be validated by a xmlhttp call to a remote server providing as input this field's value and other fields values.
In synchronous mode, the logic is simple, the blocking experienced by the user is very short and there is no problem.
In async mode, the user may change the values of any other fields while the initial one is being validated. These changes will trigger other xmlhttp calls with values from the initial field not yet validated. What happens if the initial validation failed ? Pure mess. If sync mode becomes deprecated and prohibited, the application logic becomes a nightmare to handle. Basically the application has to be re-written to manage locks (eg. disable other items during validation processes). Code complexity increases tremendously. Failing to do so may lead to logic failure and ultimately data corruption.
Basically the question is: what is more important, non-blocked UI experience or risk of data corruption ? The answer should remain with the application developer, not the W3C.
I can see a use for synchronous XHR requests to be used when a resource in a variable location must be loaded before other static resources in the page that depend on the first resource to fully function. In point of fact, I'm implementing such an XHR request in a little sub-project of my own whereas JavaScript resources reside in variable locations on the server depending on a set of specific parameters. Subsequent JavaScript resources rely on those variable resources and such files MUST be guaranteed to load before the other reliant files are loaded, thus making the application whole.
That idea foundation really kind of expands on vol7ron's answer. Transaction-based procedures are really the only time where synchronous requests should be made. In most other cases, asynchronous calls are the better alternative in which, after the call, the DOM is updated as necessary. In many cases, such as user-based systems, you could have certain features locked to "unauthorized users" until they have, per se, logged in. The those features, after the asynchronous call, are unlocked via a DOM update procedure.
I'd have to finally say that I agree with most individuals' points on the matter: wherever possible, synchronous XHR requests should be avoided as, with the way it works, the browser locks up with synchronous calls. When implementing synchronous requests, they should be done in a manner where the browser would normally be locked, anyway, say in the HEAD section before page loading actually occurs.
jQuery uses synchronous AJAX internally under some circumstances. When inserting HTML that contains scripts, the browser will not execute them. The scripts need to be executed manually. These scripts may attach click handlers. Assume a user clicks on an element before the handler is attached and the page would not function as intended. Therefore to prevent race conditions, synchronous AJAX would be used to fetch those scripts. Because synchronous AJAX effectively blocks everything else, it can be sure that scripts and events execute in the right order.
As of 2015 desktop javascript apps are becoming more popular. Usually in those apps when loading local files (and loading them using XHR is a perfectly valid option), the load speed is so fast that there is little point overcomplicating the code with async. Of course there might be cases where async is the way to go (requesting content from the internet, loading really big files or a huge number of files in a single batch), but otherwise sync works just fine (and is much easier to use).
Reason:
Let's say you have an ajax application which needs to do half a dozen http gets to load various data from the server before the user can do any interaction.
Obviously you want this triggered from onload.
Synchronous calls work very well for this without any added complexity to the code. It is simple and straightforward.
Drawback:
The only drawback is that your browser locks up until all data is loaded or a timeout happens. As for the ajax application in question, this isn't much of a problem because the application is of no use until all the initial data is loaded anyway.
Alternative?
However many browsers lock up all windows/tabs when while the javascript is busy in any one of them, which is a stupid browser design problem - but as a result blocking on possibly slow network gets is not polite if it keeps users from using other tabs while waiting for ajax page to load.
However, it looks like synchronous gets have been removed or restricted from recent browsers anyway. I'm not sure if that's because somebody decided they were just always bad, or if browser writers were confused by the WC Working Draft on the topic.
http://www.w3.org/TR/2012/WD-XMLHttpRequest-20120117/#the-open-method does make it look like (see section 4.7.3) you are not allowed to set a timeout when using blocking mode. Seems counter intuitive to me: Whenever one does blocking IO it's polite to set a reasonable timeout, so why allow blocking io but not with a user specified timeout?
My opinion is that blocking IO has a vital role in some situations but must be implemented correctly. While it is not acceptable for one browser tab or window to lock up all other tabs or windows, that's a browser design flaw. Shame where shame is due. But it is perfectly acceptable in some cases for an individual tab or window to be non-responsive for a couple of seconds (i.e. using blocking IO/HTTP GET) in some situations -- for example, on page load, perhaps a lot of data needs to be before anything can be done anyway. Sometimes properly implemented blocking code is the cleanest way to do it.
Of course equivalent function in this case can be obtained using asynchronous http gets, but what sort of goofy routine is required?
I guess I would try something along these lines:
On document load, do the following:
1: Set up 6 global "Done" flag variables, initialized to 0.
2: Execute all 6 background gets (Assuming the order didn't matter)
Then, the completion callbacks for each of the 6 http get's would set their respective "Done" flags.
Also, each callback would check all the other done flags to see if all 6 HTTP gets had completed. The last callback to complete, upon seeing that all others had completed, would then call the REAL init function which would then set everything up, now that the data was all fetched.
If the order of the fetching mattered -- or if the webserver was unable to accept multiple requests at same time -- then you would need something like this:
In onload(), the first http get would be launched.
In it's callback, the second one would be launched.
In it's callback, the third -- and so on and so forth, with each callback launching the next HTTP GET. When the last one returned, then it would call the real init() routine.
What happens if you make a synchronous call in production code?
The sky falls down.
No seriously, the user does not like a locked up browser.
I use it to validate a username, during the check that the username does not exist already.
I know it would be better to do that asynchronously, but then I should use a different code for this particular validation rule. I explain better. My validation setup uses some validation functions, which return true or false, depending if the data is valid.
Since the function has to return, I cannot use asynchronous techniques, so I just make that synchronous and hope that the server will answer promptly enough not to be too noticeable. If I used an AJAX callback, then I would have to handle the rest of the execution differently from the other validation methods.
Sometimes you have an action that depends in others. For example, action B can only be started if A is finished. The synchronous approach is usually used to avoid race conditions. Sometimes using a synchronous call is a simpler implementation then creating complex logic to check every state of your asynchronous calls that depend on each other.
The problem with this approach is that you "block" the user's browser until the action is finished (until the request returns, finishes, loads, etc). So be careful when using it.
I use synchronous calls when developing code- whatever you did while the request was commuting to and from the server can obscure the cause of an error.
When it's working, I make it asynchronous, but I try to include an abort timer and failure callbacks, cause you never know...
SYNC vs ASYNC: What is the difference?
Basically it boils down to this:
console.info('Hello, World!');
doSomething(function handleResult(result) {
console.info('Got result!');
});
console.info('Goodbye cruel world!');
When doSomething is synchronous this would print:
Hello, World!
Got result!
Goodbye cruel world!
In contrast, if doSomething is asynchronous, this would print:
Hello, World!
Goodbye cruel world!
Got result!
Because the function doSomething is doing it's work asynchronously, it returns before it's work is done. So we only get the result after printing Goodbye cruel world!
If we are depending on the result of an asynch call, we need to place the depending code in the callback:
console.info('Hello, World!');
doSomething(function handleResult(result) {
console.info('Got result!');
if (result === 'good') {
console.info('I feel great!');
}
else {
console.info('Goodbye cruel world!');
}
});
As such, just the fact that 2 or three things need to happen in order is no reason to do them synchronously (though sync code is easier for most people to work with).
WHY USE SYNCHRONOUS XMLHTTPREQUEST?
There are some situations where you need the result before the called function completes. Consider this scenario:
function lives(name) {
return (name !== 'Elvis');
}
console.info('Elvis ' + (lives('Elvis') ? 'lives!' : 'has left the building...');
Suppose we have no control over the calling code (the console.info line) and need to change function lives to ask the server... There is no way we can do an async request to the server from within lives and still have our response before lives completes. So we wouldn't know whether to return true or false. The only way to get the result before the function completes is by doing a synchronous request.
As Sami Samhuri mentions in his answer, a very real scenario where you may need an answer to your server request before your function terminates is the onbeforeunload event, as it's the last function from your app that will ever run before the window being closed.
I DON'T NEED SYNCH CALLS, BUT I USE THEM ANYWAY AS THEY ARE EASIER
Please don't. Synchronous calls lock up your browser and make the app feel unresponsive. But you are right. Async code is harder. There is, however a way to make dealing with it much easier. Not as easy as sync code, but it's getting close: Promises.
Here is an example: Two asynch calls should both complete succesfully before a third segment of code may run:
var carRented = rentCar().then(function(car){
gasStation.refuel(car);
});
var hotelBooked = bookHotel().then(function(reservation) {
reservation.confirm();
});
Promise.all([carRented, hotelBooked]).then(function(){
// At this point our car is rented and our hotel booked.
goOnHoliday();
});
Here is how you would implement bookHotel:
function bookHotel() {
return new Promise(function(resolve, reject){
if (roomsAvailable()) {
var reservation = reserveRoom();
resolve(reservation);
}
else {
reject(new Error('Could not book a reservation. No rooms available.'));
}
});
}
See also: Write Better JavaScript with Promises.
XMLHttpRequest is traditionally used for asynchronous requests. Sometimes (for debugging, or specific business logic) you would like to change all/several of the async calls in one page to sync.
You would like to do it without changing everything in your JS code. The async/sync flag gives you that ability, and if designed correctly, you need only change one line in your code/change the value of one var during execution time.
Firefox (and probable all non-IE browsers) does not support async XHR timeOut.
Stackoverflow discussion
Mozilla Firefox XMLHttpRequest
HTML5 WebWorkers do support timeouts. So, you may want to wrap sync XHR request to WebWorker with timeout to implement async-like XHR with timeout behaviour.
I just had a situation where asynchronous requests for a list of urls called in succession using forEach (and a for loop) would cause the remaining requests to be cancelled. I switched to synchronous and they work as intended.
Synchronous XHR can be very useful for (non-production) internal tool and/or framework development. Imagine, for example, you wanted to load a code library synchronously on first access, like this:
get draw()
{
if (!_draw)
{
let file;
switch(config.option)
{
case 'svg':
file = 'svgdraw.js';
break;
case 'canvas':
file = 'canvasdraw.js';
break;
default:
file = 'webgldraw.js';
}
var request = new XMLHttpRequest();
request.open('GET', file, false);
request.send(null);
_draw = eval(request.responseText);
}
return _draw;
}
Before you get yourself in a tizzy and blindly regurgitate the evil's of eval, keep in mind that this is only for local testing. For production builds, _draw would already be set.
So, your code might look like this:
foo.drawLib.draw.something(); //loaded on demand
This is just one example of something that would be impossible to do without sync XHR. You could load this library up front, yes, or do a promise/callback, but you could not load the lib synchronously without sync XHR. Think about how much this type of thing could clean up your code...
The limits to what you can do with this for tooling and frameworks (running locally) is only limited by your imagination. Though, it appears imagination is a bit limited in the JavaScript world.
Using synchronous HTTP requests is a common practice in the mobile advertisement business.
Companies (aka "Publishers") that build applications often run ads to generate revenue. For this they install advertising SDKs into their app. Many exist (MoPub, Ogury, TapJob, AppNext, Google Ads AdMob).
These SDKs will serve ads in a webview.
When serving an ad to a user, it has to be a smoothe experience, especially when playing a video. There should be no buffering or loading at any moment.
To solve this precaching is used. Where the media (picture / videos / etc) are loaded synchronously in background of the webview.
Why not do it asynchronously?
This is part of a globally accepted standard
The SDK listens for the onload event to know when the ad is "ready" to be served to the user
With the deprecation of synchronous XMLHttpRequests, ad business will most likely be forced to change the standard in the future unless another way can be determined.
Well here's one good reason. I wanted to do an http request then, depending on the result, call click() on an input type=file. This is not possible with asynchronous xhr or fetch. The callback loses the context "user action", so the call click() is ignored. Synchronous xhr saved my bacon.
onclick(event){
//here I can, but I don't want to.
//document.getElementById("myFileInput").click();
fetch("Validate.aspx", { method : "POST", body: formData, credentials: "include" })
.then((response)=>response.json())
.then(function (validResult) {
if (validResult.success) {
//here, I can't.
document.getElementById("myFileInput").click();
}
});
}
Because chrome.webRequest.*.addListener does not support asynchronous handlers.