I have heard a lot about how JS is single threaded and asynchronous, and I know about the event loop, and the callback queue.
What I don't understand is how can a single threaded language be listening to events and adding event handlers to the queue while it executes other code?
For example clicking a button while a long loop is running will add its callback to the queue even though the thread is occupied.
Thank You
The answer is that the OS (or Javascript, for code-generated events) sends events to the JS in the browser, and these events are put on a queue for for processing. Javascript's one thread takes events off the queue and processes them until there are none left, at which time it waits for the next event to come in. For more, read about message queues: https://en.wikipedia.org/wiki/Message_queue
Simply put, browsers are not written in javascript, and javascript is not what handles events.
UI events are first messaged to the browser by the OS, then the browser will handle these messages and queue a task to build up the DOM Event and dispatch it, where the related JavaScript callbacks may finally fire.
Only the last part involves JavaScript. You can very well have an User Agent (UA) that doesn't implement JavaScript but still does handle some UI events (e.g CSS pointer-events, HTML inputs, media controls etc.)
How this is handled will depend on the UA.
For very long, in the exact example you gave, browsers were unable to handle the clicks that were fired while the event loop was busy (IIRC latest IE still wasn't able to do so). It's only in modern browsers that they started to use multi-process architectures which are able to communicate (Inter-Process Communication) by sharing different message queues.
So even though the "main" renderer thread that does execute the various tasks of the event loop is busy, e.g executing js, other processes can still run in parallel and queue new tasks on the appropriate task queue.
When the renderer thread is done with its long task, a new task has been queued and it can process it.
Related
In this answer to the question -
What is non-blocking or asynchronous I/O in Node.js?
the description sounds no different from the event loop in vanilla js. Is there a difference between the two? If not, is the Event loop simply re-branded as "Asynchronous non-blocking I/O" to sell Node.js over other options more easily?
The event loop is the mechanism. Asynchronous I/O is the goal.
Asynchronous I/O is a style of programming in which I/O calls do not wait for the operation to complete before returning, but merely arrange for the caller to be notified when that happens, and for the result to be returned somewhere. In JavaScript, the notification is usually performed by invoking a callback or resolving a promise. As far as the programmer is concerned, it doesn’t matter how this happens: it just does. I request the operation, and when it’s done, I get notified about it.
An event loop is how this is usually achieved. The thing is, in most JavaScript implementations, there is literally a loop somewhere that ultimately boils down to:
while (poll_event(&ev)) {
dispatch_event(&ev);
}
Performing an asynchronous operation is then done by arranging for the completion of the operation to be received as an event by that loop, and having it dispatch to a callback of the caller’s choice.
There are ways to achieve asynchronous programming not based on an event loop, for example using threads and condition variables. But historical reasons make this programming style quite difficult to realise in JavaScript. So in practice, the predominant implementation of asynchrony in JavaScript is based on dispatching callbacks from a global event loop.
Put another way, ‘the event loop’ describes what the host does, while ‘asynchronous I/O’ describes what the programmer does.
From a non-programmer’s bird’s eye view this may seem like splitting hairs, but the distinction can be occasionally important.
There are 2 different Event Loops:
Browser Event Loop
NodeJS Event Loop
Browser Event Loop
The Event Loop is a process that runs continually, executing any task queued. It has multiple task sources which guarantees execution order within that source, but the Browser gets to pick which source to take a task from on each turn of the loop. This allows Browser to give preference to performance sensitive tasks such as user-input.
There are a few different steps that Browser Event Loop checks continuously:
Task Queue - There can be multiple task queues. Browser can execute queues in any order they like. Tasks in the same queue must be executed in the order they arrived, first in - first out. Tasks execute in order, and the Browser may render between tasks. Task from the same source must go in the same queue. The important thing is that task is going to run from start to finish. After each task, Event Loop will go to Microtask Queue and do all tasks from there.
Microtasks Queue - The microtask queue is processed at the end of each task. Any additional microtasks queued during during microtasks are added to the end of the queue and are also processed.
Animation Callback Queue - The animation callback queue is processed before pixels repaint. All animation tasks from the queue will be processed, but any additional animation tasks queued during animation tasks will be scheduled for the next frame.
Rendering Pipeline - In this step, rendering will happen. The Browser gets to decide when to do this and it tried to be as efficient as possible. The rendering steps only happen if there is something actually worth updating. The majority of screens update at a set frequency, in most cases 60 times a second (60Hz). So, if we would change page style 1000 times a second, rendering steps would not get processed 1000 times a second, but instead it would synchronize itself with the display and only render up to a frequency display is capable of.
Important thing to mention are Web APIs, that are effectively threads. So, for example setTimeout() is an API provided to us by Browser. When you call setTimeout() Web API would take over and process it, and it will return the result to the main thread as a new task in a task queue.
The best video I found that describes how Event Loops works is this one. It helped me a lot when I was investigating how Event Loop works. Another great videos are this one and this one. You should definitely check all of them.
NodeJS Event Loop
NodeJS Event Loop allows NodeJS to perform non-blocking operations by offloading operation to the system kernel whenever possible. Most modern kernels are multi-threaded and they can perform multiple operations in the background. When one of these operations completes, the kernel tells NodeJS.
Library that provides the Event Loop to NodeJS is called Libuv. It will by default create something called Thread Pool with 4 threads to offload asynchronous work to. If you want, you can also change the number of threads in the Thread Pool.
NodeJS Event Loop goes through different phases:
timers - this phase executes callbacks scheduled by setTimeout() and setInterval().
pending callbacks - executes I/O callbacks deferred to the next loop iteration.
idle, prepare - only used internally.
poll - retrieve new I/O events; execute I/O related callbacks (almost all with the exception of close callbacks, the ones scheduled by timers, and setImmediate()) Node will block here when appropriate.
check - setImmediate() callbacks are invoked here.
close callbacks - some close callbacks, e.g. socket.on('close', ...).
Between each run of the event loop, Node.js checks if it is waiting for any asynchronous I/O or timers and shuts down cleanly if there are not any.
In Browser, we had Web APIs. In NodeJS, we have C++ APIs with the same rule.
I found this video to be useful if you want to check for more information.
For years, JavaScript has been limited to client-side applications
such as interactive web applications that run on the browser. Using
NodeJS, JavaScript can be used to develop server-side applications as
well. Though it’s the same programming language which is used in both
use cases, client-side and server-side have different requirements.
“Event Loop” is a generic programming pattern and JavaScript/NodeJS
event loops are no different. The event loop continuously watches for
any queued event handlers and will process them accordingly.
The “Events”, in a browser’s context, are user interactions on web
pages (e.g, clicks, mouse movements, keyboard events etc.), but in
Node’s context, events are asynchronous server-side operations (e.g,
File I/O access, Network I/O etc.)
Last week I started learning Javascript back-end with nodejs. While working with async functions, I wanted to understand this in all aspect and started doing research on this subject.
I found Jake Archibald's presentation in jsconf and I tried to understand what an event loop is and how event loop runs in different queues and which queues can it handle.
I think one of the diagrams in this presentation is very good to understand all of the stuff.
But I am wondering how the event loop handle job queues which consist of promise callbacks.
Can we add another route into this diagram for this queue?
. (I tried to implement it into the diagram :) )
And another question that I wonder is, what does the event loop do with the render queue?
Does the event loop send it into another place? Because we know that the event loop sends functions in the event queue or job queue to the call stack in the javascript engine to run these.
But the render queue has request animation frame and other style elements.
Is event the loop sending the request animation frame part to the javascript engine and send the other layout, style and paint part to layout engine?
This diagram is an over-simplification of how the things work, to see in details what happens, I invite you to go directly to the specs, which have become actually quite readable and easily navigable these days.
From there you'll see that
[t]he microtask-queue is not a task queue.
If we take the over-simplification road ourselves to explain the event loop routine with only what we're interested in,
messages from other processes and previous tasks will queue new tasks in various task-sources, themselves ending in fewer task-queues. (This feeds the left loop of Jake's diagram).
at each iteration, the first step of the event-loop is to pick the first task from one of these task-queues (chosen as they wish, this allows task prioritization).
after this main task is completed (step 7 in the specs), the event loop will look at the microtask queue in what is called a microtask-checkpoint.
only for the Documents where the active display monitor did emit its SYNC pulse since the last iteration (once every 16.67ms on a 60Hz monitor), it updates the rendering (right loop in Jake's diagram, steps 11.6~11.15 in the specs).
During these steps it will execute a few tasks like firing UI events, updating animations and run the animation frame callbacks. Every time one of these algorithm invokes a callback, the user-agent has to perform a new microtask-checkpoint as per the
clean up after running algorithm, so for instance every animation frame callback will get interleaved with such a checkpoint, and some of these algorithms even perform the microtask-checkpoint directly.
So this means that the microtask-checkpoint is not just executed in a single point in the event loop, it is executed once after the main task, and many times after each callback execution in the update the rendering steps.
The user-agent can not just delay the microtask, it has to execute it right after the task that queued it did complete, i.e, from the event loop perspective, microtasks are executed synchronously.
In the same way, the "render queue" is not a task-queue either, it has to run when the browser has a "rendering opportunity", it can't be part of the task prioritization mechanism (more here).
Because we know that the event loop sends functions in the event queue or job queue to the call stack in the javascript engine to run these.
Not exactly. First remember that javascript execution is only a part of what the browser does, many tasks don't involve javascript at all, for instance, even if you do disable the javascript from your web-browser, the event loop still has to run, it still has to update the rendering, it still has to send forms, it still has to react to network events like the loading of a media etc.
What the event loop does is to run the task's steps, but these can be a lot of different things and they may not necessarily send anything anywhere.
Also, specs don't say at all how computation should be distributed, but regarding the rendering, we can assume that in a nutshell everything from update the rendering until its 14th step should be done on the same process (it has to be ran sequentially), but the step 15 ("update the rendering"), which will actually paint everything to the monitor can be (and I think it is in all modern browsers) passed to an other process, dedicated to this rendering task.
You can check Chromium's documentation explaining how they manage the inter-communication between the browser process and the renderer.
In order to dive into more complex concepts about Node.js, I am doing some research to make sure I understand the principles about the language and the basic building blocks it´s build upon.
As far as I´m concerned, Node.js relies in the reactor pattern to process each incoming request. This algorithm is based in the Event Demultiplexer algorithm. The second one is in charge of processing the request operation and it´s resources and then, once the operation is finished, it adds the 'result' to the Event Queue. After this process, the Event Loop is now in charge of executing the Handlers for each event.
As a single threaded process, I´m struggling to understand how does the Event demultiplexer handle all the incoming operations if the event loop is managing all the event queue tasks in parallel...
I´ve found this in the Node.js documentation:
Since most modern kernels are multi-threaded, they can handle multiple
operations executing in the background. When one of these operations
completes, the kernel tells Node.js so that the appropriate callback
may be added to the poll queue to eventually be executed. We'll
explain this in further detail later in this topic.
Does this mean that the Event Demultiplexer tasks are not handled by Node and that Node just gets the result in order to add it to the Event Queue? Doesn´t that mean that Node is not single threaded?
I've found some useful graphics on the web that clearly explain the path that each request follows, but It´s really hard to find some explaining the actual timing in which the thread processes each one.
In node.js, it runs your Javascript in a single thread (assuming we're not talking about worker threads or clustering). But, many asynchronous operations in the node.js built-in library such as file I/O or some crypto operations use their own threads to accomplish their task. So, when you call an asynchronous operations such as fs.open() to open a file, it transitions to native code, grabs a thread from an internal thread pool and that thread goes about opening the file. The fs.open() function then returns back to your Javascript (while the thread continues in the background). Sometime later when it finishes its task, the internal thread inserts an event into the nodejs event queue and when nodejs has cycles to check the event queue, it will find that event and run the Javascript callback associated with it to provide the asynchronous result back to your Javascript.
So, even though threads may be involved, your Javascript is all still single threaded through the event loop.
So, nodejs does use native code threads for some internal native code operations. Other things like networking and timers are implemented without threads.
Then if I send a request to fetch data from a database in Node, how does the Event Demultiplexer process It? Does the main thread stop the event loop to handle that operation and then resumes after the event handler is queued?
These terms like "Event Demultiplexor" are things you are applying to node.js. They are not something that node.js uses at all so I'm not entirely sure what you're asking.
Node.js runs one event at a time. It has no interrupt-driven abilities. So, it runs one event until that event returns control back to the event loop (by issuing a return from the callback that started everything). Now, the original event may not be complete - it may be doing something asynchronous that will trigger another event to announce completion, but it has finished running Javascript for now and has returned control back to the event loop.
When an incoming Fetch request (which is just an incoming http request) arrives at the server, it is first queued by the OS. Then, when the event loop has a chance to see it, it is added to the node.js event queue and is served in FIFO order when other events that were before it are done processing. Now, the nodejs event loop is not as simple as a single event queue. It actually has several different queues for different types of work and has priorities for what gets to run first, but at the simplest level, you can start out thinking of it as a single FIFO queue. And, nothing is ever interrupted.
Pull an event out of the event queue, run the Javascript callback associated with it. When that callback returns back to the event loop, get the next event and do the same.
Events may be added to the event queue either by native code threads (like might be done with file I/O or some crypto operations) or via some polling mechanisms built into the event loop as part of the event loop cycle it checks for certain things that are ready to run (as with networking and timers).
I am wondering what are the conditions under which node.js terminates the event loop. How does node.js figure out that no further events are going to be triggered ? For e.g. in the case of an http client or a file reading application.
i'd recommend you watch this great videoPhilip Roberts: What the heck is the event loop anyway? | JSConf EU 2014
To explain briefly, Javascript runtime can do one thing at a time, thus it is called "single threaded". However, you have the reason that we can do things concurrently like HTTP requests .. etc. is due what he calls on the video as "Browser APIs" for the browser and for a backend environment like Node.js it is the C++ APIs.
Now, any of these "external" API calls like the HTTP client call, is pushed to a task queue. The event loop has one very simple job. It will always take one job from the task queue given that the main callstack is empty and push that to the execution/running stack. So the event loop will terminate when there are no jobs in the task queue (external API calls)
I recommend watching the full video for better understanding :)
Right now my mobile web app on startup goes and talks to a few apis when it is lunched. These are intertwined with the javascript that renders the page.
When they fire at the same time page rendering gets all choppy, possibly because the rendering is using hardware acceleration, or maybe it's just normal on mobile (iphone) when running too much JS at the same time.
I want to architect the app in a way that if something like a user taking an actions to change the rendering of the page, or anything related to the UI is fired, it takes precedence over any other JS.
I want the experience to be quick and snappy even if it makes some action (talking with apis) take longer then it should.
Any idea how to achieve something like this?
Thanks!!!
A given javascript thread runs to completion without interruption as Javascript is single threaded (except for Web Workers, but I don't think that's what we're talking about here).
As such, the only way to prioritize work is to create a queue of work to be done. Do a unit of work from the queue and then decide which unit of work on the queue to do next. If all your work to be done is in the queue, then you can decide which items should be done first when it's time to grab the next unit of work off the queue.
The smaller/shorter the units of work are, the more granular your switching between tasks can be. If you use setTimeout() with a very short time between each item in the queue, that gives a chance for any UI events (like clicks or other timers) to run.
You can even fire off a bunch of ajax requests and as the responses come in, you can put the responses in the queue to be parsed/handled when there's time.
The reason for the choppy behavior is that you have a limit of how many Ajax requests can be outstanding.
You could write a javascript function / object that manages a priority queue of Ajax requests along with a callback for each. Requests are fired off in order of priority / position in the queue. New requests with high priority are executed before less important ones this way even if they may have been requested later, something like the jquery Ajax Manager plugin + priorities added.