Moving from Javascript to Python, and looking at asyncio has me a little confused.
As someone who is new to the fundamental concepts of concurrency, I just assumed a superficial understanding of Javascript concurrency.
A basic understanding from using async / await in Javascript:
If we run any processes inside an async function, and await the response of the function, we are essentially waiting for the function to set a value on the Promise.
Makes total sense - when the Promise is given a value, we can also use callbacks such as .then() to handle the response. Alternatively, just await.
Whatever the underlying implementation of asynchronicity here is (for example all processes running on a single thread with an event loop), should it matter how we interface with it?
Now, I move to Python and start playing with asyncio. We have Futures, just like Promises. All of a sudden, I can't use my standard libraries, such as request.get(...), but I need to use non blocking network requests in libraries such as aiohttp.
What does blocking / non-blocking mean here? I assume it means the single thread that the event loop is on is blocked, so we cant process other functions in parallel.
So my 2 questions then are:
What causes the single thread to be blocked? For example in requests.get(...)
Why are most functions non-blocking in Javascript, but not in Python (i.e we don't need specific libraries such as aiohttp).
And what about languages like Go with their goroutines? Is it just a case because its a new language with concurrency built in from the beginning, that the concept of blocking functions don't exist. Or in Go it's not a single thread, so everything can inherently be parallelised?
Thanks :)
Event loop
Javascript, and python's async io make use of a concurrency model based on event loops.
(Note the plural because you could have multiple event loops which handle different kinds of tasks e.g. disk io, network io, ipc, parallel computations etc)
The general concept of an event loop is that, you have a number of things to do, so you put those things in a queue, and once in a while (like every nanosecond), the event loop picks an event from the queue, and runs it for a short while (maybe a millisecond or so), and either shoves it back in the queue if it hasn't completed, or waits until it yields control back to the event loop.
Now to answer some of your questions:
What does blocking / non-blocking mean here? I assume it means the
single thread that the event loop is on is blocked, so we cant process
other functions in parallel.
Blocking event loop
Blocking the event loop occurs when the event loop is running a task, and the task has either not finished or given back control to the event-loop, for a period of time longer than the event loop has scheduled it to run.
In the case of python's requests library, they make use of a synchronous http library, which doesn't respect the event loop; Therefore, running such a task in the loop will starve other tasks which are patiently waiting their turn to run, until the request is finished.
Why are most functions non-blocking in Javascript, but not in Python
(i.e we don't need specific libraries such as aiohttp).
JS
Everything in Javascript can block the event loop. The only way not to block the event loop is to make heavy use of callbacks via setTimeout. However, if care is not taken, even those callbacks can block the event loop if they run too long without yielding control back to the event loop via another setTimeout call.
(If you've never had to use setTimeout, but have used promises and async network requests in JS, then you are probably making use of a library that does. Most of the popular networking libraries used in browsers (ajax, axios, fetch, etc), are based on the popular XMLHttpRequest API, which provides async network IO.)
Python
In python, the story is slightly different: Before asyncio, there was no such thing as as "event loop". Everything must run to completion before python interpreter moves on to the next thing. This is part of what makes python very easy to learn (and dare I say, to create...). The reason for this, comes in the form of the python GIL, which in simple terms enforces a single order of execution for any python program. I encourage you to click that link, and read why the GIL exists.
And what about languages like Go with their goroutines?
Note: I am not a go programmer, but I read something
How is Go different?
The only difference between the way go handles goroutines and how python asyncio/js do their event loops, is that go makes more use of os threads to ensure that threads are scheduled fairly and make full use of the machine they run in.
While js callbacks/asyncio tasks will often run in the same thread as the event loop, goroutines are able to run in seperate OS threads and over multiple cores, thus giving them higher availability and higher parallelism. (In that case, we could almost consider goroutines to be closer to OS threads in terms of how much time they actually get to run, as compared to green threads which are bound by the amount of time the event loop's thread runs.)
Related
In this answer to the question -
What is non-blocking or asynchronous I/O in Node.js?
the description sounds no different from the event loop in vanilla js. Is there a difference between the two? If not, is the Event loop simply re-branded as "Asynchronous non-blocking I/O" to sell Node.js over other options more easily?
The event loop is the mechanism. Asynchronous I/O is the goal.
Asynchronous I/O is a style of programming in which I/O calls do not wait for the operation to complete before returning, but merely arrange for the caller to be notified when that happens, and for the result to be returned somewhere. In JavaScript, the notification is usually performed by invoking a callback or resolving a promise. As far as the programmer is concerned, it doesn’t matter how this happens: it just does. I request the operation, and when it’s done, I get notified about it.
An event loop is how this is usually achieved. The thing is, in most JavaScript implementations, there is literally a loop somewhere that ultimately boils down to:
while (poll_event(&ev)) {
dispatch_event(&ev);
}
Performing an asynchronous operation is then done by arranging for the completion of the operation to be received as an event by that loop, and having it dispatch to a callback of the caller’s choice.
There are ways to achieve asynchronous programming not based on an event loop, for example using threads and condition variables. But historical reasons make this programming style quite difficult to realise in JavaScript. So in practice, the predominant implementation of asynchrony in JavaScript is based on dispatching callbacks from a global event loop.
Put another way, ‘the event loop’ describes what the host does, while ‘asynchronous I/O’ describes what the programmer does.
From a non-programmer’s bird’s eye view this may seem like splitting hairs, but the distinction can be occasionally important.
There are 2 different Event Loops:
Browser Event Loop
NodeJS Event Loop
Browser Event Loop
The Event Loop is a process that runs continually, executing any task queued. It has multiple task sources which guarantees execution order within that source, but the Browser gets to pick which source to take a task from on each turn of the loop. This allows Browser to give preference to performance sensitive tasks such as user-input.
There are a few different steps that Browser Event Loop checks continuously:
Task Queue - There can be multiple task queues. Browser can execute queues in any order they like. Tasks in the same queue must be executed in the order they arrived, first in - first out. Tasks execute in order, and the Browser may render between tasks. Task from the same source must go in the same queue. The important thing is that task is going to run from start to finish. After each task, Event Loop will go to Microtask Queue and do all tasks from there.
Microtasks Queue - The microtask queue is processed at the end of each task. Any additional microtasks queued during during microtasks are added to the end of the queue and are also processed.
Animation Callback Queue - The animation callback queue is processed before pixels repaint. All animation tasks from the queue will be processed, but any additional animation tasks queued during animation tasks will be scheduled for the next frame.
Rendering Pipeline - In this step, rendering will happen. The Browser gets to decide when to do this and it tried to be as efficient as possible. The rendering steps only happen if there is something actually worth updating. The majority of screens update at a set frequency, in most cases 60 times a second (60Hz). So, if we would change page style 1000 times a second, rendering steps would not get processed 1000 times a second, but instead it would synchronize itself with the display and only render up to a frequency display is capable of.
Important thing to mention are Web APIs, that are effectively threads. So, for example setTimeout() is an API provided to us by Browser. When you call setTimeout() Web API would take over and process it, and it will return the result to the main thread as a new task in a task queue.
The best video I found that describes how Event Loops works is this one. It helped me a lot when I was investigating how Event Loop works. Another great videos are this one and this one. You should definitely check all of them.
NodeJS Event Loop
NodeJS Event Loop allows NodeJS to perform non-blocking operations by offloading operation to the system kernel whenever possible. Most modern kernels are multi-threaded and they can perform multiple operations in the background. When one of these operations completes, the kernel tells NodeJS.
Library that provides the Event Loop to NodeJS is called Libuv. It will by default create something called Thread Pool with 4 threads to offload asynchronous work to. If you want, you can also change the number of threads in the Thread Pool.
NodeJS Event Loop goes through different phases:
timers - this phase executes callbacks scheduled by setTimeout() and setInterval().
pending callbacks - executes I/O callbacks deferred to the next loop iteration.
idle, prepare - only used internally.
poll - retrieve new I/O events; execute I/O related callbacks (almost all with the exception of close callbacks, the ones scheduled by timers, and setImmediate()) Node will block here when appropriate.
check - setImmediate() callbacks are invoked here.
close callbacks - some close callbacks, e.g. socket.on('close', ...).
Between each run of the event loop, Node.js checks if it is waiting for any asynchronous I/O or timers and shuts down cleanly if there are not any.
In Browser, we had Web APIs. In NodeJS, we have C++ APIs with the same rule.
I found this video to be useful if you want to check for more information.
For years, JavaScript has been limited to client-side applications
such as interactive web applications that run on the browser. Using
NodeJS, JavaScript can be used to develop server-side applications as
well. Though it’s the same programming language which is used in both
use cases, client-side and server-side have different requirements.
“Event Loop” is a generic programming pattern and JavaScript/NodeJS
event loops are no different. The event loop continuously watches for
any queued event handlers and will process them accordingly.
The “Events”, in a browser’s context, are user interactions on web
pages (e.g, clicks, mouse movements, keyboard events etc.), but in
Node’s context, events are asynchronous server-side operations (e.g,
File I/O access, Network I/O etc.)
I am a bit confused here I know Javascript is a single-threaded language but while reading about the event loop. I got to know that in case of setTimeout or setInterval javascript calls web API provided by the browser which spawns a new thread to execute timer on that thread. but what happens in the case of node.js environment with timers how do they execute/work?
No threads are used for timers in node.js.
Timers in node.js work in conjunction with the event loop and don't use a thread. Timers in node.js are stored in a sorted linked list with the next timer to fire at the start of the linked list. Each time through the event loop, it checks to see if the first timer in the linked list has reached its time. If so, it fires that timer. If not, it runs any other events that are waiting in the event loop.
On each subsequent cycle through the event loop, it keeps checking to see if its time for the next timer or not. When a new timer is added, it is inserted into the linked list in its proper sorted order. When it fires or is cancelled, it is removed from the linked list.
If the event loop has nothing to do, it may sleep for a brief period of time, but it won't sleep past the timer for the next timer.
Other references on the topic:
How does nodejs manage timers internally
Libuv timer code in nodejs
How many concurrent setTimeouts before performance issues?
Multiple Client Requests in NodeJs
Looking for a solution between setting lots of timers or using a scheduled task queue
Node runs on a single thread but asynchronous work happens elsewhere. For example, libuv provides a pool of 4 threads that it may use, but wont if there's a better option.
The node documentation says
Node.js runs JavaScript code in the Event Loop (initialization and callbacks), and offers a Worker Pool to handle expensive tasks like file I/O. Node.js scales well, sometimes better than more heavyweight approaches like Apache. The secret to the scalability of Node.js is that it uses a small number of threads to handle many clients. If Node.js can make do with fewer threads, then it can spend more of your system's time and memory working on clients rather than on paying space and time overheads for threads (memory, context-switching). But because Node.js has only a few threads, you must structure your application to use them wisely.
A more detailed look at the event loop
No. Timers are just scheduled on the same thread and will call their callbacks when the time expires.
Depending on what OS your are on and what javascript interpreters you use they will use various APIs form poll to epoll to kqueue to overlapped I/O on Windows but in general asynchronous APIs have similar features. So let's ignore platform differences and look at a cross-platform API that exists on all OSes: the POSIX select() system call.
The select function in C looks something like this:
int select(int nfds, fd_set *readfds, fd_set *writefds,
fd_set *exceptfds, struct timeval *timeout);
Where nfds is total number of file descriptors (including network sockets) you are waiting/listening on, readfds is the list/set of read file descriptors you are waiting on, writefds is the list/set of write file descriptors, exceptfds is the list/set of error file descriptors (think stderr) and timeval is the timeout for the function.
This system call blocks - yes, in non-blocking, asynchronous code there is a piece of blocking system call. The main difference between non-blocking code and blocking threaded code is that the entire program blocks in only one place, the select() function (or whatever equivalent you use).
This function only returns if any of the file descriptors have activity on them or if the timeout expires.
By managing the timeout and calculating the next value of timeval you can implement a function like setTimeout
I've written much deeper explanations of how this works in answers to the following related questions:
I know that callback function runs asynchronously, but why?
Event Queuing in NodeJS
how node.js server is better than thread based server
Node js architecture and performance
Performance of NodeJS with large amount of callbacks
Does javascript process using an elastic racetrack algorithm
Is there any other way to implement a "listening" function without an infinite while loop?
I recommend you at least browse each of the answers I wrote above because they are almost all non-duplicates. They sometimes overlap but explain different aspects of asynchronous code execution.
The gist of it is that javascript does not execute code in parallel to implement timers. It doesn't need to. Instead it waits in parallel. Once you understand the difference between running code in parallel and waiting (doing nothing) in parallel you will understand how things like node.js achieve high performance and how events work better.
I want to create a real thread which manages some operations in javascript.
After several search, i found 'Web Workers', 'setTimeout' or 'setInterval'.
The problem is that 'Web Workers' don't have access to global variables and therefore can't modify my global arrays directly (or i do not know how).
'setTimeout' is not really what i need.
'setInterval' sets my problem, however it is probably that after many times my operations could last longer. Therefore i am afraid that two interval overlaps.
Finally i need a infinite loop which executes a series of operations once after another. Does it exist or do I have to content myself with 'setInterval'? Is there an alternative with jQuery or other? If it is not, is what I can expect in the near future to see the developer make it available?
I'm going to assume you're talking about in a web browser.
JavaScript in web browsers has a single main UI thread, and then zero or more web worker threads. Web workers are indeed isolated from the main UI thread (and each other) and so don't have access to globals (other than their own). This is intentional, it makes both implementing the environment and using it dramatically simpler and less error-prone. (Even if that isolation weren't enforced, it's good practice for multi-threaded programming anyway.) You send messages to, and receive messages from, web workers via postMessage and the message event.
JavaScript threads (the main UI thread and any web workers) work via a thread-specific task queue (aka "job queue"): Anything that needs to happen on a JavaScript thread (the initial run of the code when a page loads, handling of an event, timer callbacks [more below]) adds a task to the queue. The JavaScript engine runs a loop: Pick up the next task, run it, pick up the next, run it, etc. When there are no tasks, the thread goes quiet waiting for a task to arrive.
setTimeout doesn't create a separate thread, it just schedules a task (a call to a callback) to be added to the task queue for that same thread after a delay (the timeout). Once the timeout occurs, the task is queued, and when the task reaches the front of the queue the thread will handle it.
setInterval does exactly what setTimeout does, but schedules a recurring callback: Once the timeout occurs, it queues the task, then sets up another timeout to queue the task again later. (The rules around the timing are a bit complex.)
If you just want something to recur, forever, at intervals, and you want that thing to have access to global variables in the main UI thread, then you either:
Use setInterval once, which will set up recurring calls back to your code, or
Use setTimeout, and every time you get your callback, use setTimeout again to schedule the next one.
From your description, it sounds as though you may be calling setInterval more than once (for instance, on each callback), which quickly bogs down the thread as you're constantly telling it to do more and more work.
The last thing is easy: webworker start their work when they get a message to (onmessage) and sit idle otherwise. (that's highly simplified, of course).
Global variables are not good for real multi-threading and even worse with the reduced thing JavaScript offers. You have to rewrite your workers to work standalone with only the information given.
Subworkers have a messaging system which you might be able to make good use of.
But the main problem with JavaScript is: once asynchronous always asynchronous. There is no way to "join" threads or a "wait4" or something similar. The only thing that can do both is the XMLHttprequest, so you can do it over a webserver but I doubt the lag that causes would do any good. BTW: synchronous XMLHttprequest is deprecated says Mozilla which also has a page listing all of the way where a synchronous request is necessary or at least very useful.
I am trying to choose a platform to code my network application on, it will be a small realtime online game server. I am not very familiar with the async theory though I know how to write a little asynchronous code.
I both know javascript and python, on the same level.
So I was reading on twisted here and he says:
During a callback, the Twisted loop is effectively “blocked” on our
code. So we should make sure our callback code doesn’t waste any time.
In particular, we should avoid making blocking I/O calls in our
callbacks. Otherwise, we would be defeating the whole point of using
the reactor pattern in the first place. Twisted will not take any
special precautions to prevent our code from blocking, we just have to
make sure not to do it. As we will eventually see, for the common case
of network I/O we don’t have to worry about it as we let Twisted do
the asynchronous communication for us.
I wanted to see how this is different to how the event loop on node.js is done. I believe node.js implements the event loop and it never blocks, or am I missing something?
I write somewhat blocking codes on my callbacks with node.js does this mean I'm making a mistake?
Why is twisted called async and event driven when it still blocks?
Cheers,
Maj
Twisted, node.js, and every other asynchronous framework behave the exact same way here: If you writing blocking code in your callbacks, the entire event loop is blocked until your callback is done.
Asynchronous frameworks are really great for doing I/O-bound work; the event loop never gets blocked waiting for I/O, because it can all be done in a non-blocking way. When there is data ready for reading, the event loop fires off your callback, the callback handles the data, and then the event loop takes control again. When you hear these frameworks being called "async" and "event-driven", it's referring to this non-blocking I/O + event loop model.
However, when you actually need to do some kind of processing with the data being sent/received, you need to be careful. Event loops are single-threaded; only one CPU-based operation can happen at a time. That means if you do some expensive calculation that takes 10 seconds in a callback, your event loop is blocked for 10 seconds. There's no extra magic in node.js that avoids this.
If you want to be able to do CPU-based operations without blocking your event loop, node.js (and twisted) have mechanisms for sending the CPU-bound work to a sub-process, and then fetching the results when the sub-process is complete. The node.js About page actually mentions this:
But what about multiple-processor concurrency? Aren't threads
necessary to scale programs to multi-core computers? You can start new
processes via child_process.fork() these other processes will be
scheduled in parallel. For load balancing incoming connections across
multiple processes use the cluster module.
I always hear that JavaScript is single-threaded; that when JavaScript is executed, it's all run in the same global mosh pit, all in a single thread.
While that may be true, that single execution thread may spawn new threads, asynchronousy reqeiving data back to the main thread, correct? For example, when an XMLHttpRequest is sent, doesn't the browser create a new thread that performs the HTTP transaction, then invoke callbacks back in the main thread when the XMLHttpRequest returns?
What about timers--setTimeout and setInterval? How do those work?
Is this single-threadedness the result of the language? What has stopped JavaScript from having multi-threaded execution before the new Web Workers draft?
XMLHttpRequest, notably, does not block the current thread. However, its specifics within the runtime are not outlined in any specification. It may run in a separate thread or within the current thread, making use of non-blocking I/O.
setTimeout and setInterval set timers that, when run down to zero, add an item for execution, either a line of code of a function/callback, to the execution stack, starting the JavaScript engine if code execution has stopped. In other words, they tell the JavaScript engine to do something after it has finished doing whatever it's doing currently. To see this in action, set multiple setTimeout(s) within one method and call it.
Your JavaScript itself is single-threaded. It may, however, interact with other threads in the browser (which is frequently written with something like C and C++). This is how asynchronous XHR's work. The browser may create a new thread (or it may re-use an existing one with an event loop.)
Timers and intervals will try to make your JavaScript run later, but if you have a while(1){ ; } running don't expect a timer or interval to interrupt it.
(edit: left something out.)
The single-threadedness is largely a result of the ECMA specification. There's really no language constructs for dealing with multiple threads. It wouldn't be impossible to write a JavaScript interpreter with multiple threads and the tools to interact with them, but no one really does that. Certainly no one will do it in a web browser; it would mess everything up. (If you're doing something server-side like Node.js, you'll see that they have eschewed multithreading in the JavaScript proper in favor of a snazzy event loop, and optional multi-processing.)
See this post for a description of how the javascript event queue works, including how it's related to ajax calls.
The browser certainly uses at least one native OS thread/process to handle the actual interface to the OS to retrieve system events (mouse, keyboard, timers, network events, etc...). Whether there is more than one native OS-level thread is dependent upon the browser implementation and isn't really relevant to Javascript behavior. All events from the outside world go through the javascript event queue and no event is processed until a previous javascript thread of execution is completed and the next event is then pulled from the queue given to the javascript engine.
Browser may have other threads to do the job but your Javascript code will still be executed in one thread. Here is how it would work in practice.
In case of time out, browser will create a separate thread to wait for time out to expire or use some other mechanism to implement actual timing logic. Then timeout expires, the message will be placed on main event queue that tells the runtime to execute your handler. and that will happen as soon as message is picked up by main thread.
AJAX request would work similarly. Some browser internal thread may actually connect to server and wait for the response and once response is available place appropriate message on main event queue so main thread executes the handler.
In all cases your code will get executed by main thread. This is not different from most other UI system except that browser hides that logic from you. On other platforms you may need to deal with separate threads and ensure execution of handlers on UI thread.
Putting it more simply than talking in terms of threads, in general (for the browsers I'm aware of) there will only be one block of JavaScript executing at any given time.
When you make an asynchronous Ajax request or call setTimeout or setInterval the browser may manage them in another thread, but the actual JS code in the callbacks will not execute until some point after the currently executing block of code finishes. It just gets queued up.
A simple test to demonstrate this is if you put a piece of relatively long running code after a setTimeout but in the same block:
setTimeout("alert('Timeout!');", 5);
alert("After setTimeout; before loop");
for (var i=0, x=0; i < 2000000; i++) { x += i };
alert("After loop");
If you run the above you'll see the "After setTimeout" alert, then there'll be a pause while the loop runs, then you'll see "After loop", and only after that will you see "Timeout!" - even though clearly much longer than 5ms has passed (especially if you take a while to close the first alert).
An often-quoted reason for the single-thread is that it simplifies the browser's job of rendering the page, because you don't get the situation of lots of different threads of JavaScript all trying to update the DOM at the same time.
Javascript is a language designed to be embedded. It can and has been used in programs that execute javascript concurrently on different operating threads. There isn't much demand for an embedded language to explicitly control the creation of new threads of execution, but it could certainly be done by providing a host object with the required capabilities. The WHATWG actually includes a justification for their decision not to push a standard concurrent execution capability for browsers.