I was automating a webpage using the Mocha testing framework, and came upon the terms Synchronous and Asynchronous code.
I'm familiar with synchronous and asynchronous events when you send an HTTP request...but I've never heard of code being synchronous and asynchronous.
Anyone care to explain...I saw on previous questions that it has something to do with a callback, but even then I'm still pretty confused about the concept.
Below is a simplified version of my server code. I demonstrate both synchronous code (after you start doing an operation, no further operations are begun until it finishes) and asynchronous code (you start doing an operation, then continue to do other operations, and at some later point you "callback" or get a result from the first operation.)
This has some important consequences. Nearly every time you call an async function:
the return value from your async function is useless, because the function will return immediately, although finding the result takes a long time.
you have to wait until the callback function is executed to have access to the result.
lines of code following the call to the asynchronous function will execute BEFORE the asynchronous callback function runs.
As an example, the order of the console.logs in my code below will be:
line 3 - before sync
line 8 - after sync, before async
line 16 - after async call, outside callback
line 14 - inside async call and callback
// synchronous operations execute immediately, subsequent code
// doesn't execute until the synchronous operation completes.
console.log('line 3 - before sync');
var app = require('express')();
var cfgFile = require('fs').readFileSync('./config.json');
var cfg = JSON.parse(cfgFile);
var server = require('http').createServer(app);
console.log('line 8 - after sync, before async');
// When you call an asynchronous function, something starts happening,
// and the callback function will be run later:
server.listen(cfg.port, function(){
// Do things that need the http server to be started
console.log('line 14 - inside async call and callback');
});
console.log('line 16 - after async call, outside callback');
Synchronous code will strictly be processed line by line whereas asynchronous code will continue to the next line while previous lines of code are still being processed.
For asynchronous code, in the following snippet, you would expect World to be logged to console before Hello because the database query requires more of the computer's resources and therefore takes more time. console.log('World') would finish executing before the query.
var Person = mongoose.model('Person', personSchema);
Person.findOne({ 'name.last': 'Ghost' }, function(err, person) {
if(err)
throw err;
else
console.log('Hello');
});
console.log('World');
In synchronous code, Hello will be logged before World because the query finishes executing before any following lines are executed.
Synchronous Code basically means the lines of code are executed in the order they are written. Synchronous events will immediately execute an associated callback like it would a direct function call so the end result is the same.
In general, Asynchronous Code is executed separately and concurrently from the calling code. e.g. Calling setTimeout will immediately return and execute the next expression but it will have started an action that at some point in the future it will fire an Asynchronous Event that calls the specified callback.
The Mocha test framework supports both types of code and it supplies the done() callback for asynchronous code. done() is the callback that lets Mocha know when the test complete.
Asynchronous test code, From http://mochajs.org/
describe('User', function() {
describe('#save()', function() {
it('should save without error', function(done) { // done is a callback
var user = new User('Luna');
user.save(function(err) { // save is async db call that starts here, completes whenever
if (err) throw err;
done();
});
});
});
});
Testing synchronous code doesn't require done(), the test code is expected to executed in order. So when the last expression is executed, the test is over.
Related
I want to preface by saying I've viewed a lot of stackoverflow questions regarding this topic, but I haven't found any 'duplicates' per se since none of them contain solutions that would solve this specific case.
I've mainly looked at How do I return the response from an asynchronous call?, and the section on 'Promises with async/await' would work inside an asynchronous function, but the function I'm working on is not async and is part of an existing codebase that I can't change easily. I wouldn't be able to change the function to async.
The section on callbacks wouldn't work either, which is explained further below. Anyway, I'll jump into my question:
I'm editing a function (standard function, not async) in JavaScript by adding an asynchronous function call. I want to wait until the async call finishes before I return from the function (since the result of the async call needs to be included in the return value). How would I do so?
I looked into using callbacks, which would allow me to write code which is guaranteed to run only after the async call completes. However, this wouldn't interrupt the flow of the program in the original function, and the original function could still return before the callback is run. A callback would allow me to execute something sequentially after the async function, but it wouldn't allow me to wait for asynchronous call to complete at the highest level.
Example code, which wouldn't return the desired result:
function getPlayers() {
... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall()
... other code ...
allPlayers.add(outfieldPlayers)
return allPlayers // the returned value may or may not include outfield players
}
The actual problem I'm facing is a little more complicated - I'm calling the async function in each iteration of a for loop, and need to wait until all calls have completed before returning. But, I think if I can solve this simpler problem, I can solve the problem with a for loop.
Sadly, it is pretty much impossible to wait for async code in a synchronous way. This is because there is no threading in JS (most JS runtimes, but some are). So code is either synchronous or asynchronous.
Asynchronous code is possible because of the event loop. The event loop is part of the javascript runtime. It works by keeping a stack of callback functions that run when events trigger them - usually either timeout events (which you can set with setTimeout()) or IO events (which happen when you make disk or HTTP requests, or on user interaction). However, these callbacks only run when no other code is running, so only when the program is idle and all functions have returned.
This means that techniques like "spin loops" (where you just run a loop until a condition is changed by another thread) that work in threaded environments don't work because the async code won't run until the spin loop finishes.
More Info: https://medium.com/front-end-weekly/javascript-event-loop-explained-4cd26af121d4
If you are using NodeJS, this is possible through execSync.
This requires you to place your asynchronous code in a separate file, spawn a separate process using execSync, which will wait until it exits.
For example, consider the following async function which prints an array.
// task.js
(async function() {
await new Promise((resolve) => setTimeout(() => {
console.log(JSON.stringify([3,4,5]));
resolve();
}, 1000));
})();
Now, you can invoke this from your main process:
function asyncGetOutfieldPlayersCall() {
const execSync = require('child_process').execSync;
return JSON.parse(execSync("node task.js"));
}
function getPlayers() {
let allPlayers = [1,2];
// ... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall();
// ... other code ...
allPlayers = allPlayers.concat(outfieldPlayers)
return allPlayers;
}
Here is my code
var x = 0
data.results[0].taxonomies.some(function(e){
if(taxo.indexOf(e.code)!=-1){ //if at least one code from the data base is in my list of codes
callback(validLic(data.results[0].taxonomies)) //return true after some other validations
return true
}else{
x++
return false
}
})
if(x==data.results[0].taxonomies.length){callback(false)}//if the entire array was processed, and I didn't find anything I was looking for, return false
I'd like someone to confirm that due the async nature of node, the last if statement is at some point, bound to fire off before I'm done processing the array.
How can I better manage this situation without the help of some sync or parallel library?
The reason I ask it that way is because I'm under the impression that if I can't write something to be completely async, then I'm not writing it efficiently, right?
EDIT:
Based on Luc Hendirks' logic, I have changed my code to this:
var flag = data.results[0].taxonomies.some(function(e){
if(taxo.indexOf(e.code)!=-1){ //if at least one code from the data base is in my list of codes
return true
}else{
return false
}
})
if(flag==true){
callback(validLic(data.results[0].taxonomies))
}else{
callback(false)
}
Because this follows the sync traits outlined below, I shouldn't have an issue with flag being undefined before the callback is called now right?
Javascript (and Node) are single threaded, meaning it has only 1 CPU available. If the functions you call only require CPU time, making it async is useless. If you need to call a function and the CPU has to wait (do nothing), then making it async is very useful. Because while it is waiting until the function is finished it can do something else.
A function that checks if a url is valid with a regular expression can be synchronous, as the CPU needs to do some calculations and you get the result. If the function actually does a GET request and checks the response code, the CPU has to wait until the response is received. In the meantime it could do something else, so this function should be made asynchronous.
The difference of a synchronous and asynchronous function is that a synchronous function returns a value:
function(a) { return a; }
and an asynchronous function returns the result using a callback function (this is an actual function that you put in as a function argument):
function(callback){
// Do something that takes time but not CPU, like an API call...
callback('Some result');
}
A synchronous function is called like this:
var a = something();
Asynchronous like this:
something(function(result){
console.log(result);
});
So to answer your question, if some() is an asynchronous function, then the last if statement can be executed before the some function is finished. Because: the CPU does not want to wait. It can do other stuff while waiting. That's the power of asynchronous programming. t
It is also important to know that "parallel" does not exist in Javascript/Node. There is only 'doing stuff instead of waiting', like executing multiple API calls at the same time. That is not parallel computing as in using multiple threads.
Here is some more info: What is the difference between synchronous and asynchronous programming (in node.js)
I am not an experienced developer and I have spent the last couple of days trying to understand few fundamental concepts about Javascript.
Single thread, blocking vs non-blocking and callback.
I have read a lot but I am still confused.
My understanding is that in a simple JS code all the statements are executed sequentially within a single thread.
In a more sophisticated situation where there are long running statements (db calls, network interaction, etc) this can be a huge problem.
The program execution (next statement) is BLOCKED until the current (long running) statement is completed.
Solution is to implement an asynchronous pattern, where the long running statement is executed in back ground (I can be brutally wrong here!) while the next statements are executed and when the long running statement has done, it passes back its result using a callback function provided in the calling statement.
Copy and pasting code from here and there I have written a trivial piece of code where I have a call back function
// function with callback
function doSomething(callback) {
callback();
}
// callback function implementation
function doSomethingElse() {
document.write("<p>Second element in the page</p>");
}
doSomething(doSomethingElse);
document.write("<p>First element in the page</p>");
The result of this code is actually what I would have expected before starting reading about non-blocking and callback (sequential code execution):
- Second element
- First element
So my question is, what's the magic fairy dust that transforms my code into asynchronous non-blocking one?
Cheers, Giovanni
what's the magic fairy dust that transforms my code into asynchronous non-blocking one?
In general, most long running operations will be handled APIs which handle asynchronous operations at a lower level than JavaScript (XMLHttpRequest for making HTTP requests being a prime example).
If you have need to implement a long running function yourself (maybe you want to do some heavy number crunching on the client) then you can use Web Workers.
var myWorker = new Worker("find_prime_numbers.js");
myWorker.onmessage = function(e) {
console.log('Next prime number is ' + e.data);
}
and in find_prime_numbers.js:
// Calculate some primes
postMessage(my_prime_number);
You have used callbacks, but it doesn't mean that this call is asynchronous.
It synchronously runs doSomething, which runs callback and outputs "Second element".
It would be asynchronous, if you had an asynchronous call there - AJAX call, file system access or a simple setTimeout:
function doSomething(callback) {
setTimeout(callback, 1000);
}
// callback function implementation
function doSomethingElse() {
document.write("<p>Second element in the page</p>");
}
doSomething(doSomethingElse);
document.write("<p>First element in the page</p>");
Now, it does the following: runs doSomething, which runs setTimeout and runs doSomethingElse right after that. At the same time, setTimeout asynchronously waits for 1 second and calls the function.
In other cases, it could be any other asynchronous operation, which requires time to complete.
I am new to node.js and relatively new to javascript. I have understood how callbacks works and wanted to try out a function myself. Here is my code:
MyScript.js:
var calledfunction = function()
{
console.log("This is a called function");
for(i=0;i<1090660;i++)
{
console.log(i);
}
console.log('done');
};
var sayHello = require('./sayhello.js');
objhello = new sayHello();
objhello.setupSuite(1,calledfunction);
console.log('Next statement;');
sayhello.js
var _ = require('underscore');
module.exports = exports = CLITEST;
function CLITEST(param1,param2)
{
}
_.extend(CLITEST.prototype, {
setupSuite: function (here,callback) {
console.log(here);
console.log('This is a callback function');
callback();
}
})
The above program is run by executing > node Myscript.js
My question is : the for loop consumes 50 secs to execute and print all the numbers in the console and then only executes the line "Next statement" which is outside the callback function .
Why is this happening? because I read theories saying that the immediate statements will be executed without having to wait for the function to get executed.
The ideal output should have been : print " Next statement" and then print the contents of the for loop
but in the above case it is vice versa ?
This is not a real callback, but rather a simple function call. Function calls are obviously synchronous, as the following statements may rely on their return values.
In order to may the callback async you can use: setTimeout or setImmediate, depending on the actual use case.
See also: Are all Node.js callback functions asynchronous?
As pointed out by one of the commenters, your code is executed in a synchronous fashion. The function calls are executed one after the other, thus no magic is happening and the console.log('Next statement;'); call is executed after the execution of the callback. If you were in a situation in which you had to call a function which executed an asynchronous call (i.e., an AJAX call with a callback) then yes, you would expect the subsequent console.log to be executed right after the asynchronous call.
In other words, in your case the code represents a simple function call, while an asynchronous call would offload the computation of the callback somewhere else, thus the execution of the code where the callback function was called keeps going by printing the statement and won't wait for the execution of the callback.
I am reading The Node Beginner Book. In the chapter Event-driven asynchronous callbacks, the author gives an example to illustrate the idea of asynchronous callbacks. The code example is like:
var result = database.query("SELECT * FROM hugetable");
console.log("Hello World");
After adding a callback function to database.query, the code becomes asynchronous:
database.query("SELECT * FROM hugetable", function(rows) {
var result = rows;
});
console.log("Hello World");
My question is why the database.query() function becomes asynchronous simply after adding a callback function.
I have no experience with Javascript and JQuery before, that might be the reason I cannot understand.
There are many functions in node.js that have both an asynchronous flavor and a synchronous flavor. For example, there are two ways to read the contents of a file (docs):
//asynchronous
fs.readFile("filename.txt", function(err, data) {
});
//synchronous
var data = fs.readFileSync("filename.txt");
The example the author provides does in fact look somewhat confusing, but its possible that the database.query makes an asynchronous call depending on whether a callback is passed in as the second argument.
For example, it could be implemented something like this:
function query(queryString, callback) {
if(callback !== undefined) {
queryInternal(queryString, callback);
return;
}
else {
return queryInternalSync(queryString);
}
}
In general, I think the convention is that a function is either asynchronous or synchronous (not both) so your intuition is right.
Note that in the synchronous case, console.log will be executed after result has the queried contents whereas in the asynchronous case, console.log will be executed as soon as query function returns and before callback is executed.
Asynchronously means it don't waits for the response and go to the next statements
to be executed
In your second example the callback function handles your response while executing this,
it doesn't waits and console.log("Hello World"); shows the output in console.
Read this:
http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/
http://nodemanual.org/latest/nodejs_dev_guide/writing_asynchronous_code.html
http://www.sebastianseilund.com/nodejs-async-in-practice