I'm having problems with using async.eachLimit. It works properly for the first 10 elements, but it doesn't continue past that; it simply ends. So, if there are 100 elements, it only does the first 10. This is clearly an issue of me misunderstanding callbacks. What is the proper way of using eachLimit with an external function that does not contain a callback? Or is it required for such a function to have one?
async.eachLimit(items, 10, function(item, callback) {
outsideFunction(item.attrOne, item.attrTwo};
//callback(); ---> leads to all running in parallel.
},
function(err) {
console.log(err);
}
);
Your issue here is you're using an async library for a function that isn't asynchronous (or isn't acting like it's asynchronous). What async.eachLimit does is go over each item in the array, only executing limit amount at a time, and waits for callback() to be called saying the current iteration is finished and can add another one to be executed.
In your code example, the callback (when uncommented) gets called immediately after it tries calling outsideFunction because the function call is non-blocking. It doesn't wait because async says "I've been told it's done, I'll go onto the next one" so all 100 will try and be executed at the same time. If outsideFunction is an asynchronous function, it needs a callback (or have it use promises) to say it has finished executing, and inside that callback you call the callback for async.eachLimit and then it will only do 10 at a time in the way you want. Here's an example:
async.eachLimit(items, 10, function(item, callback)
{
outsideFunction(item.attrOne, item.attrTwo, function(someResult)
{
// Your outside function calls back saying it's finished ...
callback(); // ... so we tell async we're done
});
},
function(err)
{
console.log(err);
});
If outsideFunction isn't your function, and the function is actually asynchronous, then it's either using promises or you need to find a library that writes asynchronous functions properly. If the function isn't asynchronous, then async.eachLimit won't work.
If it's your function, you should make it send back a callback to say it's done (or use promises).
Related
I want to make the second function await the first one to complete but I can’t think of syntax to it.
function rr(){
gr1.innerText = niz.splice(niz[Math.floor(Math.random()*niz.length)],1)
gr2.innerText = niz.splice(niz[Math.floor(Math.random()*niz.length)],1)
dl1.innerText = niz.splice(niz[Math.floor(Math.random()*niz.length)],1)
dl2.innerText = niz.splice(niz[Math.floor(Math.random()*niz.length)],1)
}
function reset(){
niz = ["jd","dv","tr","ct","pt","s","ss","os","dv","dd","jj","dvv","trr","ctt","pett","ssss","sds","ds"]
}
You can implement simple callback from first function and let second function depend on the callback. This way second function will only load when first function's loading is complete.
Refer this : https://humanwhocodes.com/blog/2009/07/28/the-best-way-to-load-external-javascript/
The functions above are synchronous functions, which means that they run synchronously, and don't let the main thread continue until they finished their work. So if you call the rr function, and then the reset function, reset will wait until rr finishes executing and only after that, will start running as well.
Asynchronous functions have the keyword async before function like this:
async function myFunction() { ... };
In your code, you don't need to use asynchronous functions, as you don't make any requests from another server. Asynchronous functions are not stopping the main thread from executing, by running in the background. They are used on loading large files, gathering data, and other operations that take a lot of time.
I want to preface by saying I've viewed a lot of stackoverflow questions regarding this topic, but I haven't found any 'duplicates' per se since none of them contain solutions that would solve this specific case.
I've mainly looked at How do I return the response from an asynchronous call?, and the section on 'Promises with async/await' would work inside an asynchronous function, but the function I'm working on is not async and is part of an existing codebase that I can't change easily. I wouldn't be able to change the function to async.
The section on callbacks wouldn't work either, which is explained further below. Anyway, I'll jump into my question:
I'm editing a function (standard function, not async) in JavaScript by adding an asynchronous function call. I want to wait until the async call finishes before I return from the function (since the result of the async call needs to be included in the return value). How would I do so?
I looked into using callbacks, which would allow me to write code which is guaranteed to run only after the async call completes. However, this wouldn't interrupt the flow of the program in the original function, and the original function could still return before the callback is run. A callback would allow me to execute something sequentially after the async function, but it wouldn't allow me to wait for asynchronous call to complete at the highest level.
Example code, which wouldn't return the desired result:
function getPlayers() {
... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall()
... other code ...
allPlayers.add(outfieldPlayers)
return allPlayers // the returned value may or may not include outfield players
}
The actual problem I'm facing is a little more complicated - I'm calling the async function in each iteration of a for loop, and need to wait until all calls have completed before returning. But, I think if I can solve this simpler problem, I can solve the problem with a for loop.
Sadly, it is pretty much impossible to wait for async code in a synchronous way. This is because there is no threading in JS (most JS runtimes, but some are). So code is either synchronous or asynchronous.
Asynchronous code is possible because of the event loop. The event loop is part of the javascript runtime. It works by keeping a stack of callback functions that run when events trigger them - usually either timeout events (which you can set with setTimeout()) or IO events (which happen when you make disk or HTTP requests, or on user interaction). However, these callbacks only run when no other code is running, so only when the program is idle and all functions have returned.
This means that techniques like "spin loops" (where you just run a loop until a condition is changed by another thread) that work in threaded environments don't work because the async code won't run until the spin loop finishes.
More Info: https://medium.com/front-end-weekly/javascript-event-loop-explained-4cd26af121d4
If you are using NodeJS, this is possible through execSync.
This requires you to place your asynchronous code in a separate file, spawn a separate process using execSync, which will wait until it exits.
For example, consider the following async function which prints an array.
// task.js
(async function() {
await new Promise((resolve) => setTimeout(() => {
console.log(JSON.stringify([3,4,5]));
resolve();
}, 1000));
})();
Now, you can invoke this from your main process:
function asyncGetOutfieldPlayersCall() {
const execSync = require('child_process').execSync;
return JSON.parse(execSync("node task.js"));
}
function getPlayers() {
let allPlayers = [1,2];
// ... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall();
// ... other code ...
allPlayers = allPlayers.concat(outfieldPlayers)
return allPlayers;
}
I've read Why do I have to use await for a method to run asynchronously. What if I don't want to wait for the method to finish before continuing? and Are callbacks always asynchronous? and am still trying to understand when a callback is actually asynchronous.
For example, doThat will need to wait for GET data before doing anything with it. And as the second link above states, javascript is single threaded.
doThis(doThat);
function doThis(callback) {
$.get('http://some_api/some_resource', function (data) {
callback(data);
});
};
function doThat(data) {
// Do something with data
};
The only truly async functionality I've seen is with promises and multiple promises where I can, for example, load other data while animations are wrapping up. I'd like help better understanding when traditional callbacks are actually asynchronous. Concrete examples help.
Starting with the definition:
Asynchrony, in computer programming, refers to the occurrence of events independently of the main program flow and ways to deal with such events. These may be "outside" events such as the arrival of signals, or actions instigated by a program that take place concurrently with program execution, without the program blocking to wait for results.
-- Davies, Alex (2012). Async in C# 5.0, via Wikipedia on Asynchrony (computer programming)
In case of JavaScript, it works like this (simplified): There is a queue of tasks waiting to be executed by the main (and only) thread. When you load a script, it is a task placed on this queue. A task runs until it exits, and no other task can interrupt it. However, a task can cause other tasks to be placed on the queue. When one task finishes, the next task on the queue starts. If the queue is empty, the first task that enters the queue afterwards gets immediately executed.
The main ways for tasks to enter the queue, besides being the main task of script being parsed and executed: triggering an event will place handlers registered for that event on the task queue, and reaching a time scheduled by setTimeout or setInterval will place the associated task on the task queue.
In JavaScript context, everything that executes within the same task ("main program flow") is said to be synchronous. Everything that is executed in a future task is called asynchronous. Another way to tell is - if the next statement executes before the callback, it is asynchronous; if the next statement executes after the callback, it is synchronous.
$.get(opts, callback) is an asynchronous call, because the task that executes the function callback will be placed on the task queue when triggered by the onreadystatechange event. If you have a statement following it, it will get executed first; it is only after that task finishes, that the task which entered the task queue because of an update to your AJAX call will have a chance to run.
In contrast, $.each(collection, callback) is a synchronous call, because callback will be directly called from the each function, without exiting the current task, and will not generate any additional tasks (not by itself anyway, callback of course can generate additional tasks). The next statement will have to wait until each finishes iterating over every element of collection.
Note that promises are simply wrappers over this mechanism. Everything you can do with promises, you can do without them, just the code will not be as pretty and legible.
I can, for example, load other data while animations are wrapping up.
This is exactly what you can do with your code. Note that you can run other code (and indeed the "running other code" is what confuses people) while waiting for doThat to execute. Exactly like your animation example:
function doThis(callback) {
$.get('http://some_api/some_resource', function (data) {
callback(data);
});
};
function doThat(function (data) {
// Do something with data
});
function doAnotherThing(function (data) {
// Do something with data
});
doThis(doThat);
// without waiting for doThat to run, you can IMMEDIATELY call:
doThis(doAnotherThing);
// without waiting for either doThat or doAnotherThing you
// can output some pretty animations:
setInterval(do_animation,100);
// and without waiting for any of the above you can do anything else:
document.body.innerHTML += 'hello';
Note that what confuses most people is that the document.body.innerHTML='hello' part runs BEFORE any of the code above it. Which means that the callbacks are asynchronous because they get executed when data arrives, not when the function is called.
Note that not all callbacks are asynchronous. For example, Array.prototype.forEach() is synchronous:
document.body.innerHTML += 'before<br>';
['hello','world'].forEach(function(x){
document.body.innerHTML += x;
})
document.body.innerHTML += '<br>after';
What determines weather a callback is asynchronous or synchronous is the function that calls it. See also: I know that callback function runs asynchronously, but why?
I am new to node.js and relatively new to javascript. I have understood how callbacks works and wanted to try out a function myself. Here is my code:
MyScript.js:
var calledfunction = function()
{
console.log("This is a called function");
for(i=0;i<1090660;i++)
{
console.log(i);
}
console.log('done');
};
var sayHello = require('./sayhello.js');
objhello = new sayHello();
objhello.setupSuite(1,calledfunction);
console.log('Next statement;');
sayhello.js
var _ = require('underscore');
module.exports = exports = CLITEST;
function CLITEST(param1,param2)
{
}
_.extend(CLITEST.prototype, {
setupSuite: function (here,callback) {
console.log(here);
console.log('This is a callback function');
callback();
}
})
The above program is run by executing > node Myscript.js
My question is : the for loop consumes 50 secs to execute and print all the numbers in the console and then only executes the line "Next statement" which is outside the callback function .
Why is this happening? because I read theories saying that the immediate statements will be executed without having to wait for the function to get executed.
The ideal output should have been : print " Next statement" and then print the contents of the for loop
but in the above case it is vice versa ?
This is not a real callback, but rather a simple function call. Function calls are obviously synchronous, as the following statements may rely on their return values.
In order to may the callback async you can use: setTimeout or setImmediate, depending on the actual use case.
See also: Are all Node.js callback functions asynchronous?
As pointed out by one of the commenters, your code is executed in a synchronous fashion. The function calls are executed one after the other, thus no magic is happening and the console.log('Next statement;'); call is executed after the execution of the callback. If you were in a situation in which you had to call a function which executed an asynchronous call (i.e., an AJAX call with a callback) then yes, you would expect the subsequent console.log to be executed right after the asynchronous call.
In other words, in your case the code represents a simple function call, while an asynchronous call would offload the computation of the callback somewhere else, thus the execution of the code where the callback function was called keeps going by printing the statement and won't wait for the execution of the callback.
I'm attempting to use async.js to manage callbacks in an application. Below is an example of that code. However, I've run into a problem where the series of functions continues executing before one of the prior functions completes. For example, the resizeImage() function is still working when saveImageData() fires. But my intention is for the image to be saved only after it is resized. Shouldn't async.js handle this? If I have to pass a callback to resizeImage() what is the value of using async.js? Can someone show how using an async library is helpful in the example that I've given?
if (req.files) {
//removed some code not relevant to the question
async.series([
function (callback) {
model.saveImageData(tempPath, originalImage, s3Headers);
callback(null);
},
function (callback) {
var src = tempPath;
dst = path.dirname(tempPath) + "\\" + newImage;
model.resizeImage(src, dst);
callback(null);
},
function (callback) {
//the next function gets called before resizeImage() finishes executing
model.saveImageData(dst, newImage, s3Headers);
callback(null);
}
]);
}
Most likely saveImageData and resizeImage are non-blocking, asynchronous calls. Given they're doing I/O.
So you call resizeImage, it starts processing, returns control while it waits for IO, and your code immediately invokes the async.js callback that indicates that the function has finished it's work.
Instead, you should only call the async callback parameter after saveImageData and resizeImage complete, which means creating and using a callback function:
// this is just a guess at the api.
async.series[function(callback) {
model.resizeImage(src,dst,function() {
callback(null);
});
}];
You still have to ensure that the code inside each async function invokes the callback at the appropriate time.