I'm working with socials API and I need to be able to run those function that return the json from the socials API endpoint in parallel, in a way that only when those function are all finished I can work on their data together.
I'm trying to work with ASync module for npm but I'm not able to understand how to make it works.
Looking at the documentation I've seen this so far:
async.parallel([
function(){ ... },
function(){ ... }
], callback);
The problems is that I didn't understand how can I use it.
I would like to do something like this below, but without timeout and with my async functions ( these function returns me the response from the APIs ).
async.series([
function(callback) {
setTimeout(function() {
console.log(“Task 1”);
callback(null, 1);
}, 300);
},
function(callback) {
setTimeout(function() {
console.log(“Task 2”);
callback(null, 2);
}, 200);
},
function(callback) {
setTimeout(function() {
console.log(“Task 3”);
callback(null, 3);
}, 100);
}
], function(error, results) {
console.log(results);
});
Cuold you please help me ?
Related
I need help understanding async.parallel(tasks, callback).
From what I understand, the list of tasks are executed, and once they're finished, all the results are then passed to callback.
An example of it in action is:
async.parallel([
function(callback) {
setTimeout(function() {
console.log('Task One');
callback(null, 1); <----- DON'T UNDERSTAND
}, 200);
},
function(callback) {
setTimeout(function() {
console.log('Task Two');
callback(null, 2); <----- DON'T UNDERSTAND
}, 100);
}
],
function(err, results) {
console.log(results);
// the results array will equal [1, 2] even though
// the second function had a shorter timeout.
});
The part that I don't understand however, are the inner calls to callback. Specifically, what does it mean when we call callback(null, 1) and callback(null, 2)? What do the null and the 1 or 2 refer to?
I was under the impression that the behavior of async.auto was that if one of the tasks returned an err, the global callback would be called with this error and all subsequent tasks would not execute. After all, why would they? The global callback has been called so the error has been reported already.
It turns out that only tasks that depend on the faulty task will not run, and the rest will.
'use strict';
var async = require('async');
async.auto({
downloadAudio: function (callback) {
console.log("downloading audio...");
setTimeout(function () {
console.log("downloaded audio.");
callback();
}, 10000);
},
downloadScript: function (callback) {
console.log("downloading script...");
setTimeout(function () {
return callback(new Error("ERROR downloading script!"));
}, 1000);
},
preprocessAudio: ['downloadAudio', function (callback) {
console.log("preprocessing audio...");
setTimeout(function () {
console.log("done preprocessing audio.");
callback();
}, 5000);
}]
}, function (err) {
if (err) {
return console.error(err.message);
}
console.log('done.');
});
In this code snippet, both downloadAudio and downloadScript will immediately run in parallel, and preprocessAudio depends on downloadAudio. The problem is that when downloadScript returns an err, preprocessAudio will still run, even though there's no point because the thing failed as a whole already:
downloading audio...
downloading script...
ERROR downloading script!
downloaded audio.
preprocessing audio...
done preprocessing audio.
I know, I could make preprocessAudio depend on downloadScript also but that means that preprocessAudio would wait for downloadScript to complete, even though they could run in parallel without a problem.
Is there any elegant way to solve this?
Thanks,
Jan
Given then that you don't want preprocessaudio to run if downloadScript has returned an error, preprocessaudio really does 'rely' on the other two functions.
Just add the dependency and all should work correctly.
'use strict';
var async = require('async');
async.auto({
downloadAudio: function (callback) {
console.log("downloading audio...");
setTimeout(function () {
console.log("downloaded audio.");
callback();
}, 10000);
},
downloadScript: function (callback) {
console.log("downloading script...");
setTimeout(function () {
return callback(new Error("ERROR downloading script!"));
}, 1000);
},
preprocessAudio: ['downloadAudio', 'downloadScript', function (callback) {
console.log("preprocessing audio...");
setTimeout(function () {
console.log("done preprocessing audio.");
callback();
}, 5000);
}]
}, function (err) {
if (err) {
return console.error(err.message);
}
console.log('done.');
});
I want to perform a set of async tasks repeatedly in a loop. I tried using async library's waterfall approach to do something like
while(somethingIsTrue) {
async.waterfall([
function(callback) { ... },
function(callback) { ... },
], function(err) {
...
}
);
}
The problem I encountered was that the the first async function never got scheduled or rather executed and the loop kept on running, obviously because the loop didnt wait for the async functions to finish. I'm not so well verse with asynchronous programming and I would need help to come up with some pattern to solve this problem.
Try a semi-recursive solution such as
function again_and_again(callback) {
if (!somethingIsTrue) return;
async.waterfall([
function(callback) { ... },
function(callback) { ... },
], function(err, result) {
if (err) return callback(err);
again_and_again(callback);
});
}
The idea is to wait for the waterfall to finish, and arrive at the callback, and repeat the "loop" there by calling again_and_again again.
I'm trying to add some mocha testing to a node module I have, but I'm new to it, and my lack of concreteness in terms of callbacks is hampering me.
I have tried to pare things back to the most straightforward example, but it's still not working.
So my main.js is
var async = require('async');
var myObject = {};
myObject.test = function(params) {
async.waterfall([
async.apply(test, params)
],
function(err, result){
if (err) {
console.log('Error: ' + err);
} else {
if (result === 200) {
return result;
}
}
});
};
function test(params, callback) {
if(params) {
callback(null, 200);
}
}
module.exports = myObject;
Then my test file
var assert = require("assert");
var myObject = require('./main');
describe('test', function(){
it("should return 200", function(done){
myObject.test({test: 'such test'}, function(err, res) {
if (err) return done(err);
assert.equal(res, 200);
done(res);
});
})
});
If I just run mocha it times out so I'm suspicious about that! Trying mocha --timeout 15000 also just stalls. Any direction you can provide would be really appreciated!
I got this far using this answer but can't get any further.
OK, I think I sorted it, but would still appreciate some feedback to see if I'm approaching it correctly, rather than just managing to get my test to pass.
var async = require('async');
var myObject = {};
myObject.test = function(params, callback) {
async.waterfall([
async.apply(test, params)
],
function(err, result){
if (err) {
console.log('Error: ' + err);
} else {
if (result === 200) {
callback(result);
}
}
});
};
function test(params, callback) {
if(params) {
callback(null, 200);
}
}
module.exports = myObject;
and the test file is
var assert = require("assert");
var myObject = require('./main');
describe('test', function(){
it("should return 200", function(done){
myObject.test({test: 'such test'}, function(res) {
assert.equal(res, 200);
done();
});
})
});
You fixed your main issue but your code still is broken. When you have an async method that takes a callback, you must always invoke the callback exactly once in all cases or your program's control flow will break. If you write an if/else clause, both branches must invoke the callback function. Both of your if statements above violate the callback contract. Check out understanding error-first-callbacks from The Node Way for a good explanation.
I'm very new to JavaScript and callbacks, so my apologies if this is a stupid question. Based on the async docs for parallel, I came up with this example code that executed the expected way based on the docs:
async = require('async')
async.parallel([
function(callback){
setTimeout(function(){
callback(null, 'one');
}, 800);
},
function(callback){
setTimeout(function(){
callback(null, 'two');
}, 100);
}
],
function(err, results){
console.log(results)
})
Running this with node foo.js prints a results array ['one', 'two'] as the docs indicate it will. The thing I don't understand is how exactly this works: when you pass callback as a parameter to each function and then callback is called as callback(null, res), what exactly is being called her? I've not actually defined any function callback, nor have I passed any sort of operating callback function as a parameter, yet everything magically works fine. Am I totally missing the point here? Or is this actually the under-the-hood magic of async?
You are not the one passing callback to the task functions, the async module is. It's a special function that the module passes to your task functions that when called, checks if any more tasks are left.
Here is something similar to what is being done inside async:
function myParallel(tasks, finalcb) {
var tasksLeft = tasks.length,
results = [],
ignore = false;
function callback(err, result) {
if (ignore) return;
if (err) {
ignore = true;
finalcb && finalcb(err);
} else if (--tasksLeft === 0) {
ignore = true;
finalcb && finalcb(null, results);
} else
results.push(result);
}
tasks.forEach(function(taskfn) {
taskfn(callback);
});
}
myParallel([
function(callback) {
setTimeout(function() {
callback(null, 'one');
}, 800);
},
function(callback) {
setTimeout(function() {
callback(null, 'two');
}, 100);
}
], function(err, results){
console.log(results)
});