So I have a situation where I need to perform a bunch of http calls, then once they are complete, continue on to the next step in the process.
Below is the code which does this and works fine.
However, I now need to wait a few seconds between each of the http calls. Is there a way to pass in a timeout with my current set up, or will it involve a good bit of refactoring?
Can post more code if needs be. I have tried passing in a timeout config varable into the http call, however, they still get fired at the same time.
Any advice would be great.
Code
var allThings = array.map(function(object) {
var singleThingPromise = getFile(object.id);
return singleThingPromise;
});
$q.all(allThings).then(function() {
deferred.resolve('Finished');
}, function(error) {
deferred.reject(error);
});
Instead of using $q.all, you might want to perform sequential calls one on success of previous and probably with use of $timeout. Maybe you could build a recursive function.
Something like this..
function performSequentialCalls (index) {
if(angular.isUndefined(array[index])) {
return;
}
getFile(array[index].id).then(function() {
$timeout(function() {
performSequentialCalls(index + 1)
}, 1000) // waiting 1 sec after each call
})
}
Inject required stuff properly. This assumes array to contain objects with ids using which you perform API calls. Also assumes that you are using $http. If using $resource, add $promise accordingly.
Hope that helps a bit!
function getItemsWithDelay(index) {
getFile(object[index].id).then(()=>{
setTimeout(()=>{
if(index+1 > object.length) { return }
getItemsWithDelay(index+1)
}, 5000)
})
}
You can make sequential calls
This is a awesome trick question to be asked in an interview, anyways I had a similar requirement and did some research on the internet and thanks to reference https://codehandbook.org/understanding-settimeout-inside-for-loop-in-javascript
I was able to delay all promise call in angularjs and the same can be applied in normal JS syntax as well.
I need to send tasks to a TTP API, and they requested to add a delay in each call
_sendTasks: function(taskMeta) {
var defer = $q.defer();
var promiseArray = [];
const delayIncrement = 1000 * 5;
let delay = 0;
for (i = 0; i < taskMeta.length; i++) {
// using 'let' keyword is VERY IMPORTANT else 'var' will send the same task in all http calls
let requestTask = {
"action": "SOME_ACTION",
"userId": '',
"sessionId": '',
};
// new Promise can be replaced with $q - you can try that, I haven't test it although.
promiseArray.push(new Promise(() => setTimeout(() => $http.post(config.API_ROOT_URL + '/' + requestTask.action, requestTask), delay)));
delay += delayIncrement;
}
$q.all(promiseArray).
then(function(results) {
// handle the results and resolve it at the end
defer.resolve(allResponses);
})
.catch(error => {
console.log(error);
defer.reject("failed to execute");
});
return defer.promise;
}
Note:: using 'let' keyword in FOR loop is VERY IMPORTANT else 'var' will send the same task in all http calls - due to closure/context getting switched
Related
There's a async call I'm making that queries a database on a service, but this service has a limit of how many it can output at once, so I need to check if it hit its limit through the result it sends and repeat the query until it doesn't.
Synchronous mockup :
var query_results = [];
var limit_hit = true; #While this is true means that the query hit the record limit
var start_from = 0; #Pagination parameter
while (limit_hit) {
Server.Query(params={start_from : start_from}, callback=function(result){
limit_hit = result.limit_hit;
start_from = result.results.length;
query_result.push(result.results);
}
}
Obviously the above does not work, I've seen some other questions here about the issue, but they don't mention what to do when you need each iteration to wait for the last one to finish and you don't know before hand the number of iterations.
How can I turn the above asynchronous? I'm open to answers using promise/deferred-like logic, but preferably something clean.
I can probably think of a monstruous and horrible way of doing this using waits/timeouts, but there has to be a clean, clever and modern way to solve it.
Another way is to make a "pre-query" to know the number of features before hand so you know the number of loops, I'm not sure if this is the correct way.
Here we use Dojo sometimes, but the examples I found does not explain what to do when you have an unknown amount of loops https://www.sitepen.com/blog/2015/06/10/dojo-faq-how-can-i-sequence-asynchronous-operations/
although many answers already, still I believe async/await is the cleanest way.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
and you might need babel
https://babeljs.io/
JS async logic syntax changed from callback to promise then to async/await, they all do the same thing, when callback nests a lot we need something like a chain, then promise come, when promise goes in loop, we need something make the chain more plain more simple, then async/await come. But not all browsers support the new syntax, so babel come to compile new syntax to old syntax, then you can always code in new syntax.
getData().then((data) => {
//do something with final data
})
async function getData() {
var query_results = [];
var limit_hit = true;
var start_from = 0;
//when you use await, handle error with try/catch
try {
while (limit_hit) {
const result = await loadPage(start_from)
limit_hit = result.limit_hit;
start_from = result.results.length;
query_result.push(result.results);
}
} catch (e) {
//when loadPage rejects
console.log(e)
return null
}
return query_result
}
async function loadPage(start_from) {
//when you use promise, handle error with reject
return new Promise((resolve, reject) => Server.Query({
start_from
}, (result, err) => {
//error reject
if (err) {
reject(err)
return
}
resolve(result)
}))
}
If you want to use a loop then I think there is no (clean) way to do it without Promises.
A different approach would be the following:
var query_results = [];
var start_from = 0;
funciton myCallback(result) {
if(!result) {
//first call
Server.Query({ start_from: start_from}, myCallback);
} else {
//repeated call
start_from = result.results.length
query_result.push(result.results);
if(!result.limit_hit) {
//limit has not been hit yet
//repeat the query with new start value
Server.Query({ start_from: start_from}, myCallback);
} else {
//call some callback function here
}
}
}
myCallback(null);
You could call this recursive, but since the Query is asynchronous you shouldn't have problems with call stack limits etc.
Using promises in an ES6 environment you could make use of async/await. Im not sure if this is possible with dojo.
You don't understand callbacks until you have written a rate limiter or queue ;) The trick is to use a counter: Increment the counter before the async request, and decrement it when you get the response, then you will know how many requests are "in flight".
If the server is choked you want to put the item back in the queue.
There are many things you need to take into account:
What will happen to the queue if the process is killed ?
How long to wait before sending another request ?
Make sure the callback is not called many times !
How many times should you retry ?
How long to wait before giving up ?
Make sure there are no loose ends ! (callback is never called)
When all edge cases are taken into account you will have a rather long and not so elegant solution. But you can abstract it into one function! (that returns a Promise or whatever you fancy).
If you have a user interface you also want to show a loading bar and some statistics!
You must await for the server response every time. Here a encapsulated method
var query = (function(){
var results = [];
var count = 0;
return function check(fun){
Server.Query({ start_from: count}, function(d){
count = d.results.length;
results.push(d.results);
if (d.limit_hit && fun) fun(results);
else check(fun);
});
};
})();
// Call here
var my_query = query(function(d){
// --> retrive all data when limit_hit is true)
});
You can use a generator function Generators to achieve this
For POC:
some basics
- You define a generator with an asterick *
- it exposes a next function which returns the next value
- generators can pause with yield statement internally and can resume externally by calling the next()
- While (true) will ensure that the generator is not done until limit has reached
function *limitQueries() {
let limit_hit = false;
let start_from = 0;
const query_result = [];
while (true) {
if (limit_hit) {break;}
yield Server.Query(params={start_from : start_from},
callback=function* (result) {
limit_hit = result.limit_hit;
start_from = result.results.length;
yield query_result.push(result.results);
}
}
}
So apparently, the generator function maintains its own state. Generator function exposes two properties { value, done } and you can call it like this
const gen = limitQueries();
let results = [];
let next = gen.next();
while(next.done) {
next = gen.next();
}
results = next.value;
You might have to touch your Server.Query method to handle generator callback. Hope this helps! Cheers!
Having trouble to understand how exactly Protractor order of execution works..
If I am having PageObject:InvitePage
And order of execution is defined like this:
InvitePage.EnterUsername()
InvitePage.EnterPassword()
InvitePage.EnterEmail()
InvitePage.Invite();
InviteHelper.waitForEmail()
browser.go(invitationUrl)
...
expect(somecondition)
All page methods are returning protractor promise(for example browser.sendKeys for entering the password)
waitForEmail also returns the promise that I have created using:
protractor.promise.defer()
Problem is that waitForEmail get executed first and methods after it don't wait for waitForEmail to finish, which I expected to be true by creating the promise using protractor method...anyway I found solution to it and it looks something like this:
lastMethodBeforeWaitForEmail.then(function(){
browser.driver.wait(InvitationHelper.waitForEmail(userEmail))
.then(function(recievedUrl){
...
//methods that I want after
expect(someCondition)
});
});
Pretty ugly don't you think?
Is there a way to do this one more nicely, any suggestions?
And which part around async nature of protractor I didn't get? Am I missing something?
getInvitationEmail
var getInvitationEmail = function (emailAddress){
var deferred = protractor.promise.defer();
mailbox.getEmailsByRecipient(emailAddress, function(err, emails) {
if (err) {
console.log('>Fetch email - call rejected');
deferred.reject(err);
}else{
console.log('>Email service fetched.')
deferred.fulfill(emails);
}
});
return deferred.promise;
};
and then waitForEmail
this.waitForEmail = function(email){
var deferred = protractor.promise.defer();
var timeout;
var interval = 3000;
var timePassed = 0;
var recursive = function () {
var message = '>Checking for invitational email';
if(timePassed>0) {
message = message + ":" + timePassed/1000 + "s";
}
console.log(message);
timePassed += interval;
getInvitationEmail(email).then(function(data){
if(data.length>0){
var loginUrl = data[0].html.links[0].href;
if(interval) clearTimeout(timeout);
console.log(">Email retrieved.Fetching stopped.")
deferred.fulfill(loginUrl);
}else{
console.log(">Still no email.");
}
});
timeout = setTimeout(recursive,interval);
};
recursive();
return deferred.promise;
};
In Protractor/WebDriverJS, there is that special mechanism called Control Flow, which is basically a queue of promises. If you have a "custom" promise, in order for it to be in the queue, you need to put it there:
flow = protractor.promise.controlFlow();
flow.await(InviteHelper.waitForEmail());
Or:
browser.controlFlow().wait(InviteHelper.waitForEmail());
One question and one remark.
Shouldn't you put other methods in the ControlFlow if you want to control the execution flow?
JavaScript engines add ; at the end of commands when needed, but it is always better to put them yourself.
In waitForEmail you have defined a promise, but you need to insert it into the controlFlow. As you may know, all of the normal webdriver actions click(), getText(), etc already know how to execute in the right order so you don't have to chain all your promises with .then every time.
... the bottom of your function should look like this
recursive();
return browser.controlFlow().execute(function() {
return deferred.promise;
});
Your ugly solution lastMethodBeforeWaitForEmail.then(function() ... works because it is one way to make sure the waitForEmail promise is executed in the right order, but the above code is the prettiness that you are looking for.
I have an array and i need to send values of array to webservice through http post request one by one . For the node.js , i'm using "async" package to do that for ex: async.eachSeries doing it well , how can i do that same thing for angular.js , my normal async code;
//this code sends all queries of array (maybe 5.000 request at same time , it is hard to process for webservice :=) ) at same time and wait for all responses.
//it works but actually for me , responses should wait others at end of loop should work one by one
//like async.eachSeries module!
for (var i = 0; i < myArr.lenght; i++) {
(function (i) {
var data = {
"myQuery": myArr[i].query
};
$http.post("/myServiceUrl", data).success(function (result) {
console.log(result);
});
})(i);
}
Both Matt Way and Chris L answers Correct , you can investigate Chris's answer for understanding about async to sync functions in for loops.
You can use $q to create a similar requirement by chaining promises together. For example:
var chain = $q.when();
angular.forEach(myArr, function(item){
chain = chain.then(function(){
var data = {
myQuery: item.query
};
return $http.post('/myServiceUrl', data).success(function(result){
console.log(result);
});
});
});
// the final chain object will resolve once all the posts have completed.
chain.then(function(){
console.log('all done!');
});
Essentially you are just running the next promise once the previous one has completed. Emphasis here on the fact that each request will wait until the previous one has completed, as per your question.
function logResultFromWebService(value)
{
$http.post("/myServiceUrl", value).success(console.log);
}
angular.forEach(myArray, logResultFromWebService);
If I understand your question correctly. You want to run a for loop in a synchronized manner such that the next iteration only occurs once the previous iteration is completed. For that, you can use a synchronized loop/callbacks. Especially if the order matters.
var syncLoop = function (iterations, process, exit) {
var index = 0,
done = false,
shouldExit = false;
var loop = {
next: function () {
if (done) {
if (shouldExit && exit) {
return exit(); // Exit if we're done
}
}
// If we're not finished
if (index < iterations) {
index++; // Increment our index
process(loop); // Run our process, pass in the loop
// Otherwise we're done
} else {
done = true; // Make sure we say we're done
if (exit) exit(); // Call the callback on exit
}
},
iteration: function () {
return index - 1; // Return the loop number we're on
},
break: function (end) {
done = true; // End the loop
shouldExit = end; // Passing end as true means we still call the exit callback
}
};
console.log('running first time');
loop.next();
return loop;
}
For your particular implementation:
syncLoop(myArray.length, function (loop) {
var index = loop.iteration();
var data = {
"myQuery": myArray[index].query
};
$http.post("/myServiceUrl", data).success(function (result) {
console.log(result);
loop.next();
});
}, function () {
console.log('done');
});
If you intend on doing something with the data once returned (such as perform calculations) you can do so with this method because you will return the data in a specified order.
I implemented something similar in a statistical calculation web app I built.
EDIT:
To illustrate the problem I had when using $q.when I have set up a fiddle. Hopefully this will help illustrate why I did this the way I did.
https://jsfiddle.net/chrislewispac/6atp3w8o/
Using the following code from Matt's answer:
var chain = $q.when(promise.getResult());
angular.forEach(myArr, function (item) {
chain = chain.then(function () {
$rootScope.status = item;
console.log(item);
});
});
// the final chain object will resolve once all the posts have completed.
chain.then(function () {
console.log('all done!');
});
And this fiddle is an example of my solution:
https://jsfiddle.net/chrislewispac/Lgwteone/3/
Compare the $q version to my version. View the console and imagine those being delivered to the user interface for user intervention in the process and/or performing statistical operations on the sequential returns.
You will see that it does not sequentially give the numbers 1,2,3,4 etc. either in the console or in the view in Matt's answer. It 'batches' the responses and then returns them. Therefore, if step 3 is not to be run depending on the response in step 2 there is not, at least in the answer provided, a way to break out or explicitly control the synchronous operation here. This presents a significant problem when attempting to perform sequential calculations and/or allow the user to control break points, etc.
Now, I am digging through both the $q libraries and the Q library to see if there is a more elegant solution for this problem. However, my solution does work as requested and is very explicit which allows me to place the function in a service and manipulate for certain use cases at my will because I completely understand what it is doing. For me, that is more important than using a library (at least at this stage in my development as a programmer and I am sure there are lots of other people at the same stage on StackOverflow as well).
If the order doesn't matter in which they are sent
var items = [/* your array */];
var promises = [];
angular.forEach(items, function(value, key){
var promise = $http.post("/myServiceUrl", { "myQuery": value.query });
promises.push(promise);
});
return $q.all(promises);
Is there a possibility to create atomic database transactions with bookshelf? I'm having a problem with duplicates in the database. The problematic code is as below:
bookshelf.transaction(function (t) {
var modelLocation = new Models.Location({'name':event.venue});
modelLocation.fetch({transacting:t})
.then(function (fetchedLocation) {
if (!fetchedLocation) {
modelLocation.save(null,{transacting:t}).then(function (savedModel) {
t.commit(savedModel)
}).catch(function (err) {
t.rollback(err)
});
}
else{
t.commit(fetchedLocation)
}
})
})
I call the method containing this code almost simultaniously and asynchronously 20 times. From these 20, there are 5 duplicate datasets. This results in around 2-3 duplicates in the database. The current workaround is to wrap the whole thing in a setTimeout with a random timout between 0 and 10 seconds which almost never gives me duplicates. But this is obviously not a production ready solution.
OK so in the end, I decided to go with the async.js library and it's queue.
The queue guarantees that maximum n async tasks are executed concurrently. In this case 1.
I made a module which exports a queue instance. This way I can use it across multiple modules. It simply waits for the promise to fulfill.
var async = require('async');
module.exports = async.queue(function (task, callback) {
task().then(function () {
callback();
});
},1);
Then in the module, where I need an "atomic" transaction I have the following code:
var queue = require('./transactionQueue');
...
...
queue.push(function(){
return bookshelf.transaction(function (t) {
var modelLocation = new Models.Location({'name':event.venue});
return modelLocation
.fetch({transacting:t})
.then(function (fetchedLocation) {
if (!fetchedLocation) {
return modelLocation
.save(null,{transacting:t});
}
});
});
});
It's important to wrap the transaction into a function so it won't get executed right away.
Since Bookshelf transactions are promises you do not need to explicitly call commit() or rollback(). Just let the fulfilled promise to commit itself, or you can force a rollback by throwing an exception.
In your code there was apparently a small bug that could be causing the trouble: an argument missing from the fetch()'s then() -- this argument is the result from the fetch() invocation, an instance if the object was found or null if not.
bookshelf.transaction(function (t) {
var modelLocation = new Models.Location({'name':event.venue});
return modelLocation
.fetch()
.then(function (fetchedLocation) {
if (!fetchedLocation) {
modelLocation
.save(null,{transacting:t});
}
})l
});
I am not able to test that now, but I hope it helps.
Simply put, I'm trying to dynamically generate an AJAX request based off a scenario that I'm retrieving via an AJAX request from a server.
The idea is that:
A server provides a "Scenario" for me to generate an AJAX Request.
I generate an AJAX Request based off the Scenario.
I then repeat this process, over and over in a Loop.
The big idea is that I can change the second AJAX request dynamically, based off the Scenario given from the server.
I have this working, but I feel like the way I'm doing this is very messy. Is there any better way to go about thinking through this problem? Perhaps promises? If anyone could please review this and provide feedback or suggestions on how to clean it up--that would be greatly appreciated.
Here is my code: http://jsfiddle.net/6jph0e98/
(please open the console to see everything in action)
As a reference, here is the scenario data I'm currently working with:
var scenario = {
"base": {
"frequency": "5000"
},
"endpoints": [
{
"method": "GET",
"type": "JSON",
"endPoint": "https://api.github.com/users/alvarengarichard",
"queryParams": {
"objectives": "objective1, objective2, objective3"
}
}
]
}
Here are my 2 cents: http://jsfiddle.net/3Lddzp9j/6/.
Yes, I think you can do this more elegantly by chaining promises. So I figured out what I think your app does, and how you can do it by chaining these promises. What is interesting that certain steps already return promises ( the jQuery AJAX calls ) but others don't. For those - we have to create our own promise that instantly resolves. And then there was the timeout which we wrapped in a promise.
Also, I tried to use some JS best practices, like keeping things out of the global space by wrapping them in an IIFE and applying the module pattern.
This makes the overall control flow of your application nice and clean IMHO:
var run = function() {
getScenario()
.then(mapToInstruction)
.then(waitForTimeout)
.then(callApi)
.then(handleResults)
.then(run);
};
And also hides the private members and only exposes the run() method:
return {
// This will expose only the run method
// and will keep all other functions private
run : run
}
Hope it helps - let me know what you think. Here's the full source, with comments:
// First of all - I'm using the javascript module pattern here
// this will all be much more easy once ES6 it out, but this will
// have to do for now.
// Also, I'm importing jQuery into the module as you can see, which
// is wrapped inside the IIFE ( Google it ) which keeps things nicely
// out of the global scope.
var App = (function ($) {
// Gets the scenario from the API - $.get is just some syntactic
// sugar for $.ajax with GET as method - NOTE: this returns a promise
var getScenario = function () {
console.log('Getting scenario ...');
return $.get('http://demo3858327.mockable.io/scenario');
};
// The result of the previous promise is passed into the
// next as we're chaining. So the data will contain the
// result of getScenario
var mapToInstruction = function (data) {
// We map it onto a new instruction object
var instruction = {
method: data.endpoints[0].method,
type: data.endpoints[0].type,
endpoint: data.endpoints[0].endPoint,
frequency: data.base.frequency
};
console.log('Instructions recieved:');
console.log(instruction);
// And now we create a promise from this
// instruction so we can chain it
var deferred = $.Deferred();
deferred.resolve(instruction);
return deferred.promise();
};
// This wraps the setTimeout into a promise, again
// so we can chain it
var waitForTimeout = function(instruction) {
console.log('Waiting for ' + instruction.frequency + ' ms');
var deferred = $.Deferred();
setTimeout(function() {
deferred.resolve(instruction)
}, instruction.frequency);
return deferred.promise();
};
// Final step: call the API from the
// provided instructions
var callApi = function(instruction) {
console.log('Calling API with given instructions ...');
return $.ajax({
type: instruction.method,
dataType: instruction.type,
url: instruction.endpoint
});
};
var handleResults = function(data) {
console.log("Handling data ...");
var deferred = $.Deferred();
deferred.resolve();
return deferred.promise();
};
// The 'run' method
var run = function() {
getScenario()
.then(mapToInstruction)
.then(waitForTimeout)
.then(callApi)
.then(handleResults)
.then(run);
};
return {
// This will expose only the run method
// and will keep all other functions private
run : run
}
})($);
// ... And start the app
App.run();