Essentially, how do you pause javascript execution without having to waste computation on something like a while loop?
For example, say I want to only perform a function next after 10 seconds and not interrupt other processes or waste computation? setTimeout won't work because I want the processes to actually pause / not continue any operations during that time.
const next = () => console.log("next");
/** pause for 10 seconds */
next()
Also what if I want to only run a method conditionally, where every couple seconds or something I can either abort the operation or continue. Notably I don't mean to use setInterval because in that case it's not actually conditionally pausing the actual javascript execution.
const next = () => console.log("next");
const start = new Date();
const startSeconds = now.getSeconds();
const checkEvery = 1; // seconds
const requiredDiff = 10; // seconds
const checker = () => {
const now = new Date();
let secondsNow = now.getSeconds();
secondsNow < startSeconds ? secondsNow += 60 : null;
const diff = secondsNow - startSeconds;
if (diff < 10) {
return false
}
return true;
}
/** something that runs checker every 1 seconds and pauses computation until checker() returns true and then runs next() */
A good way to do this is by creating a block in the pipe. i.e.
process -> pause -> next so that you can know that the flow of the program won't continue until a certain criteria is met
You can do this by pausing things inline
...
await pause();
...
where pause is something that does a timed-out promise or something along those lines. I don't know too much about this method, but it seems relatively complicated.
Another way to do this is by stopping the execution inline with a while loop
...
// pause for 10 seconds
let result;
while (!result) { result = checker() };
...
but as you alluded to, this wastes a lot of operations and can interfere with other actions in the background from running properly. Another thing is you can't only check checker every 1 second.
I suggest you do the following instead:
const max = 20; // max number of recursive calls (i.e. timeout after 20 seconds)
// checker is a function that returns true or false and is agnostic to this implementation
// timeout is the time (in milliseconds) to wait before running the checker again
// next is the next step in your pipeline that you want to prevent from e
const pause = async (checker, timeout, next, calls = 0) => {
if (calls > max) return; // prevents stack overflow
const result = await checker(); // just in case your checker is async
// if the condition was met then continue on to the next stage in your pipeline
// if the condition was not met then run this method again to re-check in timeout
result ? next() : setTimeout(() => pause(checker, timeout, next, calls + 1), timeout)
}
// with the functions you provided...
pause(checker, 1000, next)
pause will only execute operations when the timeout is met, and it won't allow the program to continue to the next stage until the checker is met.
Related
I have a REST API coded with Node.js and there is one route that applies NTFS Rights to folders. The request can take 1 second but can also last several minutes (it depends on the size of the folder).
To summarize, if the rights take less than 5 seconds to be applied, I want to return the value with the code 200.
If the rights have not finished being applied I would like to return the value with the code 202
...
inProgress = true;
ApplyRights()
.then(() => {
inProgress = false
}
// Here I want to return as fast inProgress = false otherwise wait a bit but max 5s
return ...;
I tried to wait a bit with setTimeout like this:
const sleep = s => new Promise(resolve => {
setTimeout(resolve, s * 1000)
});
let nbCheck = 0;
while (inProgress && nbCheck < 5){
await sleep(1)
nbCheck++;
}
But setTimeout is not called before the end of my previous promise (ApplyRights).
I read here that promise are executed before setTimeout
So, i tried to find a solution without setTimeout and I tried this: (I know it's not very elegant)
let dateStop = Date.now() + 5 * 1000;
while (dateStop = Date.now() && inProgress){}
return ...;
But in this case the .then() of ApplyRights is only reached at the end of the 5s.
Is there a way to let my promise ApplyRights do its job. And if the job take time, wait maximum 5 seconds before returning the response. If the job is quick, I want to return the responses without waiting.
You can use Promise.race:
Promise.race([ApplyRights(), sleep(5)]).then(result => {
if (result === undefined) { // Timeout of 5 seconds occurred
// Some message to the user?
} else { // Got reply within 5 seconds
// Do something with the result
}
});
Do realise however, that this does not interrupt the work of ApplyRights(), which eventually may still do the job.
In my code i trying to learn async, i have 700 async task to do, the job will done in random time.
My question how to limit async task in loop?
Let say i want it to do 30 job at start time. and watch event each async task done it will start 1 task to fill limit task 30 at time again until 700 task, or theres no task again.
For now loop will execute all async task in same time. its not i want.
function JobdeskAsync(){
console.log(Math.floor(Math.random() * 1000));
}
function finishedTime(max, min){
return Math.floor(Math.random() * (max - min) ) + min;
}
for ( let i = 0; i < 700; i++){
setTimeout(JobdeskAsync, finishedTime(5000, 1000));
}
#Wendelin has a great example there. For how it directly applies to your application, you currently are:
Looping through 700 asynchronous Jobs
Executing them all immediately (to finish as and when)
What you need to do is be able to recognise when you have reached your pool maximum (30) and not add/execute any more until that pool depletes. I'm not writing the code for you because there's a thousand ways to do it, but at it's base what you need:
Functionality to store executing jobs (call it a "store")
Functionality to add jobs to the store
Functionality to see if that store is full
Combine 2/3 so that when #3 is not true you can do #2
You need to start 30 jobs that continue processing data until there is no more.
Here there is a queue of data that each job pulls from. If there is no data left, it returns a simple promise that resolves to 'done'. If there is data, it processes it and returns the promise from a new call to createJob().
const queue = [];
// fill queue with data to process
for (let i = 0; i < 70; i++) {
queue.push(i);
}
function createJob() {
// pull data to work on from the queue
const data = queue.shift();
// if no data is remaining, return promsie that resolves to 'done'
if (data === undefined) {
return Promise.resolve('done');
}
// otherwise we do some work on the data
return new Promise((resolve, reject) => {
// simulate work by waiting 500-1500ms
setTimeout(() => {
console.log(`handled ${data}`);
// this resolves our promise with another promise, createJob()
resolve(createJob());
}, 500 + Math.random() * 1000);
});
}
// create 30 jobs and wait for all to complete, each job will
// create a new job when it has finished and chain the results
const jobs = [];
for (let i = 0; i < 30; i++) {
jobs.push(createJob());
}
console.log('30 jobs running...');
Promise.all(jobs).then(() => console.log('ALL JOBS COMPLETED'));
How do I make the below loop wait for current iteration to finish before it starts the next iteration?
var minute = 0
alert ("Kick Off");
var refreshIntervalId = setInterval(function() {
if (minute < 91) {
// lots of code with Ajax calls
// can take from a fraction of a second to 30 seconds to run code
// depending on conditions
} else {
clearInterval(refreshIntervalId);
}
minute++
}, 1000);
I usually use functions and '.done' to execute code after pre-requisite code has finished. I have also used functions within animations to execute code after the animation has finished.
However I can't get my head around how you make the code to wait for an iteration of a loop to finish. If I put the pre-requisite code into a function (all the code within the loop) and then use .done, there is only the closing brackets of the loop left to put within a '.done' function - which obviously will not work.
Can anyone solve this conundrum?
You can make a while loop in an async function instead and use await. In the code below, the while-loop waits at least 2 seconds at each iteration but at most as long as the task takes.
const sleep = time => new Promise(resolve => setTimeout(resolve, time))
async function main() {
while (true) {
console.log("new iteration")
const task = new Promise((resolve, reject) => {
// things that take long go here…
const duration = Math.random()*4000
setTimeout(() => {
console.log(`done, task took ${Math.round(duration)}ms`)
resolve()
}, duration)
})
// wait until task is finished but at least 2 seconds
await Promise.all([task, sleep(2000)])
}
}
main()
To bring Matthias's elegant up-voted answer to life in your specific situation:
var minute = 0
// reusable function to wait the specified amount of time (1000ms in your case)
const sleep = time => new Promise(resolve => setTimeout(resolve, time))
alert ("Kick Off");
while (minute < 91)
{
const processThisMinuteTask = new Promise((res, rej) => {
// lots of code with Ajax calls
// can take from a fraction of a second to 30 seconds to run code
// depending on conditions
// NOTE: use AWAIT for all Ajax calls
res();
});
// will wait for both:
// a) your processThisMinuteTask to finsih
// b) 1000ms to pass
// So, will mean that this iteration will wait for whichever of those is longest
await Promise.all([processThisMinuteTask, sleep(1000)]);
minute++;
}
P.S. Nothing wrong with the "Alert" if what you want to do is prompt the user that kick-off is about to start, but will not start until they acknowledge the alert.
Further to comment about "await is a reserved identifier", the enclosing function needs to be declared as async.
function playMatch() {
// will fail
await myOtherFunction();
// more code
}
Needs to be:
async function playMatch() {
// will now wait for myOtherFunction to finish before continuing.
await myOtherFunction();
// more code
}
I need my program to repeat itself continuously. My program starts to fetch proxies from servers and saves them to a database and then send those saved proxies to another server again. So I don't know how long it takes for my program to do this task.
I wanna know what happens if any problem happens that makes this startJob() function take more than 30 seconds.
Does setInterval call it again or waits for function to finish?
What's the best approach for my program to repeat itself after it's done without setInterval?
(for exmaple startJob() being called again after it's done.)
I was wondering if it is ok to put this function in a loop with a big number like:
for ( let i = 0 ; i < 999999999 ; i ++ ) {
await startJob()
}
Here is my code:
const startJob = async () => {
await postProxyToChannel()
grabProxies()
}
setInterval(function(){
startJob()
}, (30000))
grabProxies() takes about 10 seconds and postProxyToChannel() takes about 5 seconds on my server.
No matter what happens inside startJob, setInterval will call it every 30 seconds. This means that postProxyToChannel will be called every 30 seconds. If that function throws, you'll get an unhandled Promise rejection, but the interval will continue.
Even if postProxyToChannel takes, say, 45 seconds, that won't prevent startJob from being called again before the prior startJob has completed.
If you want to make sure that startJob is only called 30 seconds after it finishes, you could await it in your for loop, then await a Promise that resolves every 30 seconds:
(async () => {
for ( let i = 0 ; i < 999999999 ; i ++ ) {
await startJob();
await new Promise(resolve => setTimeout(resolve, 30000));
}
})()
.catch((err) => {
console.log('There was an error', err);
});
But it would probably make more sense just to have a recursive call of startJob, eg:
const startJob = async () => {
try {
await postProxyToChannel();
} catch(e) {
// handle error
}
grabProxies();
setTimeout(startJob, 30000);
};
startJob();
Yup an infinite loop sounds good, that can be compared with a timer to pause the loop:
const timer = ms => new Promise(resolve => setTimeout(resolve, ms));
(async function() {
while(true) {
await postProxyToChannel();
await grabProxies();
await timer(30000);
}
})();
Now that loop will run the task, wait 30secs, then do that again. Therefore the loop will not run every 30secs but will usually take longer. To adjust that, you could measure the time the task took, then await the rest of the time:
const start = Date.now();
await postProxyToChannel();
await grabProxies();
await timer(30000 - (Date.now() - start));
I have an external service I am making HTTP requests to from Node.js. The service has current limitations that only 10 requests per second can be made. I have a naive rate limiter I wrote that I am trying to test, but falling down on timing of it. I know that Javascript times are not very accurate, but I'm getting wildly different swings, in the range of up to 50 milliseconds different.
Here's the gist of what I'm doing:
var RATE_LIMIT_MS = 100 // No more than 10 requests per second
var NEXT_WAIT_MS = 0
function goGetRequest(...) {
return new Promise(function(resolve, reject) {
setTimeout(function() {
function complete() {
// Rollback next time after this request finishes
NEXT_WAIT_MS = Math.max(0, NEXT_WAIT_MS - RATE_LIMIT_MS)
}
// ... requests are fired off here asynchronously...
http.get(...).then(complete)
}, NEXT_WAIT_MS)
// Push time back on the queue
NEXT_WAIT_MS += RATE_LIMIT_MS
})
}
var chai = require('chai')
var expect = chai.expect
it('should rate limit requests properly', function() {
var iterations = [0, 1, 2, 3, 4]
var lastResponseMs = 0
var promises = iterations.map(function(i) {
return goGetRequest(...).
then(function(result) {
// Diff the times
var thisResponseMs = Date.now()
var thisDiffMs = Math.abs(thisResponseMs - lastResponseMs)
expect(wrapper.results).to.not.be.empty
expect(thisDiffMs, 'failed on pass ' + i).to.be.at.least(RATE_LIMIT_MS)
// Move last response time forward
lastResponseMs = thisResponseMs
})
})
return Promise.all(promises)
})
What happens next is that the tests will fail at random passes. A time diff on 92 milliseconds on pass 2, a time diff of 68 milliseconds on pass 4.... what am I missing here? Thanks!
Javascript setTimeout and setInterval (as with any non-realtime code) is never precise. If you're using NodeJS however, you can try using Nanotimer:
https://www.npmjs.com/package/nanotimer
var NanoTimer = require('nanotimer');
var timerA = new NanoTimer();
timerA.setTimeout(yourFunction, '', NEXT_WAIT_MS);
Alternatively, I suggest simply testing that rate limit occurs and not to worry too much about exact precision. If you spam it with 11 requests consecutively (which should hopefully take less than one second), and it gets blocked, and one second later it is fine, then your test can be considered passing.
One final solution is to use exec() and call the OS sleep command, which is significantly more precise.