Develop Reference

JavaScript while loop + setInterval doesn't work in Django

I'm developing a website using Django. I have about 50 functions. And it takes about 1 minute to get the result for all of them.
The problem is that I use heroku for my app. And heroku has a limit of 30 sec to get the request. I was suggested to use background tasks. Now I am using background jobs in Python RQ. For each function I made a separate task in Views. Here is an example of one of the tasks:
task_robots_url = q.enqueue(Robots(url).get_url)
robots_url = task_robots_url.result
And now I need to check whether the job is finished in the Template. And when it's finished, display the result.
I want to check with JS each second whether the job is finished. If it's not finished - show "Loading ... " message to the user. If it's finished - show the result.
Here is my JS code:
<script>
var h1 = document.getElementById("h1");
var job_result = "{{ robots_url }}"
var test = function () {
if (job_result == "") {
h1.innerText = "Waiting...";
} else if (job_result == "None") {
h1.innerText = "Waiting ...";
} else if (job_result == undefined) {
h1.innerText = "Waiting ...";
} else {
clearInterval(inter);
h1.innerText = job_result;
}
}
var inter = setInterval(test, 1000);
</script>
But this doesn't work. When the result page starts - I've got an empty result. And it displays Waiting. Then I have None result. And it still displays Waiting. But now after waiting there is no result.
In the documentation for Python RQ it is said I should put time.sleep() for several seconds in my code. And if I put time.sleep(2) - I got the result. But if I put time.sleep(2) for each of my 50 functions - I will get 100 sec. And the limit in Heroku is only 30 sec... So I need to check and display the result without sleep()...

The setInterval() method calls a function or evaluates an expression at specified intervals (in milliseconds).
The setInterval() method will continue calling the function until clearInterval() is called, or the window is closed.
You are calling setInterval so many times which cause your browser to crash eventually.
assuming job_result value changes after 10 sec you can write the following code:
var job_result = {{ job_result }};
var loadingMessage = function (){
if (!job_result) {
document.write("Loading ...");
}
else {
return
}
}
setInterval(() => loadingMessage, 1000);
You can call return to stop the Interval for running or you can use setTimeout and to call a function that clears the interval

Nodejs function running every 2 minutes lost over time

This is quite hard problem to describe.
I have a koajs app with a function which is created in multiple instances (10-1000 range) every 2 minutes. this scheduled job created on app startup. I use koajs because i need a few simple api endpoints for this app. It is running well for first 3-5 hours and then the count of created instances starts to decrease and some of the log output disappears.
Here is the minimal sample based on actual code:
server.ts
const bootstrap = async () => {
process.setMaxListeners(0); //(node:7310) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 uncaughtException listeners added to [process]. Use emitter.setMaxListeners() to increase limit
//appears on app startup (however seems like this setMaxListeners(0) doesnt affect anything since the warning persist)
const app = new Koa();
app.use(async ctx => {
ctx.body = "Welcome to my Server!";
});
app.listen(port);
new Main().run();
};
bootstrap();
main.ts (tried: cron npm package, node-scheduler, setInterval, recursive setTimeout) to run the scheduledJobWrapper.
isStarting: boolean = false;
async run() {
logger.info(`running the app, every 2 minutes`);
//let that = this;
// new CronJob(`*/2 * * * *`, function () {
// that.scheduledJobWrapper();
// }, null, true, 'America/Los_Angeles');
const interval = 2 * 60 * 1000;
setInterval(() => {
this.scheduledJobWrapper();
}, interval);
}
async scheduledJobWrapper() {
logger.info("here scheduledJobWrapper");
let args = {};
//some irrelevant logic to set the arguments
await this.scheduledJob(args);
}
async scheduledJob(args) {
try {
logger.info("starting");
if (!this.isStarting) {
this.isStarting = true;
const runningCount = Executor.tasks.length; //Executor.tasks is a singleton containing some info about tasks. details are irrelevant. the point is it contains the active tasks.
const tasksLimit = 100;
if (runningCount < tasksLimit) {
for await (const i of Array(tasksLimit - runningCount).keys()) {
if (Executor.tasks.length > 20)
await global.sleep(5 * 1000);
this.startWrapper(args); //calling main task here
}
}
this.isStarting = false;
logger.info(`Started: ${Executor.tasks.length - runningCount}`);
}
} catch (e) {
logger.error("Error running scheduled job: " + e.toString());
}
}
In this example the problem manifests as following:
All work as expected first 3-5 hours, later for each time the scheduled function called:
logger.info("here scheduledJobWrapper"); does now show any output.
logger.info("starting"); not in the output
this.startWrapper does run and the code inside it is being executed.
Despite that the code inside of this.startWrapper is still running, the count of newly created jobs is slowly decreasing.
Hardware (RAM/CPU) is not getting any significant load (CPU under 10%, RAM under 20%)
Any clue on possible reason?
nodejs: 12.6.0
Thanks!
UPDATE
it seems like that with the usage of setInterval the app is running OK for a longer time (6-24 hours), but after that the problem still starts.

The issue is with setInterval function. It gets slow down with the time. It has wierd behavior too. You can create custom setInterval using setTimeout or use third-party module and give try.
Sample setInterval Implementation.
const intervals = new Map();
function setInterval(fn, time, context, ...args) {
const id = new Date().getTime() + "" + Math.floor(Math.random() * 10000);
intervals.set(
id,
setTimeout(function next() {
intervals.set(id, setTimeout(next, time));
fn.apply(context, args);
}, time)
);
return id;
}
function clearInterval(id) {
clearTimeout(intervals.get(id));
}
setInterval(console.log, 100, console, "hi");
You can also enhance, by adding delta time loss in next setTimeout.
Meaning if time loss, run next setTimeout earlier.

First of all, It will be better to move instance of Main() in listen scope:
app.listen(port, () => {
new Main().run();
});
I don't know how good idea is to run setInterval function in the backend side. It's better to extract this logic and move it in cron job.
Are we sure that the machine can run 100 tasks? Please count the tasks by order and see when the problem starts. Probably you can not schedule 100 tasks and exists one limit somewhere

Simplest possible irrevocable timout / cooldown function?

I'm trying to add a 1 second cooldown to my send-message system (as in, you can send 1 message per second max). So my initial thought was simply to create a timeout, and before attempting in sending to check if it exists still. That turned out to take more line of code than I anticipated initially.
Is there something I'm missing here? Isn't there something as simple as:
//inside some message sending function
if(!mySuperCooldown)
{
//send message
mySuperCooldown = cooldown(1000);
}
Everything else I construct successfully ends up taking loads of lines, and it appears to me as something someone thought of before. Thank you, and excuse my illiteracy.

Have a flag that allows messages, and set it to false when a message is sent. Then set a timeout for 1000 milliseconds that resets the flag to true.
var allowMessage = true;
function sendMessage(msg) {
if (allowMessage) {
//do something
allowMessage = false;
setTimeout(() => allowMessage = true, 1000);
}
}

Make a higher order function that turns a normal function into one that is rate limited:
function rate_limit(delay, func) {
var last_call = null;
return function() {
if (last_call && (Date.now() - last_call <= delay)) {
return;
}
last_call = Date.now();
return func();
};
}
You can then rate limit any function:
var my_function = rate_limit(1000, function() {
console.log('foo');
});
Running my_function() will only call your original function once per second.

Gmail-like countdown timer not working

I've been trying to recreate a countdown timer similar to the one gmail uses that when you get disconnected from internet. The ajax request fails then it begin a short countdown then makes another ajax request and if it fails again then begin a longer countdown and so forth. Once a determined countdown value is reached (lets say 1 minute), the countdown is maintained at 1 minute until the internet connection is recovered or the servers comes back.
I don't want to use a plugin because this code will be embedded in a micro-controller that has limited space and also prefer not to place it as external file for practical reasons, even though jQuery library will already be external.
Everything should trigger onload, and continue automatically (i.e. no control inputs will be used).
So far I've developed code that does what I want if the ajax request succeeds or fails immediately but if there is a latency on the ajax request status (as for example having the server down) the browser won't produce an immediate result and my code fails.
I know what I stated above because I actually took the server down and was been help by firebug on Mozilla Firefox to see that the ajax result (success or failure) wasn't triggered immediately but keep waiting for several seconds.
Help please!
html code:
<div id='tempFail' ></div>
jQuery code:
$(document).ready(function()
{
//do when DOM ready - http://api.jquery.com/ready/
var timerSpeed = [1000, 5000, 10000, 20000, 30000, 60000];
// current time in ms since 1/1/1970, plus the initial reload interval
var end = (new Date).getTime() + timerSpeed[1];
var n=0;
var m=0;
setInterval(function()
{
var now = (new Date).getTime();
// Current time left to re-load in seconds, sometimes it goes to negative values, see below
var secLeft = Math.floor(( end - now ) / 1000);
// if secLeft is negative multiply by zero...equal to secLeft=0, this will produce an error of one second approximately
var timeToload = (secLeft < 0) ? secLeft * 0 : secLeft;
if (n!=0)
{
//check for failed or delayed request\n\
$('#tempFail').html('Failed or delayed response. Will auto load in: '+timeToload+ ' seconds!');
}
if( (secLeft)<=0)// if reload time is reached do
{
if (m==0)//used to prevent multiple continue reloads when ajax request status is not yet defined
{
m=1;
$.getScript('script_vars.js').done(function (data)
{
//if request succeeded
m=0;
n = 0;
end = (new Date).getTime() + timerSpeed[1];
// Time to load after the initial interval set above
$('#tempFail').html('');
//other code on success here
})
.fail(function()
{
//if request failed
m=0;
n ++;
if(n==6) n=5;
switch(n){ //timer delay for failed request\n\
case 1:
end = (new Date).getTime() + timerSpeed[1];
break;
case 2:
end = (new Date).getTime() + timerSpeed[2];
break;
case 3:
end = (new Date).getTime() + timerSpeed[3];
break;
case 4:
end = (new Date).getTime() + timerSpeed[4];
break;
case 5:
end = (new Date).getTime() + timerSpeed[5];
break;
}
});
}
}
}, 1000);
});

You asked for an example so I've written the following, you may want to wrap the contents of the function within another function so you can repeat it/don't have to worry about namespaces/etc. Didn't test so don't expect bug free!
Using window.setTimeout for every action, separated each stage into it's own function so code paths can more easily be followed.
$(document).ready(function () { // http://api.jquery.com/ready/
var $tempFail = $('#tempFail'),
delay = [1000, 5000, 10000, 20000, 30000, 60000],
delay_index = 0,
delay_ends = 0,
inform_user_ref = null,
inform_user = function inform_user() {
var now = (new Date).getTime(),
delta; // for difference, calculate later
if (delay_ends > now) { // if we're waiting for a retry
delta = Math.floor((delay_ends - now ) / 1000); // calculate time to wait
$tempFail.html('Failed or delayed response. Will auto load in: '+delta+ ' seconds!'); // let people know
window.setTimeout(inform_user, 200); // loop countdown timer
// can fast refresh this as it's just a countdown
}
},
get_success = function () {
$tempFail.html('');
// .. code on success
},
get_fail = function () {
delay_index < 5 && ++delay_index; // increment delay_index
get_initialise(); // retry
window.clearTimeout(inform_user_ref); // remove any old countdown timer
inform_user_ref = inform_user(); // and display new countdown
},
get_try = function () {
$.getScript('script_vars.js')
.done(get_success)
.fail(get_fail);
},
get_initialise = function () {
delay_ends = (new Date).getTime() + delay[delay_index];
window.setTimeout(get_try, delay[delay_index]); // retry
};
get_initialise(); // initial
});

Wow! Mr Paul S. your code was crazy good. I just made a couple of adjustments to have it perfectly working as I need it.
Added the following on ajax success:
delay_index = 0; //reset delay_index
get_initialise(); // retry
so I keep the code running every 5 seconds if everything goes ok.
2.
Added two new variables: let_cntDwn_end and ajax_rqst_status to avoid countdown number jumping (to let countdown finish before beginning the next one ) and to display a message while the ajax request haven't given any result respectively.
Here is the new code:
$(document).ready(function(){ //do when DOM ready - http://api.jquery.com/ready/
var $tempFail = $('#tempFail'),
delay = [5000, 5000, 10000, 20000, 30000, 60000],
delay_index = 0,
delay_ends = 0,
inform_user_ref = null,
let_cntDwn_end = 0,
ajax_rqst_status = 0, //ajax success or failure triggered
inform_user = function inform_user() {
var now = (new Date).getTime(),
delta; // for difference, calculated later
if (delay_ends > now) { // if we're waiting for a retry
let_cntDwn_end = 1;
delta = Math.floor((delay_ends - now ) / 1000); // calculate time to wait
if (ajax_rqst_status==0){
$tempFail.html('Failed response. Will auto load in: '+delta+ ' seconds!'); // let people know
window.setTimeout(inform_user, 900); // loop countdown timer
// can fast refresh this as it's just a countdown
}
}
else {let_cntDwn_end = 0; get_try();}
},
get_success = function () {
ajax_rqst_status =0;
$tempFail.html('');
// .. code on success
delay_index = 0; //reset delay_index
get_initialise(); // retry
},
get_fail = function () {
ajax_rqst_status =0;
delay_index < 5 && ++delay_index; // increment delay_index
get_initialise(); // retry
window.clearTimeout(inform_user_ref); // remove any old countdown timer
inform_user_ref = inform_user(); // and display new countdown
},
get_try = function () {
if (let_cntDwn_end == 0){
ajax_rqst_status=1;
$tempFail.html('Waiting for Ajax request success or failure'); // let people know
$.getScript('script_vars.js')
.done(get_success)
.fail(get_fail);
}
},
get_initialise = function () {
delay_ends = (new Date).getTime() + delay[delay_index];
window.setTimeout(get_try, delay[delay_index]); // retry
};
get_initialise(); // initial
});

There's also JS lib which handles this for you by monitoring ajax requests.
https://github.com/HubSpot/offline

Set and Clear Timeout on Node Server from Client

I'm attempting to allow a user to set an alarm from the client and pass it to the server. The server then has a setTimeout that counts down and when time runs out, executes the function.
This first part is working fine, however, I need the the ability to clear that same timeout, should the client decide to cancel that particular alarm.
Note: I've been storing various data using Redis, so that is available.
var client = redis.createClient();
io.set("store", new sio.RedisStore);
io.sockets.on('connection', function (socket) {
socket.on('alarm:config', function(list, date, time, bool) {
if (bool) {
var now = new Date().getTime();
var year = date[0],
month = date[1] - 1,
day = date[2];
var hour = time[0],
minutes = time[1];
var alarm = new Date(year, month, day, hour, minutes);
var countdown = alarm - now;
var myAlarm = setTimeout(function() {
// do stuff...
}, ( countdown ) );
} else {
clearTimeout(myAlarm);
}
});
});
The approach I have in mind is that I would use the boolean value to determine if the user is setting or canceling that particular alarm. I realize that setting a local variable "myAlarm" will not work, I just put it there to convey the idea.
I am trying to figure out a way to store a reference to that exact timeout so that the next time the "alarm:config" socket event is triggered with a false boolean value, it can cancel the timeout that was set earlier.
It might be another question all together, but how does an application like Google Calendar store a date and time and then know exactly when to trigger it as well as offer the ability to cancel it? This would essentially be the same idea.
UPDATE: I have it working using the following solution. I am open to a more elegant solution.
socket.on('alarm:config', function(list, date, time, bool) {
var alarmName = "timer:" + list;
if (bool) {
client.hset(alarmName, "status", true);
var now = new Date().getTime();
var year = date[0],
month = date[1] - 1,
day = date[2];
var hour = time[0],
minutes = time[1];
var alarm = new Date(year, month, day, hour, minutes);
var countdown = alarm - now;
setTimeout(function() {
client.hget(alarmName, "status", function(err, bool) {
if(bool == 'true') {
// do stuff...
} else {
console.log("This alarm has been canceled.");
}
});
}, ( countdown ) );
} else {
console.log('canceling alarm');
client.hset(alarmName, "status", false);
}
});

Depending on how large of an application you're building, there are a couple of options.
Processing Queue
You could restructure your application to use a job queue instead of simply setting timers. This has an advantage that you can split it in the future into multiple processes, but it does complicate the handling a bit.
A library like Kue uses just Redis and allows you to do a delayed put to set events in the future.
Going from the Kue readme:
var kue = require('kue')
, jobs = kue.createQueue();
// Create delayed job (need to store email.id in redis as well)
var email = jobs.create('email', {
title: 'Account renewal required',
to: 'tj#learnboost.com',
template: 'renewal-email'
}).delay(minute).save();
// Process job
jobs.process('email', function(job, done){
email(job.data.to, done);
});
// Cancel job
email.remove(function(err){
if (err) throw err;
console.log('removed completed job #%d', job.id);
});
Storing Reference to Timeout
This is much less robust, but could allow you to do it very simply. It leaves global variables floating around, and has no recovery if the process crashes.
Basically, you store the timer in a global variable and cancel it based on the job id.
var myAlarms = {}
socket.on('alarm:config', function(list, date, time, bool) {
var alarmName = "timer:" + list;
if (bool) {
myAlarms[alarmName] = setTimeout(function() {
// do stuff...
}, countdown);
} else {
clearTimeout(myAlarms[alarmName]);
}
}
Note: None of this code has been tested and it likely contains errors.