I use an API that has been recently rate limited and requests have to be run every 15 seconds as that is the wait time otherwise a status code of 429 rate limit exceeded is returned.
I often have more than one email address that needs to be run against this API and the email addresses are contained within an array. How would I go about running the request every say 15.5 seconds but move onto each email address until the end of the array? It's a very tricky one for sure. I've tried:
setInterval(checkEmail(email), 15500);
No joy, for some reason that just doesn't seem to work. Btw should point out that I'm using a JQuery $.ajax() within that checkEmail(email) function.
Any ideas anybody?
Thanks in advance.
You could do the following:
var emails = [];
var interval_id;
function _start() {
interval_id = window.setInterval(function() {
var email = emails.shift();
if (email) {
var r=checkEmail(email.address);
if (email.callback) email.callback(r);
}
else {
window.clearInterval(interval_id);
}
}, 15500);
}
function add_email(email, callback) {
emails.push({address:email, callback:callback});
if (!interval_id) _start();
}
Whenever you have a new email address to check, just run add_email(email). This will add it to the queue and start the interval timer if necessary. Additionally, you can say add_email(email,function(result) {....}) if you want to get notified when the check is completed.
Related
[![enter image description here][1]][1]I need to implement code to check what my throttling limit is on an endpoint (I know it's x times per minute). I've only been able to find an example of this in python, which I have never used. It seems like my options are to run a script to send the request repeatedly until it throttles me or, if possible, query the API to see what the limit is.
Does anyone have a good idea on how to go about this?
Thanks.
Note: The blank space is just data from the api calls.
[1]: https://i.stack.imgur.com/gAFQQ.png
This starts concurency number of workers (I'm using workers as a loose term here; don't # me). Each one makes as many requests as possible until one of the requests is rate-limited or it runs out of time. It them reports how many of the requests completed successfully inside the given time window.
If you know the rate-limit window (1 minute based on your question), this will find the rate-limit. If you need to discover the window, you would want to intentionally exhaust the limit, then slow down the requests and measure the time until they started going through again. The provided code does not do this.
// call apiCall() a bunch of times, stopping when a apiCall() resolves
// false or when "until" time is reached, whichever comes first. For example
// if your limit is 50 req/min (and you give "until" enough time to
// actuially complete 50+ requests) this will call apiCall() 50 times. Each
// call should return a promise resolving to TRUE, so it will be counted as
// a success. On the 51st call you will presumably hit the limit, the API
// will return an error, apiCall() will detect that, and resolve to false.
// This will cause the worker to stop making requests and return 50.
async function workerThread(apiCall, until) {
let successfullRequests = 0;
while(true) {
const success = await apiCall();
// only count it if the request was successfull
// AND finished within the timeframe
if(success && Date.now() < until) {
successfullRequests++;
} else {
break;
}
}
return successfullRequests;
}
// this just runs a bunch of workerThreads in parallell, since by doing a
// single request at a time, you might not be able to hit the limit
// depending on how slow the API is to return. It returns the sum of each
// workerThread(), AKA the total number of apiCall()s that resolved to TRUE
// across all threads.
async function testLimit(apiCall, concurency, time) {
const endTime = Date.now() + time;
// launch "concurency" number of requests
const workers = [];
while(workers.length < concurency) {
workers.push(workerThread(apiCall, endTime));
}
// sum the number of requests that succeded from each worker.
// this implicitly waits for them to finish.
let total = 0;
for(const worker of workers) {
total += await worker;
}
return total;
}
// put in your own code to make a trial API call.
// return true for success or false if you were throttled.
async function yourAPICall() {
try {
// this is a really sloppy example API
// the limit is ROUGHLY 5/min, but because of the sloppy server-side
// implimentation you might get 4-6.
const resp = await fetch("https://9072997.com/demos/rate-limit/");
return resp.ok;
} catch {
return false;
}
}
// this is a demo of how to use the function
(async function() {
// run 2 requests at a time for 5 seconds
const limit = await testLimit(yourAPICall, 2, 5*1000);
console.log("limit is " + limit + " requests in 5 seconds");
})();
Note that this method measures the quota available to itself. If other clients or previous requests have already depleted the quota, it will affect the result.
I'm using LockService when sending emails, but when multiple users run it at the same time, it throws the error: Service invoked too many times...
Here's some log, showing the failures:
function sendEmail() {
const ss = SpreadsheetApp.getActiveSpreadsheet();
var lock = LockService.getScriptLock();
try {
lock.waitLock(3000); // wait 03 seconds for others' use of the code section and lock to stop and then proceed
} catch (e) {
Logger.log('Someone has just sent an Email. Try it again 3 seconds later.');
return HtmlService.createHtmlOutput("<b> Server Busy please try after some time <p>")
// In case this a server side code called asynchronously you return a error code and display the appropriate message on the client side
//return "Error: Server busy try again later... Sorry :("
}
/*
Gets certain data from current row. These are used as criteria
*/
GmailApp.sendEmail(email, "Text", name + " BODY.", { name: 'Diplay Custom Name as Sender' });
//Looks for a matching record in another sheet to mark its adjacent column as sent ("Yes")
for (var n = 0; n < formRespValues.length; n++) {
if (formRespValues[n][1] == testNo) {
formRespSheet.getRange('M' + (2 + n)).setValue('Yes');
}
}
}
SpreadsheetApp.flush(); // applies all pending spreadsheet changes
lock.releaseLock();
}
As #Tanaike pointed out in the comments, a short time within waitLock(timeoutInMillis) or tryLock(timeoutInMillis) may prevent the Lock Service from working as expected.
To ensure the correct functioning, try setting a time that is largely greater than the execution time of your script. If it is less than this, the lock will "open" prematurely, and cause errors such as those outlined in the question.
Documentation :
Lock Service
Class Lock
When a user logs into my website, I store a "logged in until" parameter on my server for that user, which is updated each time they interact with the server. So if they are inactive for X seconds, they should be logged out and the page on their client refreshed to show the login screen. Everything works nicely except the function keeps calling itself at 1 second intervals, and not the 1700+ second interval I'm looking for. As each call hits the server, I don't want this to keep running more often than it needs to.
This function is contained within my knockout viewmodel. logoutTimer is declared as a global variable. Watching the console log I can see the delay shows the correct time, but it seems to be running about every 1 second.
self.autoLoginCheck = function() {
clearTimeout(logoutTimer);
//If not, refresh the main screen => that'll push them to the login screen automatically
myurl = "/api/checkLoginStatus.php";
var parameters = {
'userPublicID':self.userInfo().userPublicID
}
$.post(myurl, parameters, function(data) {
var getResponse = JSON.parse(data);
if (getResponse.loggedIn < 1 ) {
//This is the log out condition
location.reload(true);
} else {
//Check how many more seconds the current log in is good for, that's when we'll check it again
var msdelay = getResponse.nextCheck * 1000;
console.log("Delay is " + msdelay);
logoutTimer = setTimeout(self.autoLoginCheck(), msdelay);
}
});
}
I know this type of question has been asked many times, but reading all those answers I can't see why this function is running every second and not waiting 30 minutes. Thanks to whoever shows me the obvious!
The first parameter to setTimeout should be a function, not the result of a function.
logoutTimer = setTimeout(self.autoLoginCheck, msdelay);
Say I've a browser extension which runs JS pages the user visits.
Is there an "outLoad" event or something of the like to start counting and see how long the user has spent on a page?
I am assuming that your user opens a tab, browses some webpage, then goes to another webpage, comes back to the first tab etc. You want to calculate exact time spent by the user. Also note that a user might open a webpage and keep it running but just go away. Come back an hour later and then once again access the page. You would not want to count the time that he is away from computer as time spent on the webpage. For this, following code does a docus check every 5 minutes. Thus, your actual time might be off by 5 minutes granularity but you can adjust the interval to check focus as per your needs. Also note that a user might just stare at a video for more than 5 minutes in which case the following code will not count that. You would have to run intelligent code that checks if there is a flash running or something.
Here is what I do in the content script (using jQuery):
$(window).on('unload', window_unfocused);
$(window).on("focus", window_focused);
$(window).on("blur", window_unfocused);
setInterval(focus_check, 300 * 1000);
var start_focus_time = undefined;
var last_user_interaction = undefined;
function focus_check() {
if (start_focus_time != undefined) {
var curr_time = new Date();
//Lets just put it for 4.5 minutes
if((curr_time.getTime() - last_user_interaction.getTime()) > (270 * 1000)) {
//No interaction in this tab for last 5 minutes. Probably idle.
window_unfocused();
}
}
}
function window_focused(eo) {
last_user_interaction = new Date();
if (start_focus_time == undefined) {
start_focus_time = new Date();
}
}
function window_unfocused(eo) {
if (start_focus_time != undefined) {
var stop_focus_time = new Date();
var total_focus_time = stop_focus_time.getTime() - start_focus_time.getTime();
start_focus_time = undefined;
var message = {};
message.type = "time_spent";
message.domain = document.domain;
message.time_spent = total_focus_time;
chrome.extension.sendMessage("", message);
}
}
onbeforeunload should fit your request. It fires right before page resources are being unloaded (page closed).
<script type="text/javascript">
function send_data(){
$.ajax({
url:'something.php',
type:'POST',
data:{data to send},
success:function(data){
//get your time in response here
}
});
}
//insert this data in your data base and notice your timestamp
window.onload=function(){ send_data(); }
window.onbeforeunload=function(){ send_data(); }
</script>
Now calculate the difference in your time.you will get the time spent by user on a page.
For those interested, I've put some work into a small JavaScript library that times how long a user interacts with a web page. It has the added benefit of more accurately (not perfectly, though) tracking how long a user is actually interacting with the page. It ignore times that a user switches to different tabs, goes idle, minimizes the browser, etc.
Edit: I have updated the example to include the current API usage.
http://timemejs.com
An example of its usage:
Include in your page:
<script src="http://timemejs.com/timeme.min.js"></script>
<script type="text/javascript">
TimeMe.initialize({
currentPageName: "home-page", // page name
idleTimeoutInSeconds: 15 // time before user considered idle
});
</script>
If you want to report the times yourself to your backend:
xmlhttp=new XMLHttpRequest();
xmlhttp.open("POST","ENTER_URL_HERE",true);
xmlhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
var timeSpentOnPage = TimeMe.getTimeOnCurrentPageInSeconds();
xmlhttp.send(timeSpentOnPage);
TimeMe.js also supports sending timing data via websockets, so you don't have to try to force a full http request into the document.onbeforeunload event.
The start_time is when the user first request the page and you get the end_time by firing an ajax notification to the server just before the user quits the page :
window.onbeforeunload = function () {
// Ajax request to record the page leaving event.
$.ajax({
url: "im_leaving.aspx", cache: false
});
};
also you have to keep the user session alive for users who stays long time on the same page (keep_alive.aspxcan be an empty page) :
var iconn = self.setInterval(
function () {
$.ajax({
url: "keep_alive.aspx", cache: false });
}
,300000
);
then, you can additionally get the time spent on the site, by checking (each time the user leaves a page) if he's navigating to an external page/domain.
Revisiting this question, I know this wouldn't be much help in a Chrome Ext env, but you could just open a websock that does nothing but ping every 1 second and then when the user quits, you know to a precision of 1 second how long they've spent on the site as the connection will die which you can escape however you want.
Try out active-timeout.js. It uses the Visibility API to check when the user has switched to another tab or has minimized the browser window.
With it, you can set up a counter that runs until a predicate function returns a falsy value:
ActiveTimeout.count(function (time) {
// `time` holds the active time passed up to this point.
return true; // runs indefinitely
});
Basically, I am designing a quiz application with limited time. Use selects answer to a question and the next question loads using an Ajax request. All questions must be answered within a time frame of, say 2 minutes.
A clock ticks away to show how much time is left and as soon as it hits 0, results are shown. Now since the timer will be implemented using window.setTimeout(), it is possible that the value of timer variable be modified using an external bookmarklet or something like that. Anyway I can prevent this? I think this is implemented on file sharing sites like megaupload. Any forgery on the timer variable results in request for file being rejected.
Have .setTimeout() call an AJAX method on your server to synch time. Don't rely on the client time. You could also store the start time on the server for a quiz, and then check the end time when the quiz is posted.
You need to add a validation in your server side. When the client want to load the next question using an Ajax request, check whether deadline arrived.
The timer in client side js just a presention layer.
If the function runs as a immediately called function expression, then there are no global variables and nothing for a local script to subvert. Of course there's nothing to stop a user from reading your code and formulating a spoof, but anything to do with javascript is open to such attacks.
As others have said, use the server to validate requests based on the clock, do not rely on it to guarantee anything. Here's a simple count down that works from a start time so attempts to dealy execution won't work. There are no global variables to reset or modify either.
e.g.
(function (){
// Place to write count down
var el = document.getElementById('secondsLeft');
var starttime,
timeout,
limit = 20; // Timelimit in seconds
// Function to run about every second
function nextTick() {
var d = new Date();
// Set start time the first time
if (!starttime) starttime = d.getTime();
var diff = d.getTime() - starttime;
// Only run for period
if (diff < (limit * 1000)) {
el.innerHTML = limit - (diff/1000 | 0);
} else {
// Time's up
el.innerHTML = 0;
clearTimeout(timeout);
}
}
// Kick it off
timeout = window.setInterval(nextTick, 1000);
}());