Limit GET requests (D3.js/Twitter API) - javascript

I am currently using D3.js and have modified my chart from listening to mouseover/mouseout to mousemove. This has brought quite a few issues in the chart but none moreso than my GET statuses/show/:id requests.
Previously, I would have points on my chart to hover over and if there was a tweet within half an hour of that point (from a tweet DB in backend), it would send a GET request to get that tweet.
My problem now is that because I'm using mousemove in proximity to these points on my chart as opposed to mouseover, its firing this hundreds of times and the GET requests are limited to 900 in a 15-minute window.
var tweet_arr = [];
for(j in data_tweets){
var tweet_time = timeParser(data_tweets[j]['timestamp_s']);
var point_time = timeParser(d.timestamp);
var diff = point_time.getTime() - tweet_time.getTime();
if(diff<=1800000 && diff>=-1800000) {
tweet_arr.push(data_tweets[j]);
} else {
var tweet_list = []
d3.selectAll(".panel-body")
.data(tweet_list)
.exit()
.remove();
}
}
twitterapi.fetch().getTweets(tweet_arr, tweet_urls[0], tweet_urls[1]);
This function checks the difference between the nearest point on the x-axis and checks my collection of tweet data, if there is one in half an hour, add it to an array called tweet_arr and then pass that into my fetch() function which has an AJAX call to the Flask framework where I run my GET request by ID.
What I would ideally want it to do is have some check that if the request to fetch a specific tweet has been carried out in say, the last 5 seconds, don't run the fetch() function.
How would I go about doing something like this?

Have a look at debounce and throttle from underscore.js: http://underscorejs.org/#debounce,
http://underscorejs.org/#throttle
Here's a good, short post about debouncing requests: https://www.google.de/amp/s/davidwalsh.name/javascript-debounce-function/amp
For a comparison between throttle and debounce, see https://gist.github.com/makenova/7885923
You need to define your fetch logic in a separate function and put that one into _.debounce.
Have a look at this example: https://codepen.io/anon/pen/EQwzpZ?editors=0011
const fetchFromTwitter = function(s) { console.log(s) }
var lazyFetch = _.debounce(fetchFromTwitter, 100)
lazyFetch('This is')
lazyFetch('is')
lazyFetch('gonna be')
lazyFetch('legen ... ')
lazyFetch('wait for it')
lazyFetch('... dary')
lazyFetch('LEGENDARY')

Related

Angular Timeout Queue

I have an angular service that makes an Web API call out to retrieve my search results. The problem I'm having is the angular controller & UI is set up in a way that allows the search to be called multiple times per second causing the service to be queued up. I tried resolving/defer the http call when a new one comes in but it doesnt seem like the best solution. I would rather queue up all the search calls I get within a certain time period and then only execute the last one. Any ideas on how I could do that?
timeout(function(){
var length = queue.length
var item = queue[length - 1];
queue.splice(0, length);
processItem(item);
} , <yourtime:number>)
keep adding your requests to the queue. and add the processing logic to the processItem function.
this might do the needful
*note - please consider this as a pseudo code. might have compilations errors
Alternatively you can just create a bool variable which is referred every time a request is about to be made and done make the request till its true. Somethign like this
function processItem(item){
if(process){
process = false;
//YOUR ACTUAL PROCESSING CODE
}
}
$timeout(function(){
process = true;
}, <yourtime in milli seconds>)

Functional Javascript BaconJS, how can I push more values to an event stream?

I'm attempting to create a stack of AJAX responses in BaconJS. That processes them in a first in first out fashion, but each 'out' event should wait for user input.
This is where I'm at now: Live JSBin
var pages = Bacon.fromArray([1,2,3])
var next = $("#next").asEventStream('click').map(true);
pages.flatMapConcat(asyncFunction).zip(next).log("responses")
function asyncFunction(page) {
// Simulating something like an AJAX request
return Bacon.later(1000 + (Math.random() * 3000), "Page "+ page)
}
Currently this synchronously outputs an event from the pages EventStream each time that #next is clicked, which the behavior I want.
However, I am unable to figure out how to push more values to the pages EventStream. I have attempted to replace the pages EventStream with a Bus, and pushing values like this (which doesn't work).
var pages = new Bacon.Bus()
pages.push("value")
How do I push more values to an EventStream?
I know this is an OLD post, but bus would work. Just push the number (you had an array of numbers before) into it:
var pages = new Bacon.Bus();
// Bacon.fromArray([1,2,3])
var next = $("#next").asEventStream('click').map(true);
pages.flatMapConcat(asyncFunction).zip(next).log("responses")
function asyncFunction(page) {
// Simulating something like an AJAX request
return Bacon.later(1000 + (Math.random() * 3000), "Page "+ page)
}
pages.push(1);
pages.push(2);
pages.push(3);
I cloned your jsbin and changed it to use bus
As mentioned previously, you could stream the source of the page values using something like fromEvent or from fromBinder

Google Visualization Query - Abort Request

I am working on an interface that contains several high-intensity SQL queries, then renders multiple Google charts which are currently being called via AJAX using the google.visualization.Query object:
var query = new google.visualization.Query('/report.ashx');
query.send(callbackMethod);
//....
function callbackMethod(rs){
var data = rs.getDataTable();
var graph = new google.visualization.AreaChart(target_div);
graph.draw(data);
}
As the interface can be filtered dynamically I have encountered a scenario whereby a query can be running, whilst the user could potentially choose to re-filter and hence re-query the data source. If the first query is still running and the second query begins and returns before the first query then the chart will be drawn just fine. However, when the first query finally completes it could completely overwrite the chart with the old data, ignoring the current filters.
I have read that there is the ability to pass a jQuery AJAX object in, which exposes an XHR object allowing me to call .abort() on the XHR which would cancel the request (albeit this would still process on the server, but that's a hit I'm willing to take). Unfortunately I can find no examples of this and the Google documentation is less than helpful in this respect. Has anybody encountered the same, and if so - have they solved the problem?
Cheers,
Chris.
You can wrap callbackMethod in a higher order function to keep track of the request time, and use query.send(callback()) instead of query.send(callbackMethod).
var mostRecentReqTime;
var callback = function() {
var reqTime = Date();
return function() {
if (reqTime < mostRecentReqTime) return;
var data = rs.getDataTable();
var graph = new google.visualization.AreaChart(target_div);
graph.draw(data);
mostRecentReqTime = reqTime;
};
};

AJAX progress bar on array iteration?

I am not too familiar with AJAX but I'm sure what I am trying to do is possible.
Basically I have a form with a text area, when the form is submitted the text area is made into an array with each new line being a value. Simple stuff, now my php then performs a database query on each array value. Also simple.
Now as the operation takes a while I want to make it into an AJAX call using jquery, and for every iteration on the array I want it to display the result back on my main page as well as displaying a progress bar.
So if my text area contains a list of 20,000 names and the query is to fetch the ages of these people, the ajax call would split the textarea into an array and iterate the array. For each iteration it should perform the query and send the result to my main page. So on my main page I will see a list that grows over time.
I hope I have explained this well enough, I just need advice on where to start/what to research. Any examples would be greatly appreciated.
Thanks!
As ajax is simply a request for data you cannot create a reliable loading bar easily.
However if you are doing loads of requests as you suggest in your question, "Which by the way is not the best idea".
You could do something like the code below. Again its pretty basic but will give your user a rough idea. I stuck it in a fidddle here for you http://jsfiddle.net/6dgAF/ its a basic example but should give you a jump off point.
function Status(data)
{
this.data = data;
this.size = this.data.length;
this.current = 0;
this.bar = $("#bar");
//Get width of each step of load
this.step = (this.bar.width() / this.size) ;
for(var i =0; i < this.size; i++){
this.getData(this.data[i]);
}
}
Status.prototype.getData = function(string){
//run your ajax here on each string and on success do this
//$.get( "ajax/test.html", function( string ) {
//this.current++;
//this.updateBar();
//});
this.current++;
this.updateBar();
}
Status.prototype.updateBar = function(){
//updates the bar with the current count * step width;
$("span",this.bar).width(this.step*this.current);
}
//Init object and put in dummy data
var status = new Status(new Array("string1","string2","string3","string4","string5","string6","string7","string8","string9","string10"));
An AJAX request is nothing else than requesting data from your server, just like any HTML page. The only difference is that you use Javascript, and don't feed the data to your browser as an HTML page, but use the data some way on your page.
One possible way to do the thing you want is the following:
Set an interval with the refresh rate, which calls a function getData
In getData you perform an AJAX request, which requests a piece of data from the server, which represents the current state. You might want to make sure the server only returns incremental updates, to avoid sending large amounts of data
Use the data on your front page, by displaying it in a friendly way to the user.
The main thing to keep in mind is that the server must save it's current state (the data which it has gathered with its queries), which it must return in the event of an AJAX request.
What you want is called XHR2 - personally I am using dojo
(dojo/request/xhr here)
However it should be possible with jQuery as well, search for jQuery XHR2 progress ...

Javascript: multiple events, one ajax request

I'm somewhat new to JS and I'm trying to think of the best way to design some asynchronous interaction in an application I'm working on.
I've got a list of records that are related to some live API-backed data. I show the user a list of these records, and the user can select specific records that they want to see more information about. I load this additional data from the API via an ajax call.
To make this a more real world example, let's say what I have is a list of stocks. I've got the name and yesterday's closing price for each stock. There's a check box next to each stock name, and if the user checks this box it plots the historic price of the stock for the past year on a graph.
When the user selects one stock in the way, the behavior is simple. I send one API request for the historical data for one stock, and I plot it on the graph.
However, the user might select a bunch of stocks at once, or in rapid succesion. I don't want to fire 10 or 20 or 50 requests back-to-back, I want to make one request for 10 or 20 or 50 stock histories.
Let's say my application has an event listener that looks up the stock history when the check box is toggled, something like this:
$('input.stock_toggle').change( function(event){
var symbol = $(this).data('symbol');
lookupStockHistory(symbol);
});
How could I define a lookupStockHistory function, or some other kind of event listener etc., that would wait a second and pool all the events that came in to send a single request instead of firing many times in row?
var lookupStockHistory = (function () {
"use strict";
var qeue = [], timeoutHandler = null, timeoutTime = 1000,
sendCall = function () {
//process qeue array and trigger ajax call
//and clean qeue
qeue = [];
},
add = function (symbol) {
if (timeoutHandler) {
clearTimeout(timeoutHandler);
timeoutHandler = null;
}
qeue.push(symbol);
timeoutHandler = setTimeout(sendCall, timeoutTime);
};
return add;}());
To trigger just call lookupStockHistory(symbol). This will gather symbol to array which will be processed after 1 second since last call
You can use push your request into a "global variable" with your namespace, and then use setTimeout to delay the AJAX call (a second or two maybe?).
The setTimeout would call a function that gets the requests from the "global variable", empties the variable, and then constructs your request. Any subsequent calls to the setTimeout function would see that the "global variable" was empty and not construct future AJAX requests.
In the example below, I also remove the current pending timeout as a new one has been initiated.
Here's a pseudo-code example using jQuery for selection and event capture:
var DELAY_FOR_INPUT = 2000; // 2 seconds
var g_MyDataToRequest = [];
var g_currentAJAXCallTimeout = null;
function _callAPI = new function() {
g_currentAJAXCallTimeout = null;
var dataToGet = g_MyDataToRequest;
g_MyDataToRequest = []; // clear out global request cache
// TODO: Loop over dataToGet and construct AJAX request
// TODO: Perform AJAX call...
}
$('.myCheckbox').click(function() {
var myRequest = $(this).attr("ID"); // or Val(), or whatever you'd like to track your item
g_MyDataToRequest.push( myRequest );
// If have a pending request, kill it
if (g_currentAJAXCallTimeout != null) {
clearTimeout(g_currentAJAXCallTimeout);
g_currentAJAXCallTimeout = null;
}
g_currentAJAXCallTimeout = setTimeout( _callAPI, DELAY_FOR_INPUT );
});
This is, as noted, pseudocode and may not quite work right, but it should get you going.
You could implement a timer and start it with the first click or change event. And with each additional click or change event you can reset the timer. Also, with each event you can add or remove the symbol value to an array accordingly. Once the timer expires, you join the array to be a comma-delimited string and post that back via ajax and get a JSON result.

Categories

Resources