I have an angular service that makes an Web API call out to retrieve my search results. The problem I'm having is the angular controller & UI is set up in a way that allows the search to be called multiple times per second causing the service to be queued up. I tried resolving/defer the http call when a new one comes in but it doesnt seem like the best solution. I would rather queue up all the search calls I get within a certain time period and then only execute the last one. Any ideas on how I could do that?
timeout(function(){
var length = queue.length
var item = queue[length - 1];
queue.splice(0, length);
processItem(item);
} , <yourtime:number>)
keep adding your requests to the queue. and add the processing logic to the processItem function.
this might do the needful
*note - please consider this as a pseudo code. might have compilations errors
Alternatively you can just create a bool variable which is referred every time a request is about to be made and done make the request till its true. Somethign like this
function processItem(item){
if(process){
process = false;
//YOUR ACTUAL PROCESSING CODE
}
}
$timeout(function(){
process = true;
}, <yourtime in milli seconds>)
Related
Okay, it might be a simple question, but so far I didn't find anything helpful by searching, so I am giving it a try here.
I am using plain old javascript/jquery on asp.net core on some project I am working on.
I am currently performing some actions on some employees in a foreach loop.
For each employee I am calling synchronously via ajax an API.
What I want is the UI to be updated, showing the current employee being processed in a progress bar.
While on debug, the process seems to work fine, but during normal process, it seems that the UI thread is not updated, only after all the work has been done. As such, as soon as I start processing the employees, the screen is stuck and closes after the work has been done. No progress bar is shown.
I only managed to show the progress animation and the first employee using the below trick
$('#applyYes').click(function (e) {
e.preventDefault();
var year = $('#yearCombo').val();
$('#applyInfo').hide();
$('#applyLoad').show();
$('#applyAction').prop('innerText', 'Calculating...');
setTimeout(function () {
var employeeIDs = multipleEmployees_Array();
for (var i = 1; i <= employeeIDs.length; i++) {
employeeID = employeeIDs[i - 1];
applyAmount(employeeID, year); //1. Updates progress bar 2. Ajax sync call
}
}, 0);
})
As far as I understand the e.preventDefault seems to move the timeout function being processed after UI thread finishes.
What is the optimal way of achieving what I want?
PS: No external libraries if possible. I am working on an third-party platform, that makes it difficult to add external libraries (policies, permissions etc.)
Synchronous HTTP requests are deprecated for precisely this reason. Don't use them.
Use Asynchronous requests instead.
If you want to run them sequentially then either:
Trigger i+1 in i's success function or
Use a Promise based API and await the results
I'm currently displaying the content of an array in a view, let say $scope.array.
I load the content of an array with a request to my serv.
Sadly $scope.array contains a lot of elements an displaying every elements at once in the view takes a while.
In order to enhance user experience, I'd like to display the array part by part. At first I thought that $scope was able to handle it if I just proceed to add data chunk by chunk to $scope.array, but nope.
I figured out that the current $digest loop would only be over when my array was full. I tried with Async lib to add chunks asynchronously to $scope hoping for a way to dodge the $digest issue, but it doesn't work.
Now I kinda ran out of ideas to display datas properly, so if you had any experience with this kind of issues I'd be glad to hear about it !
Thanks a lot.
If pagination, periodic requests etc is ruled out then...
You can have all of your data in one array not bound to ui.
You then periodically add the data into a second array that is bound to the ui. Similar to how you mentioned you are already doing it but simply add the chunks in a $timeout or an $interval. This lets the $digests and page renders complete.
Simple Example:
<table>
<tr ng-repeat="item in shownItems track by $index">
<td>{{item}}</td>
</tr>
</table>
and in controller
app.controller('MainCtrl', ['$scope', '$timeout', function($scope, $timeout) {
//array that is bound to ui
$scope.shownItems = [];
//backing data
var fullItemsList = [];
//Create some fake data, you wouldn't do this in your program,
// this is where you would fetch from server
for(var ii = 0; ii < 50000; ii++){
fullItemsList.push("AYYYLMAO " + ii);
}
//How many items to add at a time
var chunkSize = 500;
//keep track of how many we have already added
var currentChunkIndex = 0;
//transfers a chunk of items to ui-bound array
function addMoreItems(){
var start = currentChunkIndex * chunkSize;
var end = (currentChunkIndex + 1) * chunkSize;
for(var ii = start; ii < end; ii++){
if(!fullItemsList[ii]){
break;
}
$scope.shownItems.push(fullItemsList[ii]);
}
currentChunkIndex++;
}
//Transfer chunk of items in a timeout, trigger another timeout to
//add more if there are stil items to transfer
function periodicAdd(){
$timeout(function(){
addMoreItems();
if(currentChunkIndex*chunkSize >= $scope.shownItems.length){
periodicAdd();
}
},0);
}
//Add the first chunk straight away, otherwise it will wait for the
//first timeout.
addMoreItems();
//Start the timeout periodic add.
periodicAdd();
}]);
Example plunkr
Keep in mind that this example is very rough. For instance the initial "load" of 50k rows will run on ui thread whereas your data load will presumably come async from your server. This also means you need to kick off the periodic adding only when your request completes. So yeah just a rough example.
Note that I use a 0 millisecond timeout.This will still push the callback to the end of the current processing queue, it wont execute straight away despite being 0 milliseconds. You might want to increase it a little bit to give your app a bit of breathing room. Remember to properly dispose of timeouts.
Use server side pagination. Even using one-time bindings is not always the solution, especially if there is complex template logic (show/hide parts depending on the data properties) and if the editing is required.
If you want to filter your array by some criteria (for example, month and year), implement this on the backend and pass your criteria to the server: GET /my_data?year=2017&month=7&page=1.
I'm trying to write back end functionality that is handling requests to particular API, but this API has some restrictive quotas, especially for requests/sec. I want to create API abstraction layer that is able of delaying function execution if there are too many requests/s, so it works like this:
New request arrives (to put it simple - library method is invoked)
Check if this request could be executed right now, according to given limit (requests/s)
If it can't be executed, delay its execution till next available moment
If at this time a new request arrives, delay its execution further or put it on some execution queue
I don't have any constraints in terms of waiting queue length. Requests are function calls with node.js callbacks as the last param for responding with data.
I thought of adding delay to each request, which would be equal to the smallest possible slot between requests (expressed as minimal miliseconds/request), but it can be a bit inefficient (always delaying functions before sending response).
Do you know any library or simple solution that could provide me with such functionality?
Save the last request's timestamp.
Whenever you have a new incoming request, check if a minimum interval elapsed since then, if not, put the function in a queue then schedule a job (unless one was already scheduled):
setTimeout(
processItemFromQueue,
(lastTime + minInterval - new Date()).getTime()
)
processItemFromQueue takes a job from the front of the queue (shift) then reschedules itself unless the queue is empty.
The definite answer for this problem (and the best one) came from the API documentation itself. We use it for a couple of months and it perfectly solved my problem.
In such cases, instead of writing some complicated queue code, the best way is to leverage JS possibility of handling asynchronous code and either write simple backoff by yourself or use one of many great libraries to use so.
So, if you stumble upon any API limits (e.g. quota, 5xx etc.), you should use backoff to recursively run the query again, but with increasing delay (more about backoff could be found here: https://en.wikipedia.org/wiki/Exponential_backoff). And, if finally, after given amount of times you fail again - gracefully return error about unavailability of the API.
Example use below (taken from https://www.npmjs.com/package/backoff):
var call = backoff.call(get, 'https://someaddress', function(err, res) {
console.log('Num retries: ' + call.getNumRetries());
if (err) {
// Put your error handling code here.
// Called ONLY IF backoff fails to help
console.log('Error: ' + err.message);
} else {
// Put your success code here
console.log('Status: ' + res.statusCode);
}
});
/*
* When to retry. Here - 503 error code returned from the API
*/
call.retryIf(function(err) { return err.status == 503; });
/*
* This lib offers two strategies - Exponential and Fibonacci.
* I'd suggest using the first one in most of the cases
*/
call.setStrategy(new backoff.ExponentialStrategy());
/*
* Info how many times backoff should try to post request
* before failing permanently
*/
call.failAfter(10);
// Triggers backoff to execute given function
call.start();
There are many backoff libraries for NodeJS, leveraging either Promise-style, callback-style or even event-style backoff handling (example above being second of the mentioned ones). They're really easy to use if you understand backoff algorithm itself. And as the backoff parameters could be stored in config, if backoff is failing too often, they could be adjusted to achieve better results.
I'm trying transmit an image file from the server to the client, but my javascript callback becomes active before the stream closes I doing this because sending it in a traditional render json: times out and takes way to long anyway. The stream takes much less time, but i keep can't get all the data before the callback fires up.
controller code
def mytest
image=ImageList.new(AssistMe.get_url(image_url))
response.stream.write image.export_pixels(0, 0, image.columns, image.rows, 'RGBA').to_s
response.stream.close
end
javascript
var getStream, runTest;
runTest = function() {
return $.post('/dotest', getStream);};
getStream = function(params) {
return document.getElementById('whatsup2').innerHTML =
"stream is here " + params.length;};
the response is an array, I can make it an array of arrays by adding a "[" at the front and a "],['finish'] at the end to be able to detect the end of the data, but I haven't been able to figure out how to get javascript to wait until the end of stream to run. I assume i need to set up some kind of pole to check for the end, but how do I attach it to the callback?
Okay, here's a blog that describes this pretty well
blog
But i decided to forgo a stream and use .to_s. Since you can pipe several actions tougher
render object.method.method.to_s you get all the server side benefits of using a stream without the complexity. If you have a slow process where you need to overlap the client and server actions, then go to the blog and do it. Otherwise to_s covers it pretty well
im making a wordpress plugin and i have a function where i import images, this is done with a $.each()-loop that calls a .load()-function every iteration. The load-function page the load-function calls is downloading the image and returns a number. The number is imported into a span-element. The source and destination Arrays is being imported from LI-elemnts of a hidden ULs.
this way the user sees a counter counting from zero up to the total number of images being imported. You can se my jQuery code below:
jQuery(document).ready(function($) {
$('#mrc_imp_img').click(function(){
var dstA = [];
var srcA = [];
$("#mrc_dst li").each(function() { dstA.push($(this).text()) });
$("#mrc_src li").each(function() { srcA.push($(this).text()) });
$.each(srcA, function (i,v) {
$('#mrc_imgimport span.fc').load('/wp-content/plugins/myplugin/imp.php?num='+i+'&dst='+dstA[i]+'&src='+srcA[i]);
});
});
});
This works pretty good but sometimes it looks like the load function isn't updating the DOM as fast as it should because sometimes the numbers that the span is updated with is lower than the previous and almost everytime a lower number is replacing the last number in the end. How can i prevent this from happening and how can i make it hide '#mrc_imp_img' when the $.each-loop is ready?
AJAX calls which have been called earlier are not guaranteed to finish earlier so the smaller number can overwrite the bigger. One solution is to simply increment the counter on each successful call:
jQuery(function($) {
$('#mrc_imp_img').click(function(){
var dstList = $("#mrc_dst li");
var srcList = $("#mrc_src li");
dstList.each(function(i) {
var dst = $(this).text();
var src = srcList[i].text();
$.post('/wp-content/plugins/myplugin/imp.php?num='+i+'&dst='+dst+'&src='+src, function() {
$('#mrc_imgimport span.fc').text($('#mrc_imgimport span.fc').text()+1);
});
});
});
});
(Changed the code to avoid unnecessary array operations, changed onready call to use shorthand, changed AJAX call to use POST which should be used for operations that change state.)
Most servers likely have a finite number of threads running. If you're firing off 10 calls at once, and your server only has 5 threads, 5 of them will fail.
Also - once you max out all the running threads, no other users can access the server, so you're essentially DOS-ing the server.
If you don't mind slowing it down to one call at a time, do what Tgr recommended which serializes the calls, waiting until each one completes before starting the next one.
I would prefer what Yoda suggested. What you can do is turn it into one server call that processes the entire array. If you really want to update a counter client-side, that one server call can update a counter in the database - and then a 2nd ajax call can poll the server every few seconds to find out where the counter is. Obviously wont be guaranteed to be sequential but will be better for your server health. You could also fake the sequential aspect (if you're on #3 and the next call yields a #6 - increment it client side one by one)
As far as not seeing an alert, there is probably a javascript error before or on the alert line. Try using firebug and the console.log statement, or even bette, step through it with the firebug debugger.