I'm currently displaying the content of an array in a view, let say $scope.array.
I load the content of an array with a request to my serv.
Sadly $scope.array contains a lot of elements an displaying every elements at once in the view takes a while.
In order to enhance user experience, I'd like to display the array part by part. At first I thought that $scope was able to handle it if I just proceed to add data chunk by chunk to $scope.array, but nope.
I figured out that the current $digest loop would only be over when my array was full. I tried with Async lib to add chunks asynchronously to $scope hoping for a way to dodge the $digest issue, but it doesn't work.
Now I kinda ran out of ideas to display datas properly, so if you had any experience with this kind of issues I'd be glad to hear about it !
Thanks a lot.
If pagination, periodic requests etc is ruled out then...
You can have all of your data in one array not bound to ui.
You then periodically add the data into a second array that is bound to the ui. Similar to how you mentioned you are already doing it but simply add the chunks in a $timeout or an $interval. This lets the $digests and page renders complete.
Simple Example:
<table>
<tr ng-repeat="item in shownItems track by $index">
<td>{{item}}</td>
</tr>
</table>
and in controller
app.controller('MainCtrl', ['$scope', '$timeout', function($scope, $timeout) {
//array that is bound to ui
$scope.shownItems = [];
//backing data
var fullItemsList = [];
//Create some fake data, you wouldn't do this in your program,
// this is where you would fetch from server
for(var ii = 0; ii < 50000; ii++){
fullItemsList.push("AYYYLMAO " + ii);
}
//How many items to add at a time
var chunkSize = 500;
//keep track of how many we have already added
var currentChunkIndex = 0;
//transfers a chunk of items to ui-bound array
function addMoreItems(){
var start = currentChunkIndex * chunkSize;
var end = (currentChunkIndex + 1) * chunkSize;
for(var ii = start; ii < end; ii++){
if(!fullItemsList[ii]){
break;
}
$scope.shownItems.push(fullItemsList[ii]);
}
currentChunkIndex++;
}
//Transfer chunk of items in a timeout, trigger another timeout to
//add more if there are stil items to transfer
function periodicAdd(){
$timeout(function(){
addMoreItems();
if(currentChunkIndex*chunkSize >= $scope.shownItems.length){
periodicAdd();
}
},0);
}
//Add the first chunk straight away, otherwise it will wait for the
//first timeout.
addMoreItems();
//Start the timeout periodic add.
periodicAdd();
}]);
Example plunkr
Keep in mind that this example is very rough. For instance the initial "load" of 50k rows will run on ui thread whereas your data load will presumably come async from your server. This also means you need to kick off the periodic adding only when your request completes. So yeah just a rough example.
Note that I use a 0 millisecond timeout.This will still push the callback to the end of the current processing queue, it wont execute straight away despite being 0 milliseconds. You might want to increase it a little bit to give your app a bit of breathing room. Remember to properly dispose of timeouts.
Use server side pagination. Even using one-time bindings is not always the solution, especially if there is complex template logic (show/hide parts depending on the data properties) and if the editing is required.
If you want to filter your array by some criteria (for example, month and year), implement this on the backend and pass your criteria to the server: GET /my_data?year=2017&month=7&page=1.
Related
I have a block of by project here in sandbox where I've used cherrio, https://cors-anywhere.herokuapp.com/, axios and manual built function to check for pagination, navigate to the next page and scrape the data and finally push the scraped data into the array of object.
The code somewhat works but gets stuck in the infinite loop while pushing the objects to the array.
The code is successful in scraping and pushing the data of the first page but gets stuck in infinite loop on other pages from pagination. Additionally The code does get in infinite loop if the url with no pagination is given.
Can anyone help me out to my mistake, it's been around a week that I've been trying to resolve this error.
Type Teacher for no pagination url and manager for pagination url.
Files to look at: src/store/modules/Site.js
Files to look at: src/store/modules/Helpers.js
The bug appears to be in makeObject(), shown below:
const makeObject = (jobs, img, org) => {
let mjobs = [];
for (let i = 0; i < jobs.length; i++) {
jobs.push({ // FIXME: Pushing into `jobs` which is being iterated
title: jobs[i],
img: "https://merojob.com" + img[i],
org: org[i]
});
}
return mjobs;
}
The for-loop iterates jobs, while pushing new objects into it. The termination condition checks that i is less than jobs.length, which is being incremented on each iteration, so it never exits the loop.
I think you meant to push into mjobs inside the for-loop.
I am currently using D3.js and have modified my chart from listening to mouseover/mouseout to mousemove. This has brought quite a few issues in the chart but none moreso than my GET statuses/show/:id requests.
Previously, I would have points on my chart to hover over and if there was a tweet within half an hour of that point (from a tweet DB in backend), it would send a GET request to get that tweet.
My problem now is that because I'm using mousemove in proximity to these points on my chart as opposed to mouseover, its firing this hundreds of times and the GET requests are limited to 900 in a 15-minute window.
var tweet_arr = [];
for(j in data_tweets){
var tweet_time = timeParser(data_tweets[j]['timestamp_s']);
var point_time = timeParser(d.timestamp);
var diff = point_time.getTime() - tweet_time.getTime();
if(diff<=1800000 && diff>=-1800000) {
tweet_arr.push(data_tweets[j]);
} else {
var tweet_list = []
d3.selectAll(".panel-body")
.data(tweet_list)
.exit()
.remove();
}
}
twitterapi.fetch().getTweets(tweet_arr, tweet_urls[0], tweet_urls[1]);
This function checks the difference between the nearest point on the x-axis and checks my collection of tweet data, if there is one in half an hour, add it to an array called tweet_arr and then pass that into my fetch() function which has an AJAX call to the Flask framework where I run my GET request by ID.
What I would ideally want it to do is have some check that if the request to fetch a specific tweet has been carried out in say, the last 5 seconds, don't run the fetch() function.
How would I go about doing something like this?
Have a look at debounce and throttle from underscore.js: http://underscorejs.org/#debounce,
http://underscorejs.org/#throttle
Here's a good, short post about debouncing requests: https://www.google.de/amp/s/davidwalsh.name/javascript-debounce-function/amp
For a comparison between throttle and debounce, see https://gist.github.com/makenova/7885923
You need to define your fetch logic in a separate function and put that one into _.debounce.
Have a look at this example: https://codepen.io/anon/pen/EQwzpZ?editors=0011
const fetchFromTwitter = function(s) { console.log(s) }
var lazyFetch = _.debounce(fetchFromTwitter, 100)
lazyFetch('This is')
lazyFetch('is')
lazyFetch('gonna be')
lazyFetch('legen ... ')
lazyFetch('wait for it')
lazyFetch('... dary')
lazyFetch('LEGENDARY')
I have an angular service that makes an Web API call out to retrieve my search results. The problem I'm having is the angular controller & UI is set up in a way that allows the search to be called multiple times per second causing the service to be queued up. I tried resolving/defer the http call when a new one comes in but it doesnt seem like the best solution. I would rather queue up all the search calls I get within a certain time period and then only execute the last one. Any ideas on how I could do that?
timeout(function(){
var length = queue.length
var item = queue[length - 1];
queue.splice(0, length);
processItem(item);
} , <yourtime:number>)
keep adding your requests to the queue. and add the processing logic to the processItem function.
this might do the needful
*note - please consider this as a pseudo code. might have compilations errors
Alternatively you can just create a bool variable which is referred every time a request is about to be made and done make the request till its true. Somethign like this
function processItem(item){
if(process){
process = false;
//YOUR ACTUAL PROCESSING CODE
}
}
$timeout(function(){
process = true;
}, <yourtime in milli seconds>)
I have a table of comments and a table of replies to comments and currently use AngularJS. I have a page for a user and load all the associated comments and replies on his/her profile. Since I'm just testing this on my local server for now, the sample size is very small (less than 10 rows returned). However, in order to create a scalable site, I'm wondering what I can do in terms of buttons or functions like "Load more comments" and "Load more replies".
Do I need to load all the comments of a user ALWAYS? What if the user has 3000 or even 10000 comments on his/her profile and there are more than 10000 users? Won't this take a long time for the profile page to even load? It seems like a waste to load all the comments and replies when less than 1/5 of them will even be seen in the first place. Is there anyway I can query the database to load the first 100 rows, than load the next 100 rows or something similar? What is the formal practice (and optimized for performance for this?)
As well, my general idea for "loading more comments" in Angular if I do always have to load all the comments is something like this:
$scope.allcomments = [some array of comments];
$scope.display = $scope.allcomments.slice(0, 5);
$scope.num = 5;
$scope.loadmore = function() {
var x = $scope.num
$scope.num += 5;
$scope.display.push($scope.allcomments.slice(x, $scope.num));
}
But is there no better way to do it?
I assume you're going to use ngRepeat to show the results, if so:
In view, i.e.:
<div ng-repeat="comment in allcomments | limitTo:num">{{comment}}</div>
In Controller:
$scope.loadmore = function() {
$scope.num += 5;
}
I am not too familiar with AJAX but I'm sure what I am trying to do is possible.
Basically I have a form with a text area, when the form is submitted the text area is made into an array with each new line being a value. Simple stuff, now my php then performs a database query on each array value. Also simple.
Now as the operation takes a while I want to make it into an AJAX call using jquery, and for every iteration on the array I want it to display the result back on my main page as well as displaying a progress bar.
So if my text area contains a list of 20,000 names and the query is to fetch the ages of these people, the ajax call would split the textarea into an array and iterate the array. For each iteration it should perform the query and send the result to my main page. So on my main page I will see a list that grows over time.
I hope I have explained this well enough, I just need advice on where to start/what to research. Any examples would be greatly appreciated.
Thanks!
As ajax is simply a request for data you cannot create a reliable loading bar easily.
However if you are doing loads of requests as you suggest in your question, "Which by the way is not the best idea".
You could do something like the code below. Again its pretty basic but will give your user a rough idea. I stuck it in a fidddle here for you http://jsfiddle.net/6dgAF/ its a basic example but should give you a jump off point.
function Status(data)
{
this.data = data;
this.size = this.data.length;
this.current = 0;
this.bar = $("#bar");
//Get width of each step of load
this.step = (this.bar.width() / this.size) ;
for(var i =0; i < this.size; i++){
this.getData(this.data[i]);
}
}
Status.prototype.getData = function(string){
//run your ajax here on each string and on success do this
//$.get( "ajax/test.html", function( string ) {
//this.current++;
//this.updateBar();
//});
this.current++;
this.updateBar();
}
Status.prototype.updateBar = function(){
//updates the bar with the current count * step width;
$("span",this.bar).width(this.step*this.current);
}
//Init object and put in dummy data
var status = new Status(new Array("string1","string2","string3","string4","string5","string6","string7","string8","string9","string10"));
An AJAX request is nothing else than requesting data from your server, just like any HTML page. The only difference is that you use Javascript, and don't feed the data to your browser as an HTML page, but use the data some way on your page.
One possible way to do the thing you want is the following:
Set an interval with the refresh rate, which calls a function getData
In getData you perform an AJAX request, which requests a piece of data from the server, which represents the current state. You might want to make sure the server only returns incremental updates, to avoid sending large amounts of data
Use the data on your front page, by displaying it in a friendly way to the user.
The main thing to keep in mind is that the server must save it's current state (the data which it has gathered with its queries), which it must return in the event of an AJAX request.
What you want is called XHR2 - personally I am using dojo
(dojo/request/xhr here)
However it should be possible with jQuery as well, search for jQuery XHR2 progress ...