In my application, I have an observableArray that loads data from DB. This observableArray fills first 25 items from DB and from scrolled down it loads another 25 and it goes on.
Now, I want to implement search, that should display the result searching the whole data from DB and not just from the displayed 25 items.
I tried to get the search result by sending the whole searching text to DB on clicking search button and there is lot of datas in DB which takes much time to load data.
Please let me know how I can get the desired result from DB within ms. Thanks in advance.
To get a well behaving search with Knockout, you should extend your searchText observable that is bound to the input with a rate-limiter
this.searchText = ko.observable('').extend({ rateLimit: { timeout: 500, method: "notifyWhenChangesStop" } })
This will call any subscribers after the input has remained unchanged for 500ms (i.e. when the user stops typing). You could also use the default method notifyAtFixedRate to call the API at least once every X seconds.
And to top it off, a fiddle!
Note: with this being said, if your query is taking 40 seconds, that sounds like a problem with your database query. It's possible that it's taking that long because you're flooding the server with requests, but that still seems awfully slow. This is the strategy we use and it works excellent, but our API response time is <200ms.
Related
I am using Node js with Express.js and I am getting data from firebase's real-time database
so How can I get data after 10 children or nodes by skipping?
example
I have a lot of data like -
Images
imageId
- {some properties ie: imgUrl:"somevalue.jpg"}
and more
So first I am taking first 10 images using limitFirst(10) function.
and if the user clicks on link or next button I want to load the next first 10 images if available
does firebase real-time database have some functionality like this
//limitToAfter(afterChild, limit);
const imageRef = query("Images", limitToAfter(10,10));
onValue(imageRef,(snapshot)=>{
//some operation here...
});
Or can anyone give me some ways I can solve this problem?
I will always appreciate any help :)
Firebase Realtime Database doesn't have an offset mechanism. Instead to get the next page of data, you need to know the last node on the current page.
For example, say that the key of the last item on page 1 is "key10", then you can get the next 10 items with:
query("Images", orderByKey(), startAfter("key10"), limitToFirst(10));
In words: take all nodes under Images, order them by their key, then start after "key10" and return the first 10 nodes.
I have a search component that gets its data in the form as a prop of PHP json_encoded data:
<search :data="<?= json_encode($jsonHotels) ?>"></search>
This way it receives the first 25 movies straight away when the page renders.
After this I make an ajax request to fetch the rest of the result if there are more than 25 movies.
The problem is that when I override the data with the data from the AJAX call, the component re-renders.
Previously to combat this I performed an ajax request with an offset of 25 to skip the first 25 movies and only give me the extra results. Pushing this onto the array didn't cause a re-render and this worked perfectly.
That was perfect, until I got to thinking:
What about people who start or refresh the page while they are on page number 2-3-4-5-etc.
I now have to not only mess with my offset, but I also have to possibly prepend data to my array and well as possibly append data with a push.
Is there a way to merge data without causing a re-render? I am wondering if anyone else has every run into the problem.
Simply fetching all results with an AJAX request takes too long and obviously also causes a re-render. Not fetching initial backend data would mean people would be starting at a blank page or a spinner for 2-3 seconds.
Edit:
Ended up with this:
mergeData(state) {
// Smart merge ajaxData and hotels.
const offSet = state.currentPage * state.itemsPerPage;
const start = state.ajaxData.slice(0, offSet - state.itemsPerPage);
const end = state.ajaxData.slice(offSet);
state.data.unshift(...start);
state.data.push(...end);
},
Pretty much slice my array in 2 and remove the currentPage section. By using unshift and push I prevent a re-render from happening.
Remember that your viewmodel is a model of your view. Here, you're not modeling your application, you're just trying to plug your data in.
If you want to display a page of results at a time, have some kind of model for a page of results. When you fetch your data, put it into the appropriate model-pages. Since you're only displaying one page at a time, populating other pages will not cause a re-render.
When using the Scroller extention for datatables, you don't have pagination but all rows in one scrollbar. I would like to know, what event is fired after scrolling down the tables, to e.g. see row 50-60 of 100.. I also need to know how to get that 10 rows out from the datatable. I'am using the lastest versions. Thanks alot.
This is how you get the rows from the current page in Datatable.net without the Scroller extention.
drawCallback: function (settings) {
var api = new $.fn.dataTable.Api(settings);
// Output the data for the visible rows to the browser's console
console.log(api.rows({ page: 'current' }).data().length);
}
Update 1: My Init of Scroller table:
initScrollerTable = function ($table, url, inclFilter, dataTableOptionsSpecific) {
$.ajax({
method: "POST",
url: url,
data: dataParameterHelper.getCommonData(),
dataType: "json"
})
.done(function (rows) {
var dataTableOptions = $.extend(
{},
{ data: rows },
dataTableOptionsSpecific
);
initTable($table, inclFilter, dataTableOptions);
});
};
Update 2 Deeper elaborating on the origional reason for the question to clarify.
Okay long story short. My table contained 26000 rows, and it took 7mins for me to load it. It contained of ALOT of DB calls and the JSON was a size of 21MB! I wanted to optimize it.
First attempt: I tripped my json to absolute minimum bringing it down to 1.5MB but it still took almost 7mins. On the second test it contained all the html, with hardcoded numbers/strings and i did 0 DB calls. It only took 3.5secs!. Yesterday I didn't knew I only had to focus on optimizing my calls to the DB.
Yesterday, when I posted the origional questing, my idea was to populate the table without any data pulled form the DB, and instead load them in a kind of lazy loading way. Let's say I showed the 10 first rows. with 3 column each where I need to call the DB for each cell, that's 30 times total. So the idea was to make 30 request for the (10) current rows, and replace the placeholder with the actually value. If you understand. I would still be better than 26.000 * 3 DB calls :)
And for that I needed to hook up on an event to get the 10 current rows-id's, I could then loop through making the 30 AJAX request. So maybe it's the scroller event I need for that, if there is someone like that.
But I don't know if it's a good idea. Usually a "good" idea is only "good" before/until you learn the best pratice :)
I think I will start focusing on reducing the DB calls with some inner joins and what have you, retrieving a big resultset I can loop through and populate all my 26.000(and later 50.000) rows, in under 15 secs!
SOLUTION
Use xhr event to handle Ajax requests and page.info() to retrieve information about the table as shown below.
Please note that event handler needs to be attached before you initialize your table to handle initial Ajax request.
$('#example').on('xhr.dt', function ( e, settings, json, xhr ) {
var api = new $.fn.dataTable.Api(settings);
var info = api.page.info();
console.log('Data has been loaded', info);
});
DEMO
See this jsFiddle for code and demonstration.
i'm currently building a website which searches an external database and brings up records which match the given search string. The search is live, so results are brought up as the user types.
now the first (and current) approach i took, is that the page actually connects to the mySQL server and retrieves content via AJAX, with EVERY letter the user types in the search box.
now i am starting to look at JSON objects (i only very recently started building websites), and was wondering if it would be a good idea, to load the entire database into a JSON object in the beginning and then look through that when searching.
is this a good idea? would it be faster? thanks in advance
It totally depends on the size of the data and the complexity of the query. If you can reasonably send the data to the client in advance and then search it locally, then sure, that's useful because it's all local and you don't have the latency of querying the server. But if you have a large amount of data, or the query is complex, it may well make more sense to do the query on the server.
There's no one-size-fits-all solution, it's data-dependent.
...and retrieves content via AJAX, with EVERY letter the user types in the search box.
That's usually overkill. Normally, you want to wait until there's a pause in the user's typing before firing off the ajax call, so that if they type "james" in rapid succession, you search for "james" rather than searching for "j", then "ja", then "jam", then "jame", and then "james".
For instance, let's say your search trigger is a keypress event. This would be a fairly common approach:
var keypressTimer = 0;
function handleKeypress() {
if (keypressTimer) {
cancelTimeout(keypressTimer);
}
keypressTimer = setTimeout(doSearch, 100); // 100ms = 1/10th of a second
}
function doSearch() {
var searchValue;
keypressTimer = 0;
searchValue = /*...get the search value...*/;
doAjaxCallUsing(searchValue);
}
This is called "debouncing" the input (from hardware engineering, related to the mechanical and electrical "bouncing" of a key as it's pressed).
I am not too familiar with AJAX but I'm sure what I am trying to do is possible.
Basically I have a form with a text area, when the form is submitted the text area is made into an array with each new line being a value. Simple stuff, now my php then performs a database query on each array value. Also simple.
Now as the operation takes a while I want to make it into an AJAX call using jquery, and for every iteration on the array I want it to display the result back on my main page as well as displaying a progress bar.
So if my text area contains a list of 20,000 names and the query is to fetch the ages of these people, the ajax call would split the textarea into an array and iterate the array. For each iteration it should perform the query and send the result to my main page. So on my main page I will see a list that grows over time.
I hope I have explained this well enough, I just need advice on where to start/what to research. Any examples would be greatly appreciated.
Thanks!
As ajax is simply a request for data you cannot create a reliable loading bar easily.
However if you are doing loads of requests as you suggest in your question, "Which by the way is not the best idea".
You could do something like the code below. Again its pretty basic but will give your user a rough idea. I stuck it in a fidddle here for you http://jsfiddle.net/6dgAF/ its a basic example but should give you a jump off point.
function Status(data)
{
this.data = data;
this.size = this.data.length;
this.current = 0;
this.bar = $("#bar");
//Get width of each step of load
this.step = (this.bar.width() / this.size) ;
for(var i =0; i < this.size; i++){
this.getData(this.data[i]);
}
}
Status.prototype.getData = function(string){
//run your ajax here on each string and on success do this
//$.get( "ajax/test.html", function( string ) {
//this.current++;
//this.updateBar();
//});
this.current++;
this.updateBar();
}
Status.prototype.updateBar = function(){
//updates the bar with the current count * step width;
$("span",this.bar).width(this.step*this.current);
}
//Init object and put in dummy data
var status = new Status(new Array("string1","string2","string3","string4","string5","string6","string7","string8","string9","string10"));
An AJAX request is nothing else than requesting data from your server, just like any HTML page. The only difference is that you use Javascript, and don't feed the data to your browser as an HTML page, but use the data some way on your page.
One possible way to do the thing you want is the following:
Set an interval with the refresh rate, which calls a function getData
In getData you perform an AJAX request, which requests a piece of data from the server, which represents the current state. You might want to make sure the server only returns incremental updates, to avoid sending large amounts of data
Use the data on your front page, by displaying it in a friendly way to the user.
The main thing to keep in mind is that the server must save it's current state (the data which it has gathered with its queries), which it must return in the event of an AJAX request.
What you want is called XHR2 - personally I am using dojo
(dojo/request/xhr here)
However it should be possible with jQuery as well, search for jQuery XHR2 progress ...