I have thousands of records in Database and I'm giving user interface with just one Textbox where he will be putting date and records will be searched on basis of Date put in textbox. Now searching logic will be like this, ajax call will go to servlet and there all records will be fetched by firing query to database and will be put in Map and send Map back to ajax with JSON format.
Ajax will iterate over the records with each function and will match the records with date equality.Those who are Matched will be shown using .html(header+data)
Now How this whole process can be made faster or is there any another way of showing the tables with thousand records in faster way at just click of the button.I need help on this.Sorry for the BIG question.
Related
I have been assigned with a task to handle large number of data and show then to a webpage tabular form. I'm using HTML/JSP and JS for Frontend and Java as backend.
Business logic is to query database (for me it's Oracle) and get data.
Query looks something like
Select field1, field2 etc.. from table where field1 = "SearchString"
Limit 30
The search string will be given by user.
So, each time the query gets executed I'm getting 30 rows and storing it in a bean.
And with field2 data from iteration 1 I'll execute the query again which will give another 30 rows, I will append those in the bean and loop continues untill there is no matching records. After that I need to display the bean data in UI in tabular form.
Now problem arises when the data is huge. Like, the iteration goes on 1000 times giving 30k records. Then the code is getting stuck in this loop for more time and UI screen is showing loading.
Is there a better approach to my situation?
Note : I can't do any operation the query. Because it's forbidden.
And the query is pseudo query not actual. If the first record has matching record of 30k rows. I need to take 30 in each iteration.
I agree with the comments that this is not the best practice when you are trying to present thousands and thousands of rows to the UI...
It really sounds like you should implement pagination on your UI. This is done by using queries... I don't know what DB system you are using but here is a guide on pagination for SQL Server.
You can explain to the business that using pagination is better for the user. Use the example of how google search gives you pages of search results instead of showing you millions of websites of cat pictures all in one page.
I need to fetch huge data(may be some 10K records) from DB and show it as report(i use DataTable), and it has data filter/search and pagination.
Question - which one is best/recommended way from the below option,
I will fetch all the records at once and store it in front end(as a object) and if filter applies i will filter from the object and display it.
Likewise i wont interact with DB if i work with pagination(Since i have all the records with myself already)
Every time i need to contact the DB when i applies filter/search.
Likewise for pagination,
For example, if i select page 5 then i will send a query to DB to get me only those data and display it. Note: Number of record per page is also the option to select.
If we have any other best way, please guide me.
Thanks,
I am not familiar with DataTable, but it appears to be similar to jqGrid, which I'm familiar with.
I prefer your proposed solution #2. You are better off fetching only what you need. If you're only displaying, say, 100 rows, it's wasteful (both in terms of bandwidth and local memory usage) to fetch 10k rows at once if you're only displaying 100.
Use LIMIT on the MySQL side to fetch only the records you need. If you want, say, records 200 through 300 for page 3, you'd add LIMIT 200, 100 to the end of your query (the first parameter to LIMIT says "start at 200" and the second says "fetch 100 rows.") If DataTable works like jqGrid, you should be able to re-query the database and repopulate your table when the user changes pages, and this fetch will be done in the background with AJAX, which conserves bandwidth. Your query will be identical except for the range specified by the LIMIT at the end of your query.
Think of it this way: say you use GMail and you never archive your messages, so your inbox contains 20,000 emails, but only shows 100 per page. Do you think Google has designed the GMail front-end so that all 20k subject and from lines are fetched at once and stored locally, or is the server queried again when the user changes pages? (It's the latter.)
I currently have a spring mvc app that gets a list of users from a database and displays their information in a table using JSP to basically loop through each object in the list and create a table row for them.
Each user has an expiry date attribute as part of their record in the database. What I want to achieve is basically a button that when toggled shows or hides all users that have expired (i.e. their expiry date is less than today's date).
For this I am trying to use AJAX calls to my controller to fetch me all users expired or not OR only users that haven't expired depending on how the button is toggled.
What I would like help on is the best way to achieve this as I can think of a few nasty ways of doing it like having a separate page and refreshing but I am confused on a few things.
Should I just ditch the JSP looping through to make the table and make a method in JavaScript that creates that table when given the data? If so how do I get the data from the controller to JavaScript, can an AJAX call to a controller return me a list of my user objects?
My best guess is that instead of adding a list of objects to a model and letting JSP do the work, that I instead return a JSON with the data and use JavaScript to build the table. I can then call an update method to re-build the table.
You are correct. You have 2 options:
Have AJAX call return html (i.e. jsp) for the table and then replace
the body of the table
Use JavaScript to build the table and then
update the table with AJAX call which returns JSON.
If you want to get more sophisticated, you could use a JavaScript framework like Knockout.js which would let you mark up the table and refresh the table without too much of JavaScript writing.
Blurgh I'm not sure why this question has received so much attention, especially now in the days of angular but if you are struggling with this then I would strongly recommend the following library:
https://www.ag-grid.com/
What I did for the CRUD in my app is that I select all the item from backend and load it to the front-end and loop the item out using js, to be specified I used ajax.
Think of my app is a todo list. Even if a user inserted a new item, I suppose still need to select all the items from db again after insert query right? same goes to delete, I may use remove() but still need to load so that my item id doesn't mess up. correct?
I using angularjs ng-repeat, I cant do like id++, then I bind the the id in ng-repeat with the object that I got from json form db.
if I have 1 thousand of item that will cause problem because I trigger the load function too much in backend, how to solve that?
Loading all the items from back end is invitation for disaster. It will kill back end and front end both. It becomes a serious usability problem if you dump 1000's of rows of data in the UI. How will the user wade through the data and act on them? Provide some way to filter the items. For example - if it is a todo list display one day at a time (default being today). For any other use case we can provide similar filtering mechanism. That way you query limited data from back end, take it to the UI and display it. If you cannot filter like this at least provide some way of pagination to limit the data you query and transport to the UI.
I'm fairly new in web design/code and I'm trying to figure out/learn how to write script that would execute my table filter. As I found out this would be easiest to do in PHP since all data comes from table itself (and not MySQL DB).
Here is example of HTML with test table and some filters I would like to implement, but I've been having problem with writing PHP script which would work properly.
http://www.bonemachineonline.com/test
(I uploaded example on my own site)
I would appreciate if someone could help me out with this in anyway! Writing example or explaining how/what to do. Anything is more than appreciate!
Try using jquery dataTable().
You can refer the below link
Jquery-dataTable()
You have 3 approaches for data filtering.
If the table is filtered by default, and all the data are not loaded, take the (1) or (2).
If you load the whole table each time, take the (3).
Approaches
(1) Server side, with page reloading : your user checks filters, then clicks on a submit button. You get all the checked filters and you send it to your PHP. Server side, you retrieve the filters and remake your query (SQL or other in your case).
(2) Server side, with AJAX : exactly the same mechanism, but you send the filters to your PHP with an AJAX request, wait a response, remove your table and remake a table with the new data.
(3) Client side, with JavaScript only : when your user checks a filter, you trigger the change event and hide the matched elements in the table.
Some tips
For the approaches (2) and (3), jQuery could be very useful.
For the approach (3), you can use a library. Here: 35+ Useful jQuery Filter and Sort Plugins.
For the approach (1), if in your case the data are in a PHP array, you have a lot of functions to manipulate this array in function of your filters. Here: Array Functions - Table of Contents.