I've tried a bunch of different React table solutions (reactable, griddle and they all work very similar. You can pass them a set of data and they populate a table. They offer pagination, but not in the since where it's from the source. You can pull 100 Rows from the database, and only show 10 at a time via table pagination. However what if the data is coming from a database / Flux store, how are you supposed to pull what you need from the database?
In the case of a relational database, you would handle the pagination on the database level, if your RDBMS supports it, in the SQL query itself. What you need to do is the following:
Make an AJAX query to the backend with the pagination settings: page size and page number
In the backend, issue your query against the database, using the aforementioned settings, then prepare a JSON result.
Return the results to the frontend, parse it with JSON.parse(), then pass it to the grid component.
In the case of a Flux store, this is a bit more complex as it has different notions than a relational database. The basic principles are the same though, pass parameters to the access component, make a query, return JSON or JavaScript objects.
Pagination in RDMSes
For example, in MySQL (MariaDB) you would do the pagination with the LIMIT qualifier:
SELECT * FROM table LIMIT 20,10
The first parameter sets the first record fetched, the second limits the maximum results. So in this example case, the records between the 20th and 29th are fetched.
Other databases work differently as there is no standard way of doing this. You should look up the documentation of your RDMS.
Related
I have been assigned with a task to handle large number of data and show then to a webpage tabular form. I'm using HTML/JSP and JS for Frontend and Java as backend.
Business logic is to query database (for me it's Oracle) and get data.
Query looks something like
Select field1, field2 etc.. from table where field1 = "SearchString"
Limit 30
The search string will be given by user.
So, each time the query gets executed I'm getting 30 rows and storing it in a bean.
And with field2 data from iteration 1 I'll execute the query again which will give another 30 rows, I will append those in the bean and loop continues untill there is no matching records. After that I need to display the bean data in UI in tabular form.
Now problem arises when the data is huge. Like, the iteration goes on 1000 times giving 30k records. Then the code is getting stuck in this loop for more time and UI screen is showing loading.
Is there a better approach to my situation?
Note : I can't do any operation the query. Because it's forbidden.
And the query is pseudo query not actual. If the first record has matching record of 30k rows. I need to take 30 in each iteration.
I agree with the comments that this is not the best practice when you are trying to present thousands and thousands of rows to the UI...
It really sounds like you should implement pagination on your UI. This is done by using queries... I don't know what DB system you are using but here is a guide on pagination for SQL Server.
You can explain to the business that using pagination is better for the user. Use the example of how google search gives you pages of search results instead of showing you millions of websites of cat pictures all in one page.
I was wondering if it is somehow possible to do MongoDB No-SQL Queries (or even SQL-queries for sql database) in the Angular frontend, like formulating the query and then sending the query string to the database and getting a subset of values back (e.g. in JSON-format).
Because the problem that I currently have is, that for each data snippet that I need, I have to create a specific route (e.g for one specific graph) and then do my middleware-function for that route that queries for the data in mongodb (e.g by mongodb aggregation framework). And then finally, do a http-request in Angular frontend, eventuelly format the data in javascript in angular and then insert the data in correct format into the appropriate charting-framework (e.g ngx-charts).
E.g. If I would like to filter the data in some chart for specific fields, or do some aggregation etc. on the same chart dynamically, I probably would just query all data from beginning, and then create my own (e.g. filter or other data manipulation) functions in javascript for my chart functions.
I heard there are some ways to do this also in the backend, e.g in Node or MongoDB Aggregation framework, there exists $facet operator that can choose some path for data operation, but I actually don't know how this works together with the Frontend...
Is there not a more direct way to query just the data that I need for a specific component without setting up routing and everything else around?
I need to fetch huge data(may be some 10K records) from DB and show it as report(i use DataTable), and it has data filter/search and pagination.
Question - which one is best/recommended way from the below option,
I will fetch all the records at once and store it in front end(as a object) and if filter applies i will filter from the object and display it.
Likewise i wont interact with DB if i work with pagination(Since i have all the records with myself already)
Every time i need to contact the DB when i applies filter/search.
Likewise for pagination,
For example, if i select page 5 then i will send a query to DB to get me only those data and display it. Note: Number of record per page is also the option to select.
If we have any other best way, please guide me.
Thanks,
I am not familiar with DataTable, but it appears to be similar to jqGrid, which I'm familiar with.
I prefer your proposed solution #2. You are better off fetching only what you need. If you're only displaying, say, 100 rows, it's wasteful (both in terms of bandwidth and local memory usage) to fetch 10k rows at once if you're only displaying 100.
Use LIMIT on the MySQL side to fetch only the records you need. If you want, say, records 200 through 300 for page 3, you'd add LIMIT 200, 100 to the end of your query (the first parameter to LIMIT says "start at 200" and the second says "fetch 100 rows.") If DataTable works like jqGrid, you should be able to re-query the database and repopulate your table when the user changes pages, and this fetch will be done in the background with AJAX, which conserves bandwidth. Your query will be identical except for the range specified by the LIMIT at the end of your query.
Think of it this way: say you use GMail and you never archive your messages, so your inbox contains 20,000 emails, but only shows 100 per page. Do you think Google has designed the GMail front-end so that all 20k subject and from lines are fetched at once and stored locally, or is the server queried again when the user changes pages? (It's the latter.)
I've developed a web application with the concept of Single Page Application but none of the modern techs and frameworks.
So I have a jQuery page that dynamically requests data to localhost - a Laravel instance that compiles the entries in the DB (within a given time interval).
So the client wants to see all the entries for last week, the app works fine. But if he wants to see the results for the whole last month... well, they're so many that the default execution time of the php ins't enough to process all the data (30 seconds). I can easily override this, of course, but then the jQuery client will loop through these arrays of objects and do stuff with them (sort, find, sum...). So I'm not even sure jQuery can handle this many data.
So my question can be broken in two:
Can laravel ->paginate() be used so the ajax request of jQuery can also chunk the data? How does this work (hopefully in a manner that doesn't force me to rewrite all the code).
How could I store large amounts of information on the client? It's only temporary but the users will hang around for a considerable amount of time on my webpage, and I don't want them to wait 5 minutes every time they press a button
Thanks.
If you want to provide an interface to a large amount of data stored in a backed, you should paginate the data. This is a standard approach, so I'm sure your client will be ok with that.
Using pagination is pretty simple - see the docs for Laravel 5.0 here: http://laravel.com/docs/5.0/pagination
In order to paginate results in the backend, you need to call paginate($perPage) on your query instead of get() in your controller, like that:
$users = User::whereIsActive(true)->paginate(15);
This will return paginated result with 15 records per page. Page number will be taken from page parameter of the request. In order to get 3rd page of users, you'll need your frontend jQuery app to send a request to URL like:
/users?page=3
I don't recommend caching data in the frontend application. The data can be changed by some other user and you won't even know about it. And with pagination, your requests should be lightweight enough to stop worrying about a request sent to fetch every page of results.
Not sure if you're subscribed to laracasts but Jeffery Way is amazing in explaining features of Laravel and I highly recommend his videos.
In short you can paginate the results, then on the view when you call the foreach on your items you can array_chunk() the results to display them how you need to. But the paginated results are going to be fetched using a query in the URL, and i'm not sure that is what you want if you're already using a lot of jQuery to keep everything on the same page.
https://laracasts.com/lessons/crazy-simple-pagination
But assuming you're already paginating the results with whatever jQuery you've already written for the json data...
You could also use a query scope to get the data you need to for the amount of time to scope to create a simple api to use with ajax. I think that's probably what you're looking for.
So here's what I would do assuming you're already doing some pagination manually with your javascript.
Create a few query scopes to filter the data for different lengths of time
Create simple routes to fetch results from URI using the query scopes
Get the json data from the route preforming an ajax requests to the URIs created
More information on Query Scopes: http://laravel.com/docs/5.1/eloquent#query-scopes
This is the first time i am using REST for any web applications.
For normal get an post and i simply call the API done in Django Rest Framework.
But i am not able to think how can i deal with situations where something more needs to be done.
Suppose I have
List of users in database and their product they have bought.
Now i have web form where if someone adds the user and then submit the button , then
I have to get the list of items bought by that user in 5 hour window
Update the row in database which says buy_succeessful to false
Then again get the list of orders from the items he has bought and then update the rows with order_successful to false
Now current in my submit actions i am doing like
call to api to add the user in override manual enrty table. This is simple post to that table
Then after getting the sucessful tehn i again call api to list of items this user has bought using Query parameters . Then i have the list
Then again i loop through the list and post to api for updating that record in datbase
and so on
I am feeling this is not right.
I have found that quite often there are some more things to do tahn just saving individual objects in database.
whats the best way to do that. DO i need to have view api for every function
Try the 3rd step of the DRF Tutorial:
http://www.django-rest-framework.org/tutorial/3-class-based-views
Here, it shows how to do a "PUT" request for updating data. And also some of the other DRF features.
Also, you can reference serializer.object which is the object instance of the django model record that you are saving to the database. This question here talks about adding extra attributes, etc... before saving to the database:
Editing django-rest-framework serializer object before save
You can also access the record post_save and there are other hooks in the framework that you can use.