What I did for the CRUD in my app is that I select all the item from backend and load it to the front-end and loop the item out using js, to be specified I used ajax.
Think of my app is a todo list. Even if a user inserted a new item, I suppose still need to select all the items from db again after insert query right? same goes to delete, I may use remove() but still need to load so that my item id doesn't mess up. correct?
I using angularjs ng-repeat, I cant do like id++, then I bind the the id in ng-repeat with the object that I got from json form db.
if I have 1 thousand of item that will cause problem because I trigger the load function too much in backend, how to solve that?
Loading all the items from back end is invitation for disaster. It will kill back end and front end both. It becomes a serious usability problem if you dump 1000's of rows of data in the UI. How will the user wade through the data and act on them? Provide some way to filter the items. For example - if it is a todo list display one day at a time (default being today). For any other use case we can provide similar filtering mechanism. That way you query limited data from back end, take it to the UI and display it. If you cannot filter like this at least provide some way of pagination to limit the data you query and transport to the UI.
Related
I have been assigned with a task to handle large number of data and show then to a webpage tabular form. I'm using HTML/JSP and JS for Frontend and Java as backend.
Business logic is to query database (for me it's Oracle) and get data.
Query looks something like
Select field1, field2 etc.. from table where field1 = "SearchString"
Limit 30
The search string will be given by user.
So, each time the query gets executed I'm getting 30 rows and storing it in a bean.
And with field2 data from iteration 1 I'll execute the query again which will give another 30 rows, I will append those in the bean and loop continues untill there is no matching records. After that I need to display the bean data in UI in tabular form.
Now problem arises when the data is huge. Like, the iteration goes on 1000 times giving 30k records. Then the code is getting stuck in this loop for more time and UI screen is showing loading.
Is there a better approach to my situation?
Note : I can't do any operation the query. Because it's forbidden.
And the query is pseudo query not actual. If the first record has matching record of 30k rows. I need to take 30 in each iteration.
I agree with the comments that this is not the best practice when you are trying to present thousands and thousands of rows to the UI...
It really sounds like you should implement pagination on your UI. This is done by using queries... I don't know what DB system you are using but here is a guide on pagination for SQL Server.
You can explain to the business that using pagination is better for the user. Use the example of how google search gives you pages of search results instead of showing you millions of websites of cat pictures all in one page.
I have an application uses viewpanels to display data. One viewpanel displays unprocessed records and the other displays processed records. The user chooses an unprocessed record (using the show values in this column as links option), and is directed to a page where they input information. Then then click on button that updates the documents using doc.replaceItemValue statements in javascript. The user is then directed back to the viewpanel that displays the unprocessed records. In order to have the just processed record not show up in the unprocessed records I have to reindex the database. I am using database.updateFTIndex(false) to accomplish this.
Is there a better way to accomplish this? If two are more users are submitting records, will their individual indexes step on each other?
I never had to worry about this when using mysql.
Thanks for any advice.
I've used that technique for a while in production and not been notified of any issues. Updating an index via the Database Properties or a View gives the message that it has been queued for update on the server, but I'm not sure if the same happens with the programmatic call. It may well do.
In my scenario, I'm consolidating a lot of data into individual documents, so although intensive use periodically, it's not a huge number of documents being updated at any one time.
I'm also running the update to the index via sessionAsSigner, I had assumed that would be needed for authority purposes.
I am using the Nodejs Cassandra driver and I want to be able to retrieve the previous and next pages. So far the documentation shows the retrieval of the next page, which is saving the pageState from the previous page and passing it as a parameter. Sadly there is no info on how to navigate to the previous page.
As I see it there are two options:
Save each pageState and page as a key-value pair and use the pageState for the page that you want to navigate to.
Save the retrieved data in an array and use the array to navigate to the previous page. (I don't think that this is a good solution as I'll have to store large chunks the data in the memory.)
Both methods does not seem to be an elegant solution to me, but if I have to choose I'll use the first one.
Is there any way to do this out of the box using the Nodejs Cassandra driver?
Another thing is that in the documentation the manual paging is used by calling the eachRow function. If I understand it correctly it gives you every row as soon as it is red from the database. The problem is that this is implemented in my API and I am returning the data for the current page in the HTTP response. So in order for me to do that I'll have to push each row to a custom array and then return the array when the data for the current page is retrieved. Is there a way to use execute with the manual paging as the above seems redundant?
Thanks
EDIT:
This is my data model:
CREATE TABLE store_customer_report (
store_id uuid,
segment_id uuid,
report_time timestamp,
sharder int,
customer_email text,
count int static,
first_name text,
last_name text,
PRIMARY KEY ((store_id, segment_id, report_time, sharder), customer_email)
) WITH CLUSTERING ORDER BY (customer_email ASC)
I am displaying the data in a grid, so that the user can navigate trough it.
As I write this I thought of a way to do this without needing the previous functionality, but nevertheless I think that this is a valid case and it will be great if there is an elegant solution to it.
Sadly there is no info on how to navigate to the previous page.
That is correct, when you make a query and there are more rows, Cassandra returns a paging state to fetch the next set of rows, but not the previous ones. While what the paging state represents is abstracted away, it is generally a pointer to where to continue reading the next set of data, there really isn't a concept of reading the previous set of data (because you just read it).
Save each pageState and page as a key-value pair and use the pageState for the page that you want to navigate to.
This is the strategy i'd recommend too, of course to get the paging state you actually have to make the queries.
Is there any way to do this out of the box using the Nodejs Cassandra driver?
Not that I am aware of unfortunately, if you want to go back to previous pages you are going to need to track the state.
Is there a way to use execute with the manual paging as the above seems redundant?
Yep, you can provide pageState in the options parameter with execute as well and it will be regarded, i.e.:
client.execute('SELECT * FROM table', [], {pageState: pageState}, function (err, result) {
...
});
There is an option to be "more" manual, this was common before paging features. You can store the last partition/clustering keys of the current page you are sending back in response (possibly encrypted depending on what it is, most likely generate the partition key and only send/receive the clustering keys to avoid security issues). Then when doing your "next page" response you just have to start your CQL query from there. To go back a page, change ORDER BY clause of clustering key and walk backwards. If you provide your schema it would be easier to give examples. Thats all the pageState really does anyway.
This is the first time i am using REST for any web applications.
For normal get an post and i simply call the API done in Django Rest Framework.
But i am not able to think how can i deal with situations where something more needs to be done.
Suppose I have
List of users in database and their product they have bought.
Now i have web form where if someone adds the user and then submit the button , then
I have to get the list of items bought by that user in 5 hour window
Update the row in database which says buy_succeessful to false
Then again get the list of orders from the items he has bought and then update the rows with order_successful to false
Now current in my submit actions i am doing like
call to api to add the user in override manual enrty table. This is simple post to that table
Then after getting the sucessful tehn i again call api to list of items this user has bought using Query parameters . Then i have the list
Then again i loop through the list and post to api for updating that record in datbase
and so on
I am feeling this is not right.
I have found that quite often there are some more things to do tahn just saving individual objects in database.
whats the best way to do that. DO i need to have view api for every function
Try the 3rd step of the DRF Tutorial:
http://www.django-rest-framework.org/tutorial/3-class-based-views
Here, it shows how to do a "PUT" request for updating data. And also some of the other DRF features.
Also, you can reference serializer.object which is the object instance of the django model record that you are saving to the database. This question here talks about adding extra attributes, etc... before saving to the database:
Editing django-rest-framework serializer object before save
You can also access the record post_save and there are other hooks in the framework that you can use.
I have a simple (yet somehow convoluted) issue. Basically I'm adding items to make my web app more "desktop-like". For instance, right now I'm trying to get a page to dynamically load info into a DIV based on previously selected items. I'm currently using a cookie to handle saving the data, but I can't for the life of me get my brain to work this problem out.
I have a scenario with the following relationships:
SITE has_many BUILDINGS
BUILDING has_many METERS
METER
All entities can have associated charts. So, in an effort to make it generic, I set up a "has_many" relationship for each to CHARTS and abstracted it like so.
SITE has_many CHARTS, as chartable
BUILDING has_many CHARTS, as chartable
METER has_many CHARTS, as chartable
Once the user selects an item from the menu on the left, I then use a method to determine what item needs charts found and I display the particular item's charts. That all works fine.
My issue now is working with cookies in order to either save data to independent keys (or perhaps Marshal objects) in order to dynamically reload the previously selected item's data whenever the page reloads. The ajax call requires several values in order for the "update" action to find the correct item and display it. I'm having trouble with whether to use Javascript directly, try to trigger an action, or use some kind of combination.
As I said, I'm sure the issue is rather simple or straightforward, but I'm just not seeing it. If this description is a bit vague, I do apologize. Feel free to ask for more info.
Best
When the user selects an item from the menu, save all the necessary information to re-select that item to a cookie. Bind a Javascript method to the page load and check the value of that cookie. If the information is there indicating that an item should be preselected, just call the same Javascript method that is called when the user selects a new item from the menu. If you're using JQuery, for example, you might do something like this to bind to the page load:
$(document).ready(function() { /* check cookie and do stuff */ }
Another thing you could do is pre-render that stuff in your RoR code if that cookie exists so you don't immediately execute an AJAX call on page load (since that is sometimes considered bad form due to the page load performance hit).
This is too big for storing in cookies, you should either:
Store an id cookie client-side and store the data on the server-side which can be accessed with a corresponding id cookie and valid authentication credentials.
Use HTML5 client-side storage such as localStorage or a local database.