I'm writing the code for a web application using PHP, MySQL, and javascript.
It's a very simple social network where users can create posts and see, like and comment other users posts. On the main page it loads posts and orders them based on an index based on the number of likes, comments and when the post was created.
Since I can't load every post at once (because ideally there can be millions of them), I load the top N posts and then when I scroll down it loads more posts (with an Ajax request) and adds them at the bottom of the page. The problem is that since the posts are ordered dynamically, if I just limit the number of posts and then offset them in the later requests, I sometimes get the same post twice, and some posts never gets shown. How can I solve this?
Right now it just checks with javascript the id of every new post and it just discards the ones that are already on the page (checking the id of the posts on the page), but I dont like it because every time it loads more posts it has to check if every single post is already on the page or not and if the number of post grows it will get very slow.
If you have a lot of processing power in the server and you're able to do the scoring for any time in the past (e.g. taking into account only likes previous this time), you can make your page send: (1) the timestamp of the first loading, and (2) the number of post already there (or, better, the score of the worst post it received, because posts could have been deleted since then).
Let the server compute the list of the past and send the next posts.
If you have more memory in the server than processing power, then you could just save a copy of the scored list for some time (1h, 24h…) whenever someone request it.
And if you want the server to send the new best post of the moment: if you know the present listing and you know what the user already got (with their timestamp and last score they got), you can make the server removes what they already have and gives them the rest.
Related
I have a page with user data, with ajax pagination using PHP. User's name is listed in descending order(last inserted data will be shown in first page). Suppose there are 3 pages having 2 data's(user name) per page . Imagine, i visited the second page it shows some data, imagine(Alan and Arun - user names). At the same time couple of data is inserted in the database and i requested the 3rd page. But it will show the same data as previous,as a couple of data is inserted in the database. My Question is - How can i handle this. It will create a confusion to the user. Hope you get my point.
One easy solution I could think of would be a timestamp, when you first requested the page. Now you keep this timestamp while navigating through the pages and only display entries older than the timestamp. This way, no new entries would pollute the result and mess with the pagination.
But you'd need to think of a mechanism to let the user include newer entries. E.g. a box display at the top saying "New entries available. Click here to refresh.". A click on this box would refresh the timestamp and navigate back to page 1.
You can create some kind of ping service in JS to see if there is any new update on the server. If there is: fetch it and update in your App. Also, you can create a unique identifier to solve the problem of duplication.
I've been doing some research about infinite scrolling and came across what people call "Lazy Loading". Now I have done it on one of my website elements (chatbox), and now I have 2 ways to do it. but I can't decide which one is more efficient than the other. Here are my ways:
Let's say I have 1,000,000 rows of data from database which I need to fetch all
1st way:
Load content from database, truncate it on the server-side code(PHP) then only show the first 50.
Upon user scroll on the page, another request will be sent to fetch the results again and display the next 50 and so on and so forth..
2nd way:
Load content from database, render it in my HTML as hidden elements but only displaying the first 50, then upon user scroll, show 50 more hidden elements.
The 1st way is requesting from the server whenever the need to display more results arises.
The 2nd way just does 1 request from the server, then hides the result except for the first few that should be shown.
I have no problem doing either of the two.
Now the dilemma, is that the 1st way is sending more HTTP requests, and the 2nd way (though sending only 1 HTTP request) is fetching huge data in a single request, which can be slow.
Which method is "better", and if there are other options, please let me know.
I know this is old question but want to give my opinion:
1st way
Is always preferred specially with that amount of rows, it is much more efficient to only request a set number of rows and if user wants to see more e.g click in next page another request is made to the server to fetch the next page, the response time will be much better, it will also make it easier to the client to manipulate the list if there is other processing that needs to be done before is returned to the user.
You also need to make sure that you are applying the limits in your DB query otherwise you will be loading all the objects into memory which is not efficient.
2nd way
If you fetch 1,000,000 rows at once the user will have to wait until the response comes back which can result in a bad user experience also as the number of rows returned keeps growing the response time will keep increasing and you can hit a time-out eventually, also consider that you will be loading all those objects into memory in your server before is returned.
The only use case I see for this approach is if you have a list that doesn't grow over time or that you have a set number of items that doesn't affect response time.
Adding a Google Map plugin to our homepage, which updates a single marker dynamically whenever there is a new product search on our site (which we read from our database). So, "has there been a new search via our site? If yes, reposition the marker based on the new search's coords".
Currently every "n" seconds (haven't settled on a seconds value yet) an Ajax call is made (using SetInterval) to determine if there has been a new search, and if there has it returns a small JSON response. The script run via the Ajax call is a PHP script, which queries the database for the last row in our searches table (order by desc limit 1).
So, my question is (not being a sysadmin), could this setup put an undesirable strain on our server? Should i incorporate a timeout session, or something, which turns off the Ajax call after 100 goes, or after 15 mins (i mean, who sits for 15 mins looking at markers dynamically generate on a Google map?!).
Our homepage only receives roughly 200 visits a day.
As you have you given the statistics that your website gets 200 visits per day and that your server is a spitting JSON that you have to extract and display it on the UI, It is a normal practice to have a set up like this one. You can rather ping the server data using AJAX in every 5 sec to get more precise data but it wont cause any performance issue at this level.
Please be sure that you dont have servers that are separated geographically else you have to use some other synchronization mechanism to track users location based on there search.
For AJAX JQuery implementation details please see the following page.
For project implementation as a tutorial please visit this tutorial.
Our website provides various data services to our clients; one of which is gauge data. Some gauges log information every 15 minutes, some every minute. This data is sent to our SQL database.
All of this data is displayed via a graph (generated server side via PHP and JPGraphs) with each individual log entry being displayed as a row in a collapsible table (jquery 1.10.2).
When a client wants to view the data, they select a date range and which gauges they would like to view. If they want to view the last 3 days of a gauge that logs every minute then it loads pretty quickly. If they want to view 2 of those then it takes around 15-30 seconds to load. The real problem comes when they want to view a months worth of data; especially more than 1 gauge. This can take upwards of 15-20 minutes to load and the browser repeatedly asks if we want to stop the script from populating the collapsible table rows(jquery).
Obviously this is a problem since clients want a relatively fast response (1-5 min max). Ideally, we would also like to be able to pull gauge data from several months at a time. The only way we can do that now is to pull data 2 weeks at a time and compile the total manually.
For reference: If I wanted to pull a months data for 2 of our once-a-minute-logging gauges, then there would be 86,400 rows added via jQuery to a collapsible table. The page takes approx. 5 minutes to load and the browser is terribly slow during this time period.
My question is: What is the best way to pull/graph/populate data using a PHP based server (Symfony 1.4 framework) and javascript?
Should we look into upgrading our allotted processing power/RAM(we are hosted by GoDaddy)? Is there a faster way to populate collapsibles than with jquery? All of our calculatoins are done server side. Should we just pull the raw data and let the client side do the data processing? Should we split the data processing between client and server?
Here's a screen shot of the web page. Its cropped so that more client-sensitive information is not displayed:
In response to my comment.
Since you need the entire data-set only on the server side (you create your graph on the server), this means that you don't actually need to send the entire data-set to the client.
Instead send a small portion to the client. Let's say the first 200 results. Then you can go ahead and cache the rest of the result-set into a JSON file (lite database, whatever you want really). Then create an interface where the user can request for more data. Infinity scroll is nice but has its own problems. Maybe just a button that says load more data. As people have said anything more than a few hundred data points in a table at one time is crazy to have because people won't look at it anyways. Then when they hit the button to get more data, you send an AJAX request to the server with the correct parameters for what data you want.
For example the first time they click getMoreData() you want to get the next 200 data points. So you send getMoreData(start=200, length=200). Your server picks up the AJAX request and finds the correct data in the JSON file or the lite database, wherever you have cached the results. And the user can keep requesting more data (making sure you update your start parameter), and you only ever return a small subset. The user doesn't even realize that they don't have the whole data-set there in front of them because it looks like they do.
One that is complicated about this is sorting and searching. If you want to implement those then you need to make sure you go to the server side and sort/search through the cached results.
So basically you have a system where you can create the entire graph on the server side which shouldn't take long. What does take long is the loading of the entire data-set to the client side. So you break up that up into small chunks. You can even easily create pagination and the such with this method.
I'm working on a sharepoint farm that has User Profiles enabled. We're creating a community feature which has a profile wall of all members of that community. I need to retrieve and display profile pictures from a search based source and display the results as they are returned in an appealing efficient way.
We have two avenues:
1: FAST search indexes the profiles of every user every 6 hours. We can run a membership query and return all members of [x] community.
2: We can use the profile API to do a search. This is slower but does not rely on the 6 hour index and therefore gives us up to date information.
We need to make this call via JavaScript, as sever side code is locked down and not an option. I'd like to write a function that calls these profiles and loads the images into a wall one at a time as they are retrieved. Possibly in a timed loop, so an image loads every 100 milliseconds.
I believe profile photos are stored as a text property containing the photo URL, so the URL can be set as an images source.
How would I go about quickly loading a set of images asynchronously to provide a good user experience?
Since you do not have server side code option, I would suggest you to go for a Jquery Script which would render these images. This javascript code can be loaded asynchronously as suggested in this article:
https://wiki.base22.com/display/btg/How+to+load+JavaScript+dynamically+with+jQuery