I've been trying to get data from my database by php and PDO. Before I ask the question, I want to show you the part of the database that i want to get in real-time:
the data I want to get is: CPU_util in real time.
** CPU_util moves between 0 to 100
In the website I'm using the plugin: highcharts in order to display the CPU_util. The chart the I picked up is this (link to the chart):
My question is:
How to get the data in CPU_util from the database and put it in the chart in real time?
(X axis should be as it is (the current time) and the Y axis moves between 0 to 100)
What I've been trying so far:
I did some coding with ajax, but after some tries the website blocks me, because I passed the number limit of http request (2000 requests).
Realtime PHP is a completely different animal compared to 'normal' web apps.
As already suggested, websockets or http-long-polling is the way to go.
The big issue to tackle is dealing with the HTTP request limit and not crashing your server and not starting a php(-fom) thread for each request you are making. To achieve this you will have to rethink your architecture a bit.
To achieve realtime php you'd want non-blocking evented php on the server (nodejs style). In the world of php the most used library for achieving this is Ratchet.
If you want to learn more:
http://socketo.me/docs/
http://www.sitepoint.com/how-to-quickly-build-a-chat-app-with-ratchet/
Also, if you're really doing ALOT of calls to mysql, you may want to move this data to a separate high performance db like Redis
Related
I am looking a proper solution to implement the below scenario.
I have a large amount of json data to process, previously the whole data is send to the server using rest apis and server perform certain action on each json row, after a very long time server sends the response back with the processing status of each row. but this approach always make the user confused, whether it is processing or not (because they are seeing the loading screen for over a 10 mins), since i am using rest apis I can't get any status while processing the data.
I am looking
Is there any good way to send the data in small batches ?
how can I get the status from server while processing the data ?
my frontend is ReactJS and backend is nodejs
As per your given situation. You can use the pagination and send the data in a limited number of data packets.
If the user wants more data he can click on the next or previous data. In this case, I would suggest ANTD library. It works perfectly for pagination. For your reference here is the link for the trial.
You can also refer to this codeSandbox by Andrew Guo.
https://codesandbox.io/s/m4lj7l2yq8?file=/src/index.js
https://ant.design/components/pagination/
AS you mentioned the data is taking too long, for this I may suggest indexing in the database.And you can show Suspence until the whole data is loaded. This may help the user to understand that the data is been loaded.
Here is the link for your reference
https://ant.design/components/spin/
Consider I have a zoo app that shows all the zoos for each city. Each city is a page with a list of zoos.
In my current solution, on each page, I have ajax call to the server that pulls the list of the zoos for that particular city.
The performance is extremely important for me and my thought was to remove the ajax call and replace it with a JSON object that will live in the app. That way I will save a call to the server and I believe the data will arrive faster.
Is this solution makes sense? There are around 40 cities with ~50 zoos for each.
Consider the data is static and will never change.
Since 900 records is not much **, you can get all the records at once during the initial load and filter the all records array by city, that way your user experience would be much smoother, since client side js processing is far better than n/w latency.
** - note: strictly considering the data set size of ~900
Other solution can be - cache the data in the session scope and when ever there is a specific request for a city check for the availability in session scope, if it's not there make a n/w call.
I think correct question is what is my performance requirements?
Because you can write all your data in json object and do everything on client side without any ajax call but in this case when any client visit your page that means it will download all data. and that is another question mark
I currently face the following issue:
After a user has uploaded his images, all images are processed through a script that optimizes every image (compresses it and removes EXIF-data).
I got everything working, the only problem is that the proces takes quite some time. I want to notify the user of the job status, e.g. a percentage of the processed images.
Currently, the user has to wait without knowing what's up in the back-end. What is the best way to accomplish this? I've thought about AJAX-calls, but I honestly have no idea where to start with implementing this, also because it looks like I need multiple calls (kinda like a heartbeat call on the processing job).
The application I am developing in is a Laravel application, I've made an API controller which handles incoming files via AJAX calls.
Any help is appreciated, thanks.
Laravel has Broadcasting for this. It uses websockets, redis or pusher to send events to the client.
This way you can send the client a message when the processing is done without them having to refresh a webpage all the time.
You'd be better off reading about the principle of how it's done, for example: Progress bar AJAX and PHP
Essentially the way it's done is that the job (processing images in your case) happens on the server through PHP. Your script will need to produce some sort of output to show how far through it is, e.g. echo some value for the percentage progress. The PHP script itself is responsible for producing this output, so you must work out how to calculate it and then code that in. It could be that it takes the number of images to be processed into account, and when each one is successfully processed, it adds 1 to a counter. When the counter equals the number of images, 100% done, or possibly some error messages if something went wrong.
On the frontend you could have an ajax script which reads the output from the PHP script. This in turn could update a progress bar, or div with some sort of percentage message - the value used coming from your PHP script.
Laravel - and other frameworks - have built-in methods to help. But you'd be better understanding the principles of how it works, such as on the link I posted.
So I found this Pie Chart that I would like to use on my website (http://canvasjs.com/docs/charts/chart-types/html5-pie-chart/)
I've already adapted the code to where it establishes a connection to the MySQL database gets the information I need from and I've saved those as variables in PHP and displays them within the Pi chart. So far so good!
Now I've really like to make this a little bit more real-time as the information changes quite rapidly, so I was thinking, of having the jQuery update its information on a regular basis. Preferably every 1000ms or so. How would I go about achieving this?
Thanks for your suggestions!
Basically you will need to establish an ajax data flow:
your main page will contain only the graph, but skip the data
request the data by sending an ajax request to a separate page
the data page should return your data object in JSON format (use json_encode())
With periodicalupdater you can update your data in the interval of your choice and automatically adjust this interval to reduce the load on your server.
There's a worked example in the documentation: http://canvasjs.com/docs/charts/how-to/live-updating-javascript-charts-json-api-ajax/
Basically, you need to use JavaScript and more specifically Ajax to query the server continuously and fetch new data in JSON format. Then update the chart using the brilliantly named 'updateChart' method. :)
Our website provides various data services to our clients; one of which is gauge data. Some gauges log information every 15 minutes, some every minute. This data is sent to our SQL database.
All of this data is displayed via a graph (generated server side via PHP and JPGraphs) with each individual log entry being displayed as a row in a collapsible table (jquery 1.10.2).
When a client wants to view the data, they select a date range and which gauges they would like to view. If they want to view the last 3 days of a gauge that logs every minute then it loads pretty quickly. If they want to view 2 of those then it takes around 15-30 seconds to load. The real problem comes when they want to view a months worth of data; especially more than 1 gauge. This can take upwards of 15-20 minutes to load and the browser repeatedly asks if we want to stop the script from populating the collapsible table rows(jquery).
Obviously this is a problem since clients want a relatively fast response (1-5 min max). Ideally, we would also like to be able to pull gauge data from several months at a time. The only way we can do that now is to pull data 2 weeks at a time and compile the total manually.
For reference: If I wanted to pull a months data for 2 of our once-a-minute-logging gauges, then there would be 86,400 rows added via jQuery to a collapsible table. The page takes approx. 5 minutes to load and the browser is terribly slow during this time period.
My question is: What is the best way to pull/graph/populate data using a PHP based server (Symfony 1.4 framework) and javascript?
Should we look into upgrading our allotted processing power/RAM(we are hosted by GoDaddy)? Is there a faster way to populate collapsibles than with jquery? All of our calculatoins are done server side. Should we just pull the raw data and let the client side do the data processing? Should we split the data processing between client and server?
Here's a screen shot of the web page. Its cropped so that more client-sensitive information is not displayed:
In response to my comment.
Since you need the entire data-set only on the server side (you create your graph on the server), this means that you don't actually need to send the entire data-set to the client.
Instead send a small portion to the client. Let's say the first 200 results. Then you can go ahead and cache the rest of the result-set into a JSON file (lite database, whatever you want really). Then create an interface where the user can request for more data. Infinity scroll is nice but has its own problems. Maybe just a button that says load more data. As people have said anything more than a few hundred data points in a table at one time is crazy to have because people won't look at it anyways. Then when they hit the button to get more data, you send an AJAX request to the server with the correct parameters for what data you want.
For example the first time they click getMoreData() you want to get the next 200 data points. So you send getMoreData(start=200, length=200). Your server picks up the AJAX request and finds the correct data in the JSON file or the lite database, wherever you have cached the results. And the user can keep requesting more data (making sure you update your start parameter), and you only ever return a small subset. The user doesn't even realize that they don't have the whole data-set there in front of them because it looks like they do.
One that is complicated about this is sorting and searching. If you want to implement those then you need to make sure you go to the server side and sort/search through the cached results.
So basically you have a system where you can create the entire graph on the server side which shouldn't take long. What does take long is the loading of the entire data-set to the client side. So you break up that up into small chunks. You can even easily create pagination and the such with this method.