I have a web application in PHP. One component of the application is submitting the data to a backend pipeline (also written in PHP). The pipeline is an external script, which my application calls using 'exec' php function. The pipeline is a multistep pipeline which executes several programs taking input from the program run before. I want to be able to display a message on the application page which submits to the pipeline, with completion of each step in the pipeline, such as
Step 1 completed... Now doing step 2.
Step 2 completed... Now doing step 3.
and so on and so forth.
I am open to use javascript/AJAX to do this, and also any other language compatible with PHP.
Any help is greatly appreciated.
Well working on the assumption that you have some kind of database backing your PHP front-end and pipeline, you don't specifically need something compatible with PHP, but rather something that could interface with your database.
Without having any further details on what you've set up/tried, etc. I can only offer an overview of the workflow I would use in this situation.
Front-end script is submitted and pushes the request into a processing queue.
User is shown a "processing, please wait" type page. This page makes a long-polling AJAX request or a Websocket connection to the front-end site to a script which polls the database for updates on the pipeline processing.
The pipeline scripts chain off each other and push into the database the details of their completion which are read off by the Websocket/long-polling front-end script and returned to the user via Javascript to display on the page.
Using the database as a go-between would be the easiest and most flexible approach. You could also use other languages if you're more comfortable with them so long as they're compatible with your database used on the PHP/pipeline side.
EDIT
While I don't have any links to a tutorial on exactly what you want to do, there are some basics behind it that you can use to piece together your solution:
I would start by getting your pipeline (processing) script to run in the background at interval using cron. Alternatively you could daemonize that pipeline using something like this PHP-Daemon framework so that it runs in the background. Perhaps having a cron task to check if it's running and restart it if needed.
From there you can build a table in your database that contains status updates on the processing tasks at hand and build a PHP script that checks the status of a given task and outputs JSON data about it. This JSON data could easily be read using AJAX requests, the simplest of which would probably be jQuery's .ajax() method. This could be called at interval on the client side "status" page using a setTimeout call at the end of each "loop" to poll for status changes every X seconds. This would be the easiest implementation of what you're after, although not the best performing or optimal way to do it.
So the general workflow here would change to this:
Front-end script is submitted and pushes the request into processing queue with status set to pending.
User is shown the processing page which pings status.php?task=12345 every X seconds.
Back-end daemon/cron script picks up the request and begins processing, pushing status updates into the database at interval.
status.php script begins to return different status information in the JSON code and it is displayed to the user.
Related
I have some PHP app. At the moment I am developing part where an user submits a request/form to the server via ajax (jQuery). Server will insert thousands of records in MySQL database that will probably take couple minutes (due to various calculations etc).
Question: Is there a way to track progress of updates on server side? I understand I can write some output once everything is finished. A workaround would be something like:
- store somewhere session ID plus progress details (example: processed 3k inserts out of 5k - maybe in some progress table in database
- have front end jQuery script requesting that information from the server every 10 seconds or so...
Any ideas? Please feel free to ask any question you may have. Thank you.
While I don't know the specifics of what you're doing server-side, a script that takes a "couple minutes" to execute likely involves several distinct steps. It might make sense to have the script execute one step at a time, return its result, and then get called again to execute the next step.
By using a process like this, you'll be able to track progress client-side. The caveat is that if the client closes the browser window or leaves the page with your ajax code, the remaining steps won't be executed.
I've created an aspx page which contains options for configuration of the final operation, the final operation on this page requires it to create backups of a number of folders; this can take a while.
Upon clicking the start button, it currently starts a timer which reads a log file (updated by the backup processes) from the system and periodically outputs this to a multi-line text box on the webpage. However, as the backing up process can take a while, this textbox does't get updated until the backup has been completed.
I just want to know, what's the best way to approach this I've been considering the following:
Webservice - As it's on the same server is this worth it? Also will this run the longer processes, allowing the timer to continue working?
Javascript to read the logfile and keep the textbox updated?
This is all on 1 server, it's just accessed/managed via this webpage.
Moving the long running operation outside the web application is a good practice, it can save threads for processing web requests. The web service is worth it, the web method of the web service that triggers the operation could run the operation on a new thread and return (fire and go). The web application could query a DB table for status (progress, result, error message) of the job.
I have a local website running that will query a mysql database via inputs given by a html form. The form values are sent to php via jQuery to call the query. Is there a way to send a command to the mysql server to kill the query if the website is refresh?
Currently if I refresh, the last mysql call will still run, so the website will be locked up until that query is finished.
Edit: To give a little more background information on the website, it is a local website that is solely running d3 visualizations based on a series of involved queries (lots of full text searches) on a large database (3 tables all in the 1 - 5 million record range). The "lockout" I'm referring to is not being able to abandon a query and try a more efficient query (i.e. something that will limit my results more).
Perhaps there is (it generally involves selecting the last process ID from the MySQL Processlist and issuing KILL QUERY on that process ID), but there are some problems that I think you should try to address first.
To begin, why in the world does one query "lock up" a website? I might be smelling a design flaw.
JavaScript could be used to make the browser "hit" the server on a refresh, but that's just adding another AJAX call and, presumably, another MySQL query (having to do with the processlist) and more PHP to write to handle the AJAX call, the processlist lookup, and the KILL QUERY query ...
I would recommend you try and make the server/MySQL query more efficient, so that you don't have to get a flying flip whether the browser is refreshed or not. As for browser security, you could probably use either PHP or JavaScript to enforce some sort of "flood limit" on repeated refreshing ....
I am trying to implement something like google public alerts at institutional level.
I will be displaying the institution map on the home page and if there's an event , I'm trying to insert a marker for that event dynamically i.e., without the user having to refresh the map.
Unfortunately to do this, I have to use a technology called IF-MAP, which forces me to use a cgi script that keeps on repeating cycles till an interrupt is passed.
so this is how my flow is:
web page with google map and ajax script that calls my cgi script for data about the markers
a perl - cgi script that keeps polling the IF-MAP database for any updates on alerts around the campus. by the end of every polling cycle, It returns some data for the web page to bring up markers.
the problem i am facing is,
the web page doesn't bring up the markers until the cgi script finishes it's execution.
Is it possible to insert dynamic alerts on a map using my approach?
if yes, can you please suggest me how?
if no, I've heard about something called socket.io and node.js which make my connection stateful. can these technologies help me work things out?
Thanks.
Can you not just keep it simple?
Poll your IF-MAP database from a cron-job or daemon
Write results to a JSON-formatted file
Make sure webserver caching/timeouts etc are setup correctly for that file
AJAX-GET the results file on a timer allowing the 304 unchanged status to do the hard work for you
You could try and do comlicated stuff with websockets (I think Perl's Mojolicious framework supports them out of the box) but you'll still have the issue of needing a separate process to query this IF-MAP database.
I am writing a web based GUI interface for a large Python script used to plumb a few activities on a server. While the application itself is very light weight, I am using Django to create the web GUI, because I can reuse many of the already builtin features such as user management among other.
I have a function, that takes number of logical steps, checks, copying of files. In the vanilla program we were using a log file to capture all statuses. Customer wants a page which shows each step with status changes in real time.
What is the right strategy. What are the steps taken? A progess bar is not what I require, it's more a progress report in real time? Can you advise the steps to be taken or point to any tutorial
You'll need to poll the backend from the browser periodically to see if there are any updates. This does assume that the long-running script is running asynchronously. The periodic update is akin to a log file tail, once in a while you check if new information has been added to the log.
There are several existing jQuery plugins that'll help you build this; PeriodicalUpdater for jQuery is a nice one, in that it'll adjust the poll interval if the server response doesn't change in a while.
Basically, with such a plugin, you'll need a Django view that returns the current status, the log file output of your process so to speak, and have PeriodicalUpdater poll that view. In the callback function for PeriodicalUpdater you'll need to add a check that the process is complete, of course; perhaps your server view could end with an easy to detect "Process complete" line at the end of the "log", or return a response that only consists of the final status.