I've created an aspx page which contains options for configuration of the final operation, the final operation on this page requires it to create backups of a number of folders; this can take a while.
Upon clicking the start button, it currently starts a timer which reads a log file (updated by the backup processes) from the system and periodically outputs this to a multi-line text box on the webpage. However, as the backing up process can take a while, this textbox does't get updated until the backup has been completed.
I just want to know, what's the best way to approach this I've been considering the following:
Webservice - As it's on the same server is this worth it? Also will this run the longer processes, allowing the timer to continue working?
Javascript to read the logfile and keep the textbox updated?
This is all on 1 server, it's just accessed/managed via this webpage.
Moving the long running operation outside the web application is a good practice, it can save threads for processing web requests. The web service is worth it, the web method of the web service that triggers the operation could run the operation on a new thread and return (fire and go). The web application could query a DB table for status (progress, result, error message) of the job.
Related
Scenario:
I have a php vagrant virtualbox server and a runnable jar that takes up to some minutes to run.
Ideally, when the user goes to a certain route in my application, the server will start running said jar in the background so the user wont have to wait looking at a blank page, and, when the jar finishes executing, the page will automatically refresh(or the data in it) giving the user information that it got from the output of the jar.
Problem:
So far, I managed to run the jar in the background using exec() function. I can even get the process id and check if it is still running or not.
The problem i have is how can the php(per say) be notified when the jar stops running without blocking the normal execution?
Is there even any way to do this? I've searched everywhere from laravel queues to forking the running process but nothing(as i understood) suits my needs.
EDIT:
I think I may have found a solution but i would like to ask the community anyway. Say I use a javascript loop (with setInterval) after the page loads, and ,inside the loop, i make an ajax async request to the server asking if the process has ended, and, if so, bringing back the output of the jar and updating the web page with the new data.
This would make for a very "pull" solution instead of the preferable "push" but I don't even know it the push approach is possible.
I'm still learning to code for web app and so this might be a silly question and for that I apologize. Please feel free to put me in the right path if this solution is completely off the rails.
Thanks in advance,
You should take a look at JS librairies already made for such purpose ex: NodeJS.
It keeps a pipe open between the user and your webserver and you can send a notice from the server to a specific user or a group of user.
Really love it myself.
I have a web application in PHP. One component of the application is submitting the data to a backend pipeline (also written in PHP). The pipeline is an external script, which my application calls using 'exec' php function. The pipeline is a multistep pipeline which executes several programs taking input from the program run before. I want to be able to display a message on the application page which submits to the pipeline, with completion of each step in the pipeline, such as
Step 1 completed... Now doing step 2.
Step 2 completed... Now doing step 3.
and so on and so forth.
I am open to use javascript/AJAX to do this, and also any other language compatible with PHP.
Any help is greatly appreciated.
Well working on the assumption that you have some kind of database backing your PHP front-end and pipeline, you don't specifically need something compatible with PHP, but rather something that could interface with your database.
Without having any further details on what you've set up/tried, etc. I can only offer an overview of the workflow I would use in this situation.
Front-end script is submitted and pushes the request into a processing queue.
User is shown a "processing, please wait" type page. This page makes a long-polling AJAX request or a Websocket connection to the front-end site to a script which polls the database for updates on the pipeline processing.
The pipeline scripts chain off each other and push into the database the details of their completion which are read off by the Websocket/long-polling front-end script and returned to the user via Javascript to display on the page.
Using the database as a go-between would be the easiest and most flexible approach. You could also use other languages if you're more comfortable with them so long as they're compatible with your database used on the PHP/pipeline side.
EDIT
While I don't have any links to a tutorial on exactly what you want to do, there are some basics behind it that you can use to piece together your solution:
I would start by getting your pipeline (processing) script to run in the background at interval using cron. Alternatively you could daemonize that pipeline using something like this PHP-Daemon framework so that it runs in the background. Perhaps having a cron task to check if it's running and restart it if needed.
From there you can build a table in your database that contains status updates on the processing tasks at hand and build a PHP script that checks the status of a given task and outputs JSON data about it. This JSON data could easily be read using AJAX requests, the simplest of which would probably be jQuery's .ajax() method. This could be called at interval on the client side "status" page using a setTimeout call at the end of each "loop" to poll for status changes every X seconds. This would be the easiest implementation of what you're after, although not the best performing or optimal way to do it.
So the general workflow here would change to this:
Front-end script is submitted and pushes the request into processing queue with status set to pending.
User is shown the processing page which pings status.php?task=12345 every X seconds.
Back-end daemon/cron script picks up the request and begins processing, pushing status updates into the database at interval.
status.php script begins to return different status information in the JSON code and it is displayed to the user.
I am trying to implement something like google public alerts at institutional level.
I will be displaying the institution map on the home page and if there's an event , I'm trying to insert a marker for that event dynamically i.e., without the user having to refresh the map.
Unfortunately to do this, I have to use a technology called IF-MAP, which forces me to use a cgi script that keeps on repeating cycles till an interrupt is passed.
so this is how my flow is:
web page with google map and ajax script that calls my cgi script for data about the markers
a perl - cgi script that keeps polling the IF-MAP database for any updates on alerts around the campus. by the end of every polling cycle, It returns some data for the web page to bring up markers.
the problem i am facing is,
the web page doesn't bring up the markers until the cgi script finishes it's execution.
Is it possible to insert dynamic alerts on a map using my approach?
if yes, can you please suggest me how?
if no, I've heard about something called socket.io and node.js which make my connection stateful. can these technologies help me work things out?
Thanks.
Can you not just keep it simple?
Poll your IF-MAP database from a cron-job or daemon
Write results to a JSON-formatted file
Make sure webserver caching/timeouts etc are setup correctly for that file
AJAX-GET the results file on a timer allowing the 304 unchanged status to do the hard work for you
You could try and do comlicated stuff with websockets (I think Perl's Mojolicious framework supports them out of the box) but you'll still have the issue of needing a separate process to query this IF-MAP database.
I am writing a web based GUI interface for a large Python script used to plumb a few activities on a server. While the application itself is very light weight, I am using Django to create the web GUI, because I can reuse many of the already builtin features such as user management among other.
I have a function, that takes number of logical steps, checks, copying of files. In the vanilla program we were using a log file to capture all statuses. Customer wants a page which shows each step with status changes in real time.
What is the right strategy. What are the steps taken? A progess bar is not what I require, it's more a progress report in real time? Can you advise the steps to be taken or point to any tutorial
You'll need to poll the backend from the browser periodically to see if there are any updates. This does assume that the long-running script is running asynchronously. The periodic update is akin to a log file tail, once in a while you check if new information has been added to the log.
There are several existing jQuery plugins that'll help you build this; PeriodicalUpdater for jQuery is a nice one, in that it'll adjust the poll interval if the server response doesn't change in a while.
Basically, with such a plugin, you'll need a Django view that returns the current status, the log file output of your process so to speak, and have PeriodicalUpdater poll that view. In the callback function for PeriodicalUpdater you'll need to add a check that the process is complete, of course; perhaps your server view could end with an easy to detect "Process complete" line at the end of the "log", or return a response that only consists of the final status.
All my research so far suggests this can't be done, but I'm hoping someone here has some cunning ideas.
I have a form on a website which allows users to bulk upload lots of URLs to add to a list on the server. There's quite a lot of server-side processing to do on each URL, so to avoid timeouts and to display progress, I've implemented the upload using jQuery to submit the URLs one at a time using ajax.
This is all working nicely. However, part of the processing on each URL is deduplicating it against the complete list. The ajax call returns a status indicating either a successful upload or a rejection due to duplication. As the upload progresses, I tell the user how many URLs have been rejected as duplicates (along with overall progress and ETA).
The problem now is how to give the user a complete list of the failed duplicate URLs. I've kept them in an array in my jQuery, and would like the user to be able to click on a link on the form to download a text file containing those URLs. Is this possible just using client-side processing?
The server-side processing basically handles a single keyword at a time. I'd rather not have to store the duplicates in a database table with some kind of session key which gets sent with every ajax call, and is then used at the end to generate the text file server-side (and then gets cleaned up some time later). I can see how to do this, but it seems very clunky and a bit 20th century.
I haven't used it myself yet, but Downloadify was built for exactly this purpose I think.
Downloadify is a tiny JavaScript + Flash library that enables the generation and saving of files on the fly, in the browser, without server interaction.
It was created by Doug Neiner who is also pretty active on Stack Overflow.
It needs Flash 10 to work.