I have a web application based on Django. I use the Scrapy Crawler to crawl webpages. My goal, at the moment, is to be able to control the crawler from within a webpage using jQuery and AJAX requests.
My theoretical setup is the following:
On the webpage, I have a button. When I click the button, the crawler is started on the server-side.
Once the crawler has started, I periodically send AJAX GET requests to the server using window.setInterval to find out how many webpages have been crawled so far.
Once the crawler has finished, the GET requests should stop by using window.clearInterval.
These are the relevant lines from my current code:
$(document).ready(function() {
// This variable will hold the ID returned by setInterval
var monitorCrawlerId;
$startCrawlerButton.on('click', function(event) {
// This function should be run periodically using setInterval
var monitorCrawler = function() {
$.ajax({
type: 'GET',
url: '/monitor_crawler/',
// ...
success: function(response) {
// if the server sends the message that the crawler
// has stopped, use clearInterval to stop executing this function
if (response.crawler_status == 'finished') {
clearInterval(monitorCrawlerId);
}
}
});
};
// Here I send an AJAX POST request to the server to start the crawler
$.ajax({
type: 'POST',
url: '/start_crawler/',
// ...
success: function(response) {
// If the form that the button belongs to validates correctly,
// call setInterval with the function monitorCrawler defined above
if (response.validation_status == 'success') {
monitorCrawlerId = setInterval('monitorCrawler()', 10000);
}
}
});
});
});
The problem: When I execute this code, I get this in Firefox's web console:
ReferenceError: monitorCrawler is not defined
The strange thing, however, is that the function monitorCrawler gets periodically executed anyway. But with every execution, I get the same error message again. If I put monitorCrawler outside of $startCrawlerButton.on() I still get the same errors. How can I resolve this? Since I'm a JavaScript newbie, any help is appreciated. Thank you very much!
setInterval, when first parameter is string, resolves in the global (window) context. You can give it a variable pointing to the function to be called or even:
setInterval(function(){monitorCrawler();}, 10000);
this will create a closure where local variable monitorCrawler will still exist when interval fires.
Change
setInterval('monitorCrawler()', 10000);
to
setInterval(monitorCrawler, 10000);
Never ever pass strings to setInterval, but function references! They will be evaled every time, and that in global scope - yet your monitorCrawler function is local to the click-handler (and I guess by "putting it outside" you meant "into the ready-callback").
Try
monitorCrawlerId = setInterval(monitorCrawler, 10000);
With parameters:
monitorCrawlerId = setInterval(function(){
//prepare params.
monitorCrawler(/* param1, param2*/);
}, 10000);
Related
I've looked into a great deal of solutions which make use of setInterval and setTimeout, however I thought I could give it a go adding a delay between function calls with use of a python script.
To explain the code:
I have a dropdown menu which contains the football matches being played today. When a user selects an option from this dropdown list i.e. it detects a change, then it executes a bunch of code to update the page with the relevant information.
Within this function I have another function liveCommentaryCall()
which uses AJAX to get the latest updates in the football match selected from the dropdown menu.
Of course I need to call this every X seconds to update the webpage
regarding the latest events.
At the end of the function I make an AJAX POST request to a
specific URL /delayRequest, which I then have that route in flask
simply perform a sleep()and return.
After this I then call the function liveCommentaryCall()again to
restart the update procedure.
JAVASCRIPT
$("#teamDropdownSelector").change(function(){
.
.
.
function liveCommentaryCall(){
alert("called");
.
. // bunch of code to update the page
.
$.ajax({
url: "/delayRequest",
type: "POST",
data: JSON.stringify(""),
success: function(response) {
alert("RESPONSE RECIEVED");
},
error: function(delayError) {
alert("Something's gone wrong!");
}
});
liveCommentaryCall();
}
//Now out of the function scope.
//Below is the first time the liveCommentaryCall function gets called.
liveCommentaryCall();
});
PYTHON
#app.route("/delayRequest", methods=['POST', 'GET'])
def delay():
time.sleep(10)
return;
However, opposed to calling the function, it running, then having a delay of 10 seconds before being called again, the webpage just peppers me with "called" alerts, meaning liveCommentaryCall is just getting called instantly on loop somewhere?
I can't see anywhere in code which should be causing this, which makes me think theres some underlying principle I'm overlooking.
Question
Is the error in the fact that I cant use a python script like this, or am I just not coding this idea correctly?
Instead of implementing liveCommentaryCall recursively, you can use the setInterval function in the script:
<script>
setInterval("liveCommentaryCall()",1000); //call every second
function liveCommentaryCall(){
.
. // bunch of code to update the page
.
$.ajax({
url: "/delayRequest",
type: "POST",
data: JSON.stringify(""),
success: function(response) {
alert("RESPONSE RECIEVED");
},
error: function(delayError) {
alert("Something's gone wrong!");
}
});
}
</script>
Here's my issue. I have a js function that performs an $.ajax call to fetch some data from a server. When it gets that data back, I need to pass control back to the browser in order to show an update to a div.
The js function is itself within a for loop, and I need to ensure that the for loop does not advance until the js function has updated the div and allowed the Browser to display that update, at which point the for loop advances and the js function (with its ajax call) is called again, continuing until the for loop test causes the loop to end.
I've tried many different approaches - callbacks, promises etc, but to date I can't seem to get a handle on ensuring that the loop doesn't advance until the js function gets its server data, updates the div, causes the browser to display that update and fully completes.
Here's a simple stripped-down version of the function:
function myFunction (email) {
var request = $.ajax( {
url: 'getit.php',
cache: false,
async: false,
method: "post",
timeout: 1000,
data: "requesttype=getemailname&email="+encodeURIComponent(email)
});
request.done(function(response) {
$("#myDiv").html(response);
});
}
and here's part of the js that calls it:
.....
var emailscount = emails.length;
for(var i=0;i<emailscount;i++) {
myFunction (emails[i]);
}
.....
So, my issues are:
1) myFunction must allow the browser to display the updated div html - I'm not sure how to achieve that?
2) the for loop should only proceed when myFunction has received the data back from the server, updated the div html, AND allowed the browser to display that div.
At the moment, I have set the $.ajax call async flag set to "false" to stop execution until the data comes back, but how do I ensure the browser displays the new div content, and that the for loop does not proceed to call myFunction again until the previous myFunction call fully completes?
Any help you can give me would be very welcome, as right now I can't get this all to work!
Sounds like you need a recursive function, not a for loop with synchronous ajax calls
(function myFunction(i) {
$.ajax({
url: 'getit.php',
method: "post",
timeout: 1000,
data: {
requesttype : 'getemailname',
email : emails[i]
}
}).done(function(response) {
$("#myDiv").html(response);
if (emails[++i]) myFunction(i); // continue when this one is done
});
})(0);
Thanks for everyone's help! I'm making good progress (including taking care of JQuery deprecations!) but have run into a further problem. As I need to hand control back to the browser in order to show the refreshed div as I recurse, I'm calling a setTimeout as follows:
var nextBitOfWork = function () {
return myFunction(email);
};
setTimeout(nextBitOfWork, 0);
where myFunction (which recurses) now returns a promise when it's done doing it's $.ajax call.
If I simply call:
return myFunction(email);
without the setTimeout function construct above, the promise is passed through and all my promises are captured and allow me to get the array output I need and everything works great. But without the setTimeout I don't get the browser refresh. Using it as above I get the div update refresh displaying, but seem to lose the promise and so the script continues and I don't get to fill the array I use to capture values as I recurse.
Any thoughts on how to make sure the setTimeout passes on the promise?
Thanks
I've read some answers on this problem, but I'm not sure what to do in my case.
I have a function where the idea is it will keep attempting to connect via AJAX, and if it fails it will keep trying again every 5 seconds to submit information. This is important for a web app I'm developing where the users are offline 50% of the time and online 50% of the time.
The issue is in my current setup, I want to call setTimeOut to delay the execution of the function, but I think because I'm inside another function it has no idea about the location of startAjaxSubmitLoop() function? But when I run my code and check in console, I see millions of connections to my server one after another without a delay. It's as if the setTimeout() function delay properties was not working at all but it was still running the function?
Any ideas what I'm doing wrong?
function startAjaxSubmitLoop(id,tech_id){
//TODO: Make POST instead of GET because of pictures.
var request = $.ajax({
url: "script.php",
type: "GET",
data: { }
});
//If Successfully Sent & Got Response.
request.done(function( msg ) {
//Sometimes because of weak connections response may send, but the message it worked might not come back.
//Keep sending until for sure SUCCESS message from server comes back.
//TODO: Make sure server edits existing entries incase of double sends. This is good in case they decide to edit anyways.
$( "#log" ).html( msg );
});
//If Failed...
request.fail(function( jqXHR, textStatus ) {
//Color the window the errorColor
//alert( "Request failed: " + textStatus );
setTimeout(startAjaxSubmitLoop(id,tech_id),5000);
console.log('test');
});
}
You're calling setTimeout incorrectly. You're calling your startAjaxSubmitLoop immediately on the setTimout line and passing the result to setTimeout instead of your function. Modern setTimeout implementations (es5) let you pass args to setTimeout as below:
Instead of:
setTimeout(startAjaxSubmitLoop(id,tech_id),5000);
Use:
setTimeout(startAjaxSubmitLoop, 5000, id, tech_id); //call startAjaxSubmitLoop in 5000ms with id and tech_id
The standard way of doing this to support older browsers where setTimeout doesnt take params is to just wrap your function startAjaxSubmitLoop
setTimeout(function(){startAjaxSubmitLoop(id, tech_id)}, 5000); //equivalent to above
two ways to do this:
1) with a callback function:
setTimeout( function(){startAjaxSubmitLoop(id,tech_id);},5000);
2) with parameters listed after the function name (no parentheses) and timeout period:
setTimeout(startAjaxSubmitLoop,5000, id, tech_id);
I have two questions regarding the following function. Obviously, it is about a chat. In the function chat() different functions are called, one to establish a connection, one to search for someone to chat with (randomly), and one to get messages every second.
function chat()
{
//Open connection
var openconnection=openConnection();
//Stop function if connection could not be established
if(openconnection==false) return;
//Search for someone to chat with
searchforcontact=searchForContact();
if(searchforcontact==false) return;
//Constantly get messages
setTimeout(function(){
setInterval(getMessages(),1000);
},2000);
}
function openConnection() {
//Establish connection to php script
$.ajax({
type: 'POST',
url: 'action/chat/openconnection.php',
success: function(connection) {
//Let user know that someone to chat with is searched for
$('#chatTextDiv').append('bla');
//Return that the connection was successfull
return true;
}
}).error(function() {
//Let user know that a connection could not be established
$('#chatTextDiv').append('bla');
//Return false
return false;
});
}
Here are my questions:
1: I use return to stop the function chat() if, e.g., a connection could not be established. However the functions goes on to searchForContact(), and even if that fails, still goes on. How come?
2: The function getMessages() only runs once, I wonder why? FYI, I use the timeout for usability.
Most likely openConnection() doesn't return false. Since synchronous API are very uncommon and not really useable from within JavaScript, I am pretty sure, that openConnection does not work the way you use it. Please provide more information on the openConnection function.
Also, instead of passing the getMessages function to the call to setInterval, you invoke getMessages and pass whatever it returns to setInterval. This is most likely not what you want. You should change that call to the following:
setTimeout(function(){
setInterval(getMessages,1000);
},2000);
You should really read up on how AJAX and asynchronous APIs in general work. To give you a head-start here is an update to your code that should demonstrate what you're doing wrong:
function openConnection(success, error) {
//Establish connection to php script
$.ajax({
type: 'POST',
url: 'action/chat/openconnection.php',
success: success
}).error(error||function () {});
}
function chat(ready) {
//Open connection
openConnection(function () {
// If this function is called, then the connection is established
//Search for someone to chat with
searchforcontact=searchForContact();
if(searchforcontact==false) return;
// I STRONGLY SUPPOSE searchForContact is also an asynchronous API!
//Constantly get messages
setTimeout(function(){
setInterval(getMessages,1000);
ready();
},2000);
}, function () {
// when this function is called, something went wrong.
});
}
chat(function () {
// when this function is called, chat is ready to be used!
});
Sorry for answering a question with a question, but what are the values of openconnection and searchforcontact? It appears that they are not satisfying the conditional, so your return statement is not running. You can verify the value with FireBug or Chrome Developer Tools.
getMessages runs only once because your invoking it instead of passing it to setInterval. It should be setInterval(getMessages, 1000);.
Okay, so I appreciate that Javascript is not C# or PHP, but I keep coming back to an issue in Javascript - not with JS itself but my use of it.
I have a function:
function updateStatuses(){
showLoader() //show the 'loader.gif' in the UI
updateStatus('cron1'); //performs an ajax request to get the status of something
updateStatus('cron2');
updateStatus('cron3');
updateStatus('cronEmail');
updateStatus('cronHourly');
updateStatus('cronDaily');
hideLoader(); //hide the 'loader.gif' in the UI
}
Thing is, owing to Javascript's burning desire to jump ahead in the code, the loader never appears because the 'hideLoader' function runs straight after.
How can I fix this? Or in other words, how can I make a javascript function execute in the order I write it on the page...
The problem occurs because AJAX is in its nature asynchronus. This means that the updateStatus() calls are indeed executed in order but returns immediatly and the JS interpreter reaches hideLoader() before any data is retreived from the AJAX requests.
You should perform the hideLoader() on an event where the AJAX calls are finished.
You need to think of JavaScript as event based rather than procedural if you're doing AJAX programming. You have to wait until the first call completes before executing the second. The way to do that is to bind the second call to a callback that fires when the first is finished. Without knowing more about the inner workings of your AJAX library (hopefully you're using a library) I can't tell you how to do this, but it will probably look something like this:
showLoader();
updateStatus('cron1', function() {
updateStatus('cron2', function() {
updateStatus('cron3', function() {
updateStatus('cronEmail', function() {
updateStatus('cronHourly', function() {
updateStatus('cronDaily', funciton() { hideLoader(); })
})
})
})
})
})
});
The idea is, updateStatus takes its normal argument, plus a callback function to execute when it's finished. It's a reasonably common pattern to pass a function to run onComplete into a function which provides such a hook.
Update
If you're using jQuery, you can read up on $.ajax() here: http://api.jquery.com/jQuery.ajax/
Your code probably looks something like this:
function updateStatus(arg) {
// processing
$.ajax({
data : /* something */,
url : /* something */
});
// processing
}
You can modify your functions to take a callback as their second parameter with something like this:
function updateStatus(arg, onComplete) {
$.ajax({
data : /* something */,
url : /* something */,
complete : onComplete // called when AJAX transaction finishes
});
}
I thinks all you need to do is have this in your code:
async: false,
So your Ajax call would look like this:
jQuery.ajax({
type: "GET",
url: "something.html for example",
dataType: "html",
async: false,
context: document.body,
success: function(response){
//do stuff here
},
error: function() {
alert("Sorry, The requested property could not be found.");
}
});
Obviously some of this need to change for XML, JSON etc but the async: false, is the main point here which tell the JS engine to wait until the success call have returned (or failed depending) and then carry on.
Remember there is a downside to this, and thats that the entire page becomes unresponsive until the ajax returns!!! usually within milliseconds which is not a big deals but COULD take longer.
Hope this is the right answer and it helps you :)
We have something similar in one of our projects, and we solved it by using a counter. If you increase the counter for each call to updateStatus and decrease it in the AJAX request's response function (depends on the AJAX JavaScript library you're using.)
Once the counter reaches zero, all AJAX requests are completed and you can call hideLoader().
Here's a sample:
var loadCounter = 0;
function updateStatuses(){
updateStatus('cron1'); //performs an ajax request to get the status of something
updateStatus('cron2');
updateStatus('cron3');
updateStatus('cronEmail');
updateStatus('cronHourly');
updateStatus('cronDaily');
}
function updateStatus(what) {
loadCounter++;
//perform your AJAX call and set the response method to updateStatusCompleted()
}
function updateStatusCompleted() {
loadCounter--;
if (loadCounter <= 0)
hideLoader(); //hide the 'loader.gif' in the UI
}
This has nothing to do with the execution order of the code.
The reason that the loader image never shows, is that the UI doesn't update while your function is running. If you do changes in the UI, they don't appear until you exit the function and return control to the browser.
You can use a timeout after setting the image, giving the browser a chance to update the UI before starting rest of the code:
function updateStatuses(){
showLoader() //show the 'loader.gif' in the UI
// start a timeout that will start the rest of the code after the UI updates
window.setTimeout(function(){
updateStatus('cron1'); //performs an ajax request to get the status of something
updateStatus('cron2');
updateStatus('cron3');
updateStatus('cronEmail');
updateStatus('cronHourly');
updateStatus('cronDaily');
hideLoader(); //hide the 'loader.gif' in the UI
},0);
}
There is another factor that also can make your code appear to execute out of order. If your AJAX requests are asynchronous, the function won't wait for the responses. The function that takes care of the response will run when the browser receives the response. If you want to hide the loader image after the response has been received, you would have to do that when the last response handler function runs. As the responses doesn't have to arrive in the order that you sent the requests, you would need to count how many responses you got to know when the last one comes.
As others have pointed out, you don't want to do a synchronous operation. Embrace Async, that's what the A in AJAX stands for.
I would just like to mention an excellent analogy on sync v/s async. You can read the entire post on the GWT forum, I am just including the relevant analogies.
Imagine if you will ...
You are sitting on the couch watching
TV, and knowing that you are out of
beer, you ask your spouse to please
run down to the liquor store and
fetch you some. As soon as you see
your spouse walk out the front door,
you get up off the couch and trundle
into the kitchen and open the
fridge. To your surprise, there is no
beer!
Well of course there is no beer, your
spouse is still on the trip to the
liquor store. You've gotta wait until
[s]he returns before you can expect
to have a beer.
But, you say you want it synchronous? Imagine again ...
... spouse walks out the door ... now,
the entire world around you stops, you
don't get to breath, answer the
door, or finish watching your show
while [s]he runs across town to
fetch your beer. You just get to sit
there not moving a muscle, and
turning blue until you lose
consciousness ... waking up some
indefinite time later surrounded by
EMTs and a spouse saying oh, hey, I
got your beer.
That's exactly what happens when you insist on doing a synchronous server call.
Install Firebug, then add a line like this to each of showLoader, updateStatus and hideLoader:
Console.log("event logged");
You'll see listed in the console window the calls to your function, and they will be in order. The question, is what does your "updateStatus" method do?
Presumably it starts a background task, then returns, so you will reach the call to hideLoader before any of the background tasks finish. Your Ajax library probably has an "OnComplete" or "OnFinished" callback - call the following updateStatus from there.
move the updateStatus calls to another function. make a call setTimeout with the new function as a target.
if your ajax requests are asynchronous, you should have something to track which ones have completed. each callback method can set a "completed" flag somewhere for itself, and check to see if it's the last one to do so. if it is, then have it call hideLoader.
One of the best solutions for handling all async requests is the 'Promise'.
The Promise object represents the eventual completion (or failure) of an asynchronous operation.
Example:
let myFirstPromise = new Promise((resolve, reject) => {
// We call resolve(...) when what we were doing asynchronously was successful, and reject(...) when it failed.
// In this example, we use setTimeout(...) to simulate async code.
// In reality, you will probably be using something like XHR or an HTML5 API.
setTimeout(function(){
resolve("Success!"); // Yay! Everything went well!
}, 250);
});
myFirstPromise.then((successMessage) => {
// successMessage is whatever we passed in the resolve(...) function above.
// It doesn't have to be a string, but if it is only a succeed message, it probably will be.
console.log("Yay! " + successMessage);
});
Promise
If you have 3 async functions and expect to run in order, do as follows:
let FirstPromise = new Promise((resolve, reject) => {
FirstPromise.resolve("First!");
});
let SecondPromise = new Promise((resolve, reject) => {
});
let ThirdPromise = new Promise((resolve, reject) => {
});
FirstPromise.then((successMessage) => {
jQuery.ajax({
type: "type",
url: "url",
success: function(response){
console.log("First! ");
SecondPromise.resolve("Second!");
},
error: function() {
//handle your error
}
});
});
SecondPromise.then((successMessage) => {
jQuery.ajax({
type: "type",
url: "url",
success: function(response){
console.log("Second! ");
ThirdPromise.resolve("Third!");
},
error: function() {
//handle your error
}
});
});
ThirdPromise.then((successMessage) => {
jQuery.ajax({
type: "type",
url: "url",
success: function(response){
console.log("Third! ");
},
error: function() {
//handle your error
}
});
});
With this approach, you can handle all async operation as you wish.