I have an application that is scraping websites for data. It is scraping the website in an iFrame. I need to scrape to subsequent website from that same iframe that both have an entry adress that leads to a page with a field and a button. On that page I enter a number into the field and push the button to go to the next page. The problem I have is that when I (coded) push the button, it doesn't wait for the page to load and scrapes before there is anythong there.
setTimeOut and Delay seem to be ignored completely.
$(document).ready(function() {
$('#startbutton').click(function() {
$('#iframe').attr('src', 'www.pageone.com');
// Here I start a load because I need to wait for the button to become available.
$('#iframe').one('load', function() {
$('#okbutton').click();
});
// This is where it goes wrong. I would assume that it waits for the second page
// to load. But it doesn't. Adding another load block didn't help.
scrapeFunction();
});
});
How do I get these timings right?
when I (coded) push the button, it doesn't wait for the page to load and scrapes before there is anythong there
Actually, it doesn't even wait for the button to be pressed. Instead, you want to wait for another load event:
var $if = $('#iframe');
// (1) assign a load handler
$if.one('load', function() { // that says "When loaded (first), …
// (3) assign the next load handler
$if.one('load', function() {
// (5) which then (second load) can scrape the content
scrapefunction();
});
// (4) and load it
$('#okbutton').click();
}); // …"
// (2) and load it
$('#iframe').attr('src', 'www.pageone.com');
This will execute the code inside when the iFrame has been loaded.
$("iframe").ready(function() {
//iFrame loaded
scrapeFunction();
});
Related
I’m wanting to use an AJAX get method to retrieve contents from a page URL. The method works fine, the issue I’m getting is that call returns data before the page is completely loaded
Is there a way I can have the AJAX call only retrieve contents after say 5 seconds for example? (To ensure the page is fully loaded)
You can see on the page below, the page loads and then an additional loading symbol appears which loads the content. I want to retrieve the page data once all content has been loaded
https://pool.pm/addr1qy2465r5axxz92f0rlfyymz5zvsxad2v038slu70qeegm4aac2mvf8qqsgva9exmwhwpxymt896v5anudtr2wee77y6qgudr8k
You can use the onload global event handler. The load event fires at the end of the document loading process. At this point, all of the objects in the document are in the DOM, and all the images, scripts, links and sub-frames have finished loading.
For example
window.onload = function() {
doSomething();
};
And if you need to do the call after 5 secs, do the following
window.onload = function() {
setTimeout(() => {
doSomething();
}, 5000);
};
In ASP.NET MVC C#, I used Context.Request.Headers["Referer"] to get the referrer information from which page it is navigated to. But when I refresh the page, it still shows the old referrer url.
Is there any way that I can differentiate the page refresh and page load by navigation?
JS:
$(document).ready(function () {
debugger;
var referrer = '#Context.Request.Headers["Referer"]';
}
You could use window.onbeforeload to set a cookie/sessionStorage value, the event is triggered before page refreshes or a new page is to be loaded.
// Vanilla JavaScript
window.addEventListener('onbeforeload', function() {
// your code to set value here
});
// jQuery
$(window).on('beforeunload', function() {
// your code to set value here
});
After the page is loaded (window.onload), you can check for the value. If it matches you know the page is refreshed. You must also delete it at this point.
// Vanilla JavaScript
window.onload = function() {
// your code to check value here
// remember to delete the value too
});
// jQuery
$(window).on('load', function() {
// your code to check value here
// remember to delete the value too
});
The load event fires at the end of the document loading process - all of the objects in the document are in the DOM at this point. If you want to perform the check as soon as possible you can use an IIFE:
(function refreshCheck() {
// your check here
// remember to delete the value too
})();
I am using .load to call content from a section of my site when someone clicks a link. I am then using getscript to load in any scripts that are needed that .load strips out.
However, the content call function is needed on links that are loaded from this loaded content.
I am therefore in a situation where there is a loop being created and multiple executions.
This isn't an issue if the user clicks on one or two links to load content from other pages, but if they do it for several links then I get a spike in CPU, and therefore need to find a solution.
I am not sure why there is this issue, as the script executions are done through user clicks, so there shouldn't be continual execution.
The script looks like this:
$(document).ready(function () {
$('.Select a').one("click", function () {
var toLoad = $(this).attr('href') + ' .containerPlus';
$("body").append($("<div class='loadedcontent'>").load(toLoad, function () {
$.getScript("scripts required for loaded content", function () {
$.getScript("load this script again");
}));
});
});
basically the user clicks on a link, this calls the content and loads it into a new container, scripts needed on that page are then loaded, and then finally this code is applied to any links within this loaded content that need it.
This is a loop, but as there is a click element, I wouldn't have thought the second execution wouldn't take place until another click event.
Thanks for any advice
Richard
the contained of my pages are dynamic, then how to display the page if the page contained is complete, if not I wait message appears.
i used jquery mobile.
$("#A1").live('pageshow', function()
{
// code for fill the page
}
You could handle the pagebeforeshow event. Your code would be executed before the page is shown:
$(document).on('pagebeforeshow', '#A1', function(ev){
// your code
});
I need to detect the first time a page loads in jQuery so that I can perform some actions only when the page loads the first time a user navigates to that page. Similar to server side code page.ispostbasck. I have tested $(document).ready and it fires every time the page loads so this will not provide what I need. I have also tried the jQuery Load function - it also fires every page load. So by page load an example is that I have an HTML input tag on the page of type button and it does not fire a postback (like an asp.net button) but it does reload the page and fires $(document).ready
Thanks
You will have to use cookie to store first load information:
if (! $.cookie("cookieName")){
// do your stuff
// set cookie now
$.cookie("cookieName", "firstSet", {"expires" : 7})
}
Note: Above example uses jQuery Cookie plugin.
An event doesn't exist that fires only when the page is loaded for the first time.
You should use jQuery's .ready() event, and then persist the fact that you've handled a first time page load using your method of choice (i.e. cookie, session variable, local storage, etc.).
Note: This method will never be fool proof unless you can store this information at the user level in a DB. Otherwise, as soon as the user clears their cookies, or whatever method you choose, the "first time loaded" code will fire again.
I just ran into this problem and this is how I handled it. Keep track of the first time the page loads by using a variable initialLoad:
var initialLoad = true;
$(document).ready(function() {
...
...
...
initialLoad = false;
});
Then in other functions, you can do this:
if (initialLoad) {
//Do work that is done when the page was first refreshed/loaded.
} else {
//Do work when it's not the initial load.
}
This works well for me. If the user is already on the page and some jQuery functions run, I now know if that user just loaded the page or if they were already on the page.
The easy solution is to use jQuery ‘Once’ plugin
$(element).once('class-name', function() {
// your javascript code
});