Develop Reference

Selenium is returning an old state of PageSource and is not updating after Javascript Execution

I have a console program in C# with Selenium controlling a Chrome Browser Instance and I want to get all Links from a page.
But after the Page has loaded in Selenium the PageSource from Selenium ist different to the HTML of the Website I have navigated to. The Content of the Page is asynchronously loaded by JavaScript and the HTML is changed.
Even if I load the HTML of the Website like the following the HTML is still different to the one inside the Selenium controlled Browserwindow:
var html = ((IJavaScriptExecutor)driver).ExecuteScript("return document.getElementsByTagName('html')[0].outerHTML").ToString();
But why is the PageSource or the HTML returned by my JS still the same as it was when Selenium loaded the page?
EDIT:
As #BinaryBob has pointed out I have now implemented a wait-function to wait for a desired element to change a specific attribute value. The Code looks like this:
private static void AttributeIsNotEmpty(IWebDriver driver, By locator, string attribute, int secondsToWait = 60)
{
new WebDriverWait(driver, new TimeSpan(0, 0, secondsToWait)).Until(d => IsAttributeEmpty(d, locator, attribute));
}
private static bool IsAttributeEmpty(IWebDriver driver, By locator, string attribute)
{
Console.WriteLine("Output: " + driver.FindElement(locator).GetAttribute(attribute));
return !string.IsNullOrEmpty(driver.FindElement(locator).GetAttribute(attribute));
}
And the function call looks like this:
AttributeIsNotEmpty(driver, By.XPath("/html/body/div[2]/c-wiz/div[4]/div[1]/div/div/div/div/div[1]/div[1]/div[1]/a[1]"), "href");
But the condition is never met and the timeout is thrown. But inside the Chrome Browser (which is controlled by Selenium) the condition is met and the element has a filled href-Attribute.

I'm taking a stab at this. Are you calling wait.Until(ExpectedConditions...) somewhere in your code? If not, that might be the issue. Just because a FindElement method has returned does not mean the page has finished rendering.
For a quick example, this code comes from the Selenium docs site. Take note of the creation of a WebDriverWait object (line 1), and the use of it in the firstResult assignment (line 4)
WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
driver.Navigate().GoToUrl("https://www.google.com/ncr");
driver.FindElement(By.Name("q")).SendKeys("cheese" + Keys.Enter);
IWebElement firstResult = wait.Until(ExpectedConditions.ElementExists(By.CssSelector("h3>div")));
Console.WriteLine(firstResult.GetAttribute("textContent"));
If this is indeed the problem, you may need to read up on the various ways to use ExpectedConditions. I'd start here: Selenium Documentation: WebDriver Waits

Test a self deleting/dynamic Webelement in Selenium

What I'm doing
I've been experimenting with Selenium and making a simple program to make my Selenium testing life easier. Part of this is testing webelements and figuring out what methods (clicking submitting ect) make them reload the page, remain static, or become stale without reloading the page. In this question I'm particularly interested in the third case as the first two are already implemented by me.
The problem I'm having
The problem I have is finding a Webelement that goes stale and doesn't cause a page reload. I can't think of a good way to search for one, I don't have the HTML and javascript skills to make one (yet anyways) and I can't verify my code works unless I actually test it.
What I've done/tried
The first thing I thought to look for was a popup but those aren't actually part of the webpage and they're also quite unreliable. I want something thats going to behave consistently because otherwise the test won't work. I think dynamic Webelements, those that change their locators when acted upon will suit my needs but I have no good way of finding them. Any google results for "Self deleting webelement exmaple" or "Webelement goes stale doesn't cause page reload example" or similar, will only give me questions on stackoverflow like this one rather than what I want - concrete examples. The code I'm running simply waits for a staleReferenceException and for an onload event in javascript. If the staleReferenceException occurs but the onload event does not, then I know I've found a self-deleting / dynamic webelement (at least thats what I think is the proper way to detect this). Here is the code I'm running:
try {
//wait until the element goes stale
wait.until(ExpectedConditions.stalenessOf(webElement));
//init the async javascript callback script
String script = "var callback = arguments[arguments.length - 1];" +
"var classToCall = 'SeleniumTest.isPageReloaded';" +
"window.addEventListener('onload'," + "callback(classToCall));";
//execute the script and wait till it returns (unless timeout exceeded)
JavascriptExecutor js = (JavascriptExecutor) driver;
//execute the script and return the java classname to call
//if/when the callback function returns normally
String classToCall = (String) js.executeAsyncScript(script);
clazz = Class.forName(classToCall);
callbackMethod = clazz.getMethod("JavascriptWorking");
callbackMethod.invoke(null,null);
//page reloaded
threadcase = 1;
}
//waiting until webElement becomes stale exceeded timeoutSeconds
catch (TimeoutException e) {
//page was static
threadcase = 2;
}
//waiting until webElement Reloaded the page exceeded timeoutSeconds
catch (ScriptTimeoutException e) {
//the webElement became stale BUT didn't cause a page reload.
threadcase = 3;
As you can notice above there is an int variable named threadcase in this code. The three 'cases' starting from 1 (0 was the starting value which represented a program flow error) represent the three (non-error) possible results of this test:
the page reloads
the page remains static, webelement doesn't change
the page remains static, webelement changes
And I need a good example with which to test the third case.
Solutions I've considered
I've done some basic research into removing webelements in javascript but I A: don't even know if I can act on the page in Selenium like that and B: I'd rather get a test case that just uses the Webpage as is since introducing my edits makes the validity of my testcase reliant on more of my code (which is bad!). So what I need is a good way of finding a webelement that matches my criteria without having to scour the internet with the f12 window open hoping to find that one button that does what I need.
Edit 1
I just tried doing this test more manually, it was suggested in an answer that I manually delete a webelement at the right time and then test my program that way. What I tested was the Google homepage. I tried using the google apps button because when clicked it doesn't cause the whole page to reload. So my thinking was, I'll click it, halt program execution, manually delete it, run the rest of my code, and since no onload events will occur, my program will pass the test. To my suprise thats not what happened.
The exact code I ran is the below. I had my debug stop on the first line:
1 Method callbackMethod = null;
2 try {
3 //wait until the element goes stale
4 wait.until(ExpectedConditions.stalenessOf(webElement));
5 //init the async javascript callback script
6 String script = "var callback = arguments[arguments.length - 1];" +
7 "var classToCall = 'SeleniumTest.isPageReloaded';" +
8 "window.addEventListener('onload', callback(classToCall));";
9 //execute the script and wait till it returns (unless timeout
10 //exceeded)
11 JavascriptExecutor js = (JavascriptExecutor) driver;
12 //execute the script and return the java classname to call if/when
13 //the callback function returns normally
14 String classToCall = (String) js.executeAsyncScript(script);
15 clazz = Class.forName(classToCall);
16 callbackMethod = clazz.getMethod("JavascriptWorking");
17 callbackMethod.invoke(null,null);
18 //page reloaded
19 threadcase = 1;
20 }
21 //waiting until webElement becomes stale exceeded timeoutSeconds
22 catch (TimeoutException e) {
23 //page was static
24 threadcase = 2;
25 }
26 //waiting until webElement Reloaded the page exceeded
27 //timeoutSeconds
28 catch (ScriptTimeoutException e) {
29 //the webElement became stale BUT didn't cause a page reload.
30 threadcase = 3;
31 //trying to get the class from javascript callback failed.
32 }
whats supposed to happen is that a Stale webelement causes the program to stop waiting on line 4, the program progresses, initializes the Javascript callback in lines 6-11 and then on line 14 the call to executeAsyncScript is SUPPOSED to wait untill an 'onload' event which should only occur if the page reloads. Right now its not doing that or I'm blind. I must be confusing the program flow because I'm 99% certain that there are no page reloads happening when I manipulate the DOM to delete the webelement I'm clicking on.
This is the URL I'm trying:
https://www.google.com/webhp?gws_rd=ssl
Simple google homepage, the button I'm deleting is the google apps button (the black 9-grid in the top right)
some info on that element:
class="gb_8 gb_9c gb_R gb_g"
id="gbwa"
Its the general container element for the button itself and the dropdown it creates. I'm deleting this when my program hits the STOP on line 1. Then I go through my program in the debugger. Note (you may have to click inspect element on the button more than once to focus in on it). I'm going to try deleting lower level elements rather than the whole container and see if that changes anything but still this behavior baffles me. The goal here is to get the program flow to threadcase 3 because thats the one we are testing for. There should be no page reloads BUT the webelement should become stale after I manually delete it. I don't have any clue why the javascript callback is running when I can't see a page reload. Let me know if you need more info on what exactly I'm deleting on the google homepage and I'll try sending a picture (with optional freehand circles of course).

I would think that you could debug through a test, place a breakpoint at a suitable point, then use the browsers dev tools to manually update the HTML.
Obviously, if you want this to be a repeatable process it is not an option, but if you are just investigating, then a manual intervention could be suitable

How to calculate page load time of a website which includes the total time and not just document.ready() time? [duplicate]

This question already has answers here:
Chrome: API for performance data
(2 answers)
Closed 4 years ago.
I was trying to write a function either in Javascript or in Python using Selenium to calculate the page load time of a website. document.ready() will only give the DOM load time but there might be some AJAX calls which cannot be detected using document.ready().
There is even an extension in chrome web store named 'Page Load Time', which will calculate the total time, as per my requirements. How do I replicate same kind of functionality?

You can use load like as follows.
$(window).load(function() {
//code in here
});
See jQuery docs here. Also, another answer that will show you how to set up a page timer can be found here.

driver.execute_script("return $.active == 0") should help you.
$.active returns the number of active Ajax requests. Link

You can try it with selenium and with execute_script method, we will get information from window.performance.timing, it will return the milliseconds, but if we divide it by 1000 we will get the seconds of the loaded page.
from selenium import webdriver
driver=webdriver.Chrome()
driver.get("https://example.com")
load_time = driver.execute_script(
"""
var loadTime = ((window.performance.timing.domComplete- window.performance.timing.navigationStart)/1000)+" sec.";
return loadTime;
"""
)
print(load_time)
Output
# something like this
0.803 sec.

Jquery/Ajax Code which check the internet [duplicate]

This question already has answers here:
How to detect online/offline event cross-browser?
(15 answers)
Closed 8 years ago.
Jquery Code which check the internet/network is there or not(mobile/PC/Tablet).It must just check on page load.I thinkAjax will good because Ajax will check after certain interval.
I am looking just like http://tomriley.net/, But it is plugin, I am looking for simple jquery/Javascript.
Its static page which check system internet only.
Any idea is appreciated.

You might try a $.ajax() invoication with a .fail() handler, for example JQuery's getJSON():
var network_available; // bool
var url = '/some/json/call'; // must be relative to the site that
// you are already addressing
$.getJSON(url, function(data) {
network_available = true;
})
.fail(function() {
network_available = false;
});
Though I doubt this will solve all of your problems. The Javascript engine won't allow 'foreign' URL's, just the domain that the script or page was received from. So you'd not be really testing network availability, but also whether your site is up and responding within a reasonable time.

Jquery code to check internet connection and change the button as per result [duplicate]

This question already has answers here:
How to detect online/offline event cross-browser?
(15 answers)
Closed 8 years ago.
Jquery Code which check the internet/network is there or not(mobile/PC/Tablet).It must just check on page load.I thinkAjax will good because Ajax will check after certain interval.
I am looking just like http://tomriley.net/, But it is plugin, I am looking for simple jquery/Javascript.
Its static page which check system internet only.
Any idea is appreciated.

You might try a $.ajax() invoication with a .fail() handler, for example JQuery's getJSON():
var network_available; // bool
var url = '/some/json/call'; // must be relative to the site that
// you are already addressing
$.getJSON(url, function(data) {
network_available = true;
})
.fail(function() {
network_available = false;
});
Though I doubt this will solve all of your problems. The Javascript engine won't allow 'foreign' URL's, just the domain that the script or page was received from. So you'd not be really testing network availability, but also whether your site is up and responding within a reasonable time.