Develop Reference

waiting until page loads completely in selenium for node js

How can i make my code wait until the page load completely in Nodejs?
I use selenium-webdriver version 4.0.0
const driver = new Builder().forBrowser("firefox").build();
await driver.get("http://www.tsetmc.com/Loader.aspx?ParTree=15131F");
// here we should wait but how?

There are 2 concepts here - there is:
Page Load time - time for the page and content to download (...this downloads the scripts), then....
Script load time - time for scripts to run, get more data and populate the page
Page load time is relatively easy - but this is enabled by default and might not help much if you're already experiencing issues.
For script load time, you need an object to synchronise on.
The selenium docs give a good outline of different wait strategies and provide code examples for different languages.
Looking at your page, you will mostly likely want to try the explicit wait. In Javascript this is:
await driver.wait(until.elementLocated(By.id('foo')), 30000);
You can update the By identifier to be the LAST item on the page or, use this approach for each item you need to interact with.
The same pages also lists other expected conditions too depending on the state you need.
A quick final note, if you're not aware selenium 4 is still in alpha. You can access it if you need the new features but they're still being testing and it is subject to frequent change. If you don't need selenium 4's new features, you might want to roll back to latest stable version.

How to debug timed out waiting for asynchronous Angular tasks? Failure to find elements on angular page occurring

Edit: Note that I found the root of my problem after help from #ernst-zwingli, so if you have this same error one of his noted fixes might help you out. My problem is a known issue with Protractor itself, if you think this may be you, I've expanded on my steps to pinpoint the root of the problem after my original question.
I'm trying to use Protractor in an Angular2 (just Angular) application built using angular-cli.
My problem: Elements on an Angular app page are not being found when browser.waitForAngularEnabledis at it's default setting of true (as in 'I believe I am on an angular page and would like for Protractor to do it's magic'). They are being found just fine if I set browser.waitForAngularEnabledto false (as in 'I am not on an angular page and would like to handle this myself, take a seat Protractor'). How do I track down what's causing this on my definitely Angular pages?
I have a product with a non-Angular Auth0 login page that gates access to the rest of the product that is written in Angular (Angular 4.3.2 to be exact). I have successfully traversed logging in on the non-Angular login page. I flipped the waitForAngularEnabled switched to false to facilitate the non-Angular login. I turned it back to true at the point where I expected my initial landing page (Angular) to be loaded, after clicking the submit button. Code is as follows:
browser.waitForAngularEnabled(false);
browser.driver.get('https://dashboard.net/projects');
browser.driver.sleep(10000);
browser.driver.findElement(By.css("[type='email']"));
browser.driver.findElement(By.css("[type='email']")).sendKeys("email#example.com");
browser.driver.findElement(By.css(".auth0-label-submit")).click();
browser.driver.findElement(By.id("passwordInput")).sendKeys("password");
browser.driver.findElement(By.id("submitButton")).click();
browser.driver.sleep(5000); // needed if not waiting for Angular
//browser.waitForAngularEnabled(true); // Back to Protractor land we go
let elementToFind = element(by.className("header-text"));
elementToFind.isDisplayed().then(function() {grabTheDarnLocalStorage()});
expect(elementToFind.isDisplayed()).toBeTruthy();
If I uncomment the browser.waitForAngularEnabled(true); line to state that I'm back in Angular code I get the error trace as follows:
Failed: Timed out waiting for asynchronous Angular tasks to finish after 30 seconds. This may be because the current page is not an Angular application. Please see the FAQ for more details: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
While waiting for element with locator - Locator: By(css selector, .header-text)
ScriptTimeoutError: asynchronous script timeout: result was not received in 30 seconds
(Session info: chrome=61.0.3163.100)
(Driver info: chromedriver=2.32.498550 (9dec58e66c31bcc53a9ce3c7226f0c1c5810906a),platform=Windows NT 10.0.14393 x86_64)
at WebDriverError (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\error.js:27:5)
at ScriptTimeoutError (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\error.js:203:5)
at Object.checkLegacyResponse (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\error.js:505:15)
at parseHttpResponse (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\http.js:509:13)
at doSend.then.response (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\http.js:440:13)
at process._tickCallback (internal/process/next_tick.js:109:7)
From: Task: Protractor.waitForAngular() - Locator: By(css selector, .header-text)
I've referenced the FAQ: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
I have my devs stating that they don't use $timeout (they use (Edit: NOT $interval) Observable Interval thank you very much) and they're not sure about $http.
I found this solution about the canonical way to debug protractor Angular sync issue issue: Canonical way to debug Protractor-to-Angular sync issues
but I'm not sure the solution works without access to modifying the dev code to run the programmatic tracker. (Edit: I never did figure out how to get this to work)
I also found this about a long timeout you add before each test, but I feel this is unnecessary overhead that makes your overall test execution take longer than it should without understanding the root cause of the problem: https://stackoverflow.com/a/37217167/2718402 (Edit: yeah, this is a bad idea and adds unnecessary time to your tests, please don't do this)
The frustrating bit is that this seems to be a common occurrence and there doesn't seem to be a streamlined documentation on how to deal with it. Logging in with a non-Angular page only to transition to an Angular page. Angular pages not being picked up properly by Protractor. All of the examples I find online are bits of code that I don't have a reference for where they should be at in my overall test framework. I would kill for a full example of someone testing a non-Angular login that transitions to a fully Angular website, with a setup config and real world test cases. (Edit: This is still true, but I can't make one myself as my application is in a bad grey area, note my RCA below for more details.)
I just want the ability to do my login and then successfully transition over to my Angular pages and be able to rely on Protractor to work with my Angular pages. I need to know what to look for that may be a long running asynchronous process (What specifically can I check for in the Chrome dev tools?). I would love to understand what Protractor needs as defaults in order to successfully work with the Angular parts of my app/website (Is there something beyond the presence of <app-root _nghost-c0="" ng-version="4.3.2"> in the HTML?). Before this job I worked in Java, so all of this asynchronicity and Angular is new to me, so I know I'm missing the known things that a seasoned Javascript dev is aware of.
My Solution/Root Cause Analysis
Starting down the list suggested by #ernst-zwingli:
for Angular(2) Check if the object window.getAllAngularRootElements returns at least one value.
It returned at least one value, so I moved on.
useAllAngular2AppRoots: true,
I tried this and still ran into the async timeout.
And if $interval or other long lasting asynchronous tasks are used, there can be issues, because of the zones
Previously #ernst-zwingli also mentioned looking at the testability method, except it was the old way. Through research and testing I found the window object also has a getAllAngularTestabilities method. This led down an interesting rabbit hole. An example output from the Chrome console (put window.getAllAngularTestabilities() in the Chrome console window, look at the resulting list) is as follows:
t:
_callbacks:...,
_didWork:true,
_isZoneStable: true (this looks promising, but why isn't Protractor working then?!?)
_ngZone:
hasPendingMacrotasks: true,
hasPendingMicrotasks: false,
isStable: true
I would think isZoneStable would be enough, but apparently not so for Protractor. Then looking at Macrotasks being true, I had to look up what the heck a Macrotask was: What does hasPendingMacrotasks and hasPendingMicrotasks check for?.
A macrotask can be:
i.e. setTimeout, setInterval, setImmediate
Thus #ernst-zwingli's note about interval's causing problems in the zones was remembered and something finally clicked.
First github issue, about zone instability
Another github issue complaining about the necessity of using browser.driver to get things done along with browser.waitForAngularEnabled. Apparently this is expected behavior, it led me to issue #3349
Issue #3349 - The actual root cause of my issue. My developers do not actively jump in and out of zones around observables. Even though these observables only have one subscriber. Since they live in the angular zone at this time, they are a long running "Macrotask" that Protractor waits infinitely on.
I can't rewrite the code with these wrappers as I am not currently versed enough in Angular to do it safely and we are currently hurtling toward a November deadline. I think I'll have to deal with using browser.driver for the time being and hope I can't get it fixed later. Hopefully my RCA was helpful for you.

In the following I list a set of potential causes and possibilities to fix/resolve them.
How does AngularJS and Angular(2) Work / What can I check in the Browser Dev Mode
I can't explain it as well as Andrey Agibalov in his Blog here, so check it out (also for developers).
Basically, the objects required by Protractor you can check in your Chrome Dev.
for AngularJS
Check if the object window.angular is properly defined, i.e. lookup window.angular.version and also try window.angular.getTestability of your Root element
for Angular(2)
Check if the object window.getAllAngularRootElements returns at least one value.
Root Element (AngularJS)
Potentially your Angular App is somewhere wrapped within the Body as something like <div ng-app="my-app">.
In that case, you must adjust your rootElement: body inside config.ts. Check this answer for details.
Angular(2)
If you're using Angular (aka Angular2), then there are ngZone's introduced. In this case your config.js should additionally contain this:
exports.config = {
framework: 'jasmine',
seleniumAddress: 'http://localhost:4444/wd/hub',
specs: ['spec.js'],
useAllAngular2AppRoots: true,
// rootElement: 'root-element'
};
check in your browser for window.getAllAngularRootElements as the additional line in conf.js is about this.
If you can, maybe use the advantage of multiple zones possible. Create a 2nd zone, configure rootElement: 'root-element'
to only focus on one zone and then move some asynchronous tasks into the other zone until you found, which task(s) lead to timeout. Keep those tasks (if possible) in the separate zone, so Protractor ignores those tasks.
And if $interval or other long lasting asynchronous tasks are used, there can be issues, because of the zones. Repeatedly or long lasting tasks should be started outside the zone and then be moved into the zone as else Protractor could run into timeouts. There is a workaround for developers to apply, in order to avoid these problems for Protractor.
read all about it here
browser.driver. - side remark
browser.driver.get() works as if ignoreSynchronization = true, since you directly assign the Browser Driver and you kind of bypass the synchronization logic of Protractor.
Read more about it in this answer here.
Hope I could give you some more input and you can solve your issue. Please let me know the results.

Could you please set
browser.ignoreSynchronization = true;
and try

non iframe html transition when using SCORM

i want to use SCORM (ver. 2004) using multiple html pages and i need to switch them using location.href
when im using only 1 html file its working as intended.
when using multiple files and switching them off with location.href, we get no connection on the new page and cannot initialize new connection because its already initialized.
Thank you very much for your help.

So the connection being Initialized isn't a big deal. But, each page loading and trying to initialize just generates a SCORM warning/error. That technically is a non-actionable error.
Cons of this approach
JavaScript has to instantiate on each page - each time. This means it has to pull back down (depending on the features your using) bookmarking, suspend data, etc...
So this is where mitigating all this becomes problematic.
When do you terminate?
How can you bookmark or support bookmarking?
What happens if curriculum adds or removes a page later?
Can I limit the number of times I try to initialize?
Will the LMS even allow this (since sometimes they salt and pepper values in the query string)?
The share-ability barometer on doing this I'd say is ripe with failure and I'd caution against it. Some LMS systems even detect the unload. Can you over come some of the above - sure. But will you be over taken by the rest... absolutely.
SCO = Shareable content object. And anything that diminishes the shareable part will hurt downstream.
Alternative
Use a single page SCO collection defined in a imsmanifest.xml. See https://github.com/cybercussion/SCOBot/wiki/Single-Pages-Managed-by-LMS-Navigation
Comment
Hope that helps. I was involved with a project a very long time ago where a architect wanted to do things simple like this, and it really requires some added elbow grease to either support single pages managed by the LMS, a AJAX or IFRAME approach to do it right.

Report all used Javascript functions

I am working on a WYSIWIG animation editor for designing sliders / ad banners that includes a lot of dependencies, which also means a lot of extra bloated code that isn't ever used. I am hoping to run a report on the code that helps me identify the important things. I have a couple cool starts that will search through javascript for all functions and return each function by parts:
https://regex101.com/r/sXrHLI/1
Then some PHP that will sort it by size:
Sort preg_match_all by named group size
The thought is that by identifying large functions that aren't being used, we can remove them. My next step is to identify the function tree of what functions are invoked on document load, and then which are loaded and invoked on actions such as clicks / mouseovers and so on.
While I have this handy function that tells me all functions loaded in the DOM, it isn't enough:
var functionArray;
$(document).ready(function(){
var objs = [];
for (var obj in window){
if(window.hasOwnProperty(obj) && typeof window[obj] === 'function') objs.push(obj);
};
console.log(obj));
});
I am looking for a solution that I can use to script in PHP / shell to emulate page load - now here is where my knowledge of terminology fails me, am I looking for "Call Stack", do I need a timeline, interpreter, framework, engine or a parser?
I next need to emulate a click / hover event on all elements, or all elements that match something like this regex:
(?|\$\(['"](\.\w*)["']|getElementsByClassName\('(\w*)'\))
(?|\$\(['"](\#\w*)["']|getElementsById\('(\w*)'\))
to find any events that trigger functions so I can make a master list of functions that need to be in the final code.

I was watching a talk from a Google Developer and I thought of your post. The following link has more intel on Dev Tools Coverage Profiler, but the following is the high level.
Google dev tools ships out a neat feature for generating reports on used and unused JS and CSS code -- which is right along the essence of what you were searching to do (just a slightly different medium -- it'd be a bit harder to automate, but it otherwise contains, I believe, exactly what you were looking for).
Open Dev tools and then open up the ellipse in the bottom left corner (see image) and then click the record button [see image 1]. Go through the steps you want to capture. You'll get an interactive screen to which you can go through all the code and see what was used (green) and what was not used (red) [see image 2]
Image 1 - Ellipse drop down to get to coverage tool
Image 2 - Full screenshot of the interactive report for this StackOverflow page while editing this post.

I'd suggest you to take a look at this tool:
Istanbul
With it you can do the following:
create an instrumented version of your code
deploy it on the server and run the manual test (coverage information is collected in one of the global variables inside the browser)
copy coverage information into a file (its an lcov data as far as I remember)
generate code coverage report with it
If you feel like going further, you can actually use something like jvm-cucumber with selenium to automate UI tests. You will need to dump the coverage data every time you reload the page, however. Then you will have to combine coverage from different runs and use this combined lcov data to generate the overall report.
The whole process is a bit heavy, but it is way better then starting to reinvent it.
Even more, you can combine this data with unit test coverage information to produce joint reports.
As a step further, you may want to setup sonar server so that you store multiple versions of the coverage reports and compare differences between tests.

How can I manage MSI session state within Javascript Custom Actions?

I have an ISAPI DLL, an add-on to IIS. I build the installer for it using WIX 3.0.
In the installer project, I have a number of custom actions implemented in Javascript. One of them, run at the initiation of the install, stops any IIS websites that are running. Another starts the IIS websites at the end of the install.
This stuff works, the CA's get invoked at the right times and under the right conditions. but the logic is naive. It stops all websites in the beginning (even if they are already stopped) and starts all websites at the end (even if they were previously stopped). This is obviously wrong.
What I'd like to do is keep track in the session of which websites required a stop at the beginning, and then, at the end, only try to restart those websites. Getting the state of a website is easy using the ServerState property on the CIM object. The question I have is, how should I store this information in the MSI session?
It's easy to stuff a single piece of information into a session Property, but what's the best way to store a set of N pieces of information, one for each website? In some cases there can be 1 website, in some cases, 51 websites.
I suppose I could use each distinct website name to create a distinct property name. Just not sure that is the best, most-efficient, most efficacious way to do things. Also, is it legal to use slashes in the name of an MSI Session property? (the website names will have slashes in them)
Suggestions?

You might want to check out:
VBScript (and Jscript) MSI CustomActions suck
C++ or C# is a much better choice. If your application already has dependencies on the framework then adding dependencies in your installer is a good logical choice. WiX has Deployment Tools Foundation ( DTF ) that has a custom action pattern that feels a lot jscript. You could then create a dictionary of websites and their run state and serialize it out to a single property. On the back side you could reconsitute that collection and then act upon it.
Not to mention the debugging story is MUCH better in DTF.

There's a simple solution. I was having a brain cramp.
All of the items I needed to store were strings - actually the names of websites that had been stopped during the installation. I just used the Javascript String.join method to create a single string, and the stuffed that into the session variable. Like this:
Session.Property("CA_STOPPEDSITES") = sitesThatWereStopped.join(",");
Then to retrieve that information later in another custom action, I do
var stoppedSites = Session.Property("CA_STOPPEDSITES");
if (stoppedSites != null) {
var sitesToStart = stoppedSites.split(",");
....
Simple, easy.