My goal is to systematically collect information about every element present on a web page. Specifically, I would like to perform el.getBoundingClientRect() and window.getComputedStyle(el) for each element.
I have been using Selenium WebDriver for NodeJS to load the pages and manage the browser interaction. To simplify, let's just focus on getComputedStyle:
driver.findElements(By.xpath("//*"))
.then(elements => {
var elementsLeft = elements.length;
console.log('Entering async map');
async.map(elements, el => {
driver.executeScript("return window.getComputedStyle(arguments[0]).cssText",el)
.then((styles: any) => {
//stuff would be done here with the styles
console.log(elements.indexOf(el));
});
});
});
This code will loop through all the elements and retrieve their styles, but it is very slow. It may take a few minutes to complete for a page. I would like the driver to execute the scripts asynchronously, but this does not appear possible because each Selenium driver has a 'ControlFlow' that ensures each command to the driver is only started after the last has completed. I need to find a workaround for this so I can execute javascript asynchronously on the page (and make my data gathering faster).
Note: I have also tried Selenium's executeAsyncScript, which turns out to be just a wrapper around executeScript and will still block until it is finished. Here is my code using executeAsyncScript - it performs just as well as the previous code:
driver.findElements(By.xpath("//*"))
.then(elements => {
var elementsLeft = elements.length;
async.map(elements, el => {
driver.executeAsyncScript(
function(el: any) {
var cb = arguments[arguments.length - 1];
cb(window.getComputedStyle(el).cssText);
}, el)
.then((styles: any) => {
//stuff would be done here with the styles
console.log(elements.indexOf(el));
});
});
});
I am looking for a way to either bypass Selenium's ControlFlow in order to execute my javascript asynchronously, find a way to extract the objects and not be bound by the driver, or to find an alternative tool/solution for getting the data that I need.
Since executeScript can take in WebElements, did you see if it is faster to do all the work in one call rather than repeatedly calling executeScript?
driver.findElements(By.xpath("//*"))
.then(elements => {
driver.executeScript("return arguments[0].map(function(el) {return [el, window.getComputedStyle(el).cssText];})", elements)
.then((styles: any) => {
//stuff would be done here with the styles
console.log(styles);
});
});
});
If that is too slow, did you consider locating all the elements inside the script instead of passing them in?
driver.findElements(By.xpath("//*"))
.then(elements => {
driver.executeScript("return Array.prototype.slice.call(document.getElementsByTagName('*'))" +
".map(function(el) {return [el, window.getComputedStyle(el).cssText];})", elements)
.then((styles: any) => {
//stuff would be done here with the styles
console.log(styles);
});
});
});
Related
I'm trying to interact with some elements inside an iframe with cypress.
If I use the approach in https://bparkerproductions.com/how-to-interact-with-iframes-using-cypress-io/ for only one element per test, everything is fine.
# commands.js
Cypress.Commands.add(
'iframe',
{ prevSubject: 'element' },
($iframe) => {
return new Cypress.Promise(resolve => {
$iframe.on('load', () => {
resolve($iframe.contents().find('body'))
})
})
})
# landing_page.spec.js
cy.get('iframe').iframe().find('#happybutton').should('be.visible')
However, I want to look for multiple elements, click on them, and check if they are rendered correctly, but if I assign the iframe contents to a variable and reuse it to locate another element (for example, a button), cypress tries to locate the second element (for example, a menu) from the first element (the button, which is doomed to fail, because the button does not contain the menu).
# landing_page.spec.js
let iframeContent = cy.get('iframe').iframe()
iframeContent.find('#happybutton').should('be.visible')
iframeContent.find('#myMenu').should('be.visible')
I tried using different variables, or calling directly cy.get('iframe').iframe(), every time I wanted to interact with different elements, but cypress gets trapped in an infinite loop and the test never ends (but no errors or warnings are produced).
Does anybody knows a way to avoid this infinite loop? As I want to reproduce a sequence of steps to build a test case, it is not possible to isolate each interaction in a different test.
Or does anybody knows of a framework that is more suitable for working with iframes?
The problem is $iframe.on('load', only fires once, so you can't call cy.get('iframe').iframe() twice which is effectively what both .find() commands are doing.
let iframeContent = cy.get('iframe').iframe() doesn't store the iframe body, it stores a "chainer" which is treated like a function or getter.
The "infinite loop" is Cypress waiting for the promise resolve() call the second time, which never happens.
So you can nest the commands like this
cy.get('iframe').iframe().then(body => {
cy.wrap(body).find('#happybutton').should('be.visible')
cy.wrap(body).find('#myMenu').should('be.visible')
});
or you can enhance the command by adding a tag when the load event fires
Cypress.Commands.add('iframe', { prevSubject: 'element' }, ($iframe) => {
return $iframe._hasloaded
? $iframe.contents().find('body')
: new Cypress.Promise(resolve => {
$iframe.on('load', () => {
$iframe._hasloaded = true;
resolve($iframe.contents().find('body'))
})
})
})
Thanks to Marion's answer I found a way to refactor my code, so now it works!
Note: the iframe() function was left untouched
# commands.js
Cypress.Commands.add(
'iframe',
{ prevSubject: 'element' },
($iframe) => {
return new Cypress.Promise(resolve => {
$iframe.on('load', () => {
resolve($iframe.contents().find('body'))
})
})
})
# landing_page.spec.js
cy.get('iframe').iframe().as('iframeContent')
cy.get('#iframeContent').then((iframeContent) => {
cy.get(iframeContent).find('#happybutton').click()
cy.get(iframeContent).find('#myMenu')
cy.get(iframeContent).find('#anotherElement').should('be.visible')
})
The above answers pointed me to the right direction. By omittimg the 'then' phrase and the first cy.get('#iframeContent'), Semiramis' solution can be simplified a bit and made easier to understand like this:
cy.get('iframe').iframe().as('iframeContent')
cy.get('#iframeContent').find('#happybutton').click()
cy.get('#iframeContent').find('#myMenu')
cy.get('#iframeContent').find('#anotherElement').should('be.visible')
For Cypress newbees (like me): Cypress Variables and Aliases
I came across the following code:
let timeoutHandler;
clearTimeout(timeoutHandler);
timeoutHandler = setTimeout(() => {...});
This is an overly simplification since the original code is contained in a Vue application as follow:
public handleMultiSelectInput(value): void {
if (value === "") {
return;
}
clearTimeout(this.inputTimeoutHandler);
this.inputTimeoutHandler = setTimeout(() => {
axios.get(`${this.endpoint}?filter[${this.filterName}]=${value}`)
.then(response => {
console.log(response);
})
}, 400);
}
Does this mean this is some kind of cheap-ass debounce function?
Could someone explain what this exactly means.
Yes, it is a debounce function, which is when we wait for some amount of time to pass after the last event before we actually run some function.
There are actually many different scenarios where we might want to debounce some event inside of a web application.
The one you posted above seems to handle a text input. So it's debouncing the input, meaning that instead of fetching that endpoint as soon as the user starts to enter some character into the input, it's going to wait until the user stops entering anything in that input. It appears it's going to wait 400 milliseconds and then execute the network request.
The code you posted is kind of hard to read and understand, but yes, that is the idea of it.
I would have extracted out the network request like so:
const fetchData = async searchTerm => {
const response = await axios.get(`${this.endpoint}?filter[${this.filterName}]=${value}`);
console.log(response.data);
}
let timeoutHandler;
const onInput = event => {
if (timeoutHandler) {
clearTimeout(timeoutHandler);
}
timeoutHandler = setTimeout(() => {
fetchData(event.target.value);
}, 400);
}
Granted I am just using vanilla JS and the above is inside a Vuejs application and I am not familiar with the API the user is reaching out to. Also, even what I offer above could be made a lot clearer by hiding some of its logic.
I need synchronous notifications of DOM changes a la MutationEvents for an extension capability. MutationEvents, however, are deprecated. MutationObserver is limited in usefulness because of the way it aggregates changes and delivers them after the changes have been made.
So, simple question. Is synchronous notification of Element style changes possible in current (2019) browser extensions?
There's no API other than those you've mentioned. The only additional approach is to hook Node.prototype.appendChild, and a bunch of other methods to alter DOM in page context. Naturally you'll have to hook things like innerHTML/outerHTML setters as well.
Redefining prototype methods may break some sites that do similar low-level things.
Theoretically, at least, so be warned.
Here's a simplified content script that intercepts a few common methods:
const eventId = chrome.runtime.id + Math.random().toString(36);
const script = document.createElement('script');
script.textContent = `(${eventId => {
let reportingEnabled = true;
// only simple data can be transferred, not DOM elements, not functions, etc.
const sendReport = detail => dispatchEvent(new CustomEvent(eventId, {detail}));
const makeHook = (name, fn) =>
function () {
if (reportingEnabled) sendReport({name, phase: 'pre'});
const res = fn.apply(this, arguments);
if (reportingEnabled) sendReport({name, phase: 'post'});
return res;
};
const {appendChild} = Node.prototype;
Node.prototype.appendChild =
Element.prototype.appendChild = makeHook('appendChild', appendChild);
const {append} = Element.prototype;
Element.prototype.append = makeHook('append', append);
const innerHTML = Object.getOwnPropertyDescriptor(Element.prototype, 'innerHTML');
innerHTML.set = makeHook('innerHTML', innerHTML.set);
Object.defineProperties(Element.prototype, {innerHTML});
}})('${eventId}')`;
document.documentElement.appendChild(script);
script.remove();
window.addEventListener(eventId, e => {
console.log(e.detail);
});
Obviously you'll need to hook all the other methods like removeChild, insertBefore, and so on.
DOM elements cannot be transferred via messaging from the page context to the content script. Only trivial types like strings, numbers, boolean, null, and arrays/objects that consist of such types are transferable. There's a trick though for an existing DOM element: you can transfer its index [...document.getElementsByTagName('*')].indexOf(element) and then use it immediately as document.getElementsByTagName('*')[index]. For ShadowDOM you'll have to make a recursive indexer.
How do I focus an input with Cycle? Do I need to reach inside the DOM and call .focus() either with or without jQuery, or is there some other way with Cycle/RxJS?
Yes, you do need to reach inside the DOM and call .focus() either with or without jQuery. However this is a side-effect and it is Cycle.js convention to move these kinds of side effects to a so-called driver.
The two questions the driver needs to know are:
which element do you want to focus?
when do you want to focus the element?
The answer to both questions can be provided by a single stream of DOM elements.
Create the driver
First make your driver. Let's call it SetFocus. We'll make it a so-called read-only driver. It will read from the app's sinks but it will not provide a source to the app. Because it is reading, the driver's function will need to accept a formal parameter that will be a stream, call it elem$:
function makeSetFocusDriver() {
function SetFocusDriver(elem$) {
elem$.subscribe(elem => {
elem.focus();
});
}
return SetFocusDriver;
}
This driver takes whatever DOM element arrives in the stream and calls .focus() on it.
Use the Driver
Add it to the list of drivers provided to the Cycle.run function:
Cycle.run(main, {
DOM: makeDOMDriver('#app'),
SetFocus: makeSetFocusDriver() // add a driver
});
Then in your main function:
function main({DOM}) {
// setup some code to produce the elem$ stream
// that will be read by the driver ...
// [1]: say _when_ we want to focus, perhaps we need to focus when
// the user clicked somewhere, or maybe when some model value
// has changed
// [2]: say _what_ we want to focus
// provide the textbox dom element as actual value to the stream
// the result is:
// |----o-----o-----o--->
// where each o indicates we want to focus the textfield
// with the class 'field'
const textbox$ = DOM.select('.field').observable.flatMap(x => x); // [2]
const focusNeeded = [
clickingSomewhere$, // [1]
someKindofStateChange$ // [1]
];
const focus$ = Observable.merge(...focusNeeded)
.withLatestFrom(textbox$, (_, textbox) => textbox); // [2]
// ...
// [*]: Add driver to sinks, the driver reads from sinks.
// Cycle.js will call your driver function with the parameter
// `elem$` being supplied with the argument of `focus$`
return {
DOM: vtree$,
SetFocus: focus$, // [*]
};
}
You can then configure focusNeeded to say when you want .field to be focused.
You can tailor for your own situation, but this should illustrate how to solve your problem. Let's assume you have a text input and a button. When the button is clicked, you want the focus to remain on the text input.
First write the intent() function:
function intent(DOMSource) {
const textStream$ = DOMSource.select('#input-msg').events('keyup').map(e => e.target);
const buttonClick$ = DOMSource.select('#send-btn').events('click').map(e => e.target);
return buttonClick$.withLatestFrom(textStream$, (buttonClick, textStream) => {
return textStream;
});
}
Then the main which has a sink to handle the lost focus side effect
function main(sources) {
const textStream$ = intent(sources.DOM);
const sink = {
DOM: view(sources.DOM),
EffectLostFocus: textStream$,
}
return sink;
}
Then the driver to handle this side effect would look something like
Cycle.run(main, {
DOM: makeDOMDriver('#app'),
EffectLostFocus: function(textStream$) {
textStream$.subscribe((textStream) => {
console.log(textStream.value);
textStream.focus();
textStream.value = '';
})
}
});
The entire example is in this codepen.
Here's one example, written by Mr. Staltz himself: https://github.com/cyclejs/cycle-examples/blob/master/autocomplete-search/src/main.js#L298
I've created this custom command for my UI testing in Nightwatch. Here it is in full:
exports.command = function(element, callback) {
var self = this;
try {
this.waitForElementVisible('body', 15000);
console.log("trying..");
window.addEventListener('load', function() {
var selects = document.getElementsByName("select");
console.log(selects);
}, false);
} catch (err) {
console.log("code failed, here's the problem..");
console.log(err);
}
this
.useXpath()
// click dropdowns
.waitForElementVisible(element, 15000)
.click(element)
.useCss()
.waitForElementVisible('option[value="02To0000000L1Hy"]', 15000)
// operation we want all select boxes to perform
.setValue('option[value="02To0000000L1Hy"]', "02To0000000L1Hy")
.useXpath()
.click(element + '/option[4]');
if (typeof callback === "function") {
callback.call(self);
}
return this; // allows the command to be chained.
};
What I'm attempting to do is after I load the page, I want to retrieve all the select boxes and perform the same operation on them. Everything is working correctly except for the code in the try/catch block. I keep getting '[ReferenceError: window is not defined]' and am unsure of how to get past that.
The 'window' property is undefined in the global scope because it's being run via command line Node and not in the browser as one might assume initially.
You could try to use this.injectScript from the Nightwatch API but I would suggest using the Selenium Protocol API 'elements'
Hey there #logan_gabriel,
You could also use the execute command which I use when I need to inject a bit of JavaScript on the actual page. As #Steve Hyndig pointed out, your tests are running in the Node instead of on an actual browser window (somewhat confusing, since a window is generally open while tests are being run! Except if using PhantomJS to headless test, of course).
Here is an example custom command which will inject some JavaScript onto the page based on your original post:
exports.command = function(callback) {
var self = this;
this.execute(function getStorage() {
window.addEventListener('load', function() {
let selects = document.getElementsByName('select');
return selects;
}
},
// allows for use of callbacks with custom function
function(result) {
if (typeof callback === 'function') {
callback.call(self, selects);
}
});
// allows command to be chained
return this;
};
Which you could call from your test using the below syntax, including an optional callback to do something with the result:
client
.setAuth(function showSession(result) {
console.log(result);
});
You could opt to just do the work inside of the custom function, but I run into problems due to the async nature of Nightwatch sometimes if I don't nest stuff inside of callbacks, so it's more a safety thing.
Good luck!