Develop Reference

Selenium [JavaScript] - Find elements under embedded element with Shadow DOM [ #shadow-root (user-agent) ] [duplicate]

I am using Selenium (Python) for automating a web page. I am trying to get text from an input field that is under #shadow-root (user-agent).
Xpath I used:
driver.find_element_by_xpath("**//*/p-calendar/span/input**").text
is not returning anything.
Attached the screenshot of my DOM element.
Requirement: To get text from the shadow root : 01:01

As per #hayatoito's (creator of Shadow DOM) comment:
The original motivation of introducing a closed shadow tree is "Never allow an access to a node in a closed shadow tree, via any APIs, from outside", AFAIK. Like that we can not access a node in the internal hidden shadow tree which is used in <video> element, in Blink.
In fact, I designed a closed shadow tree in such a way. If there is a way to access a node in a closed shadow tree, it should be considered as a bug of the spec.
I think it's totally okay to have an API to allow an access in the layer of Chrome apps or extensions.
However, for a normal web app, I think the current agreement is "Never allow it".
If we allowed it, that means we do not need a closed shadow tree. Just having an open shadow tree is enough, I think.
Furhter #Supersharp in his comment below his own answer within the discussion How to get element in user-agent shadow root with JavaScript? mentions:
#shadow-root (user-agent) are browser vendors native implementation so they are not documented and will never be accessible. Only open Shadow DOM are, as per the specs
WebDriver perspective
Recently, #AutomatedTester [David Burns, Chief Bacon Officer, Mozilla Corporation] initiated a discussion on WebDriver - Testability of web components
Requests
Proposals
Issue Tracker
Currently Selenium Team is open for accepting pull requests for the same.
Reference
You can find a couple of relevant detailed discussions in:
Need help to click on the element under the shadow Root (closed) type
Outro
Here you can find a relevant discussion on How to automate shadow DOM elements using selenium?

You can use driver.execute_script to inject JavaScript code that returns the ShadowRoot then use find_element to get the children element of the shadow root you are looking for.
input_shadow = driver.execute_script('''return document.querySelector("$1").shadowRoot''')
div_text = inputShadow.find_element_by_tag_name("div").text
$1 - Your element's identifier or selector.
If you are keen to using xpath to find elements
input_shadow = driver.execute_script('''return $x(\"//*/p-calendar/span/input\")[0]''')
div_text = inputShadow.find_element_by_tag_name("div").text

How to make JS tooltips work in Shadow DOM?

I am using Vue & Bootstrap for an app where I generate web components according to the official Vue documentation (https://cli.vuejs.org/guide/build-targets.html#web-component). For the most part Bootstrap and my business logic is working fine within the #shadow-roots of the web components as if it were in the light DOM.
However, Bootstrap tooltips (which are based on Popper.js https://popper.js.org/) are not working within the Shadow DOM at all. I have also tried to invoke tooltips directly with Popper.js and Tippy.js (https://atomiks.github.io/tippyjs/) in the Shadow DOM encapsulated code, sidestepping Bootstrap altogether, and I still cannot get them to work.
See example here: https://jsfiddle.net/mfep6rg9/
I can guess why -- the 3rd party tooltip libraries most likely aren't finding the target DOM element because it's in a Shadow DOM.
Is there a 3rd party solution out there that accounts for Shadow DOM / web component encapsulation?

Your guess is correct. 3party solutions using document. are not querying shadowDOM.
And there probably is no 3rd party solution as a solution requires either
WebComponents to communicate Mouse positions to the outside world.
Host querying shadowDOM (and nested shadowDOMs and nested shadowDOMs)
Not much different from an (even more restricted) IFRAME

I had the same problem while building LitElement-based Web Components and found the following solution:
$(this.shadowRoot.querySelectorAll("[data-toggle='tooltip']")).tooltip();
Make sure to target the respective element's shadowRoot and run a querySelectorAll to listen to all shadowRoot child elements that listen to "data-toggle='tooltip'".

What is this code for in the head of the DiscoverWestworld.com website? [duplicate]

In Google Chrome’s Developer Tools, I see a #shadow-root right under <html lang="en"> tag. what does it do and what is it used for? I don’t see it in Firefox nor in IE; only in Chrome, is this a special feature?
If I open it, it shows <head> and <body> and a link beside named reveal, by clicking, it points to the <head> and <body>, nothing else.

This is a special indicator that a Shadow DOM exists. These have existed for years, but developers have never been given APIs into it until recently. Chrome has had this functionality for a while, other browsers are still catching up. It can be toggled in the DevTools Settings under the "Elements" section. Uncheck the "Show User Agent Shadow DOM". This will at least hide away any Shadow DOMs created internally (like select elements.) I am unsure right away if it affects user-created ones, such as custom elements.
These come up in things like iframes as well, where you have a separate DOM tree nested inside of another.
The Shadow DOM is simply saying that some part of the page, has its own DOM within it. Styles and scripting can be scoped within that element so what runs in it only executes in that boundary.
This is one of the primary pieces needed for Web Components to work. Which is a new technology allowing developers to build their own encapsulated components that developers can use just like any other HTML element.

As an example of Shadow DOM, when you have a <video> tag on a web page, its shown as just one tag in the main DOM, but if you enable Shadow DOM, you will be able to see the video player's HTML(player DOM).
This is explained aptly in this article, http://webcomponents.org/articles/introduction-to-shadow-dom/

In the case of web components, there is a fundamental problem that makes widgets built out of HTML and JavaScript hard to use.
Problem: The DOM tree inside a widget isn’t encapsulated from the rest of the page. This lack of encapsulation means your document stylesheet might accidentally apply to parts inside the widget; your JavaScript might accidentally modify parts inside the widget; your IDs might overlap with IDs inside the widget and so on.
Shadow DOM addresses the DOM tree encapsulation problem.
For example, if you had markup like this:
<button>Hello, world!</button>
<script>
var host = document.querySelector('button');
var root = host.createShadowRoot();
root.textContent = 'こんにちは、影の世界!';
</script>
then instead of
Hello, world!
your page looks like
こんにちは、影の世界!
Not only that, if JavaScript on the page asks what the button’s textContent is, it isn’t going to get “こんにちは、影の世界!”, but “Hello, world!” because the DOM subtree under the shadow root is encapsulated.
NOTE: I have picked up above content from https://www.html5rocks.com/en/tutorials/webcomponents/shadowdom/ as it helped me understand shadow DOM a little better than answers already here. I have added relevant content here so that it helps others but do take a look at the link for detailed discussion on same.

Polymer create element dynamically

I need to create an custom-element dynamically.
I tried
var newElement= document.createElement('custom-element');
this is work. but My problem is when I want to add attribute to this element, to bind an array to this element.
I tried
newElement.setAttribute('data','{{data}}')
But it says that it expected to array and received '{{data}}'
How can I add this binding to dynamically elements?

I dont think this is possible right now, please see from kevinpschaaf:
https://github.com/Polymer/polymer/issues/1778
No, we don't currently support this, outside of dom-bind, which is the
only template implementation that late-binds instance children. You
can document.createElement('template', 'dom-bind'), then you can
dynamically append children with binding annotations to its content,
and the bindings will only be evaluated once the dom-bind is attached
to the document. See tests here that show this usage of it:
https://github.com/Polymer/polymer/blob/master/test/unit/dom-bind.html#L95
Note that dom-bind does not currently allow binding to outer scope, so
it has limited use in custom element templates (it's main use case is
for binding between elements in the main document), and that's not
likely to change short-term.
We are achieving a lot of performance optimization by baking the
binding connections into the prototype at registration time for an
element (rather than at instance time), and we haven't built up enough
of the machinery to easily allow runtime addition/removal of bindings.

Shadow dom in HTML

I've been hearing about Shadow DOM recently. While watching a video about the release of Angular 2 the presenter repeatedly made mention of Shadow DOM without clear expressions. What does Shadow DOM really mean?

The best explanation I've found is this from What the Heck is Shadow DOM? shown below:
Shadow DOM refers to the ability of the browser to include a subtree
of DOM elements into the rendering of a document, but not into the
main document DOM tree.
An important use case would be with video controls on a web page. The markup only shows a video tag, with some attributes and source tags. The addtional code for all the video operations to work is hidden in the Shadow DOM and not available to the rest of the page. The actual markup, javascript, and styles for the tag are encapsulated, which hides the implementation details of the video controls that each browser vendor has already written.
So while it's there in the DOM, its hidden from the page that renders it. So, to view the Shadow DOM, you can enable it under Dev Tools in Chrome.
The short answer is that the Shadow DOM is one of four technologies that make up Web Components.
For a definition, Web Components are: A component platform from the W3C that allows Web sites to be constructed from standardized building blocks. Web Components comprise Custom Elements, Shadow DOM and HTML Imports and Templates.
Shadow DOM is a technology of Web Components (although each can be used separately):
Custom Elements: is a capability for creating your own custom HTML tags and elements. They can have their own scripted behavior and CSS styling. They are part of Web Components but they can also be used by themselves.
HTML Templates: The HTML template element is a mechanism for holding client-side content that is not to be rendered when a page is loaded but may subsequently be instantiated during runtime using JavaScript. Think of a template as a content fragment that is being stored for subsequent use in the document.
Shadow DOM: provides encapsulation for the JavaScript, CSS, and templating in a Web Component. Shadow DOM makes it so these things remain separate from the DOM of the main document. You can also use Shadow DOM by itself, outside of a web component.
HTML Imports: is intended to be the packaging mechanism for Web Components, but you can also use HTML Imports by itself. You import an HTML file by using a tag in an HTML document.
See Introduction to the Shadow DOM.

It refers to the ability to create a "child" DOM completely sandboxed from the rest of the page. Useful for web components, reusable "widgets" which allow to not worry about their css/js affecting things they shouldn't. http://glazkov.com/2011/01/14/what-the-heck-is-shadow-dom/
Check out https://www.polymer-project.org/ if you want to see it in action.

Think of shadow DOM is as an encapsulated (private) DOM.
You can't access the shadow DOM in the manner you access regular DOM, like 'document.querySelector()'.
Let's say, you defined a reusable custom element, (which contains its DOM trees). Then you use the custom element within your app HTML.
Now, the DOM under (which is now called a "host element") has become a shadow DOM subtrees (under a shadow root), hidden from the parent structure!
I hope this help you a bit.