Develop Reference

Convenient way to get input for puppeteer page.click()

Challenge
When using puppeteer page.click('something') one of the first challenges is to make sure that the right 'something' is provided.
I guess this is a very common challenge, yet I did not find any simple way to achieve this.
What I tried so far
In Google Chrome I inspect the element that I want to click. I then get an extensive element description with a class and such. Based on an example I found, my approach is now:
Take the class
Replace all spaces with dots
Try
If it fails, check what is around this and add it as a prefix, for example one or two instances of button.
This does not exactly feel like it is the best way (and sometimes also fails, perhaps due to inaccuracies from my side).
One thing that I notice is that Chrome actually often seems to give a hint hovering over the thing I want to click, I am not sure if that is right but I also did not see a way to copy that (and it can be quite long).
If there is a totally different recommended way (e.g. Looking in the browser for what the name roughly is, and then using puppeteer to list all possible things), that is also fine. I just want to get the right input for page.click()
If you need an example of what I am trying: If you open this question in an incognito tab, you get options like share or follow. Or if you go to a web shop like staples and want to add something to cart.

When using puppeteer page.click('something') one of the first challenges is to make sure that the right 'something' is provided.
Just to be clear, "something" is a CSS selector, so your question seems to reduce to how to write CSS selectors that are accurate. Or, since Puppeteer offers XPath and traditional DOM traversals, we could extend it to include those selection tools as well.
Broader still, if there's a data goal we're interested in, often times there are other routes to get the data that don't involve touching the document at all.
I guess this is a very common challenge, yet I did not find any simple way to achieve this.
That's because there is no simple way to achieve this. It's like asking for the one baseball swing that hits all pitches. Web pages have messy, complex, arbitrary structures that follow thousands of different conventions (or no conventions at all). They can serve up a slightly or completely different page structure on any request. There's no silver-bullet strategy for writing good CSS selectors, and no step-by-step algorithm you can apply to universally "solve" the problem of accurately and robustly selecting elements.
Your goal should be to learn the toolkit and then practice on many different pages to develop an intuition for which tools and tricks work in which contexts and be able to correctly discern the tradeoffs in different approaches. Writing a full guide to this is out of scope, and articles exist elsewhere that cover this in depth, but here are a few high-level rules of thumb:
Look at context: consider the goals of your project, the general structure of the page and patterns on the page. Too many questions on Stack Overflow regarding CSS selectors (but also in general) omit context, which severely constrains the recommendation space, often leading to an XY problem. A few factors that are often relevant:
Whether the scrape is intended to be one-off or a long-running script that should try to anticipate and be resillient to page changes over time
Development time/cost/goal tradeoffs
Whether the data can be obtained by other means than the DOM, like accessing an API, pulling a JSON blob from a <script> tag, accessing a global variable on the window or intercepting a network response.
Considering nesting: is the element in a frame or shadow DOM?
Considering whole-page context: which patterns does the site tend to follow? Are there parent elements that are useful to selecting a child? (often, this is a distant relationship, not visible in a screenshot as provided by OP)
Consider all capabilities provided by your toolkit. For example, OP asked for a selector to close a modal on Stack Overflow; it turns out that none of the elements have particularly great CSS selectors, so using Puppeteer to trigger an Esc key press might be more robust.
Keep it simple: since pages can change at any time, the more constraints you add to the selector, the more likely one of those assumptions will no longer be true, introducing unnecessary points of failure.
Look for unique identifiers first: ids are usually unique on a page (some Google pages seem to scoff at this rule), so those are usually the best bets. For elements without an id, my next steps are typically:
Look for an id in a close parent element and use that, then select the child based on its next-most-unique identifier, usually a class name or combination tag name and attribute (like an input field with a name attribute, for example).
If there are few ids or none nearby, check whether the class name or attribute that is unique. If so, consider using that, likely coupled with a parent container class.
When selecting between class names, pay attention to those that seem temporary or stateful and might be added and removed dynamically. For example, a class of .highlighted-tab might disappear when the element isn't highlighted.
Prefer "bespoke" class names that seem tied to role or logic over generic library class names associated with styling (bootstrap, semantic UI, material UI, tailwind, etc).
Avoid the > operator which can be too rigid, unless you need precision to disambiguate a tree where no other identifiers are available.
Avoid sibling selectors unless unavoidable. Siblings often have more tenuous relationships than parents and children.
Avoid nth-child and nth-of type to the extent possibe. Lists are often reordered or may have fewer or more elements than you expect.
When using anything related to text, generally trim whitespace, ignore case and special characters where appropriate and prefer substrings over exact equality. On the other hand, don't be too loose. Usually, text content and values are weak targets but sometimes necessary.
Avoid pointless steps in a selector, like body > div#container > p > .target which should just be #container .target or #container p .target. body says almost nothing, > is too rigid, div isn't necessary since we have an id (if it changes to a span our new selector will still work), and the p is generic--there are probably no .targets outside of ps anyway.
Avoid browser-generated selectors. These are usually the worst of both worlds: highly vague and rigid at the same time. The goal is to be the opposite: accurate and specific, yet as flexible as possible.
Feel free to break rules as appropriate.

How to add Keyboard navigation( Accessibility Access) to existing website using JavaScript, CSS or HTML?

How to add Keyboard navigation to an existing website using JavaScript, CSS, or HTML?
I don't want to add any paid apps. if you know free javascript library or something let me know.
similar techs I used:
tabindex="0",
a:focus{border : solid 2px blue}

Some concrete advice:
Use standard HTML5 elements where possible. Read up on HTML5 semantics, in particular headings and navigation. Before reaching for custom classes and CSS display/visibility features, make use of HTML attributes like hidden and disabled to control the visibility and availability of content and UI.
By all means use <div> and <span> elements for presentation and layout, but pay attention to document structure. Let semantic markup reflect/express the meaning you intend. ARIA landmark roles can help, although most of them are implicit in HTML already.
Note that the "Outline algorithm" mentioned in the HTML5 spec is not implemented on any browser, so don't get too hung up on perfectly sequential heading levels, just be consistent and logical. I find that starting each landmark with a new <h1> tends to be easiest to manage, but others prefer to have only one <h1> per page (thereafter starting distinctly meaningful areas or landmarks with <h2>). Both approaches are acceptable, and the exact choice should depend on complexity and did I mention consistency?
Add ARIA attributes sparingly, and only when HTML does not already express the intended semantic. (e.g. adding aria-pressed to a <button> turns it into a toggle button). No need to add aria-disabled="true" if the element already has a HTML5 disabled attribute.
It's worth knowing that ARIA attributes work well in CSS attribute selectors. If you use them, you can style your content based on their values, which keeps things synchronised.
Ensure that all operable/interactive elements are clearly labeled using standard HTML mechanisms like <label> if possible. If a visible label is not desired, or is ambiguous without visual cues, you may use aria-label to provide an 'offscreen' one.
More complex interactions may need some special keyboard handling in javascript.
Pay attention to the many articles offering guidance about color and contrast (for users with low-vision or color blindness). In particular, make sure the keyboard focus indicator is always clear and obvious. (You mentioned the CSS outline property in your question, so you are on the right track).
Provide text alternatives for all non-decorative images. If you're using an <img> tag, the alt attribute works very well. Decorative images need this too, but with a null value. (e.g. alt="")
Bear in mind different screen sizes and orientations. Consider using responsive CSS features such as media queries, grid and flex layouts. Avoid re-ordering content visually when it contradicts the meaning of the sequence in the underlying HTML code.
Consider that users may want to increase text size or line spacing using custom settings. This means leaning on relative units such as %, em and rem, rather than px, which should be reserved for hairline borders, or media query breakpoints.
VALIDATE YOUR CODE using an HTML validator. Free dev tools such as axe-core will also give very useful feedback and helpful guidance.
Use stack overflow for concrete questions. It's pretty reliable. Many good accessibility questions have already been asked and answered. Search first!
If you follow this advice, and if your product is not too outlandish in design, you will most likely be able to approach a high level of conformance with WCAG A/AA without too much pain.
Finally: Transparency is more important than full conformance. If you have WCAG violations you can't fix in the current iteration, be open and honest about it, and you will be in the clear, legally speaking. Perfect conformance is rare, and especially so for a first time accessibility implementation. Welcome to this challenging and rewarding field, and good luck!

Is it always necessary for overlay elements to be located in end of HTML body?

I have noticed that in javascript frameworks elements such as dialogs, tooltips and alerts mostly appear at end of body.
I'm making my own implementation of these elements and trying to make it failproof. I'm repeating some techniques like using transparent iframe to overlay embeded objects in old browsers, and so on.
What restrictions could I face if I place my dialog/tooltip somewhere deep inside of the DOM tree with {position: fixed}? I'm afraid if there are some dangers to this approach, because big frameworks never use it.
I want to support IE8+.

Aside from z-ordering that is a very valid point made by Teemu, another major consideration in JS frameworks is speed of execution / speed of lookup.
The DOM in JS terms is one large object. The deeper into an object javascript needs to go to get what it's being asked for, the less performant the script gets, take a look at this answer.
Therefore it makes sense to keep everything that is probably going to be cloned or deep copied at a sensible nesting level and in the correct z-order. That happens to be toward the end of the body and usually wrapped by at most one containing element.
There may be other reasons but the depth / nesting sprung to mind as a consideration I'd take into account.

Short answer - very few techniques like this are "always necessary". JavaScript can easily remove items from their natural position in the DOM and relocate them at will.
Long answer - I don't think approaching this from a JavaScript first angle is correct. Look at it in terms of where the content belongs naturally within the hierarchy of the rest of the DOM.
For example, if you are talking about a modal dialog, then the chrome (the container elements) usually do not belong within the rest of the DOM - they exist only to contain and provide modal overlay functionality for the content within. This chrome does not participate in the outline of the DOM and the rest of the content. In that case, unless you are able to load them separately via ajax or embed the chrome HTML within the JavaScript, then the closest you will come to removing them from the main DOM is to append them to the bottom of the main DOM content. Note that this disregards the upcoming TEMPLATE element (http://www.html5rocks.com/en/tutorials/webcomponents/template/) which is designed for just this purpose.
However, the content of your dialog might very well belong within the main content of the DOM - either as an element, or as an attribute (i.e. title or data-) to an associated element. This would especially be true for tooltip text.

Using custom HTML attributes for JavaScript purposes? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
One of my colleagues doesn't like to use HTML classes and ids for javascript/jQuery purpose. Therefore I've seen that he had created custom html attribute such as
<div id="myid" class="cssClasses ..." [some-purpose-id]="mycustomId">...</div>
I asked him if it was really a good idea and he replied that he considers that classes and Ids should be reserved for styling.
Personally, I would have used classes. Some classes would have been used for styling, and some other classes would have been used for programming (jQuery selectors). The idea is to keep things appart also. And of course jQuery could set styling classes but if possible not use them for selection. Of course I also use id when appropriate but since an id is unique on a page, I like to do generic stuff using classes.
I would like to know you opinions on the better approach (if there is one). Thank you.

I would recommend against that, but rather use the HTML5 data attribute. Read about it here.

You should use classes and IDs for both javascript and CSS. Your colleague is wrong.
It helps to remember that you want to keep separation between data (html), presentation (css styling), and behavior (javascript). Classes and IDs are there for a reason, and it's not for styling (and not for javascript, either!). They are there to allow you to semantically mark up your document.
When writing proper HTML, your classes and IDs should reflect the actual content and purpose of the data those nodes contain. Then, when you need to style them, you can specify the proper styling for, say the .zip_code class. And when you need special behavior, you can use javascript to apply the required behavior to your .deletable list items.
Of course, sometimes in the real world, there are people who feel very strongly against things, and you end up having to go along with what they want. In this case, you may have to compromise with your colleague and use HTML5's data-* attributes, as someone else suggested. However, while this is a solution that will result in valid HTML, it is not what data-* attributes were designed for, so it isn't technically the correct solution.

No, for the most part, you shouldn't use made-up attributes. These are not valid HTML, and will show up as errors if you try to validate your code.
However in HTML5 you can use custom attributes that begin with data-. You can read more about them on the html5doctor website. data- attributes, however, are more often advocated for storing information, not labels. e.g. for a product you might say data-brand="gucci". Or something.
Using attributes for styling or javascript is misleading, because it's not a dichotomy. Often, you will need to add an id or class to style your markup. If possible, you can reuse these attributes to select elements in javascript without adding attributes. It keeps your code clean, but this is obviously just a preference.
If you need to select an element that doesn't already have a unique id or class that could be used for selection, you could use the data attribute. However using ids and classes is still standard practice.

It may be useful if you want to supply another data inside attributes for multiple purposes, look at the following example:
<script>
function getIt(){
ob = document.getElementById('myid')
alert(ob.getAttributeNode('anotherTitle').value)
}
</script>
<div id="myid" class="cssClasses" anotherTitle="new Title">The div</div>
Test
I used the anotherTitle attribute to save or store another set of data in-which it may be used in another usage.
This is a working example http://jsfiddle.net/vxknh/
The refrence: http://www.w3schools.com/jsref/prop_attr_value.asp

Maybe your colleague was inspired by examples of the declarative style used in the dojo toolkit.
data- is a good way to "stamp" your attributes for a specific purpose (for example all dojo specific attributes are of the form data-dojo-whatever). This reduces the risk of conflict with other scripts or css.
So yes it can be a good idea, but no classes and ids are not reserved for styling.

Classes and ID have their advantages and disadvantages (it is also very dependant on how you use them).
Classes - they were designed to convey styling info, but with advent of jQuery now they have dual purpose, sometimes signifying action (or data) and sometimes styling. This is quite confusing especially on large scale projects. Imagine a designer trying to "re-factor" the CSS and changing these class names on elements that have javascript/jQuery hooked up to it - bad things ensue. You could mitigate against that prefixing your non-styling class names with a "namespace" like ".donttouchthis-jqueryclass", which differentiates the logic from presentation is a very primitive manner. The advantage of using a class is that it can be "stacked" thus it can convey more complex "logic" than an ID for example. From a pure javascript (no jQuery) point of view reliance on classes in your code introduces some backward compatibility issues (getElementsByClassName).
IDs - as the name implies were designed to uniquely identify elements. From a programming point of view IDs offer the fastest way to access elements in the DOM (getElementById). They can also be styled, however the potential to remove an ID when re-factoring is lower (as you can always remove ID styling and give it a class). IDs are unique thus making extensive use of them to store any complex data is very, very limited. But they are great to bind click handlers or uniquely identify chunks of the DOM as long as you only need one of elements identified by them.
Both Classes and IDs have one important rule which limits their usefulness as means for conveying any business logic - they cannot start with a number (and often you just need that database id conveniently assigned to an element)
Custom attributes (not data-) - they can be anything which is kinda cool, however if you care about "validity" of your HTML you will be restricted to certain DTD types and you will need to specify them in your DTD. In reality they do not harm your HTML as browsers are specifically obliged (under the W3C convention) to ignore unknown attributes. The only problem being a clash with potential future developments. If you are using pure javascript they will also be a bit of a pain to use (but this is a subjective option - as suggested in one of the answers above this is a good supplementary option to use them as data storage). The potential for a mixup when re-factoring CSS is small.
Custom data- attributes - these are great if you are using jQuery because you can store really complex data against it (objects etc), when used correctly they really kick ass and give you a nice vocabulary to describe function of your DOM elements. "data-" namespace nicely protects it from any future conflicts with non name-spaced attributes.
Having said all that the best solution is not to overtly rely on any of these, but implement javascript patterns which allow you to pass a parent DOM element and later apply functionality to it based on markup (like a standard jQuery plugin pattern). This way you remove a lot of dependencies on specific attributes/classes/ids that are a part of static HTML source.

Is loose coupling possible for a web app?

I understand that content, presentation, and behavior are supposed to be separated by using HTML, CSS, and JS to implement them independently. Is there a best practice for binding these elements concretely while maintaining loose coupling in a dynamic web app? Is it even possible? Optimally without depending on jQuery?
To clarify what I'm talking about: HTML has no mechanism for the definition of new tags, so when your JavaScript model needs to generate content, the tags and structure for its presentation are necessarily defined in JS or somehow made accessible to it, tightly coupling behavior to content.
Before someone calls over-OOP, the reason I want to keep tags and CSS classes out of JS is that I don't even want to commit to HTML as a front end. There are at least five different representations of my underlying model that I'll eventually want, and tightly coupling the model to HTML rules out at least two of them.
Is there some way to late bind the model to the target representation without sacrificing dynamics or loose coupling? Like an XML document corresponding model elements to template HTML snippets?
UPDATE
Though it seems backward to provide concrete details of a question about factoring and OO best practice, I want to provide a much fresher example of the entanglement of form & function inherent to HTML. The entanglement of content & behavior between HTML and JS is the problem, though, not the example, and I want an answer that doesn't break when I change the details. It's from a very different project than the one I was working when I first asked this question, but the gist is the same:
I have some HTML for a UI widget. It is a view that represents a model object - a selection of palettes - to the user. Palettes themselves are part of the app's task, so they're not view, they're model.
I want to use this widget in at least 2 different dialogs (and any I might want in the future). In one, I need a changed selection to cause immediate action, the recoloring of a set of images in canvas elements that are part of the same dialog. In another, I just need to have two different sets to select from - one selection in each set.
I do not want to copy and paste this widget's HTML form or JS behavior because I want maintenance and revision to propagate from one base thing to all of its instances. Thus, only one instance of the widget, whatever its form may be, may exist in my source.
I do not want to have my JS rendering HTML tags directly because doing so will forfeit my IDE's error checking and content assistance on the HTML. Further, if there are problems with the HTML output, their source in my JS will be less clear because the process of generating the HTML doesn't look like HTML or necessarily reflect its structure. Finally, I want to test the effects of changes to the widget's CSS and HTML independently of my JS.
I do not want to involve additional dependencies e.g. separate templating & parameterization languages in my source because changes to various standards or evolving needs may break their compatibility, or they may themselves become unmaintained, in the future, leaving me with a massive quantity of useless source. Depending on jQuery is okay because it does more for normalizing between browsers (not to mention for convenience) than it is likely to ever break given its ubiquity, but other libraries will be regarded with extreme suspicion.
I want to specify the details of altering and retrieving the widget's state (what colors compose the palettes, how the user chooses among them) in JS that references only the widget and its component parts, since it's going to be a part of multiple dialogs that use its information in different ways.
Simultaneously, other JS code (the dialog's behavior) needs to be able to observe and initialize the widget's state - but without including any specific references to the widget's internal structure, which may some day change to accommodate new features, fix bugs, etc.
For simply making multiple copies of the widget's HTML, I can settle on calling jquery.clone on a template node that exists in an HTML file, and inserting the contents where I need them, from the function that sets up the dialog. HTML makes things difficult when any of the cloned elements have id or name attributes, but careful handling during cloning can make unique identifiers, point their corresponding label elements and href attributes at the new identifiers, and keep the old ones accessible in data attributes. So all is well and good for structure & content, and obviously the CSS presentation propagates to the clone just fine.
The rub comes when I try to implement the widget's interface with its dialog's controller. What is the most future-proof way to expose the widget's state's abstraction? I.e. I don't want to use widgetJquery.find('input:checked').val() to get the selected color from code that doesn't absolutely have to commit to a selection being based on a checked input element whose value attribute is a set of serialized colors.
I hope that makes it more clear what I'm looking for without muddling the question with details.

You can always have different XSLTs parse an XML.
But, it seems you try to abstract tooo much.
use the MVC, use different views to parse.
The methodology you write about is common in old systems, I worked with more than 10 years ago, there is probably a good reason evolution killed it.

Loose coupling is best achieved archtecturally using design patterns. A great example of how to achieve loose coupling is found in the Mediator Pattern. However, there are several patterns to use.
I suggest reading Addy Osmani's 'Essential JavaScript design Patterns' for comprehensive coverage of a range of patterns.
Enjoy!