I was working on a website, and I thought of how funny it would be if I completely randomized the CSS rules on the page as a joke. (Not just the elements' styles, but the CSS rules themselves, because the website has a lot of dynamically created elements.) Each time you loaded the page the result would be completely different, and most of them would look terrible. So my question has two parts:
Using JavaScript/JQuery, How do you programmatically get a list of all CSS rules? As a sort of dictionary, with the rules paired to the selectors.
Then, after you have broken down the list and randomly assigned each rule to a different selector, how do you delete the previous rules and substitute in your own?
NOTE: I mean, using JavaScript/JQuery, how do you randomize the rules on the client side, not just a single CSS file.
You can access and traverse all the stylesheets with document.styleSheets. See the API documentation on MDN
Mind you this is a bit psuedo-ey, also note that you can do this using pure JS.
foreach (var e in document.getElementsByTagName("*")) {
foreach (var p in el.style) {
var r = Math.random(0, 255);
e.style[p] = r;
}
}
also note that not all css properties take 0 - 255 so you might have to create your own algorithm, but this'll sure get you started.
Looping through all the elements in the page would be trivial (document.getElementsByTagName('*')).
Looping through all the styles available for each element would be trivial (element.style).
Setting random values for any given style would be harder.
You would need to have hard-coded lists of styles and possible values for each of them, because the values that can be set vary so wildly. Some can be in a variety of different units (px, em, %). Some of them have pre-defined keywords as the possible values.
And a random value is no good at all if you don't have limits to it. Setting a random width sounds easy, but you have to know what ranges you're going to work to. width:5743731px isn't going to be very useful even for a randomised page.
And then you have the properties that can fetch external resources. A CSS background image is going to be virtually impossible to randomise, and fonts would need to be loaded in a separate #font-face declaration, so you would only be able to randomise fonts that you know are loaded.
And then you have to think about how randomised you're going to be. Are you going to randomise every possible style on every element? (crazy, but hey, this whole thing is crazy so why not) Or just one style per element?
Don't forget that a lot of styles work in conjunction with each other. So text-overflow:ellipsis does nothing unless you also have white-space:nowrap and overflow:hidden. And setting a border-color is pointless unless you've also set the other border attributes.
So yes, I think your first task here would be to go through the list of CSS styles and work out which ones could be randomised and what the possible randomised values for them could be. That's the difficult bit. Once you've got that, hard code it into your program, and the rest should be fairly simple.
Related
Challenge
When using puppeteer page.click('something') one of the first challenges is to make sure that the right 'something' is provided.
I guess this is a very common challenge, yet I did not find any simple way to achieve this.
What I tried so far
In Google Chrome I inspect the element that I want to click. I then get an extensive element description with a class and such. Based on an example I found, my approach is now:
Take the class
Replace all spaces with dots
Try
If it fails, check what is around this and add it as a prefix, for example one or two instances of button.
This does not exactly feel like it is the best way (and sometimes also fails, perhaps due to inaccuracies from my side).
One thing that I notice is that Chrome actually often seems to give a hint hovering over the thing I want to click, I am not sure if that is right but I also did not see a way to copy that (and it can be quite long).
If there is a totally different recommended way (e.g. Looking in the browser for what the name roughly is, and then using puppeteer to list all possible things), that is also fine. I just want to get the right input for page.click()
If you need an example of what I am trying: If you open this question in an incognito tab, you get options like share or follow. Or if you go to a web shop like staples and want to add something to cart.
When using puppeteer page.click('something') one of the first challenges is to make sure that the right 'something' is provided.
Just to be clear, "something" is a CSS selector, so your question seems to reduce to how to write CSS selectors that are accurate. Or, since Puppeteer offers XPath and traditional DOM traversals, we could extend it to include those selection tools as well.
Broader still, if there's a data goal we're interested in, often times there are other routes to get the data that don't involve touching the document at all.
I guess this is a very common challenge, yet I did not find any simple way to achieve this.
That's because there is no simple way to achieve this. It's like asking for the one baseball swing that hits all pitches. Web pages have messy, complex, arbitrary structures that follow thousands of different conventions (or no conventions at all). They can serve up a slightly or completely different page structure on any request. There's no silver-bullet strategy for writing good CSS selectors, and no step-by-step algorithm you can apply to universally "solve" the problem of accurately and robustly selecting elements.
Your goal should be to learn the toolkit and then practice on many different pages to develop an intuition for which tools and tricks work in which contexts and be able to correctly discern the tradeoffs in different approaches. Writing a full guide to this is out of scope, and articles exist elsewhere that cover this in depth, but here are a few high-level rules of thumb:
Look at context: consider the goals of your project, the general structure of the page and patterns on the page. Too many questions on Stack Overflow regarding CSS selectors (but also in general) omit context, which severely constrains the recommendation space, often leading to an XY problem. A few factors that are often relevant:
Whether the scrape is intended to be one-off or a long-running script that should try to anticipate and be resillient to page changes over time
Development time/cost/goal tradeoffs
Whether the data can be obtained by other means than the DOM, like accessing an API, pulling a JSON blob from a <script> tag, accessing a global variable on the window or intercepting a network response.
Considering nesting: is the element in a frame or shadow DOM?
Considering whole-page context: which patterns does the site tend to follow? Are there parent elements that are useful to selecting a child? (often, this is a distant relationship, not visible in a screenshot as provided by OP)
Consider all capabilities provided by your toolkit. For example, OP asked for a selector to close a modal on Stack Overflow; it turns out that none of the elements have particularly great CSS selectors, so using Puppeteer to trigger an Esc key press might be more robust.
Keep it simple: since pages can change at any time, the more constraints you add to the selector, the more likely one of those assumptions will no longer be true, introducing unnecessary points of failure.
Look for unique identifiers first: ids are usually unique on a page (some Google pages seem to scoff at this rule), so those are usually the best bets. For elements without an id, my next steps are typically:
Look for an id in a close parent element and use that, then select the child based on its next-most-unique identifier, usually a class name or combination tag name and attribute (like an input field with a name attribute, for example).
If there are few ids or none nearby, check whether the class name or attribute that is unique. If so, consider using that, likely coupled with a parent container class.
When selecting between class names, pay attention to those that seem temporary or stateful and might be added and removed dynamically. For example, a class of .highlighted-tab might disappear when the element isn't highlighted.
Prefer "bespoke" class names that seem tied to role or logic over generic library class names associated with styling (bootstrap, semantic UI, material UI, tailwind, etc).
Avoid the > operator which can be too rigid, unless you need precision to disambiguate a tree where no other identifiers are available.
Avoid sibling selectors unless unavoidable. Siblings often have more tenuous relationships than parents and children.
Avoid nth-child and nth-of type to the extent possibe. Lists are often reordered or may have fewer or more elements than you expect.
When using anything related to text, generally trim whitespace, ignore case and special characters where appropriate and prefer substrings over exact equality. On the other hand, don't be too loose. Usually, text content and values are weak targets but sometimes necessary.
Avoid pointless steps in a selector, like body > div#container > p > .target which should just be #container .target or #container p .target. body says almost nothing, > is too rigid, div isn't necessary since we have an id (if it changes to a span our new selector will still work), and the p is generic--there are probably no .targets outside of ps anyway.
Avoid browser-generated selectors. These are usually the worst of both worlds: highly vague and rigid at the same time. The goal is to be the opposite: accurate and specific, yet as flexible as possible.
Feel free to break rules as appropriate.
I know we can use document.querySelectorAll(yada yada yada) to get a list of elements that match any CSS selector, but is it possible to come at the problem from another angle, and match any element with a certain CSS property?
I've written a function to shrink any flex-wrap containers to the correct size, without the ugly gap on the right hand side that current CSS means we're always stuck with. (Aside: is there something in an upcoming spec to fix this issue?)
I'd like it to run automatically on any flex-wrap containers on my site.
I don't want to add a class="flex-wrap" to each element. I know tailwind et al are all about this type of classes these days, but I'm old school. I believe in separation of design/content and that class/IDs should be semantic, not stylistic.
But for this "problem", the only solutions I can see are:
manually add IDs/classes of elements which I know need the fix in my JavaScript
add a class that querySelectorAll can latch on to
loop through every element calling getComputedStyles() to check for this CSS property. I've not tried that, as some of my pages have many thousands of DOM elements, and I know it's a terrible idea.
Is there a better 4th option that I've not considered?
I extract several blocks that are spread across the page, draw them onto a canvas, via CanvasRenderingContext2D.drawImage(), and then I export the canvas to PDF/JPG.
Now, while drawing them, I also want to tidy their looks and/or remove/display parts of some blocks, because the styling no longer makes sense from a static point of view.
I can append a class on body, style blocks differently while body has that class, build the canvas, export it to a content type of choice and remove body class, but this makes the page look quirky for 1 - 2 seconds. (I could just cover the page with a loader, I guess)
I can clone the block, style it differently or apply whatever styling (inline or via classes) and draw an image from that, but this is not optimal.
Any idea on how to do this properly? I'm inclined to go for something like emulation. Have the whole "export styling" under the native #print and emulate that before drawing the blocks onto the canvas, without affecting what gets rendered in the browser, but I'm not sure if this is possible.
I know these don't answer your specific question, but perhaps these are viable alternatives to the problem.
It sounds like you're trying to make something printable. I assume this is triggered by a user interaction of some kind so...
This means you have a few interface options. For example, you could "hide" the screen by placing a modal over the entire thing with a message that says "processing just a moment". Then the body (or another element) class solution works.
You could copy the elements as you suggested. If you go that route I would move the copies off the screen while you change them.
.element {
position: fixed;
left: -100vw;
}
Without knowing how many duplicates you need to make it's hard to recommend this option.
Alternately, could you offload the effort to a service worker? This would require a copy into memory BUT its completely detached from the DOM and runs in a different thread.
It's a really interesting problem though!
I think you can use node.cloneNode(true) to make a deep clone for all these blocks, put them in a classname scoped common root that's hide away from user's view, then you mod their style secretly.
Since you mod the cloned version of nodes, the original remains untouched. Plus, using the classname scope, your css can target these clones accurately.
I have poor eyesight. So I want to enlarge font size on any page if it less than minimum. I am using "tampermonkey" plugin to automate this script.
People either advising to find specific element and edit it like this:
document.getElementById("p1").style.font="italic bold 20px arial,serif";
or to change all fonts like this:
document.body.style.fontSize = "220%";
But it's not affecting text inside divs like in here(only titles are effected): http://mashable.com/
Should I iterate through all page elements or is there better way? Thnx
Your current approach is dependent on a site only using relative units (or inheritance) to determine what font size to use.
Given (not HTML, I'm using simple element names for ease of references):
<a> Alpha
<b> Beta
<c> Charlie
</c>
</b>
</a>
with:
b { font-size: 0.8em; }
c { font-size: 12px; }
Any changes you make to a will be used to calculate b (since em is a relative unit). However, c is defined using an absolute* unit so it will always be 12px.
Should I iterate through all page elements or is there better way?
I can see a couple of options for that rough approach.
Iterate through all the style rules (both document.styleSheets and the style property of every element in the page) and remove any references to absolute units.
Iterate through every element on the page, testing its computed style and setting element.style.fontSize = "something !important" as you go.
The problem is that both are going to take quite a while to run, especially on a large page (that's a lot of elements to test).
You might want to consider throwing out the default styles entirely, possibly by grabbing all the HTML from the page and then document.writeing it to clear the existing content.
Rather than write my own from scratch, I'd probably look at using existing tools on the market. I tend to use Mobile Safari's Reading Mode and Clearly for Chrome when I come across an article I want to read that has too small text or contrast problems.
Alternatively, Firefox has a Minimum Font Size preference.
* OK, so technically in strict CSS terms, pixels are a relative unit, but they aren't relative to the font-size of another element, so they are effectively absolute for our purposes.
Usually we search elements by selectors in libraries like jquery but what happens if we want to make the other way around: given an element we want to find all rules applied to it like firebug does it!
This kind of job is necessary if we want to bring a css file and update it so we can see live results in our webpage.
In firebug by selecting an element we can see in reverse order all the rules that were applied to it.
First, we have to find the document's style rules and retrieve their contents as a text so user can update them but the real problem is to find the relation between an element and all the rules....
Is this possible even with the help of a library like jquery?
I was looking for opinions about what tools can one use to tackle such a problem in the restrictions applied by a browser environment...
My opinion is that we have to make a brute search attack at the css files/selectors and construct the dependencies: suppose we have 100 rules for a page with 100 elements the possible combinations are 10^4 as the relations are many-to-many.
Then, we can build 'tables' in memory (hash arrays) that might excced 10^4 records if we want to keep the cascading order of rules for an element.
Anyway, my point is that I wouldn't dare to put jquery in such a pain! If we can 'transplant' the heart of jquery, it's search engine I mean.
It seems that it's heart listens to the name of 'sizzle' but it's a waste of time here (regular expressions? no thanks). I think the real 'heart' is a simple word: 'querySelectorAll' and now we are running with native speeds here...
just an opinion since I don't care about wide/old browser support.