Unable to find some html elements using Jsoup - javascript

I am trying to find elements corresponding to "Apply" button in this page(https://gwlabs.taleo.net/careersection/gw_ext_career_section/jobsearch.ftl) using Jsoup. I am tried using Document.getElementsMatchingOwnText("Apply") but it didn't return the elements. I have tried to list all the anchor elements and print their text, i couldn't see any elements with the text "Apply". Output was the following
Text of anchor is Sign In
Text of anchor is Job Search
Text of anchor is My Jobpage
Text of anchor is OK
Text of anchor is OK
Text of anchor is Submit a friend's profile
Text of anchor is View All Jobs
Text of anchor is Advanced Search
Text of anchor is Add Organization
Text of anchor is Add Location
Text of anchor is Add Job Field
Text of anchor is Cancel
Text of anchor is Add
Text of anchor is OK
Text of anchor is Clear
Text of anchor is Save this Search
Text of anchor is Single-line
Text of anchor is Save this Search
Text of anchor is Cancel
Text of anchor is Access My Saved Searches
Text of anchor is Confirm
Text of anchor is Cancel
Text of anchor is OK
Text of anchor is OK
Text of anchor is OK
Text of anchor is You can also view all available job openings.
Text of anchor is Refer a friend
Text of anchor is Previous
Text of anchor is Next
Text of anchor is OK
The same approach was giving desired results for https://healthfirst.taleo.net/careersection/hf_ext_cs/jobsearch.ftl?lang=en
I am clueless as to why is this so, can somebody take a look please

The data you are trying to retrieve are not part of initial html, but is set by JavaScript after page is loaded. You can check it by disabling JavaScript in your browser. Jsoup only gets static html, does not execute JavaScript code.
Jsoup does not currently support JavaScript, which means that pages on which data is loaded with JavaScript will not be available when parsing using Jsoup.
If you want to get such dynamically loaded data, you can:
-Use an alternative, such as HtmlUnit, Selenium WebDriver or ui4j.
-Use the website's API, if it offers one,
-To find out from where the website loads its data, usually all you need to do is send an HTTP request somewhere to get the data as JSON.
More details can be found here:
https://stackoverflow.com/tags/jsoup/info
or you can use hints from this answer:
https://stackoverflow.com/a/50831894

Related

How to capture the *anchor text* of a right-clicked link for a Chrome extension context menu

According to the chrome.contextMenus api page you can get linkUrl, srcUrl, pageUrl, etc. If you select the anchor text before clicking you could use selectionText but I don't want to have to do that.
The best answer I've been able to find after much googling is here, which is to use document.getElementsByTagName('a') to get all the links on the page, then get the innerHTML of the one that had the url you got via linkUrl. But this would break if there were more than one link on the page to the same target url with different anchor texts.
I also found this suggestion to use document.activeElement but when I tried that it returned the entire body of the page, apparently right-clicking on a link does not properly make it the "active element".

Scrape / extract data from hidden divs in scrapy

Hi I am trying to scrape a website where there is an input text. Whenever, I click on the input text there are dropdown suggestions for the value of the input text. It is not on select tag.
The value of those suggestion is inside a div tag elements. There were almost 200 divs/suggestions of it.
What I did is scrape from it using scrapy using xpath / css selectors. I found out that these 200 divs are actually hidden when I view the code using "View page source" instead of "Inspect elements".
Please help. Thank you
These elements are generated on the fly by some dropdown library, so you have to investigate the website source code and/or the HTTP requests it's making. All the data you are looking for should be there (most likely in JSON format), not in the HTML itself.
For example, if you are using Chrome:
Press F12 to open devtools while you are on the website
Press F5 to reload the page
Navigate to Network or Source tab
Try to locate the data (CTRL+F would be really helpful here)

Html element to allow Crtl+A of the text inside it

I'm creating a website that allows a user to enter some data into a form, and the form once submitted will return a JSON. My website has a header, a footer, etc. Right now, I'm displaying the JSON inside a div.
My problem: the JSON can potentially be very large. The user of my website would ideally be able to copy the JSON result and use it for other purposes. If the user tries to select the whole JSON via Ctrl+A, it will select all the text in my website, i.e., header and footer. If the user just wants to get the JSON (and it is very large), he/she will have to manually select the JSON.
Is there a HTML element that I can use to render the JSON in (instead of div) so that, when the cursor is focused on this element, it will allow Ctrl+A to just select all the text inside this element? Something similar to this: http://www.jsoneditoronline.org/
I've been looking for different HTML elements, such as pre, but they don't achieve my goal. Maybe I'm not using the right keywords to search in a search engine.
Thanks.
A textarea element will do the trick or add contenteditable="true" to any element and they will be able to select all that way as well. Additionally check out https://clipboardjs.com/ which will allow you to automatically copy content to the users clipboard for them.

Javascript: Keep text selected in div when focusing textbox

I'm working on a wysiwyg editor for my blog application on my website.
I've already written some basic editor commands like bolding, italic style and that stuff...
Now the problem is, when I wan't to do a command which needs additional information like a hyperlink.
I got a div with contentEditable on true and a textbox where I can put in the URL for the hyperlink.
I looked up in this forum and other places but I didn't find any possible solution on how to select text in the editable div, then put the desired url for the hyperlink in the textbox and finally click on a div button to call the document.execCommand("CreateLink", null, $URL); function in javascript.
Is there a possibility to select the text, then on focusing the textbox, save the position of the start and end of the selection and on clicking the button select the text again, with the saved range positions to affect the hyperlink?

Highlight text inside iframe while clicking same text on the document using jquery

I wanted to highlight text inside iframe if the user clicks the same text outside iframe in the webpage.
I could search for the contents using the below code.
$("#iframe").contents().text().search("text_to_be_searched");
But how to highlight the text?
ASSUMING that your code for finding the text is going to work (and it is going to be a little complicated to get that to work), then all you have to do is this:
$("#iframe").contents().text().search("text_to_be_searched").
wrap("<span class='highlight-me'>");
And then use CSS to highlight anything with a class of 'highlight-me'.
If that doesn't work, we'd have to see how you are finding the text, you need to get into the form of a node that jQuery can work with.

Categories

Resources