Develop Reference

How to browse over a page using PhantomJS and Selenium

I got some DIV elements on a web page. Totally there are abound 30 DIV blocks of the following similar structure:
<div class="w-dyn-item">
<a href="/project/soft" class="jobs-wrapper no-line w-inline-block w-clearfix">
<div class="jobs-client">
<img data-qazy="true" src="https://global.com/test.jpg" alt="Soft" class="image-9">
<div style="background-color:#cd7f32" class="job-time">Level 1</div>
</div>
<div class="jobs-content w-clearfix">
<div class="w-clearfix">
<div class="text-block-19 w-condition-invisible">PROMO</div>
<h3 class="job-title">Soft</h3>
<img height="30" data-qazy="true" src="https://global.com/test.jpg" alt="Soft" class="image-15 w-hidden-main w-hidden-medium w-hidden-small"></div>
<div class="div-block w-clearfix">
<div class="text-block-4">Italy</div>
<div class="text-block-4 w-hidden-small w-hidden-tiny">AMB</div>
<div class="text-block-4 w-hidden-small w-hidden-tiny">GTL</div>
<div class="text-block-13">January 10, 2017</div><div class="text-block-14">End date:</div></div><div class="space small"></div><p class="paragraph-3">Text text text</p></div>
</a>
</div>
I am trying to access a href and click on the link. However, the problem is that I cannot use find_element_by_link_text, because the link text does not exist. Is it possible to access a href by class class="jobs-wrapper no-line w-inline-block w-clearfix"? When I used find_element_by_class_name, I got the error Message: {"errorMessage":"Compound class names not permitted","request
from selenium import webdriver
driver = webdriver.PhantomJS()
driver.set_window_size(1120, 550)
driver.get("https://myurl.com/")
driver.find_element_by_link_text("//a href").click()
print driver.current_url
driver.quit()

The error you're getting is because Selenium's find_element_by_class_name does not support multiple classes.
Use a CSS selector with find_elements_by_css_selector instead:
driver.find_elements_by_css_selector('.jobs-wrapper.no-line.w-inline-block.w-clearfix')
Will select all tags with your wanted class, then you can iterate over them and use click() or any other wanted action
EDIT
Following your comment, new snippet to help you do what you want:
result = {}
urls = []
# 'elements' is a the list you previously obtained using the css selector
for element in elements:
urls.append(element.get_attribute('href'))
# Now you can iterate over all extracted hrefs:
for url in urls:
url_data = {}
driver.get(url)
field1 = driver.find_element_by_id('wanted_id_1')
url_data['field1'] = field1
field2 = driver.find_element_by_id('wanted_id_2')
url_data['field2'] = field2
result[url] = url_data
Now, result is a dictionary in a structure similar to what you wanted.
Note that field1 and field2 are of type WebElement so you'll probably need to do something with them first (extract attribute, text, etc).
Also, on personal note, Look into the requests together with BeautifulSoup, they might be a way better fit than Selenium for this or future similar cases.

If your only requirement is to click the a tag inside a tag with w-dyn-item class, then you could do it like this:
driver.find_element_by_class_name("w-dyn-item").find_element_by_tag_name("a").click()
To iterate over all tags with w-dyn-item class -> click the a inside them -> do something -> go back, do this:
tags = driver.find_elements_by_class_name("w-dyn-item")
for i in range(len(tags)):
tag = driver.find_elements_by_class_name("w-dyn-item")[i]
tag.find_element_by_tag_name("a").click()
# Do what you want inside the page...
driver.back()
The key here is of course to go back to the root page after you're done with the inner page.

To access and click the a href you can use the following line of code :
driver.find_element_by_xpath("//div[#class='w-dyn-item']/a[#href='/project/soft']").click()

How to print the content of a div that changes based on a foreach statement?

I have an ArrayList named conversations_client, I want to be able to get the value of conversation[6] for each div.
Each one the the media div represent a conversation.
Here is a part of my code :
<c:forEach var="conversation" items="${conversations_client}" >
<div class="media">
<div class="media-left">
<input class="checkbox" type="checkbox" name="selected-conv">
</div>
<div class="media-body">
<h4 class="media-heading">${conversation[0]}</h4>
<span class="hidden">${conversation[6]}</span>
......
I had first tried to use this code :
<script type="text/javascript">
$('.media').click(function(){
var text = $(this).text();
alert(text);
});
</script>
But it would print not only conversation[6] but others as well since they're all inside the div with class media.
Then I tried this :
<script type="text/javascript">
$('.media').click(function(){
var text = $('.hidden').text();
alert(text);
});
</script>
But it would print the same id for all divs, which is the id of the first one.
How can I do that? and would it be better to wrap those media divs with tags to be able to use a specific action for each one of them? because I am displaying all conversations, then once the user will click on one them I'll have to recover its id to be able to display the messages of that specific conversation and it isn't advisable to write Java code inside of jsp.
Thank you :)

Use find from current element.
$('.media').click(function()
{
var text = $(this).find('#hidden').text();
alert(text);
});
UPDATE:
You are using loop to repeat the markup. It's not good to use ID hidden multiple times as per W3C standards. ID should be unique and should be used once in the page.
Use class instead of id for multiple usages. In this case class="hidden"
Better change <span id="hidden">${conversation[6]}</span> to <span class="hidden">${conversation[6]}</span>
Also change JS to find .hidden
$('.media').click(function()
{
var text = $(this).find('.hidden').text();
alert(text);
});

Google Tag Manager - CSS selector challenge

I am trying to get the URL of a link in the source code. The challenge is that the URL is hidden behind a image, and thus only letting me fetch the image-url.
I've been trying to figure a way to solve this issue by using the new CSS selector in the trigger system and also made a DOM variable that should get the URL when the image is clicked. There can also be multiple downloads.
Here is an example of what I am trying to achieve:
<div>
<div class="download">
<a href="example.com/The-URL-I-Want-to-get-if-top-image-is-clicked.pdf" target="_blank">
<img src="some-download-image.png"/></a>
<div class="download">
<a href="example.com/Another-URL-I-Want-to-get-if-middle-image-is-clicked.pdf" target="_blank">
<img src="some-download-image.png"/></a>
<div class="download">
<a href="example.com/Last-URL-I-Want-to-get-if-bottom-image-is-clicked.pdf" target="_blank">
<img src="some-download-image.png"/></a>
</div>
</div>
There are much code above and below this snippet, but with the selector it should be fairly easy to get the information I want. Only that I don't.
If anyone have met this wall and solved it, I really would like to know how. :)

This is one possible solution. So as I understand it, you would like to grab the anchor element's href attribute when you click the "download" image.
A Custom Javascript variable would need to be created so that you can manipulate the click element object:
function(){
var ec = {{Click Element}};
var href = $(ec).closest('a').attr('href');
return href;
}
So you will need to do your due diligence and add in your error checking and stuff, but basically this should return to you the href, and then you will need to parse the string to extract the portion that you need.

Chrome Extension: Reading text between a tag

I'm new to making Chrome extensions. I'm trying to read text between a class tag, such as:
<div id="AssetThumbnail" class="thumbnail-holder" data-3d-thumbs-enabled data-url="/thumbnail/asset?assetId=111795617&thumbnailFormatId=6912&width=320&height=320" style="width:320px; height:320px;">
<span class="thumbnail-span" **data-3d-url=**"/asset-thumbnail-3d/json?assetId=111795617" data-js-files='http://js.rbxcdn.com/a552a24cb2c7a47ad748fd129a2e9624.js.gzip' ><img class='' src='http://t7.rbxcdn.com/7cfa58047697662d12f33d68b71e5f42' /></span>
<span class="enable-three-dee btn-control btn-control-small"></span>
</div>
I'd like to get the text from data-3d-url=. I have surrounded it with asterisks in my code so you can see what I'm referring to.

Using jQuery you could retrieve an attribute like that with the following snippet.
alert($('.thumbnail-span').attr("data-3d-url"));
You can then change the alert function to set a variable or whatever else you want to use that value for.
Here's a JSFiddle example: http://jsfiddle.net/r1umr3s5/1/

show element not working

Hi I have a problem with my function, when I call it to show my hidden div it does not work. It does not show the hidden div. I followed previous examples from what have been posted in stackoverflow but still my code does not work.
This is my html file
<div id="playTheGame" class="css/outer" style="display:none;" >
<div class="css/inner">
<h1>Choose!</h1>
<section id="hand">
<img src="images/rock.png">
<img src="images/paper.png">
<img src="images/scissors.png">
</section>
</div>
</div>
My Function
<script>
function logSuccess(){
document.getElementById("playTheGame").style.visibility="visible";
}
</script>
The Button I used for the function
<input type="button" onclick="logSuccess()" value="Show">

Change your code to this
document.getElementById("playTheGame").style.display = "block";
Since you hid it using the display property, show it using the display property.

There are two options for this:
One using JavaScript:
object.style.display="block"; // where object will be the playThemGame id element..
And the other one using jQuery; JavaScript library.
$("#playTheGame").show();
The option two won't work, because you will have to write the event function too, So just use the first one as:
document.getElementById("playTheGame").style.display="block";
Edit:
Since you are using
document.getElementById("playTheGame").style.display="block";
To disply the result, then you must use this to remove the display!
document.getElementById("playTheGame").style.display = "none"; to hide it back!
The basic idea is that, this will just shift the object-->style-->display's value to none Its not going to add any more attribute. Its just going to shift current attribute's value.