Develop Reference

Scrape web data from mouseover popup

I'm trying to use JavaScript to scrape data from the following page, specifically the "free shipping free returns" text that appears when you hover your mouse over the cart icon:
Whenever I hover over the cart icon, new HTML is added to the DOM.
And when I move my mouse away, the the previously added HTML goes away. I want to be able to parse data from the HTML that gets added without having the popup visible. How would I be able to scrape this text data even if someone does not hover over the cart icon? Is there a way to access all the HTML data at once?

You can try to catch the JavaScript function being executed when you hover your mouse over the cart icon. You can do this via the developer tools. Add break points to code execution if the DOM changes (on the parent element in which the new element is added).
Once you get the function, just execute it directly on that page and you'll probably be able to see the popup and extract it's contents.
You could also try to simulate a hover as explained in these answers: How do I simulate a mouseover in pure JavaScript that activates the CSS ":hover"?

Scraping a page for data is not usually recommended since they can change over time (especially ones not written directly in HTML, but are rather generated (usually they have CSS classes like 8h2H1)).
If this is not supposed to be a long-term solution, the above answer by #nvkrjn is a good answer. Or, you can just check for an element with the id name free-shipping-label.
But, if this is supposed to be a long-term solution, then I would suggest using an API (this site doesn't seem to have one) or querying the database like how to Javascript does. Also, if you're using a non-browser environment (eg BeautifulSoup), it may not run the JS required to get the data.

Modify Contents of Editable Div on Twitter Post

I'm writing a chrome extension which helps the user type things on twitter. When writing a tweet on twitter, twitter opens an editable div container. When the user types into it, twitter (which is using some web-framework presumably) generates sub-divs and spans with the text the user types and places them within the content-editable div.
The thing is when one manually changes the span value (for instance, through inspect elements), and then types something again, the value in the span will just revert back to what it previously was (before the inspect elements edit). This is probably because the actually typed string is stored somewhere in javascript, and everything gets overwritten again when the user types into the div.
I've been trying to find a way around this using JQuery but with no success. I don't really know how to start. If it were just a regular input tag, you could call something like $("input").val("new value"), easy-peasy... but I don't know how one could go about doing that for an editable div that gets updated by javascript running somewhere on the page.
For a while, I just thought it would be impossible...
BUT NOW I do know it is possible. If you download the Grammarly extension and use the Grammarly popup-editor (which opens a new window to edit text), then submit that, the twitter editable-content div updates appropriately and everything works like magic.
Sorry if this isn't a standard programming question, but I couldn't find anything on the web that comes close to what I'm trying to do. Maybe I'm just not experienced enough and am missing something really obvious. I tried looking at the twitter and Grammarly source code but it's all minified garbled javascript that I can't read...
Thanks for any help and insight!
EDIT: the twitter url in question is: https://twitter.com/compose/tweet The div in question is the one with contenteditable="true" attribute (you can search it in the inspector)

How to get text boxes to except text added through JavaScript (Twitch TV chat)

So, I am trying to write a simple command chat bot for Twitch.tv chat, but I cannot seem to be able to get it to actually submit the text I add. Although I can add text and "click" the send button with JavaScript, it will not actually submit it.
For example, I can use http://www.twitch.tv/example to test it. Apparently it is not good practice on Stack Overflow to link to sites, but I do not know what else to do. I cannot recreate the problem in JS Fiddle because I do not know exactly what the problem is.
I can get these elements to seemingly work by using the Chrome console:
//finding the textbox
var textbox = document.getElementsByClassName("chat_text_input mousetrap ember-view ember-text-area")[0];
//then assign some text to it.
textbox.value = "Hello, World.";
This works fine and the text appears.
//to send it I should merely find a way to click the "send" button
var send = document.getElementsByClassName("button primary float-right send-chat-button")[0];
send.click();
Well, this does not really work. It only actually submits the text if I physically either type or use a character (like shift) inside the textbox, or physically click the button myself. So, from what I understand there is some sort of event that is triggered when I physically type and click. Unfortunately, I do not understand how to find/trigger this event through the console. This is what I want to do.
Preferably I would like to do this in pure JavaScript, but I believe it may be simpler in jQuery. If it absolutely has to be jQuery that's OK and I will just have to suck it up and learn.
As a side note, I was able to successfully use this (with the elements changed of course), for YouTube live streaming chat. However, several other websites' chat react similarly to Twitch. These include web skype.

I've spent the last few hours poking around Ember and Javascript to figure this out. I too was running into an issue where I had to either click into the chat area or the send button itself.
I thought I had a lead with a script made for Twitch Plays Pokemon which would send preset messages into the chat. It used Jquery to input the value into the textarea and click on the send button, but that no longer works as Twitch now uses Emberjs 2
I am not too familiar with the Ember framework, but it seems Twitch utilizes a Textarea component(in Ember terms), which associates some framework specific API to the Textarea HTML element.
Since the Twitch webapp is written with Ember, it would seem that using vanilla javascript to set the value of the textbox would not notify the property change to whatever observers are registered internally to Ember
The workaround would then be to change the value property via Ember internal methods. Luckily poking around the views associated on the Textarea DOM element gives access to the set method.
http://www.ember-doc.com/classes/Ember.TextArea.html#method_set
Ember.view.views worked pre-Ember 2 but was deprecated and now requires lookup
App.__container__.lookup('-view-registry:main')[document.querySelector(".chat_text_input").id].set("value","test");
document.querySelector(".chat-interface__submit").click();

Hide WSS 3.0 Webpart Using JavaScript

I am using WSS 3.0 in my application. I am displaying a List as a DataView Webpart. My objective here is to make this webpart visible to a selected group of individuals. As there is no option for Target Audience in WSS 3.0, I went to edit Permissions for List and gave Read permissions only to selected users. This doesn't hide the web part from the page, rather shows an Access Denied message to other users.
Access denied. You do not have permission to perform this action or access this resource.
As I said, I want to hide this webpart, as in make it invisible on the web page from other users who do not have permissions to view it. As this message will be displayed only to those users who do not have permissions!, my approach is to search for the above message in the html and identify and hide the parentnode, thereby hiding the webpart.
I am not quite sure how to do this. Any ideas? Thanks in advance!

I'm going to assume you're in a situation where you can add additional web parts to the page and not trying to add JavaScript to the DataView Web Part directly. My suggestion won't work on a separate page if a Designer adds another view of this list.
Upload a blank .js file to your Site Assets. Add a Content Editor Web Part to your page, point it at that file. Add JQuery from a provider or host it yourself, adding the reference in your file. From there, you have 3 directions in which to work: first, explore the web part with Internet Explorer's F12 Developer Tools, keeping a particular eye on divs and tables with good unique ids, names, or classes that would solve your problem if hidden. Also keep an eye on the id of the div or table or cell or whatever that contains your access denied text. Second, (assuming you're new to JQuery) do some JQuery tutorials and then start playing with selecting the above items and, say, changing their background color. Once you have both of those, you're 90% there: (try to) select the object that would contain the access denied text, and if the innerHTML is present and equals that string, then set display:none for the div or tables to hide your web part. The third tool you have is editing the page directly with SharePoint Designer: you can toss a div with an id of your choosing around any xsl:template, which might help in your JQuery selecting.
I'm sorry I can't give you the specific code, since I'm not in a position to test it. If that changes, I'll try and give a more detailed response.
Old, misdirected answer: Do either of the answers here work for you? Alternatively, this answer has some great resources to solve your problem. Just change the message to an empty string.

Thanks Aron :D
I found the id for the webpart and hard coded it. It provided the solution, but I was hoping to programmatically fetch the id instead by searching the innerhtml, as I have more than one web parts that have to be hidden.
I found a partial solution here:
Hide SharePoint web part using javascript onclick method
I put a CEWP on the page and added the following script in it:
<script>
function hide()
{
var content = document.getElementById("webpartID").innerHTML;
var n = content.search("Access denied. You do not have permission to perform this action or access this resource");
if(n!=-1)
{ document.getElementById("webpartID").style.display="none";
}
}
_spbodyonloadfunctionnames.push("hide");
</script>
In my case, I picked up the webpart id from the aspx page or view source for the page.

How to target a input box with selenium when the ID changes?

Trying to automate certain tests dealing with Yammer. What I want to do is to use selenium to post something, but when I click on the textbox it changes it's id. Also, everytime the page reloads, or a post is made, that textbox changes it's id.
So what I did next was to use wildcards by telling selenium to look for
//input[starts-with(id, 'yamjs')]
However, I quickly found out that yamjs is used for both the body of the update as well as the "+ Add people to notify" textbox.
The difference between the two seems to be the class. The one that we're interested in has a class="yj-tapf-textarea" whereas the add people to notify textbox has a class="yj-callout-bar-entry-field"
How do I get selenium to find the input field with the id that starts with yamjs within the class yj-tapf-textarea, and not the one in the class yj-callout-bar-entry-field?

Seems like I wasn't paying close enough attention to the classes and their locations. Here's what I ended up with:
driver.findElement(By.className("yj-auto-width-fake-textarea")).click();
driver.findElement(By.className("yj-tapf-textarea")).sendKeys(Text);
driver.findElement(By.id("yj-yam.ui.publisher.old.SubmitButton")).click();

If the id changes then perhaps it is a hint to not use id as the locator.
Using CssSelectors and XPath can give you more stable and reliable locators