I'm trying to use JavaScript to scrape data from the following page, specifically the "free shipping free returns" text that appears when you hover your mouse over the cart icon:
Whenever I hover over the cart icon, new HTML is added to the DOM.
And when I move my mouse away, the the previously added HTML goes away. I want to be able to parse data from the HTML that gets added without having the popup visible. How would I be able to scrape this text data even if someone does not hover over the cart icon? Is there a way to access all the HTML data at once?
You can try to catch the JavaScript function being executed when you hover your mouse over the cart icon. You can do this via the developer tools. Add break points to code execution if the DOM changes (on the parent element in which the new element is added).
Once you get the function, just execute it directly on that page and you'll probably be able to see the popup and extract it's contents.
You could also try to simulate a hover as explained in these answers: How do I simulate a mouseover in pure JavaScript that activates the CSS ":hover"?
Scraping a page for data is not usually recommended since they can change over time (especially ones not written directly in HTML, but are rather generated (usually they have CSS classes like 8h2H1)).
If this is not supposed to be a long-term solution, the above answer by #nvkrjn is a good answer. Or, you can just check for an element with the id name free-shipping-label.
But, if this is supposed to be a long-term solution, then I would suggest using an API (this site doesn't seem to have one) or querying the database like how to Javascript does. Also, if you're using a non-browser environment (eg BeautifulSoup), it may not run the JS required to get the data.
Related
I have an existing website composed of individual pages (each page is a different tool that requires some user input (ie forms), and each with it's own set of javascript functions to populate dropdown lists, etc on that page). Each of the tools is accessed from the main index.html.
Instead of each tool being its own "stand-alone" page that is invoked from index.html, I'd like each tool to be displayed in an iFrame instead on the main page. This way the main page remains static, while only updating the iframe with whatever tool the user selects. So say on the main index page, I have a 3 tools menu (collect logs, collect KPIs, collect status), along with an iFrame. If the user selects collect logs for example, the menu containing "collect logs" stays there, but the "collect logs" page is displayed in the iFrame.
My problem is that all the HTML content works fine, but none of the javascript code in the selected tool page works (ie none of the drop downs get populated since it's the javascript code in the page that does that by reading a file on the server).
Is there an easy way to port each tool page (html+javascript) to an iFrame without having to re-write tons of code (in my naivety I thought simply invoking the page inside an iFrame using target='' in the href would work)? Or is there a better method of accomplishing what I'm trying to do? Maybe iFrame isn't the solution.
Content in iframes remain autonomous from the wrapper app, so it makes sense that it's not working correctly. Other than building a listener for a click event associated with the div wrapped around the iframe, the iframe document isn't accessible if it points to a different origin. (See [same-origin policy]
(https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy))
To stay with simple html/css/js solution:
You can use a regular div to wrap each 'stand-alone' content and then just use whatever button/navigation target you have display that div and hide the previous by changing their css display style with the onClick event.
More invasive option:
You may want to consider using a more modular JS approach, like React JS, to build components vs pages and utilize React's structure to toggle components.
With react you can render each 'tool' when the user selects it . You would be able to utilize React component state as well to help in storing data and such for the life-cycle of the component.
If I do
<a href ... oncontextmenu="myFunction(event)">
It gets called when the user right clicks on a link, before the context menu appears. But I want to get an event after a context menu item has been selected. Ideally, only when they have selected "Open link in ...", so having access to what they actually clicked is desirable.
Would prefer a non-JQuery answer, but if needed, JQuery is o.k.
Added: Explanation what I'm trying to do:
When the user directly clicks a link to another (commercial hotel finder) site, I bring up a little alert "If you book a room, please enter XXXX as the Gift Code"). Would also like to put up this alert if they right click and open the link in a new tab etc.,.,
Added#2: Thanks for the prompt and direct answers. See my "answer" below for what I ended up doing. I'll leave this question open in case there are other options.
This is not possible to do in Javascript, but you might be able to achieve what you are looking for by replacing the default context menu with a custom one.
You can't. The primary purpose of the contextmenu event is to allow a page to replace the browser's context menu with a menu of its own rendered as HTML. The contents of the browser's context menu are not specified by any HTML standard, and will vary from browser to browser. Most of the options in the context menu refer to actions outside the scope of the page (e.g, "copy", "view page source", "open link in new window", "save as", etc.), and there is no event fired on the page when they are selected unless the action being performed would normally fire an event. (For instance, selecting "Print" might fire onbeforeprint and onafterprint events.)
The WHATWG HTML5 specification specifies a menu element which can be used to add items to the browser's context menu, but it's currently only supported under Firefox. So that's probably not really an option either.
Just an idea, and I'm not sure exactly what you are trying to do, but I would try calling the function on a set delay, like this:
Right-Click Me, then wait for 3 seconds.
The only downside is that once the alert() is called, the context menu is hidden.
Hope this helps!
What I ended up doing is inserting an intermediate page on my site.
Instead of directly going to the external site, the route (I'm using node & Express) brings up a intermediate page with the explanatory text, a checkbox "Got it, don't show this page again", (in my case, the answer gets stored in localStorage, others might store on the server) and a big green OK button that takes you to the external site.
If the route includes the query "?directXXX=true", (read from localStorage) you skip over the intermediate page and are immediately redirected to the external page.
This way I get a much nicer explanatory page, with an image, etc... (dialogs, even if you use something like alertify, look klunky) and I get to "intercept" all calls to the external site.
I set up a web service using LAMP for personal use.
Basically, I have a scrollable list with some database information I pull once in the PHP script. I'd like to be able to click on one of the items, and have more information appear in a preview pane . (preview pane is set up)
I don't expect to ever have more than 100 entries in my database. Also, the data I'm pulling is very small...about 10 fields, all text data.
My question is as follows:
If I pull all the data I need in one shot, how can I store it so that each block of data I define (2-3 rows are what I need in the preview pane) is "cached" and I can access that given block at will so I can display it upon clicking its corresponding list entry?
Basically, this is about the same as clicking an email in a web-browser and having the rest of the message show up in a preview pane.
Thanks.
nb: the fact that I'm using a preview pane or a list is irrelevant. Just describing what I'm doing.
If I understand correctly, you would like to create a simple page, where several details are hidden until you click on different buttons. If you would like to do this in a "cached" way, you can try the following:
Fetch the complete data so that everything (with details) ends up in the result HTML. Everything: I mean, the parts which should be visible at all times and also the details which should be visible only after pressing a button.
Now, to the design. You will have to write CSS code, which will hide the details. (Of course, for that, you will have to create HTML in step 1 which will let you target the details via CSS classes, for example.) You will also have to figure out how to create buttons.
The most complicated part is to bind the buttons to Javascript actions, which will perform showing the hidden details. You can control all CSS properties from Javascript, so for example you can alter the position, the size, the text color etc. of a <div> dynamically.
Most people nowadays use jQuery for tasks like the one in step 3, or even software libraries built on the top of jQuery. That could help a lot if you're just starting out with tasks like this one. On the jQuery site, you will find a great place to start, called the jQuery Learning Center.
Edit: I've created a very basic fiddle to let you test the JS part of my concept, see it here: http://jsfiddle.net/eL9mj/22/
I am using WSS 3.0 in my application. I am displaying a List as a DataView Webpart. My objective here is to make this webpart visible to a selected group of individuals. As there is no option for Target Audience in WSS 3.0, I went to edit Permissions for List and gave Read permissions only to selected users. This doesn't hide the web part from the page, rather shows an Access Denied message to other users.
Access denied. You do not have permission to perform this action or access this resource.
As I said, I want to hide this webpart, as in make it invisible on the web page from other users who do not have permissions to view it. As this message will be displayed only to those users who do not have permissions!, my approach is to search for the above message in the html and identify and hide the parentnode, thereby hiding the webpart.
I am not quite sure how to do this. Any ideas? Thanks in advance!
I'm going to assume you're in a situation where you can add additional web parts to the page and not trying to add JavaScript to the DataView Web Part directly. My suggestion won't work on a separate page if a Designer adds another view of this list.
Upload a blank .js file to your Site Assets. Add a Content Editor Web Part to your page, point it at that file. Add JQuery from a provider or host it yourself, adding the reference in your file. From there, you have 3 directions in which to work: first, explore the web part with Internet Explorer's F12 Developer Tools, keeping a particular eye on divs and tables with good unique ids, names, or classes that would solve your problem if hidden. Also keep an eye on the id of the div or table or cell or whatever that contains your access denied text. Second, (assuming you're new to JQuery) do some JQuery tutorials and then start playing with selecting the above items and, say, changing their background color. Once you have both of those, you're 90% there: (try to) select the object that would contain the access denied text, and if the innerHTML is present and equals that string, then set display:none for the div or tables to hide your web part. The third tool you have is editing the page directly with SharePoint Designer: you can toss a div with an id of your choosing around any xsl:template, which might help in your JQuery selecting.
I'm sorry I can't give you the specific code, since I'm not in a position to test it. If that changes, I'll try and give a more detailed response.
Old, misdirected answer: Do either of the answers here work for you? Alternatively, this answer has some great resources to solve your problem. Just change the message to an empty string.
Thanks Aron :D
I found the id for the webpart and hard coded it. It provided the solution, but I was hoping to programmatically fetch the id instead by searching the innerhtml, as I have more than one web parts that have to be hidden.
I found a partial solution here:
Hide SharePoint web part using javascript onclick method
I put a CEWP on the page and added the following script in it:
<script>
function hide()
{
var content = document.getElementById("webpartID").innerHTML;
var n = content.search("Access denied. You do not have permission to perform this action or access this resource");
if(n!=-1)
{ document.getElementById("webpartID").style.display="none";
}
}
_spbodyonloadfunctionnames.push("hide");
</script>
In my case, I picked up the webpart id from the aspx page or view source for the page.
In Google Reader, you can use a bookmarklet to "note" a page you're visiting. When you press the bookmarklet, a little Google form is displayed on top of the current page. In the form you can enter a description, etc. When you press Submit, the form submits itself without leaving the page, and then the form disappears. All in all, a very smooth experience.
I obviously tried to take a look at how it's done, but the most interesting parts are minified and unreadable. So...
Any ideas on how to implement something like this (on the browser side)? What issues are there? Existing blog posts describing this?
Aupajo has it right. I will, however, point you towards a bookmarklet framework I worked up for our site (www.iminta.com).
The bookmarklet itself reads as follows:
javascript:void((function(){
var e=document.createElement('script');
e.setAttribute('type','text/javascript');
e.setAttribute('src','http://www.iminta.com/javascripts/new_bookmarklet.js?noCache='+new%20Date().getTime());
document.body.appendChild(e)
})())
This just injects a new script into the document that includes this file:
http://www.iminta.com/javascripts/new_bookmarklet.js
It's important to note that the bookmarklet creates an iframe, positions it, and adds events to the document to allow the user to do things like hit escape (to close the window) or to scroll (so it stays visible). It also hides elements that don't play well with z-positioning (flash, for example). Finally, it facilitates communicating across to the javascript that is running within the iframe. In this way, you can have a close button in the iframe that tells the parent document to remove the iframe. This kind of cross-domain stuff is a bit hacky, but it's the only way (I've seen) to do it.
Not for the feint of heart; if you're not good at JavaScript, prepare to struggle.
At it's very basic level it will be using createElement to create the elements to insert into the page and appendChild or insertBefore to insert them into the page.
You can use a simple bookmarklet to add a <script> tag which loads an external JavaScript file that can push the necessary elements to the DOM and present a modal window to the user. The form is submitted via an AJAX request, it's processed server-side, and returns with success or a list of errors the user needs to correct.
So the bookmarklet would look like:
javascript:code-to-add-script-tag-and-init-the-script;
The external script would include:
The ability to add an element to the DOM
The ability to update innerHTML of that element to be the markup you want to display for the user
Handling for the AJAX form processing
The window effect can be achieved with CSS positioning.
As for one complete resource for this specific task, you'd be pretty lucky to find anything. But have a look at the smaller, individual parts and you'll find plenty of resources. Have a look around for information on modal windows, adding elements to the DOM, and AJAX processing.