I want to fetch this site
https://www.film-fish.com/modern-mindless-action
to fetch the IMDB IDs of all movies listed there.
The problem is that the page loads all movies listed there just after scrolling down. So, a simple wget doesn't work.
Even if I scroll to the bottom of the page and view the source code, I do not see the last movie in the list (Hard Kill (2020)).
So the problem seems to be that the content is being created via JavaScript.
Has anybody a tip on how to achieve that?
So the problem seems to be that the content is being created via a js
script. Has anybody a tip on how to achieve that?
Indeed, executing JavaScript code is beyond scope of GNU Wget. You would need browser automation tool. If you know some Node.js or JavaScript I suggest taking look at PhantomJS Quick Start, Page Automation. Please take look at first example in 2nd link, you should be probably able to rework to your needs, i.e. instruct page to scroll down using JavaScript then extract what you need using JavaScript.
I am making a site at 000webhost.com, but the problem is that they add a script at the bottom of every page to show ads, and even if I deleted it manually, it comes back.
It is placed at the very bottom of the page, even under the </html> tag ( is not included).
Is there a way to cancel it's function with another JavaScript or something? Here is the code that is placed:
<!-- Hosting24 Analytics Code -->
<script type="text/javascript" src="http://stats.hosting24.com/count.php"></script>
<!-- End Of Analytics Code -->
Though you should not remove the adds(as its against the policy and probably will take down your site),
you can do a simple trick if you want to hide the adds, Trick is based on anchors, which are used as adds by 000.webhost
change all your anchor tags to add a class: for example
<a class='myAnchor' />
this class will not be applied to any add than 000.webhost provides.
add a css to your page
a:not(.myAnchor){
display:none;
}
You can set timer to constantly check for injected scripts and remove them, or you can hide ads with CSS rules or you can just leave it there and don't break their policy. There are plenty of hostings without any ads.
you are on free hosting server and they are injecting adds script into your page please provide injected code here, i mean whatever html code injected by this script.other wise you need full access to server to know which service or things inject script into your page and stop them....
I will start with admitting that although I have been researching, reading, and trouble-shooting my issue I think I am missing something fundamental with my site. I apologize for my ignorance in advance but this is the only way to learn--by admitting you don't know what to do.
I'm running a VB.NET site which has a master page and content pages. I am still very much in development of the entire site trying to add features.
What I'd like to do:
I would like to add google analytics and jquery magnific popup. I've followed all the instructions from google and from magnific pop-up and I'm still having issues.
What I've done to try and make this work so far:
My head on masterpage:
<script src="Scripts/googleanalytics.js" type="text/javascript"></script>
<script src="Scripts/jquery-2.1.3.js" type="text/javascript"></script>
<script src="Scripts/jquery-magnific-popup.js" type="text/javascript"></script>
My footer on masterpage:
<script type="text/javascript">
$('.test-popup-link').magnificPopup({type: 'image'});
</script>
Added class to image:
<a class="test-popup-link" href="img/bigdog.jpg"><img class="centerimg" src="img/smalldog.jpg" /></a>
My issues:
Google script fails to find script. It's definitely there, it was
copy/pasted directly from google's site into a file I am calling.
jquery and magnific pop-up only load on my root pages.
"/products/product1.aspx" pages for example refuse to load scripts.
Although they load on root pages, the script does not appear to work
for the image pop up. I've followed magnific-popup's instructions word for word for "initializing in html"
Instructions for magnific-popup: http://dimsemenov.com/plugins/magnific-popup/documentation.html#mfp-build-tool
Lastly, before I tried adding the image pop-up google analytics appeared to be working as I was getting reporting on the web analytics page.
Any help would be appreciated.
Solution found via talking it out on IRC.
I restarted my local server and it worked on my root directory. Race conditions or caching causing the problem?
I needed to add "/" before "Scripts" in my head which fixed non-root script loading.
Thanks to robert on freenode for talking me through it. :D
I have developed a small component which can be put in to any website. Now, I want to develop a code that could demonstrate how would my component look like on any website.
So, the person would come to my page and put in his URL and then my code should embed my custom JS/CSS in to the downloaded HTML and display it. Something like this.
Here, like the feedback tab, I want to show my component any where on that page.
Try a bookmarklet.
Create a piece of javascript that adds your code into the page such as the following:
javascript:(function(){var%20script=document.createElement('script');script.src='http://www.example.org/js/example.js';document.getElementsByTagName('head')[0].appendChild(script);})()
Add it as the href of a link like so:
Link Text Here
Tell your users to drag the link to their bookmark toolbar and click on it on different websites to try your code out.
Some examples: http://www.reclaimprivacy.org/, http://www.readability.com/bookmarklets
In the example you linked, they are requesting the page specified in the url querystring parameter on the server, and then doing more or less the following steps:
In the <head> tag they are adding a <base href="url" /> tag to the document. The base tag will make any relative links in the document treat the value in the href attribute as their root. This is how they are getting around broken css / images. (The base tag is supported by all browsers)
At the end of the document (IE the </body> tag) they are injecting the javascript that runs their demos.
They serve the modified HTML requested to the browser.
All of this is pretty straight forward in implementation. You could use regular expressions to match the <head> and </body> tags for steps 1 and 2 respectively. Depending on the server platform how you actually request the page will vary, but here are some links to get you started:
C# - HttpWebRequest object documentation
PHP - HttpRequest::send
Nathan's answer is the closest to how we have done the demo feature at WebEngage. To make such a demo functional, you'll need to create a Javascript widget that can be embedded on third party sites. syserr0r's answer on creating a bookmarklet is the simplest approach to do so. Our's is a JAVA backend and we use HttpClient to fetch the responses. As Nathan suggested, we parse the response, sanitize it and add our widget Javascript to the response. The widget JS code takes it on from there to render the Feedback tab and load a demo short survey.
Disclosure: I am a co-founder and ceo at WebEngage.
You can not do this with JQuery due to cross site scripting restrictions.
I suggest you write a PHP script that downloads the URL specified by the user and includes your widget code and then echo it back to the user.
I recommend using bookmarklets. I've made a bookmarklet generator for adding jQuery-enabled bookmarklets to a page to make development easier.
There's a caliper bookmarklet on the page that you can mess around with just to show an example of it working.
Full disclosure, this is something I've made, I'm not trying to be spammy as I think it's relevant: zbooks
You could make an iframe page, which loads their page in the iframe, and uses javascript to inject your code into the iframe.
Here is my approach...
http://jsfiddle.net/L2kEf/
html
<iframe src="http://www.bing.com"></iframe>
<div>I am div</div>
css
div { background: red; position: absolute; top: 20px; width: 100px; left:20px;}
iframe{width: 100%; height: 500px;}
you can add javascript/jquery too, so you could do something like,
jQuery //not 100% sure it would work coz of cross browser thingy, but you know, worth a try.
$('div').click(function (){
$('iframe').contents().html('changed');///
});
if this can't change any of the contents, you can display a dialog, to say it would normally work if it was in your website, then use #syserr0r approach for bookmarked users, for better results, since you are offering this kinda services, to developers, im sure they would know about bookmarking, my approach would be rarely used :) so hope it helps.
I had a problem of a similiar nature, and the main obstacle is the cross-domain policy.
You have to ask the user to put your code in a <script src="..."> or create a proxy solution that would add your code for them.
I went for the proxy and here are my observations:
it's easy to create a basic proxy in php - there are some php proxies on sourceforge and Ben Alman has created a simple php proxy for AJAX. Based on those I was able to create a php proxy altering the content properly in one day.
after that I spent a lot of time making it work with more and more sites with issues. You can never create a perfect proxy.
As an alternative (sa long as you are non-commercial) you can use http://www.jmarshall.com/tools/cgiproxy/ and put the site in an iframe and then do whatever you want to do with the iframes document, as it's in your domain thanks to the proxy. You can access iframeDOMnode.contentWindow.document then, etc.
You can create a Crossrider extension which your users can download.
Then simply add this to your App/Extension code:
appAPI.dom.addRemoteJS("http://yourdomain.com/file.js")
Your users can then download the extension (it works cross-browser for Internet Explorer, Chrome and Firefox) and it will load your JS code on every page load.
You can get an approximation of what it will look like using a iframe. Take a look at that link for an example.
http://jsfiddle.net/jzaun/5PjRy/
The issue with this appoch is that you can't move your DIV(s) when the page scrolls, they are in effect just floating over the iframe. There is no way around this as cross-domain scripting wont let you access the iframe's document to monitor scroll events.
The only other option you have for a better fitting example would be to load the page from the server side in whatever scripting language you are using and load that into the iframe (or into a div, etc.) and you can use javascript all you want as the page is coming from your domain.
For your example of what will your widget look like I imagine floating your DIV(s) over an iframe would give enough of an idea.
Please note the example you gave is using the server side method, not the iframe method.
I agree with the bookmarklet strategy.
I'm a fan of http://bookmarklets.heroku.com/, which lets you generate bookmarklets easily, inject jQuery, etc.
I am using the addthis widget on my site:
<!-- AddThis Button BEGIN -->
<a class="addthis_button" href="http://www.addthis.com/bookmark.php?v=250&username=abcdefg" rel="nofollow">
<img src="http://s7.addthis.com/static/btn/v2/lg-share-en.gif" width="188" height="24" alt="Bookmark and Share" style="border:0"/></a>
<script type="text/javascript" src="http://s7.addthis.com/js/250/addthis_widget.js"></script>
<!-- AddThis Button END -->
Now, the javascript addthis_widget.js, is useful only when someone actually uses the share button. How do I ensure that it doesn't load everytime, but only when someone clicks on the share button?
I am a noob in javascript, so some sample code will be much appreciated :)
ADDED:
I wish to avoid using jquery or other libraries, because as you can imagine, I am trying to reduce the load time / number of requests / data on my pages.
thanks
The simple answer is don't bother.
What you're asking for is possible, and there are a bunch of script loader tools out there, but that's not the point. The script there sets up the button so that it is clickable in the first place.
If you really wanted to only load the script if someone clicks on the button, then you'd have to:
Add your own code first to detect a click, and stop the browser from redirecting to the default href
Then load the script, which will cause a noticeable delay for the user
Then invoke the correct behaviour on that script. I presume the script is external, and therefore probably isn't designed to work that way without some modification.
Sorry if this isn't the answer you're looking for, but unless that script is actually causing some delays, then really, don't bother :).
(You can, however, add that script to the end of your own JS file and load a single script at the end of your <body>).
A simple solution would be to generate the script tag when the use clicks the link and place in on the site. Can you use external libraries or frameworks such as jQuery to help you out?