Opening web page as I see it in browser with Python - javascript

Using the google CSE utilities I've made a HTML file that is a barebones version of Google Search (about 1/10th page size), which is as follows:
<script>
(function() {
var cx = '011947631902407852034:gq02yx0e1mq';
var gcse = document.createElement('script');
gcse.type = 'text/javascript';
gcse.async = true;
gcse.src = 'https://cse.google.com/cse.js?cx=' + cx;
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(gcse, s);
})();
</script>
<gcse:searchresults-only></gcse:searchresults-only>
I can open this in my browser like C:\Users\Me\Documents\MyWebpage.html?q=MyQuery and it shows something like this:
This is obviously not what's in the HTML file, so I assume it's generated via JavaScript. Indeed, when I look at the page in my browser there's a lot more HTML than is in the actual file. The 1/10th size comment refers to the size of all of this, if I download it using Inspect Element, selecting the topmost node, Edit HTML, and copying all the HTML and placing it in a file.
My question is: how can I read in all of this HTML for a given query with Python?

Related

Javascript (Remote include) : HTML Elements creation on-the-fly? [duplicate]

This question already has answers here:
Execute write on doc: It isn't possible to write into a document from an asynchronously-loaded external script unless it is explicitly opened.
(4 answers)
Closed 6 years ago.
I'm trying to make a floating HTML button, along with the interaction on it (like: Popping up a form when clicked) by letting the people just EMBED some Javascript inside their Page.
So it's like, my HTML Elements should be generated on top of their page (at different domain) remotely.
For example, i have seen something like Uservoice Feedback Button and Form, it looks like:
We can have that Button generated on our site. The only thing we need to do is to embed their Javascript in our page:
<!-- USERVOICE -->
<script type="text/rocketscript">
var uvOptions = {};
(function() {
var uv = document.createElement('script'); uv.type = 'text/javascript'; uv.async = true;
uv.src = ('https:' == document.location.protocol ? 'https://' : 'http://') + 'widget.uservoice.com/xxxxxxxxxxxxxxxxxxxxx.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(uv, s);
})();
</script>
I tried this
Then in order to achieve similar thing, i tried with my own 2 domains. On my Webpage at www.domain-a.com, i embed:
<script type="text/javascript">
(function() {
var myButtonJS = document.createElement('script'); myButtonJS.type = 'text/javascript'; myButtonJS.setAttribute('defer', 'defer');
myButtonJS.src = ('https:' == document.location.protocol ? 'https://' : 'http://') + 'www.domain-b.com/myButton.js?3';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(myButtonJS, s);
})();
</script>
On the www.domain-b.com, inside the myButton.js:
document.write('<div style="z-index:99999999;padding:5px 10px 2px 10px;border:1px solid #000000;position: fixed;bottom: 10px;right: 10px;_position:absolute;_top: expression(offsetParent.scrollTop+document.documentElement.offsetHeight-60);direction:ltr;background:#7fff00;">Click me!</div>');
But when i run, http://www.domain-a.com, the browser says:
Failed to execute 'write' on 'Document': It isn't possible to write into a document from an asynchronously-loaded external script unless it is explicitly opened.
So please kindly let me know:
What are the standard (proper) ways to achieve such ones?
(In other words) how do i properly generate the HTML elements on top of remote sites, when they include the Javascript i provided to them?
Thanks all :)
IMHO, the better way to do it is create the element on the fly from your externa js then load the script in the body of the html, that way you are sure that the document is loaded and then just hook in the body of document e.g:
document.getElementsByTagName('body')[0]
in your external script. then append your element there. :)
You can create A DOM element, use innerHTML to set its contents ant then attach to the DOM when the content is ready:
var myElement = document.createElement('div');
myElement.innerHTML = '<div style="z-index:99999999;padding:5px 10px 2px 10px;border:1px solid #000000;position: fixed;bottom: 10px;right: 10px;_position:absolute;_top: expression(offsetParent.scrollTop+document.documentElement.offsetHeight-60);direction:ltr;background:#7fff00;">Click me!</div>';
document.addEventListener("DOMContentLoaded", function() {
document.body.appendChild(myElement);
});

Google site search

I need to implement google site search I am using the below code
<script>
(function() {
var cx = '012847953619635190580:vrz-2wloub8';
var gcse = document.createElement('script');
gcse.type = 'text/javascript';
gcse.async = true;
gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
'//www.google.com/cse/cse.js?cx=' + cx;
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(gcse, s);
})();
</script>
<gcse:searchbox></gcse:searchbox>
Problem is shows a warning saying "unknown tag".
Pls help me out
I've never had that issue before, but I use Google Site Search and can make some educated guesses.
Place that script tag at the end of the 'head' section of the enclosing file. It should be there ideally.
This is your best friend: https://developers.google.com/custom-search/docs/element
That page shows many different ways to implement Google Custom Search, and even gives you examples that you can toy with.
And on a side note, do you have a corresponding section somewhere?
To quote that page: "two-column and A two-column layout with search results on one side and a search box on the other. If you plan to insert multiple elements in two-column mode in your webpage, you can use the gname attribute to pair a search box with a block of search results."
Basically, if you are using a 'searchbox' section, you will also need a 'searchresults' section.

reduce page weight of google plus button

I am currently developing a website and am wanting to include the google plus button. My problem however is that the google plus button adds an extra 300kb to the page weight. A lot of this comes from one single file that the http://apis.google.com/js/plusone.js file calls.
I've managed to keep the page load time down to a minimum by loading it asynchronously but I would like to know if there's any way to adapt the code to minimise the page weight?
Here is the code I am using...
<script>
window.___gcfg = {lang: 'en-GB'};
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
</script>
<div id="google-plusone">
<g:plusone annotation="none" size="medium"></g:plusone>
</div>
You shouldn't really worry about it as it doesn't affect your code startup time. Most users will most likely just have it in cache already and load it from there. Trying to provide your own somehow "minimized" copy is actually counter-productive, because it won't profit from being cached from visiting other sites.

Executing a javascript that is injected as a script element via bookmarklet?

I am using bookmarklet to inject a element in document with a custom JS script file. I did it like this:
var newscript = document.createElement('script');
newscript.type = 'text/javascript';
newscript.async = true;
newscript.src = 'http://www.myurl.com/my_js_script.js';
var oldscript = document.getElementsByTagName('script')[0];
oldscript.parentNode.insertBefore(newscript, oldscript);
But I can't figure out how to actually execute it. Can someone tell me how can I execute that JS file?
Note: Since this can be a Greasemonkey script as well, I am tagging this question for Greasemonkey as well.
Script tags are automatically downloaded and executed when they're added to the document. Note, however, that the script you're using may fail if the document you're injecting into doesn't already contain any <script> tags, as oldscript will be undefined.

JavaScript is blocking images from loading

I'm using the following code to allow parallel JavaScript downloading on my website
var head = document.getElementsByTagName("head")[0];
var sTag1 = document.createElement("script");
sTag1.type = sTag1.type = "text/javascript";
sTag1.src = "http://example.com/one.js";
var sTag2 = document.createElement("script");
sTag2.type = sTag2.type = "text/javascript";
sTag2.src = "http://example.com/two.js";
var sTag1 = document.createElement("script");
sTag3.type = sTag3.type = "text/javascript";
sTag3.src = "http://example.com/three.js";
head.appendChild(sTag1);
head.appendChild(sTag2);
head.appendChild(sTag3);
However, using YSlow, it shows that even though one.js, two.js and three.js are downloading in parallel - images are not loading until the last JavaScript is fully downloaded.
What can I do to allow images to not be blocked from loaded due to my JavaScript files downloading.
Load your Javascript files right above the </body> tag.
Where are you triggering that code from? Because you could wait to execute your quoted code until you see the window.load event, e.g.:
<script type='text/javascript'>
function loadMyScripts() {
/* ...your loading code here...*/
}
window.onload = loadMyScripts; // Or use addEventListener/attachEvent to do it
</script>
The window.load event isn't fired until all of the images are loaded, so you'll be sure the scripts aren't getting in the way. Of course, it also leaves quite a large margin of time for the user to start doing things with the page, so you need to be sure the page doesn't need that JavaScript to be functional.

Categories

Resources