I am looking for a way to, give a URL, get the source of a webpage back after the JavaScript has been run on it. For example:
I have a webpage with a .
On loading the page, some JavaScript populates the div.
Viewing the source of the page through a browser will not give the information which is within the div.
As far as I know, in order for the browser to render the page the div must have been filled with (X|D)HTML which would mean that the source of the page after being rendered is still just nested markup, so theoretically there should be a "final" version of the page source.
I have considered using a rendering engine like WebKit or Gecko and somehow adapting these to do this, however this is a fairly large task and I don't really want to duplicate something which has already been done. Does anyone know of a way of performing this task.
Regards.
Update: I am aiming to use Selenium (as mentioned in the comments to the accepted answer) to do this automatically for several pages. My project is a web spider which by design needs to target a number of pages in which the content I am aiming to reach is not available until after the JavaScript has populated everything.
Such addons for Firefox as the WebDev toolbar, or Firebug have options like 'View generated source'.
As far as timing it goes, just about the only option you have is to have a snippet of javascript code. You could set a start-time as soon as is possible on the page-load, and check again when the page is completed (either for dom-ready or page completely downloaded). It's going to be highly variable however, and if you are trying to time it in order to improve the speed (which is good to know, and to do) - just getting Firebug + Yslow would be far more useful.
Within Firefox you can get the final rendered DIV by waiting the browser to finish rendering, then pressing ctrl-A to select all content on the page and finally selecting "Show selection source" from the right-click menu.
This shows you the manipulated/populated DOM-code of the page.
Related
Is there an application that allows me to select a section of a web page, and then outputs all js used there? I've been told I can do this with Chrome Inspector, but haven't had any success so far.
Example:
On this page - http://preview.oklerthemes.com/porto/2.7.0/page-left-sidebar.html - there is a tabbed box in the sidebar. I want to easily grab all the JS/CSS needed for that box. I usually use Inspector to look at all the styles, and go and grab theme from each CSS file, but I don't know how to do this for the JS.
It's not quite clear from your question what you're asking.
Are you trying to see what JS causes writes or changes to a particular part of a web page? The easiest way would be to open the page with the element inspector, right-click a particular chunk of HTML and stick a breakpoint on modifications.
The next time a function causes any changes, the breakpoint will trigger and you'll be able to crawl up the call stack to see what the cause was.
I have two problems with the same php/ajax solution. The first is a weird problem. Or maybe I'm just missing something.
i have a php page that displays data from a database.
so let's say when it loads the first time, "myavalue" = 1.
On the same page i have a refresh button that triggers an ajax function to get the latest data from the database. If I change the value in the database from 1 to 2, and then use this button, the page does display 2. however, when i do a view source on the page, it shows the value 1 instead of 2.
Why is this? i've tried it both in IE 9 and also in fire fox.
The second issue may be related... the problem is with the same page that I mentioned above. When the refresh button is clicked, i'm actually generating html in my controller and then having my view display it. what i'm noticing is that in the view source, the html is missing a class name that i assign to my table cells. However, in my console, i can see that the html that the controller is generating does infact include the proper class names.
Here's what the console is showing:
<tr><td>10.14.193.0/27</td><td>Vancouver Island</td><td class='deadclient'>2012-06-13 12:12:12</td></tr>
And here's what the view source is showing:
<tr>
<td>10.14.193.0/27</td>
<td>Vancouver Island</td>
<td>2012-06-13 12:12:12</td>
</tr>
The view source does not show the effects of JavaScript on the original source code. You'll need to view the altered source code through a tool like Firebug or chrome's dev tools to see the effects of any JavaScript you are using.
"View Source" is better worded as "View Original Source" (and I believe it is in some browsers). It is the source of the page before JavaScript had any say on the matter.
To view real-time source, use whatever Developer Tools your browser has. IE has the F12 Tools, Chrome has the Inspector, and Firefox recently added the Web Developer.
The point is that there is a difference between "View Source" and what is actually on the page. It's like trying to look at the source of a stream by looking at where it meets the ocean.
The view-source option of Firefox will do another request to the page to get the HTML response. To confirm what I'm saying, you can try a page with a form, it'll ask you if you want to re-send the data twice.
What you want is something like Chrome built-in developper tool, you will see the source in real-time, as it is modified by Javascript and even make change to the markup and see it live.
Have fun, your problem is not a real problem :]
Is there any reason to NOT have a webpage retrieve it's main content on the fly?
For example, I have a page that has a header and a footer, and in the middle of this page is an empty div. When you click on one of the buttons in the header, an http GET is done behind the scenes and the .innerHTML() of the empty div is replaced with the result.
I can't think of any reason why this might be a bad idea, but I can't seem to find any pages out there that do it? Please advise!
It's not unheard of, but there are issues.
The obvious one is that some users have javascript turned off for security reasons, and they will not be able to use your site at all.
It can also negatively impact handicapped users that are using assistive technology such as a screen reader.
It can make it harder for the browser to effectively cache your static content, slowing down the browsing experience.
It can make it harder for search engines to index your content.
It can cause the back and forward buttons to stop working unless to take special steps to make them work.
It's also fairly annoying to debug problems, although certainly not impossible if you use a tool such as Firebug.
I wouldn't use it for static content (a plain web page) but it's certainly a reasonable approach for content that is dynamically updated anyway.
Without extra work on your part it kills the back and forward history buttons, and it makes it difficult to link to the pages each button loads. You'd have to implement some sort of URL changing mechanism, for example by encoding the last clicked page in the URL's hash (e.g. when you click a button you redirect to #page-2 or whatever).
It also makes your site inaccessible to users with JavaScript disabled. One of the principles of good web design is "graceful degradation"--enhancing your site with advanced features like JavaScript or Flash or CSS but still working if they are disabled.
Two considerations: Search engine optimization (SEO) and bookmarks.
Is there a direct URL to access your header links? If so, you're (almost) fine. For example, the following code is both SEO friendly and populates your page as you desire:
Header Link
The catch occurs when people attempt to bookmark the page they've loaded via JavaScript... it won't happen. You can throw most of those potential tweets, email referrals, and front page Digg/Reddit articles out the window. The average user won't know how to link to your content.
Where did you read it is a bad idea? It purely depends on requirements whether or not content will be populated on-the-fly. In most cases, however, the content is loaded along with the page not on-the-fly but if you need your content on-the-fly, it shouldn't be a bad idea.
If your content is loaded via javascript and javascript is disabled on users' browser then definitely it is a bad idea.
I cant think of a bad reason for this either (other than possibly SEO), one thing that would probably be a good idea is to load the data only once. ie
Show Div1 - do ajax/whatever only if the innerhtml is blank
Show Div2 - do ajax/whatever only if the innerhtml is blank
<div1></div>
<div2></div2>
This should keep the server load down so the divs content is only loaded once.
Cheers
This is pretty standard behavior in ajax enabled sites.
Keep in mind however that extra effort will be needed to:
ensure the back button works
link to (and bookmark) specific content
support browsers with javascript disabled.
I want to have a popup div with iframe content.
Can search engines read this when I'm using jquery to create it?
Alternatively, is there a way to detect a search engine on the server side and remove the option of this popup?
The best way would be to degrade gracefully, e.g. by using a standard
<a id='mylink' href='xyz.html'>
link that points to the resource that is opened in the popup.
You would then add the JQuery code to the link, causing it to open in the pop-up.
That way, even users that do not have JavaScript turned on can access your popup.
Most Lightbox clones like Thickbox work that way.
Generally speaking, search engines do not execute JavaScript, so there's no way for them to index anything contained in a popup div.
You can, however, inspect User-Agent header to see if the page is requested by web spider, but this is something that's not considered best practice.
Search engines do not play well with javascript but you can see how google bot would fetch your page using Google's webmaster tools.
Rendering a different page to bots is not considered a best practice too. The best you can do is graceful degradation.
Just to say it at once: DON'T use the iframe, as it is bad-ass technology and search engines won't index the iframe'd page.
First question:
No, it cannot, if the data are loaded simultaneously with the popup "popping up" (as search engines, as said already, generally doesn't execute javascript). If the data are already loaded, and the popup div are somehow hidden at page load, the search engine will index the content.
Second question:
Don't do that. That's called cloaking, and will be punished by search engines if detected - they don't like content customized just for them, and then you're back at scratch.
Hi.
I'm new to Java/AJAX etc.
I have a page with links down the left and a DIV on the right.
I want content (other pages) to load in the DIV when users click on the links on the left... beginner AJAX stuff I guess.
I played around with a few JQery plugins and found one that allows pages to load with a fade effect, which is perfect. I have a problem though:
The plugin works fine when I click links on the parent page, but when I click links in one of the loaded pages, after one link deep, it breaks out of the div and replaces my parent page. (This issue was described on the plugin page, supposedly solved, but is still cropping up on my page). I suspect it has something to do with the "bind" variable.
I've uploaded a stripped down example of my site here:
This is the plugin website: www.thecreativeoutfit.com/index.php?view=Simple-AJAX-Content-Changer-with-EZJax (Because I'm a new user I can't add any more links, sorry for the long-hand).
For those who are willing to look at my site or the plugin, I'd appreciate your insight.
If that's a hassle maybe you could recommend a similar simple ajax plugin that allows the loading of content with a fade-in effect, but also allows links within the loaded content to stay contained within the original div.
Many thanks!
Max
I was going to post a comment but it got too long so, what the heck..
Your website worked just fine for me (except for the pages that were not available) in Firefox running on Windows XP.
However, I would strongly recommend against that type of design - it will be a pain for you to maintain in the long run and it is generally considered bad design because it is against the functioning principles of the web: different pages of your website should be represented by distinct URLs which users of your site could use to link to. It also breaks browser back button functionality which is a big usability issue (at least for me).
Plus, it will not be SEO friendly - which means that search engines like Google won't think highly of your website - which means that you won't show up in searches.
You nested pages are breaking because the JavaScript click events are not being reattached to the paging controls after the first page causing the normal href attribute to be used.