Develop Reference

C# + WebBrowser + JavaScript - full load

Good time of day!:).Net 4.0, console. I need to write a parser page in console mode to get the code page in the form in which it is displayed to the user after download, without clicking the buttons, scrolling, and other events. I use this code, but it returns absolutely not what you need. In what ways can I get my desired result? I use other methods or components?
wb = new WebBrowser();
wb.Navigate(linkNorm);
wb.ScriptErrorsSuppressed = true;
wb.DocumentCompleted += new
WebBrowserDocumentCompletedEventHandler(w_DocumentCompleted);
while (wb.ReadyState != WebBrowserReadyState.Complete)
{
Application.DoEvents();
}
originalText = wb.DocumentText;
wb.Dispose();
void w_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
Trace.WriteLine(wb.DocumentText);
}
Change method to wb.Document.Body.OuterHtml - not help me. Result is so bad.
I think this is a pretty simple task, probably someone already solved.
Note - i need full HTML text in string-variable after work ALL JS

Xamarin form webview Eval not calling javascript

I use following code to load page to MyWebView:
private void DisplayLocalPage(string filename)
{
var html = new HtmlWebViewSource();
html.BaseUrl = DependencyService.Get<IBaseUrl>().Get();
html.Html = ReadFile(filename);
MyWebView.Source = html;
MyWebView.Eval("alert(200)");
}
Page is rendered well - all scripts run too, but this alert isn't fired, what are possible reasons?

Please try this:
webView.Navigated += (o, s) => {
webView.Eval("alert('text')");
};

I think I figured out why it might not work. The Eval is triggered before the WebView is fully initialized or web page is rendered, so the script is trigerred, but doesn't have any effect.

C# WebBrowser control - document does not contain html input control [duplicate]

Most of the answers I have read concerning this subject point to either the System.Windows.Forms.WebBrowser class or the COM interface mshtml.HTMLDocument from the Microsoft HTML Object Library assembly.
The WebBrowser class did not lead me anywhere. The following code fails to retrieve the HTML code as rendered by my web browser:
[STAThread]
public static void Main()
{
WebBrowser wb = new WebBrowser();
wb.Navigate("https://www.google.com/#q=where+am+i");
wb.DocumentCompleted += delegate(object sender, WebBrowserDocumentCompletedEventArgs e)
{
mshtml.IHTMLDocument2 doc = (mshtml.IHTMLDocument2)wb.Document.DomDocument;
foreach (IHTMLElement element in doc.all)
{
System.Diagnostics.Debug.WriteLine(element.outerHTML);
}
};
Form f = new Form();
f.Controls.Add(wb);
Application.Run(f);
}
The above is just an example. I'm not really interested in finding a workaround for figuring out the name of the town where I am located. I simply need to understand how to retrieve that kind of dynamically generated data programmatically.
(Call new System.Net.WebClient.DownloadString("https://www.google.com/#q=where+am+i"), save the resulting text somewhere, search for the name of the town where you are currently located, and let me know if you were able to find it.)
But yet when I access "https://www.google.com/#q=where+am+i" from my Web Browser (ie or firefox) I see the name of my town written on the web page. In Firefox, if I right click on the name of the town and select "Inspect Element (Q)" I clearly see the name of the town written in the HTML code which happens to look quite different from the raw HTML that is returned by WebClient.
After I got tired of playing System.Net.WebBrowser, I decided to give mshtml.HTMLDocument a shot, just to end up with the same useless raw HTML:
public static void Main()
{
mshtml.IHTMLDocument2 doc = (mshtml.IHTMLDocument2)new mshtml.HTMLDocument();
doc.write(new System.Net.WebClient().DownloadString("https://www.google.com/#q=where+am+i"));
foreach (IHTMLElement e in doc.all)
{
System.Diagnostics.Debug.WriteLine(e.outerHTML);
}
}
I suppose there must be an elegant way to obtain this kind of information. Right now all I can think of is add a WebBrowser control to a form, have it navigate to the URL in question, send the keys "CLRL, A", and copy whatever happens to be displayed on the page to the clipboard and attempt to parse it. That's horrible solution, though.

I'd like to contribute some code to Alexei's answer. A few points:
Strictly speaking, it may not always be possible to determine when the page has finished rendering with 100% probability. Some pages
are quite complex and use continuous AJAX updates. But we
can get quite close, by polling the page's current HTML snapshot for changes
and checking the WebBrowser.IsBusy property. That's what
LoadDynamicPage does below.
Some time-out logic has to be present on top of the above, in case the page rendering is never-ending (note CancellationTokenSource).
Async/await is a great tool for coding this, as it gives the linear
code flow to our asynchronous polling logic, which greatly simplifies it.
It's important to enable HTML5 rendering using Browser Feature
Control, as WebBrowser runs in IE7 emulation mode by default.
That's what SetFeatureBrowserEmulation does below.
This is a WinForms app, but the concept can be easily converted into a console app.
This logic works well on the URL you've specifically mentioned: https://www.google.com/#q=where+am+i.
using Microsoft.Win32;
using System;
using System.ComponentModel;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace WbFetchPage
{
public partial class MainForm : Form
{
public MainForm()
{
SetFeatureBrowserEmulation();
InitializeComponent();
this.Load += MainForm_Load;
}
// start the task
async void MainForm_Load(object sender, EventArgs e)
{
try
{
var cts = new CancellationTokenSource(10000); // cancel in 10s
var html = await LoadDynamicPage("https://www.google.com/#q=where+am+i", cts.Token);
MessageBox.Show(html.Substring(0, 1024) + "..." ); // it's too long!
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
// navigate and download
async Task<string> LoadDynamicPage(string url, CancellationToken token)
{
// navigate and await DocumentCompleted
var tcs = new TaskCompletionSource<bool>();
WebBrowserDocumentCompletedEventHandler handler = (s, arg) =>
tcs.TrySetResult(true);
using (token.Register(() => tcs.TrySetCanceled(), useSynchronizationContext: true))
{
this.webBrowser.DocumentCompleted += handler;
try
{
this.webBrowser.Navigate(url);
await tcs.Task; // wait for DocumentCompleted
}
finally
{
this.webBrowser.DocumentCompleted -= handler;
}
}
// get the root element
var documentElement = this.webBrowser.Document.GetElementsByTagName("html")[0];
// poll the current HTML for changes asynchronosly
var html = documentElement.OuterHtml;
while (true)
{
// wait asynchronously, this will throw if cancellation requested
await Task.Delay(500, token);
// continue polling if the WebBrowser is still busy
if (this.webBrowser.IsBusy)
continue;
var htmlNow = documentElement.OuterHtml;
if (html == htmlNow)
break; // no changes detected, end the poll loop
html = htmlNow;
}
// consider the page fully rendered
token.ThrowIfCancellationRequested();
return html;
}
// enable HTML5 (assuming we're running IE10+)
// more info: https://stackoverflow.com/a/18333982/1768303
static void SetFeatureBrowserEmulation()
{
if (LicenseManager.UsageMode != LicenseUsageMode.Runtime)
return;
var appName = System.IO.Path.GetFileName(System.Diagnostics.Process.GetCurrentProcess().MainModule.FileName);
Registry.SetValue(#"HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_BROWSER_EMULATION",
appName, 10000, RegistryValueKind.DWord);
}
}
}

Your web-browser code looks reasonable - wait for something, that grab current content. Unfortunately there is no official "I'm done executing JavaScript, feel free to steal content" notification from browser nor JavaScript.
Some sort of active wait (not Sleep but Timer) may be necessary and page-specific. Even if you use headless browser (i.e. PhantomJS) you'll have the same issue.

Get the navigating url in webview WinRt

I am injecting this JS to get the navigating url (destination) at run time. I am getting JS exceptions. What is wrong?
static String NAVIGATING_FUNCTION = "window.onbeforeunload = function(){ window.external.notify(' + location.href + ''); };";
webView.InvokeScript("eval", new String[] { NAVIGATING_FUNCTION });
Also please tell me how to cancel the navigation and return back to the previous page using this JS?

If I understand correctly, since I haven't used the WebView (yet), you want to report the current location before leaving and then cancel it?
You could try this:
window.onbeforeunload = function(){ window.external.notify(location.href); return false; };
I am not sure why you had quotes in there; location.href would be in that web view, and it would be passed as a string.
Also, take a look at the responses to this thread. One of them says:
Did you get your problem solved? In the Release Preview, you need to
add some code that looks like this:
List<Uri> allowedUris = new List<Uri>();
allowedUris.Add(e.Uri);
allowedUris.Add(new Uri("http://www.bing.com"));
Browser.AllowedScriptNotifyUris = allowedUris;
Another thing, this thread: Can I get the current HTML or Uri for the Metro WebView control?, talks about the LoadCompleted event handler.
See if that helps.

How to load a script into a XUL app after initial loading

Greetings,
my xul app needs to load scripts dynamically, for this I derived a function that works in regular html/js apps :
function loadScript(url)
{
var e = document.createElement("script");
e.src = url;
e.type="text/javascript";
document.getElementsByTagName("head")[0].appendChild(e);
}
to something that ought work in XUL :
function loadScript( url)
{
var e = document.createElement("script");
//I can tell from statically loaded scripts that these 2 are set thru attributes
e.setAttribute( 'type' , "application/javascript" ); //type is as per MDC docs
e.setAttribute( 'src' , url );
//XUL apps attach scripts to the window I can tell from firebug, there is no head
document.getElementsByTagName("window")[0].appendChild(e);
}
The script tags get properly added, the attributes look fine,but it does not work at all, no code inside these loaded scripts is executed or even parsed.
Can any one give a hint as to what might be going on ?
T.

Okay,
as usual whenever I post on stack overflow, the answer will come pretty soon thru one last desperate Google search.
This works :
//Check this for how the url should look like :
//https://developer.mozilla.org/en/mozIJSSubScriptLoader
function loadScript( url)
{
var loader = Components.classes["#mozilla.org/moz/jssubscript-loader;1"].getService(Components.interfaces.mozIJSSubScriptLoader);
//The magic happens here
loader.loadSubScript( url );
}
This will only load local files, which is what I need for my app.
I am fairly disappointed by Mozilla, why not do this the same way like html, in a standard way ?

I've tried this, and I think you're right - I can't seem to get XUL to run dynamically appended script tags - perhaps it's a bug.
I'm curious as to why you would want to though - I can't think of any situation where one would need to do this - perhaps whatever you're trying could be achieved another way. Why is it they need to be dynamically loaded?
Off-topic: on the changes you made to the script.
e.setAttribute('src',url); is valid in normal webpages as well, and is actually technically more "correct" than e.src=url; anyway (although longer and not well supported in old browsers).
Types application/javascript or application/ecmascript are supposed to work in normal webpages and are more "correct" than text/javascript, but IE doesn't support them so they're not normally used.

Inside xul environment you are only allowed to use XHR+eval like the following:
function loadScript (url) {
var xhr = new XMLHttpRequest();
xhr.open("GET", url, false); // sync
xhr.send(null);
if (xhr.status && xhr.status != 200)
throw xhr.statusText;
try {
eval(xhr.responseText, window);
} catch (x) {
throw new Error("ERROR in loadScript: Can't load script '" + url+ "'\nError message is:" + x.message);
}
};