I am trying to create an Android app for a Website, Which is not mine. But is a search engine for Restaurants. They have no API to work with. And i want to heedlessly browse their website and put the search query in the HTML Form and Click the Submit Button. And then Parse the Results and Use it with my Application Code. After doing loads of research here, i am finally asking for it. Question 1, Question 2, Question 3 and many more that i have looked so far. So all i know so far is if i want to do the same on Google.com i would write:
myWebView.getSettings().setJavaScriptEnabled(true);
myWebView.loadUrl("http://www.google.com/");
myWebView.setWebViewClient(new WebViewClient() {
#Override
public void onPageFinished(WebView view, String url) {
//Load HTML
myWebView.loadUrl("javascript:document.getElementById('q') =" + "StackOverFlow" + "; document.getElementByName('btnK').click();");
}
});
In the above code i am trying to put the search term "StackOverFlow" and Click the Search Button. But its not working. Kindly Help me out in this code or either point me in the right direction.
It's been a while, but for the sake of letting others know, webviews no longer use loadUrl to run Javascript. Try using evaluateJavascript.
Since you've also mentioned headless browsing, I would recommend overriding shouldInterceptRequest in your client to redirect all unnecessary files (such as css, images, and perhaps js depending on the site) to a blank inputstream
myWebView.loadUrl("http://www.google.com/");
after overriding onPageFinished method not before
Related
I'm currently trying to get the source of some page from android.
From pre-checking the source, it contains javascript in it.
In order to be able to parse it correctly by using jsoup, I had to do the following steps:
Load the url into a webView.
use jsoup.parse() on the webView to get the source with the javascript in it.
wb_result.setVisibility( View.GONE );
wb_result.getSettings().setSaveFormData( false );
wb_result.getSettings().setBlockNetworkLoads( true );
wb_result.addJavascriptInterface( new MyJavaScriptInterface( this ), "HtmlViewer" );
wb_result.setWebViewClient( new WebViewClient() {
#Override
public void onPageFinished(WebView view, String url) {
wb_result.loadUrl( "javascript:window.HtmlViewer.showHTML" +
"('<html>'+document.getElementsByTagName('html')[0].innerHTML+'</html>');" );
}
} );
In my opinion, it is a little weird to have a webView in my activity and keep its visibility GONE and only to use it as a middle step in order to get to the source I need + it is slow.
I was wondering what is the fastest solution to obtain a source that contains javascript.
I read about Chrome Custom Tabs and that it should be faster but then I can't hide the opened tab from what I saw and it will disturb the flow of the app.
Specifically, the url im trying to get its source is - link.
Any modern ideas? all the solutions I saw are from 2016.
Thank you
Me an my team are currently working on a software project at university and my present task is to bind our desktop javafx application with Facebook.
Basically I have an fxml method in a controller that is called when the user hits a "Share" button in my GUI. In the method I'd like to simply open up my .html file using a WebView:
#FXML
public void shareFacebookClicked() throws Exception{
// Setting up the webview
WebView webView = new WebView();
final WebEngine webEngine = webView.getEngine();
webEngine.setJavaScriptEnabled(true);
// Read the html file and let the web engine load it.
File file = new File(getClass().getClassLoader().getResource("facebook.html").toURI().getPath());
webEngine.load(file.toURI().toURL().toString());
Stage stage = new Stage();
stage.initOwner(this.stage);
stage.setScene(new Scene(webView, 1000, 800));
stage.show();
}
There is no problem with it, my "facebook.html" file is loaded and displayed correctly (well, almost correctly) in a web view.
The actual problem is that I'm constantly getting the 191 Facebook error saying that the link is not owned by the application. Since there are tons of posts and questions on this around the Internet (and yes I checked and read all of them) here are the things that I'm already aware of:
I registered my application on the Facebook Developer site. I know about the AppID and Secret
I know that this error mainly comes from the fact that people forget to set their website URL and domain in the Settings. The problem is that I don't have a website. I just have a simple .html file which I'd like to use in a web view inside of javafx. However, I tried all possible combinations advised on stackoverflow, facebook help centre and other forums which include: Setting website URL to http://localhost/, domain to localhost, enabling Embedded browser OAuth Login, setting the redirect URI to localhost too, etc.
I assume that my goal could be achieved by using RESTfb, Facebook4j or Graph API. When I tried those I had to stop because I faced problems with the user authentication plus I thought this current option would be the easiest way (considering this feature has LOW-priority in our software).
None of this solved my problem therefore I've given up researching the answer and decided to post my very own personal question.
In my opinion there must be some error in the .html file and/or I completely misunderstand something in the way this works. The .html file:
<html>
<head>
<title> Share on Facebook </title>
<script src="http://ajax.aspnetcdn.com/ajax/jquery/jquery-1.5.1.js"></script>
<script src="https://connect.facebook.net/en_US/all.js"></script>
<script type="text/javascript">
$(document).ready(function () {
$('#shareonfacebook').click(function (e) {
FB.ui({
appId: 'MY_APP_ID',
display: "popup",
method: "feed",
name: "Test",
link:"",
caption:"Test",
description: "Test",
});
});
});
</script>
</head>
<body>
<div id="fb-root"></div>
<button id="shareonfacebook" >Share</button>
<script>
FB.init({
appId : 'MY_APP_ID'
});
</script>
</body>
</html>
Partially I have this code from a tutorial site. Theoretically it should work. All I want is a dialog to come up where the user can publish the results of the workout he/she completed using our software. Currently when the .html file is opened up there is a simple button to click. This and all the "Test" strings inside of the javascript are only for testing. I just want to achieve that I can post something on my wall. The next step would be of course to somehow set the posting text dynamically etc.
Please tell me what I'm doing wrong or how I should approach the whole thing. Like I said, the task is minimal therefore it shouldn't be that difficult but I've been sitting in front of my laptop for 2 days without any success. I'm ready to post more code or give more information if it's needed.
Thank you for the help in advance!
I want to run a function to open the link in the android browser system. Here is an illustration of my questions. See the illustration What should I do? I only use javascript not java. Please help
Some time ago I encountered the same problem. To do this I modified the InAppBrowser source code.
you should override the shouldOverrideUrlLoading method in the InAppBrowserClient class found in InAppBrowser.java
This will allow you to hook in to the request before the url is being loaded and choose an alternate behavior. In your case loading the URL in the system browser.
Your code will look something like this:
#Override
public boolean shouldOverrideUrlLoading (WebView view, String url){
if(url.equals("Your URL to be loaded")){
openExternal(url);
return true;
}
return false;
}
I want to get the xpath of a facebook post using HtmlUnit. You can refer these two questions to get more ideas on what I want to do:
Supernatural behaviour with a facebook page
HtmlUnit commenting out lines of facebook page
To simulate what I did, you can follow q-1. The pastebin link of HTML code(of facebook page) is http://pastebin.com/MfXsYSJQ.
Or simply you can go to https://www.facebook.com/bhramakarserver .
I just want to get the xpath of the span containing the post with text:"Hi! this is the first post of this page."
What I tried was this:
public class ForStackOverflow {
public static void main(String[] args) throws IOException {
WebClient client=new WebClient(BrowserVersion.FIREFOX_17);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setRedirectEnabled(true);
client.getOptions().setThrowExceptionOnScriptError(true);
client.getOptions().setCssEnabled(true);
client.getOptions().setUseInsecureSSL(true);
client.getOptions().setThrowExceptionOnFailingStatusCode(false);
client.setAjaxController(new NicelyResynchronizingAjaxController());
HtmlPage page1=client.getPage("https://www.facebook.com/bhramakarserver");
System.out.println(page1.asXml());
//getting the xpath of span of class="userContent"
HtmlInput input=(HtmlInput)page1.getByXPath("/html/body//input[#type='submit']").get(0);
System.out.println(input.asXml());
//This line gives error as the xpath evaluates to null
HtmlSpan span=(HtmlSpan)page1.getByXPath("/html/body//span[#class='userContent']").get(0);
}
}
The problem which seems is that the page1 has the static html. In this, the span element:
<span data-ft="{"tn":"K"}" class="userContent">Hi! this is the first post of this page.</span>
is generated dynamically. So it appears as commented in html of page1.But on inspection via inspect element, it appears as normal. Hence its dynamically uncommented.Is there no way that I can get page1's html to be in the state after all its dynamic contents have been loaded so that I may get the xpath correctly? Can it be done using selenium web-driver?
Given that information, it seems fair to assume that some AJAX call is not being fired or that you're not properly waiting for the AJAX to execute. I haven't gotten the best results using that AJAX controller. Sadly, a loop is usually the best way to go.
I've explained how to do that in this question: Get the changed HTML content after it's updated by Javascript? (htmlunit)
If this doesn't do the trick, then probably you're getting a JavaScript exception. I've written some possible workarounds to that situation in this other question: How to overcome an HTMLUnit ScriptException?
If none of these work... then I'd recommend using something else rather than HTMLUnit. Any real browser drive would do the trick. Or maybe using some other alternative such as PhantomJS or ZombieJS.
I am working on taking readings about web browser performance and so need to access the window.performance object of the browser.
To collect this data i have written a javascript file, collect.js which i need to add to the DOM of the page that i need to test eg. www.google.com, www.facebook.com and so on...
Also i need to run this test for about 1000 websites, any manual approach is out of the question. I need it to be automated somehow.
How could i go about doing this?
EDIT: I need to run these tests on an android browser, so i need mobile oriented solutions.
You can create a simple android app with a WebView component. This way you can control which URLs are loaded and also insert your JS code.
http://developer.android.com/guide/tutorials/views/hello-webview.html
EDIT
You can run any javascript like this:
Implement a custom WebView:
public class WebClient extends WebViewClient {
#Override
public boolean shouldOverrideUrlLoading(WebView view, String url) {
view.loadUrl(url);
return true;
}
#Override
public void onPageFinished(WebView view, String url) {
// Execute your javascript below
view.loadUrl("javascript:...");
}
}
If you are looking for an automated solution, try PhantomJs this provides an automated headless web browser. Also has access to network traffic
perhaps you can try "bookmarklet"
http://www.bookmarklets.com/
the advantage over greasemonkey script is that it can run on
firefox and explorer