Login to a website and scraping its js generated html in android - javascript

I am really new to android and I am building an app that will take the username and password of a website(https://lms.mrt.ac.lk).
When the user hits the login button of my android app, the app should itself logged in to the above website and retrieve the complete(including js generated html) html source code.
As a start I tried to print the html source code in the android studio. I tried to use HtmlUnit, but later I found that it is not supported by android. Then, I tried to use Selenium on android, but when tried to build the project it gave me an error "More than one file was found with OS independent path 'META INF' ".
And then tries android's WebView to do the thing, as described in this question, but I don't know how to view the complete html output.
So, my question is -
How to login to a website inside an android app
How to get the complete html source code of the logged in website in android(scraping the js)
I want to do the above in same app. I searched everywhere and still could not find anything complete, that helps a beginner.
Please help me! your suggestions/solutions are highly appreciated.

I'll answer your questions in order.
I don't know what javascript android framework you are using, but the solution should be similar to the java solution. For your specific website, the login endpoint is https://lms.mrt.ac.lk/login.php. You should send a POST request to the endpoint with header parameters LearnOrgUsername and LearnOrgPassword. On success, this should return your desired LoginSessionCookies.
Instead of trying to interpret the javascript stuff, you should rather try to understand how the javascript code retrieves the information you need. And then directly access your information by a web request.

Related

How can i code Zapier to follow a link that was provided in an email?

I am using Mailparser to scrape a booking email, then use Zapier to create a booking in our system.
In the email is a link to confirm the booking so am wanting to use the code action to click on the link or follow the link.
Nothing needs to be seen, just follow the link & allow the page to load. This confirms the reservation
I have attempted using python & JS to no avail.
Can you point me in the right direction?
Thanks for reading!
David here, from the Zapier Platform team.
When you click on a link and it opens in your browser, all that really is is a GET request. If the site you're trying to load is simple, then this can be accomplished with a Webhook by Zapier step that make the request to the provided URL.
The big issue that'll pop up is if the site is more complex. A browser loads and execute all the javascript on the page, which our GET request won't do. If the site uses javascript to confirm the reservation (not just the page loading itself), then this isn't going to work. If that's the case, it's a lot more involved. You'd probably need to make a CLI app and use a tool like PhantomJS that can load pages and everything in them. Note that I haven't tested that on the CLI and there's a chance it's incompatible with the way we run apps, so do a proof of concept before you really dive in.
Hope this helps. ​Let me know if you've got any other questions!

Google Analytics: Do I need URL address?

I am a complete Google Analytics beginner and would appreciate a help with a basic question.
I am developing HTML, CSS and JavaScript based applications which are further uploaded into an iOS application to present your applications in a fancy way. Therefore my application is a hybrid application (half JS web site, half mobile app).
I would love to see users' activity in my app when they are browsing through it and I thought GA might work well with it - but the problem is, that the outer app doesn't provide me with any URL of my inner JS app (the inner web site's URL is file:///).
At this page (link), I found that URL is not really important, that it is the tracking code which is important. So I used a dummy URL, added the GA snippet into my application and uploaded it in iPresent. I can't see no live activity though... :/ It also says the measuring is not installed (not used at a home page).
So I am wondering - is the URL really important?
Any ideas?
Thanks!
URL (or page path) is only important if you want to report on data based on which URLs your visitors went to.
If you app doesn't use URLs at all, perhaps it fits better with the "app" model where you are sending screen name data instead of page data. You can read more about the differences between web and app views here:
https://support.google.com/analytics/answer/2649553
I found out that URL is not needed. This type of problem can be solved by using GA Measurement Tool
https://developers.google.com/analytics/devguides/collection/protocol/v1/
Validate your hit here:
https://ga-dev-tools.appspot.com/hit-builder/

Issue with editing of Path authentication page

So I have next issue. My app export some images to Path(some social network, as I think), and I use Amazon appstore for sharing my app. But the authentication page of Path, which I fetch in webview has a link to Apple Store and Google Play Market, that's very bad, because of amazon policy disallows such links and I can't publish my app. Here is example of such authentication page - https://partner.path.com/oauth2/authenticate?response_type=code&client_id=1b81bf92642176a90222d90c5ca5c2c6937f4dd7.
First of all, I tried to find some way to edit downloaded page - use javascript to get code of page, delete a link and fetch it back to webview. But after such manipulations webview stop to respond on clicks and I can't log in. I done smth like in this post - Is it possible to get the HTML code from WebView.
So I ask your help guys in next issue - I need working variant of authentication page of Path without any links to any markets.
everybody! It's not possible to edit the page in way it will working correctly after it. So I was forced to delete this function from my app.
Also I have written an email to Path team - they just answered that they don't going to change their authentication page in near future.

How can I hide my App tag on a post I created on facebook

I've created a facebook app from which anyone can post to his wall or his friend's wall. But at that post at the bottom the post it displays the app hyperlink. But want to remove it so that it should look like the user create it himself and not by my app.
Is there any way to do so. I use faecbook graph api with Javascript sdk.
Please help me out.
First: Posting to the wall of a friend is highly spammy and should be avoided as much as possible. With the PHP SDK it´s not even possible anymore afaik, or will be removed very soon because it is deprecated. The only way is to use the FB.ui feed dialog of the JavaScript SDK:
https://developers.facebook.com/docs/reference/dialogs/feed/
That being said, there is no way to remove the App Link. Postings made through an App always have to be easily identifiable. It may be possible that posting via the PHP SDK does not show the App Link, but as i said: that function is deprecated for spam reasons and should not be used anymore.

Creating a testing script which 'skips' file uploads

I am creating a testing script for my team. So far it works fine - it goes to the login form and attempts to login using the details you entered, if that fails it goes to a job description and attempts to apply for a job, selecting a random answer for each screening question until it gets to the actual application form. Unfortunately the application form includes a file upload control, which I don't appear to be able to skip over. Does anyone have any idea if I could actually skip over it or somehow click it? I am using FF7 and from previous posts I can see that apparently FF4 does it - but FF7 doesn't appear to make file uploads clickable through JS.... any ideas? Thanks in advance.
Regards,
Richard
Javascript can't access file upload inputs for security reasons (e.g. prevent malicious upload of user's files). For this kind of testing you should use a browser automation tool like Selenium http://seleniumhq.org/ (or in web service form https://saucelabs.com/).

Categories

Resources