I'm need to automatically perform the login at: http://gsc.klub-modul.dk/. After I am logged in, I need to able to fetch pages from this domain, which requires me to be logged in. As far as I have gathered, the site is using ASP.NET to interact with the server. I am open to all suggestions. I'm looking for a script to do this, as I am still new to this.
What you are trying to do is botting (Automated access of site or online game),so follow these steps,
Generic steps
Monitor the HTTP request being made by the browser (I use
Fiddler,you might also find browser built-in network tool(press F12) handy)
You need to look for cookies especially.
In short you should be able to make same HTTPRequest on each event
(by event I mean first time page request ,page submit for login and
etc)
Use python for botting
use these libraries Mecanize / twill /scrapy / Beautiful Soup
Tutorials to get started
http://docs.python-guide.org/en/latest/scenarios/scrape/
http://www.pythonforbeginners.com/python-on-the-web/web-scraping-with-beautifulsoup/
http://arunrocks.com/easy-practical-web-scraping-in-python/
https://classic.scraperwiki.com/docs/python/python_intro_tutorial/
Stackoverflow links
Scraping sites that require login with Python
Scrape a web page that requires they give you a session cookie first
Fake a cookie to scrape a site in python
Python Scraping Web with Session Cookie
My Advice
I think if you haven't made a web scraper (web-bot or bot-tool) before than you should start with small things like fetching a specific information from a page which doesn't require login then move on to more complex scenarios
As usual Asp.Net uses cookies for session, authorization needs. So basically you just need to make POST request to login page with Login, Password parameters, then you need to obtain cookies from response, and then make new requests for needed pages with these cookies.
By default these cookies are named ASP.NET_SessionId and ASPXAUTH
I have worked on a similar problem but used a generic approach. I used greasemonkey addon in firefox to login on a game and save data on different db. Later I found out that I could've wrote a addon myself (mozilla ref)
Both of these technique will rely on javascript only.(example of a login script of fb in greasemonkey)
The website was on asp.net too! You just need to receive the cookies and login from javascript code. Jsoup is another approach to parse data and login.
Related
I am developing a web application using Facelets and Entity-Controller-EJB structure. In the application, there are contents which are reachable only if you are logged. A bean checks the login state every time you click on a button/link for the restricted contents, and redirects you either to the selected page or to the login page.
I thought that this way is not safe, as you can write the link directly in the browser instead of generating it from a button that checks the bean. So what should I do? Is there a render option embeddable in each page or should I write a javascript function? In this case, what should I do? I have studied js fundamentals but don't really know how to implement this control!
Thank you for reading!
You cannot rely only on frontend to deny access to some parts of a web application.
This because all the HTML/CSS/Javascript is downloaded on users' browser, so they can read your code and your authentication mechanism, and understand how to bypass it (or just disable it).
More on this: Why are browsers allowed to display client-side source code?
What you need is implementing some security mechanism in the backed.
The simplest one is to delegate this to your webserver (here the instructions for Apache) and then use something similar to this to do login.
Another way is to have a proper backend: you send data to it (email/password) and it provides you a token that you use to access protected resources.
Or also, dinamically create your documents on server side, only if the user is authenticated.
I am trying to do a login like Google does, which includes a cross-domain-authentication. When loggin into google.com you are automaticly logged in for different domains like youtube.com. For me it looks like:
you enter credentials at accounts.google.com
Google sends a request via AJAX to check them
if they are correct, it 1. sets a cookie and 2. does requests to the other domains like youtube and sends some identifiers (SID or token?) and sets the cookies for those domains as well
after all other domains also have the cookie in place Google redirects you back where you came from
I was researching a lot and it seems like easyXDM (http://easyxdm.net/wp/) is a good solution for this (do you think it is?). But I am not sure how to do the cross-domain-authentication. When the user logs in correctly, should i just do requests to the other domains and pass the SID and create cookie on the domains? Or is it not secure? Or should I create a token for the user after loggin in and store it in my DB. Then send the token with to all the domains to authenticate him and then delete the token?
Some other stuff i found was a solution via iframes in iframes.. also Googles analytics.js (https://developers.google.com/analytics/devguides/collection/analyticsjs/cross-domain) looked exciting... but i saw it's even integreated in easyXDM.
I hope some if you guys are having experience with that stuff. I am curious to read what you think :) gerti
What you want is OAuth2 (this is the protocol you're referring to -- what Google login does).
If you're looking for a simple solution, you might want to try using something like Stormpath + IDsite. It's a free API service you can sign up for / use.
Basically, they'll host a domain for you where you redirect your users when you want them to create an account or log in. They'll then handle all the authentication / authorization (across domains), and send you back an authenticated user with a JSON web token (JWT).
Stormpath has a bunch of libraries for a variety of programming languages, you can get started here: https://docs.stormpath.com/home/
The other solutions people typically go with require quite a bit of time / effort to get running. I've actually helped build quite a few of the Stormpath libraries (in Node / Python), so I'm pretty familiar with it -- you really can't beat it in terms of simplicity.
I want to create a web widget that will display information from my site.
The widget will be included in the client's website HTML using JavaScript, and should only be usable for my clients -- web sites that were registered at my site.
The information in the widget should be specific to the user who is currently visiting the client's site.
So, I need to authenticate both the client (website owner) and the resource owner (website visitor). This seems to map nicely to OAuth 2.0, but I couldn't find a complete example or explanation for such an implementation.
Any resources or pointers to such information will be appreciated.
Update: I've stumbled upon this article, which provides an outline for an approach that uses OAuth. However, it is not detailed enough for me to really understand how to use this with OAuth 2.
There are many large organizations that have done this, and I'm sad to see no other answers for this question since it's such an important web pattern.
I'm going to presume that you are not rolling your own OAuth 2.0 provider from scratch, if you are - well done otherwise you should be using something kickass like Doorkeeper to do this for you.
Now, in OAuth 2.0 you have the following entities:
Users registered on your website
Applications registered on your website (who subscribe to your oauth2)
User Permissions which is a list of Applications that a user has 'allowed'
Developer (who is consuming your auth API / widgets and building an Application)
The first thing to note is you must have a domain name associated with each Application. So if a developer registers for a API token / secret on your website, the Application he creates is mapped to a unique domain.
Now, I presume that the flow for an application to authenticate users via your website is already clear. That being said, you don't need to do much for this to work.
When an Application sends the user to your website (in order to sign in) you place a session cookie on the user's computer. Lets call this "Cookie-X".
Now the user is authenticated by your website and goes back to the Application. There we want to show a custom widget with information pertaining to that user.
The developer will be need to copy paste some code into this app.
The flow is like this:
The code will contain a url to your website with his Application ID (not secret) which he got when registering his application on your website.
When that code runs, it will ping your website with his appId. You need to check that AppID with your database, and additionally check that the referrer url is from the same domain as that which is registered in your website for that AppID. Edit: Alternatively or additionally, the code can check for document.domain and include it in the ping to your website, allowing you to verify that the request has come from the domain that has registered with the given AppID.
If that is correct, you reply back with some JS code.
Your JS code looks for the session cookie your website had set when the user had signed in. If that cookie is found, it pings back to your website with the session and your website responds with the custom view content.
Edit: as rightfully mentioned in a comment, the cookie should be HttpOnly to safeguard against common XSS attacks.
Additional Notes
The reasons this is a secure approach:
The AppId and domain name are a good enough combination to verify that other people are not fetching this information. Even thou the appId is visible in the applications html source, the domain name would have to be spoofed by anyone attempting to use someone else's AppID.
Presuming someone takes an AppID which is not his, and writes code to spoof the domain name of the referrer when requesting for your widget, he still won't be able to see any information. Since you are showing user specific information, the widget will only render if your website can find the session cookie it placed on the users browser which can't really be spoofed. There are ways around like session-hijacking, etc. But I think that's beyond the scope of this question.
Other Methods
Just by looking at Facebook's Social Plugins, you can tell that there are other options.
For example, one might be to use an Iframe. If you ask the developer to add an Iframe to his application, you can even reduce a few of the steps mentioned above. But you will have to add JS along with it (outside the iframe) to grab the correct domain, etc. And ofcourse from an accessibility and interface standpoint I'm not very found of Iframes.
I know this has been asked a bunch of times, but I have only seen serverside solutions.
I'm running an iframe app that is embedded into a page as a tab. I want be test to see if the page is liked or not without prompting the user for anything.
Is there a way to do this with just JavaScript? The platoform we are building on is ASPX and I dont really have the option of going serverside.
Its not available because you need to inspect the http post parameter called signed_request and this isn't available on the client side. If the user has authenticated with your app and given you permissions to read their likes/interests, then you could then check with javascript api but I'm guessing you wouldn't want to make them approve your app just for this.
I am creating a chrome extension, rather a chrome webapp. This application just contains the html, js, image and css files. The application connects to a server to fetch data. I chose to do this as it would reduce the amount of files downloaded by the user. Using Backbone.js I have an MVC architecture in my application. Thus the application just sends json.
Now having said this, I need a session management. I plan to use Google authentication as the organization has Google Apps. I need a method that once the user has logged in using google auth the server get the user name every time the application makes a request.
Is it a good idea to add the user name in request header, if possible. Or should I use cookies? Can any one tell me how I could go about using cookies in this case?
This might be a late response but I want to present a more elegant solution to you given that the user has cookies enabled in their browser.
First read my answer on another question.
Now that you can send cross origin xhr from your content scripts all you need to do is store all your authentication and session management at server only. That is right, you just need to display whether the user is logged in or not and a logout button at client based on server response.
Just follow these steps.
At client Whenever user accesses your chrome web app, blindly make XmlHttpRequests to your server without worrying about authentication, just keep a tab on response from server which I describe below.
At server whenever you receive a request check for valid sessions or session cookie. If session is valid send proper response, if not send error, 401 or any other response to communicate to your client that session is not valid. It is better if you send an error code like 401 since then you can put a generic script at client to inform them that they are not logged in.
At Client If response from server is proper, display it, else display login link to your website.
IMPORTANT: Display logout button if user is logged in.
Check out my implementation of this in my extension
For help using Google authentication in your app take a look at Google's OAuth tutorial which comes with all you need (took me no time to set it up using this).
As for session management. The implementation of OAuth used by Google stores the tokens in localStorage. Also, as briefly mentioned in the extensions overview we are expected to use localStorage to store data. Thus, I suggest you store the users name here as it will be accessible throughout the app's lifetime (until it is uninstalled). However, you may need to manage the name stored here and consider what should happen when users log in and out. That said; I'm not sure if sessionStorage would be a better option as I've never used it before, let alone in an extension.
Note
localStorage and its counterparts only store strings so I suggest using a wrapper which uses JSON to parse and stringify to get and set your values respectively.