I am trying to build a console application using c# .net.. I need one particular functionality which is to retrieve values from an external website. The thing is we need to sign in to that website. I am able to open the page that i need using process.start in chrome signed in with the values i need.. but problem is when retrieving the value from page.. i thought of getting source code but every way i try it does not take the session and hence i am getting just error page source code as i am just entering the URL n not accessing the already opened tab? Is there any other way available either using JavaScript or c#?
Use WebClient API to login and download page data. You will need to make it cookie aware in order to maintain session. You will need to add reference to System.Web Assembly to use this API in console application.
public class CookieAwareWebClient : WebClient
{
public CookieAwareWebClient()
{
CookieContainer = new CookieContainer();
}
public CookieContainer CookieContainer { get; private set; }
protected override WebRequest GetWebRequest(Uri address)
{
var request = (HttpWebRequest)base.GetWebRequest(address);
request.CookieContainer = CookieContainer;
return request;
}
}
Now use it like
using (var client = new CookieAwareWebClient())
{
var values = new NameValueCollection
{
{ "username", "john" },
{ "password", "secret" },
};
client.UploadValues("http://domain.loc/logon.aspx", values);
// If the previous call succeeded we now have a valid authentication cookie
// so we could download the protected page
string result = client.DownloadString("http://domain.loc/testpage.aspx");
}
I borrowed this piece of code from WebClient accessing page with credentials
Related
I'm trying to implement a google Oauth2 authentication on a vueJs SPA (front) and a ASP.NET CORE WEB API (back).But I don't know what to do with the authorization code sent after the user properly signedIn.
What I want to achieve :
I want to implement an Oauth2 process using the GSI library to get the user unique OPENID code.
<script src="https://accounts.google.com/gsi/client" async defer></script>
I want to be able to create a custom google SignIn Button that open an authentication popin when clicked on it.
What did I managed to achieve for now ?
I already setup a google sign in flow using the following functions :
google.accounts.id.initialize({...})
Then, I'm force to render a google button using this method :
google.accounts.id.renderButton(buttonWrapper, {
theme: 'outline',
size: 'medium',
type: 'standard',
});
When the user clicks on the button, it opens a signIn popin.
When he register properly, The callback function gives me a response.credential which is a jwtToken
I send the JWTToken to my ASP NET CORE WEB API and validate it :
[...]
var payload = await GoogleJsonWebSignature.ValidateAsync(token);
[...]
The payload gives me user information like his email, his name and... his openId. Success !
Why am I not satisfied with the previous method?
The previous method force me to use the google.accounts.id.renderButton.
I am not satisfied by the customization options that google gave us with the button.
I did not used the Oauth2 standard authentication method which was also the point of my exercise.
What's happening when I try the Oauth2 ways ?
First, I'm setting up a client.
this.client = google.accounts.oauth2.initTokenClient({
client_id: this.clientId,
scope:
'https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile openid',
callback: this.onTokenReceivedCallback,
});
Then I create a custom button that triggers this method on click
function(){
this.client.requestAccessToken();
}
If the user click on the button, the sign in popin appears.
If the user signIn, the callback function triggers and I get a response that looks like this.
access_token: "ya29.A0ARrdyM_x7n9uh12345678901234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ12345678901234567890ABCDEFGUMZabqaXVDmmomNrfO_bhLIP-9llQNExSZFNZA2mH5Pzeaq_UiE1mNlfQfhVXQyBv0Hbr1dgYUNnWUtBVEFTQVRBU0ZRRl91NjFWNTNYaVpRa3k0cGRXTWlqZ0pwZGd0Zw0165"
authuser: "1"
expires_in: 3599
prompt: "none"
scope: "email profile https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile openid"
token_type: "Bearer"
Now, the problems starts because I have absolutely no idea what I should do with that access_token.
This code is not a JwtToken so my previous method does not works.
I guess I should send this accessToken to my ASP .NET CORE WEB API.
The doc mention many apis like google calendar, google drive, email etc etc but I'm not interested in any of that.
What I want is very simple :
I would like to retrieve the user OPENID code.
I will use it to search for an existing user in my database.
Every tutorial I read are not clear (for me) about that and they always consider that I'm using an ASP .NET CORE WEB APP which is REALLY different than an ASP .NET CORE WEB API
because the user never directly interact with the last one.
Any help would be appreciated.
Thank you for your time and your help.
I finally managed to find a solution that works for me.
The Access_Token I received was an Authorization Code I could use to consume google apis.
The only thing I had to do was to send the AccessToken to my personal ASP.NET CORE WebApi.
Then I just had to write a simple function that my controller could execute :
public class GoogleUserInfos
{
public string Id { get; set; }
public string Email { get; set; }
public string Verified_Email { get; set; }
public string Name { get; set; }
public string Given_Name { get; set; }
public string Family_Name { get; set; }
public string Picture { get; set; }
public string Locale { get; set; }
}
public async Task<GoogleUserInfos?> GetGoogleUserInfos(string authCode)
{
try
{
using (var client = new HttpClient())
{
var uri = "https://www.googleapis.com/oauth2/v1/userinfo?access_token=" + authCode;
using (var response = await client.GetAsync(uri))
{
if (response.StatusCode == HttpStatusCode.OK)
{
string responseString = await response.Content.ReadAsStringAsync();
var obj = JsonConvert.DeserializeObject<GoogleUserInfos>(responseString);
return obj;
}
}
}
}
catch
{
return null;
}
return null;
}
The function GetGoogleUserInfos take the Access_Token as parameter (named authCode).
Then it consume the googleApi oauth2 with the following endpoint https://www.googleapis.com/oauth2/v1/userinfo.
It also send the accessToken as argument in the getRequest.
Once it get the response, the function deserialize the object in a custom object GoogleUserInfos.
The Id field of the response is the google user unique OPEN ID which will allow me to link the user with the users I store in my database.
I am creating web application, which let's users to communicate with so called chat.
In order to enable such communication, I use SignalR library. I create connection at first visit to my page (main page). So JS code, which creates connection, creates variable used to configure connection.
Then user enters chat room, which is different page, so new JSs are loaded etc. The one, which held connection variable is now unavailable. But now I need that connection to send messages in my chat room.
So this variable must be "transfered" over to next scripts.
So, what would be the way to actually persist connection through whole session on the website?
Finally I got my answer.
It was suggested that I should use ASP NET Identity, which is very valid, but I already created simple authentication and users management. It isn't safe and as half good as ASP NET Identity (I looked throught it and got my head around it), but it's just personal project, not production one, so if it evolves I might switch to Identity or even implement one myself ;) But it's not the case.
It required little bit of extra steps, so:
I needed to enable sessions in ASP.NET Core, I used this article for this purpose. Having that, I can persist my user login and provide it to user ID provider for signalR
Adding cutsom user ID provider for SignalR in .NET:
I needed to create such class
public class UserIdProvider : IUserIdProvider
{
public static readonly string SESSION_LOGIN_KEY = "loggedUser";
private readonly IHttpContextAccessor _httpContextAccessor;
public UserIdProvider(IHttpContextAccessor httpContextAccessor)
{
_httpContextAccessor = httpContextAccessor;
}
public string GetUserId(HubConnectionContext connection)
{
var session = _httpContextAccessor.HttpContext.Session;
session.TryGetValue(SESSION_LOGIN_KEY, out byte[] loginBA);
if (loginBA == null)
{
return null;
}
return new string(loginBA.Select(b => (char)b).ToArray());
}
}
So, after logging I can set login in Session, so it becomes "state" variable, and use it in above class.
Also, one need to add it in ASP services, like below (in Startup.ConfigureServices):
services.AddSingleton<IUserIdProvider, UserIdProvider>();
There also one thing, which still need to be set: in UserIdProvider we need to access Session through HttpContext. In order to use HttpContextwe need to specify it like below (also in Startup.ConfigureServices):
services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();
which passes HttpContextAccessor to serices constructors.
After all this, you can access users in SignalR Hub with their logins, which is set in Context.UserIdnentifier
This also enables sending messages to specific user, just by passing their login (frontend client just chooses user), like below:
public async Task SendMessage(string message, string user)
{
await Clients.User(user).SendAsync("ReceiveMessage", message).ConfigureAwait(false);
}
NOTE There was one problem though. Browsers on computer didn't persist sessions, which I solved by (also in Startup.ConfigureServices):
services.Configure<CookiePolicyOptions>(options =>
{
// This lambda determines whether user consent for non-essential cookies is needed for a given request.
options.CheckConsentNeeded = context => false; // <~~~~ This needs to be set to false
options.MinimumSameSitePolicy = SameSiteMode.None;
});
Without it, you need to be carefull with cookies, if you don't accept them on a site, it won't work, as user's login won't be persisted.
On the server side, you can send the message to specific user via user id:
var user = await _userManager.GetUserAsync(Context.User);
await Clients.User(user.Id).SendCoreAsync("msg", new object[] { user.Id, user.Email });
On the client side, whenever the connection is started and the hub msg is listened, the user will receive the message:
connection.on('msg', function (...data) {
console.log('data:', data);
});
By sending message(s) via user id, you don't need to care where the target user is.
public class ChatHub : Hub
{
private IUserManager<ApplicationUser> _userManager;
public ChatHub(UserManager<ApplicationUser> userManager)
{
_userManager = userManager;
}
public async Task GetInfo()
{
var user = await _userManager.GetUserAsync(Context.User);
await Clients.User(user.Id).SendCoreAsync("msg", new object[] { user.Id, user.Email });
}
}
Unfortunately, there are many snippets, but no complete code examples. I'm trying to allow my group's events to be seen in a webpage outside of FB, and following the Javascript SDK example quick start I'm able to put the login and share buttons. But when I try to access my group's events, I get the error - "An access token is required to request this resource." Where do I put the access token in this code?
window.fbAsyncInit = function() {
FB.init({
appId : '{myappID}',
xfbml : true,
version : 'v2.8'
});
FB.AppEvents.logPageView();
FB.api(
'/myGroupID/events',
'GET',
{},
function(response) {
// Insert your code here
}
);
};
A couple of things:
I don't need my users to log in, as my group is public; my events are public. If someone is casually browsing through my website I want them to be able to see the events.
One of the other things that I've had trouble with is extremely short answers. Coming from the .NET community, I'm used to seeing lots of tutorials, and lots of complete code examples for how to do things. There are few "long form" answers or tutorials - even stackoverflow answers (like this one) don't contain enough detail on how to do this stuff. Especially the access token thing.
Is there a complete example of how to do this?
Thanks in advance.
You COULD add the Token like this:
FB.api(
'/myGroupID/events',
'GET',
{access_token: 'xxx'},...
...but that would expose your Token (which is always meant to be kept secret) to everyone visiting the website. You have to do that API call server side. Check out the PHP SDK or just use PHP cURL. The Token is just a GET parameter.
Ok, I figured it out. The point of all of this is to get the Access Token that Facebook says it wants. The Access Token is really the appsecret_proof (and not the access_token - the access_token is a different thing. See later in this post), so be aware of that. I cobbled together some different code examples (like this one, and a super, very careful reading of the Facebook graph api docs, to reach an answer. I coded in C# rather than Javascript because a lot of this needs to be done server side, and I'm more comfortable there anyway.
I created a console app as a proof of concept. A few notes:
the page_id is, in my case, for a group, not a (capital P) Page,
which is a different thing.
The access_token and app_secret are from the app
you've (hopefully) already created. (If you need to get the access_token in the first place, there
are some docs out there to help you get this part started.) You should also
make sure to use (as of .NET 1.0) the dotnet core secret manager
to protect your secrets when you develop so you don't pass them
around via source control.
This last one is big - the appsecret_proof is a combination of your access_token and your app_secret hashed together (with the app_secret being the key) and then made part of the query string. This is what Facebook wants when it says "An access token is required to request this resource."
public static string page_id = {your_page_id};
public static string access_token = {your_app_access_token};
public static string app_secret = {your_app_secret};
public static string appsecret_proof = FaceBookSecret(access_token, app_secret);
static void Main(string[] args)
{
Console.WriteLine("Making API Call...");
using (var client = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate }))
{
client.BaseAddress = new Uri("https://graph.facebook.com/" + page_id + "/");
HttpResponseMessage response = client.GetAsync("events?access_token=" + access_token + "&appsecret_proof=" + appsecret_proof).Result;
response.EnsureSuccessStatusCode();
string result = response.Content.ReadAsStringAsync().Result;
Console.WriteLine("Result: " + result);
}
Console.ReadLine();
}
internal static string FaceBookSecret(string content, string key)
{
byte[] keyBytes = Encoding.UTF8.GetBytes(key);
byte[] messageBytes = Encoding.UTF8.GetBytes(content);
byte[] hash;
using (HMACSHA256 hmacsha256 = new HMACSHA256(keyBytes))
{
hash = hmacsha256.ComputeHash(messageBytes);
}
StringBuilder sbHash = new StringBuilder();
for (int i = 0; i < hash.Length; i++)
{
sbHash.Append(hash[i].ToString("x2"));
}
Console.WriteLine(sbHash);
return sbHash.ToString();
}
So after all of this runs, I get a nice json string with all of my events.
I have an asp.net mvc site where users can select a number of objects, they are identified by a unique id.
What I would like to do is:
User A opens an object
When user B tries to open the same object, a message will appear saying this is already open by user A, therefore is inaccessible.
I guess the server would have a sort of 'shared' session that can be queried in every Ajax request, to check if the object is already open by a different user. When the object is closed then that id should be removed from the shared list of opened objects. What would be the best approach for this?
In a typical scenario, assuming that you are working with plain CLR objects, this could be a solution
// User1 and User2 are trying to access objectA
private static int objectACounter; // Counter to decide whether object A is occupied
if(objectACounter != 0) // Object A is occupied
return user message object A is occupied
lock(objectA)
{
objectACounter ++;
// User logic processing
objectACounter --;
}
Above mentioned solution can be further refined by using a static dictionary to suggest which user has currently locked the object A and at what time, which can be returned to the incoming user.
In this case you plan to use something like Shared session, in my understanding there's nothing like a shared session, since session is user specific, above mentioned mechanism is a way to create one. You can plan to use Application Cache instead, where for each object you can define a key for each object and add the details in same way as you would do to the static dictionary, since Cache is a key value pair and it persist across the multiple user requests on the server.
Logic will be something like:
if(Cache.ContainsKey(objectAKey))
// Return error to the incoming user with details of object holding user
lock(objectA)
{
Cache.Insert(objectAKey, UserValues);
// User1 logic processing
Cache.Remove(objectAKey, UserValues);
}
Please let me know if I have misunderstood your question and you are expecting a different solution, We can adapt the current solution in multiple scenarios
I had to do something similar recently, but what I ended up doing was to not prevent a second person from accessing a submitted application, but to notify them that someone else is currently working on it when they view it (passive feedback) and if they attempt to modify it, prevent them from doing so, either after the postback occurs and possibly rendering the UI in a readonly fashion (active feedback).
I tracked the activity of who was using what with SignalR, this way I could dynamically update the list of applications when someone accessed one for the first time, or release an application when they leave the page, without forcing the user viewing the list of projects to refresh the page to be made aware of the change.
I setup a signalR Hub in my code to track which project was claimed by whom. I implemented the tracking using ConcurrentDictionary class to tackle any sort of concurrency problems of modifying the collection.
public class RetainedApplicationsHub : Hub
{
private static readonly ConcurrentDictionary<Guid, RetainedApplication> RetainedApplications = new ConcurrentDictionary<Guid, RetainedApplication>();
public static bool IsApplicationRetained(Guid applicationId)
{
return RetainedApplications.ContainsKey(applicationId);
}
public void RetainApplication(Guid applicationId, Guid personId)
{
var retainedSuccessfully = RetainedApplications.TryAdd(applicationId, new RetainedApplication{ConnectionId = Context.ConnectionId, PersonId = personId});
if(retainedSuccessfully)
Clients.All.applicationRetained(applicationId);
}
public void ReleaseApplication(Guid applicationId)
{
RetainedApplication temp;
RetainedApplications.TryRemove(applicationId, out temp);
Clients.All.applicationReleased(applicationId);
}
public ICollection<Guid> GetRetainedApplicationIds()
{
return RetainedApplications.Keys;
}
public override Task OnDisconnected(bool stopCalled)
{
RetainedApplication temp;
var applicationId = RetainedApplications.Single(x => x.Value.ConnectionId == Context.ConnectionId).Key;
RetainedApplications.TryRemove(applicationId, out temp);
Clients.All.applicationReleased(applicationId);
return base.OnDisconnected(stopCalled);
}
}
public class RetainedApplication
{
public string ConnectionId { get; set; }
public Guid PersonId { get; set; }
}
On my Application list page
<script src="~/signalr/hubs"></script>
<script>
$(function () {
'use strict';
var retainedApplicationsHub = $.connection.retainedApplicationsHub;
retainedApplicationsHub.client.applicationRetained = function(applicationId) {
$("td:contains('" + applicationId + "')").parent().addClass("active text-muted");
};
retainedApplicationsHub.client.applicationReleased = function(applicationId) {
$("td:contains('" + applicationId + "')").parent().removeClass();
};
$.connection.hub.start().done(function() {
retainedApplicationsHub.server.getRetainedApplicationIds()
.done(function(applicationIds) {
$.each(applicationIds, function() {
retainedApplicationsHub.client.applicationRetained(this);
});
});
});
});
</script>
On my application details page
<script src="~/signalr/hubs"></script>
<script>
$(function () {
'use strict';
var retainedApplicationsHub = $.connection.retainedApplicationsHub;
retainedApplicationsHub.client.applicationRetained = function(applicationId) {
//do nothing, must subscribe to at least one event to have the OnDisconnected event on the hub register correctly
};
$.connection.hub.start().done(function () {
retainedApplicationsHub.server.retainApplication("#Model.RegistrationApplication.ObjectId", "#User.GetIdentityId()");
});
});
</script>
This pertains to cookies set inside a script (maybe inside a script tag).
System.Windows.Forms.HtmlDocument executes those scripts and the cookies set (like document.cookie=etc...) can be retrieved through its Cookies property.
I assume HtmlAgilityPack.HtmlDocument doesn't do this (execution). I wonder if there is an easy way to emulate the System.Windows.Forms.HtmlDocument capabilities (the cookies part).
Anyone?
When I need to use Cookies and HtmlAgilityPack together, or just create custom requests (for example, set the User-Agent property, etc), here is what I do:
Create a class that encapsulates the request/response. Let's call this class WebQuery
Have a private CookieCollection (in your case public) property inside that class
Create a method inside the class that does manually the request. The signature could be:
...
public HtmlAgilityPack.HtmlDocument GetSource(string url);
What do we need to do inside this method?
Well, using HttpWebRequest and HttpWebResponse, generate the http request manually (there are several examples of how to do this on Internet), create an instance of a HtmlDocument class using the constructor that receives an stream.
What stream do we have to use? Well, the one returned by:
httpResponse.GetResponseStream();
If you use HttpWebRequest to make the query, you can easily set the CookieContainer property of it to the variable you declared before everytime you access a new page, and that way all cookies set by the sites you access will be properly stored in the CookieContainer variable you declared in your WebQuery class, taking in count you're using only one instance of the WebQuery class.
Hope you find useful this explanation. Take in count that using this, you can do whatever you want, no matter if HtmlAgilityPack supports it or not.
I also worked with Rohit Agarwal's BrowserSession class together with HtmlAgilityPack.
But for me subsequent calls of the "Get-function" didn't work, because every time new cookies have been set.
That's why I added some functions by my own. (My solution is far a way from beeing perfect - it's just a quick and dirty fix) But for me it worked and if you don't want to spent a lot of time in investigating BrowserSession class here is what I did:
The added/modified functions are the following:
class BrowserSession{
private bool _isPost;
private HtmlDocument _htmlDoc;
public CookieContainer cookiePot; //<- This is the new CookieContainer
...
public string Get2(string url)
{
HtmlWeb web = new HtmlWeb();
web.UseCookies = true;
web.PreRequest = new HtmlWeb.PreRequestHandler(OnPreRequest2);
web.PostResponse = new HtmlWeb.PostResponseHandler(OnAfterResponse2);
HtmlDocument doc = web.Load(url);
return doc.DocumentNode.InnerHtml;
}
public bool OnPreRequest2(HttpWebRequest request)
{
request.CookieContainer = cookiePot;
return true;
}
protected void OnAfterResponse2(HttpWebRequest request, HttpWebResponse response)
{
//do nothing
}
private void SaveCookiesFrom(HttpWebResponse response)
{
if ((response.Cookies.Count > 0))
{
if (Cookies == null)
{
Cookies = new CookieCollection();
}
Cookies.Add(response.Cookies);
cookiePot.Add(Cookies); //-> add the Cookies to the cookiePot
}
}
What it does: It basically saves the cookies from the initial "Post-Response" and adds the same CookieContainer to the request called later. I do not fully understand why it was not working in the initial version because it somehow does the same in the AddCookiesTo-function. (if (Cookies != null && Cookies.Count > 0) request.CookieContainer.Add(Cookies);)
Anyhow, with these added functions it should work fine now.
It can be used like this:
//initial "Login-procedure"
BrowserSession b = new BrowserSession();
b.Get("http://www.blablubb/login.php");
b.FormElements["username"] = "yourusername";
b.FormElements["password"] = "yourpass";
string response = b.Post("http://www.blablubb/login.php");
all subsequent calls should use:
response = b.Get2("http://www.blablubb/secondpageyouwannabrowseto");
response = b.Get2("http://www.blablubb/thirdpageyouwannabrowseto");
...
I hope it helps when you're facing the same problem.