HtmlAgilityPack how to get div that gets added by js?

HtmlAgilityPack how to get div that gets added by js? - javascript

Today I've wanted to write a webscraper that searches through a calendar website and finds the tags of events, so I can search through them and get info about who worked for the event.
The Problem is: The div I want to search for gets added by js, so how do I get it with htmlagilitypack?
The Calendar Website: https://esel.at/termine
My Code:
using System;
using HtmlAgilityPack;
using System.Linq;
using System.Diagnostics;
using System.Threading;
namespace ESEL_Scraper
{
class Program
{
static void Main(string[] args)
{
string Url = $"https://esel.at/termine";
HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load(Url);
HtmlNode[] nodes = doc.DocumentNode.SelectNodes("//div[#class='content']").ToArray();
for(int i = 0; i < nodes.Length; i++) {
Console.WriteLine(nodes[i].InnerText);
}
}
}
}

SelectNodes returns null when it doesn't find what you're looking for. So that's why you get the null exception. There are no "div" elements with class = "content". If you change to a class that is used by a div element on that page you'll get results.
With HtmlAgility pack "SelectNodes" you need to do a null check in some way before using the result.

Short anwser: You can't. Parsing the web page for data, that gets added when the page loads is not possible by using HtmlAgilityPack - the initial source code of the page doesn't have the data.
Long anwser: There is probably some API call that gets the data for the events, and is then pushed via javascript to the page. Try to figure out what URL is used, and try to parse that. That would be this one: https://esel.at/api/termine/data?date=05.09.2020&selection=false

As it's stated Javascript append the content. Using basic network inspecting you will see that there is another network request.
What you get here is data in JSON format which gets appended in HTML using Javascript.
Instead of using HtmlAgility pack you will need to parse JSON. In the example below I have used Newtonsoft.Json package to do that.
Here is the code:
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Text;
namespace ESEL_Scraper
{
internal class Program
{
private static void Main(string[] args)
{
//Simply create request to the API and deserialize JSON using the Root class
ServicePointManager.Expect100Continue = true;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
CookieContainer cookies = new CookieContainer();
// Set the date you want in the link, in this example it's 06.09.2020
var request = (HttpWebRequest)WebRequest.Create("https://esel.at/api/termine/data?date=06.09.2020&selection=false");
request.CookieContainer = cookies;
request.Method = "GET";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.ContentType = "application/json";
request.Headers.Add("accept-language", "en,hr;q=0.9");
request.Headers.Add("accept-encoding", "");
request.Headers.Add("Upgrade-Insecure-Requests", "1");
WebResponse response = request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
string responseFromServer = reader.ReadToEnd();
reader.Close();
response.Close();
//Deserialize Json
Root myDeserializedClass = JsonConvert.DeserializeObject<Root>(responseFromServer);
foreach (var el in myDeserializedClass.termine)
{
//Get any field you need
Console.WriteLine(el.title);
Console.WriteLine(el.location);
}
Console.ReadLine();
}
}
// Based on the JSON response https://pastebin.com/Xa5gSp50 I have generated classes using this website: https://json2csharp.com/
public class Termine
{
public int id { get; set; }
public string title { get; set; }
public string category { get; set; }
public string startdate { get; set; }
public string startdatetime { get; set; }
public string starttime { get; set; }
public string enddate { get; set; }
public List<object> runtime { get; set; }
public string thumbnail { get; set; }
public string thumbnail_credits { get; set; }
public string type { get; set; }
public string recommended { get; set; }
public bool online_event { get; set; }
public object feed_urls { get; set; }
public string status { get; set; }
public string tags { get; set; }
public string url { get; set; }
public string sort_date { get; set; }
public string sort_category { get; set; }
public string location_url { get; set; }
public string location { get; set; }
}
public class Meta
{
public List<string> next { get; set; }
public DateTime now { get; set; }
public List<string> date { get; set; }
public DateTime end { get; set; }
public DateTime runtime { get; set; }
public int upcoming { get; set; }
public int running { get; set; }
public int termine { get; set; }
}
public class Root
{
public List<Termine> termine { get; set; }
public Meta meta { get; set; }
}
}

Related

Deserialize javascript string rather then JSON using C#

Actually I've implmented ios in-app purchase & upon success, I receive json response.
The json response contains PurchaseToken, which is basically a base64 encoded javascript string.
{
"Id": "1000000872920320",
"TransactionDateUtc": "2021-09-06T14:42:17Z",
"ProductId": "1",
"AutoRenewing": false,
"PurchaseToken": "ewoJInNpZ25hdHVyZSIgPSAic29tZSBsb25nIGJhc2U2NCBzdHJpbmciOwoJInB1cmNoYXNlLWluZm8iID0gImV3b0pJbTl5YVdkcGJtRnNMWEIxY21Ob1lYTmxMV1JoZEdVdGNITjBJaUE5SUNJeU1EQXdMVEF4TFRBeElEQXhPakF4T2pBeElFRnRaWEpwWTJFdlRHOXpYMEZ1WjJWc1pYTWlPd29KSW5WdWFYRjFaUzFwWkdWdWRHbG1hV1Z5SWlBOUlDSnpiMjFsVlVsRUlqc0tDU0p2Y21sbmFXNWhiQzEwY21GdWMyRmpkR2x2YmkxcFpDSWdQU0FpVTI5dFpWUkpSQ0k3Q2draVluWnljeUlnUFNBaU15NDJJanNLQ1NKMGNtRnVjMkZqZEdsdmJpMXBaQ0lnUFNBaVUyOXRaVlJKUkNJN0Nna2ljWFZoYm5ScGRIa2lJRDBnSWpFaU93b0pJbWx1TFdGd2NDMXZkMjVsY25Ob2FYQXRkSGx3WlNJZ1BTQWlVRlZTUTBoQlUwVkVJanNLQ1NKdmNtbG5hVzVoYkMxd2RYSmphR0Z6WlMxa1lYUmxMVzF6SWlBOUlDSnpiMjFsSUc1MWJXVnlhV01nZEdsdFpTQnpkR0Z0Y0NCcGJpQnRjeUk3Q2draWRXNXBjWFZsTFhabGJtUnZjaTFwWkdWdWRHbG1hV1Z5SWlBOUlDSnpiMjFsSUZWSlJDSTdDZ2tpY0hKdlpIVmpkQzFwWkNJZ1BTQWlNU0k3Q2draWFYUmxiUzFwWkNJZ1BTQWljMjl0WlNCVmJtbHhkV1VnU1hSbGJTQkpSQ0k3Q2draWRtVnljMmx2YmkxbGVIUmxjbTVoYkMxcFpHVnVkR2xtYVdWeUlpQTlJQ0l3SWpzS0NTSnBjeTFwYmkxcGJuUnlieTF2Wm1abGNpMXdaWEpwYjJRaUlEMGdJbVpoYkhObElqc0tDU0p3ZFhKamFHRnpaUzFrWVhSbExXMXpJaUE5SUNKemIyMWxJRzUxYldWeWFXTWdkR2x0WlNCemRHRnRjQ0JwYmlCdGN5STdDZ2tpY0hWeVkyaGhjMlV0WkdGMFpTSWdQU0FpTWpBd01DMHdNUzB3TVNBd01Ub3dNVG93TVNCRmRHTXZSMDFVSWpzS0NTSnBjeTEwY21saGJDMXdaWEpwYjJRaUlEMGdJbVpoYkhObElqc0tDU0p2Y21sbmFXNWhiQzF3ZFhKamFHRnpaUzFrWVhSbElpQTlJQ0l5TURBd0xUQXhMVEF4SURBeE9qQXhPakF4SUVWMFl5OUhUVlFpT3dvSkltSnBaQ0lnUFNBaWFYQmhJR0Z3Y0NCaWRXNWtiR1VnYm1GdFpTSTdDZ2tpY0hWeVkyaGhjMlV0WkdGMFpTMXdjM1FpSUQwZ0lqSXdNREF0TURFdE1ERWdNREU2TURFNk1ERWdRVzFsY21sallTOU1iM05mUVc1blpXeGxjeUk3Q24wPSI7CgkiZW52aXJvbm1lbnQiID0gIlNhbmRib3giOwoJInBvZCIgPSAiMTAwIjsKCSJzaWduaW5nLXN0YXR1cyIgPSAiMCI7Cn0=",
"State": 0,
"ConsumptionState": 0,
"IsAcknowledged": false,
"Payload": null
}
I deserialize the above object & decode the base64 purchaseToken, but the decoded string is javascript string rather then json (look at the = & ;).
{
"signature" = "some long base64 string";
"purchase-info" = "ewoJIm9yaWdpbmFsLXB1cmNoYXNlLWRhdGUtcHN0IiA9ICIyMDAwLTAxLTAxIDAxOjAxOjAxIEFtZXJpY2EvTG9zX0FuZ2VsZXMiOwoJInVuaXF1ZS1pZGVudGlmaWVyIiA9ICJzb21lVUlEIjsKCSJvcmlnaW5hbC10cmFuc2FjdGlvbi1pZCIgPSAiU29tZVRJRCI7CgkiYnZycyIgPSAiMy42IjsKCSJ0cmFuc2FjdGlvbi1pZCIgPSAiU29tZVRJRCI7CgkicXVhbnRpdHkiID0gIjEiOwoJImluLWFwcC1vd25lcnNoaXAtdHlwZSIgPSAiUFVSQ0hBU0VEIjsKCSJvcmlnaW5hbC1wdXJjaGFzZS1kYXRlLW1zIiA9ICJzb21lIG51bWVyaWMgdGltZSBzdGFtcCBpbiBtcyI7CgkidW5pcXVlLXZlbmRvci1pZGVudGlmaWVyIiA9ICJzb21lIFVJRCI7CgkicHJvZHVjdC1pZCIgPSAiMSI7CgkiaXRlbS1pZCIgPSAic29tZSBVbmlxdWUgSXRlbSBJRCI7CgkidmVyc2lvbi1leHRlcm5hbC1pZGVudGlmaWVyIiA9ICIwIjsKCSJpcy1pbi1pbnRyby1vZmZlci1wZXJpb2QiID0gImZhbHNlIjsKCSJwdXJjaGFzZS1kYXRlLW1zIiA9ICJzb21lIG51bWVyaWMgdGltZSBzdGFtcCBpbiBtcyI7CgkicHVyY2hhc2UtZGF0ZSIgPSAiMjAwMC0wMS0wMSAwMTowMTowMSBFdGMvR01UIjsKCSJpcy10cmlhbC1wZXJpb2QiID0gImZhbHNlIjsKCSJvcmlnaW5hbC1wdXJjaGFzZS1kYXRlIiA9ICIyMDAwLTAxLTAxIDAxOjAxOjAxIEV0Yy9HTVQiOwoJImJpZCIgPSAiaXBhIGFwcCBidW5kbGUgbmFtZSI7CgkicHVyY2hhc2UtZGF0ZS1wc3QiID0gIjIwMDAtMDEtMDEgMDE6MDE6MDEgQW1lcmljYS9Mb3NfQW5nZWxlcyI7Cn0=";
"environment" = "Sandbox";
"pod" = "100";
"signing-status" = "0";
}
And the purchase info:
{
"original-purchase-date-pst" = "2000-01-01 01:01:01 America/Los_Angeles";
"unique-identifier" = "someUID";
"original-transaction-id" = "SomeTID";
"bvrs" = "3.6";
"transaction-id" = "SomeTID";
"quantity" = "1";
"in-app-ownership-type" = "PURCHASED";
"original-purchase-date-ms" = "some numeric time stamp in ms";
"unique-vendor-identifier" = "some UID";
"product-id" = "1";
"item-id" = "some Unique Item ID";
"version-external-identifier" = "0";
"is-in-intro-offer-period" = "false";
"purchase-date-ms" = "some numeric time stamp in ms";
"purchase-date" = "2000-01-01 01:01:01 Etc/GMT";
"is-trial-period" = "false";
"original-purchase-date" = "2000-01-01 01:01:01 Etc/GMT";
"bid" = "ipa app bundle name";
"purchase-date-pst" = "2000-01-01 01:01:01 America/Los_Angeles";
}
I'm facing this issue with deserializing of the javascript string into a C# object.
Any proper way to handle this deserialization?
Newtonsoft.Json is not able to handle it & I have looked its docs again & again to find any suitable methods but no success.
Is there any way other then to replace characters & make its json like?
Any help would be highly appreciated.

As others already mentioned, it's not JSON. But based on the example you provided you should be able to transform it to a readable JSON. It's not the most pretty solution, but it seems to work. It have some draw-backs:
It will only work when " = " is the separator.
It will only work when each field is separated by ; followed by line-break (windows-style).
private string MakeReadableJson(string base64Input)
{
var data = System.Convert.FromBase64String(base64Input);
string readableString = Encoding.UTF8.GetString(data);
var json = readableString.Replace("\" = \"", "\":\"").Replace(";\n", ",");
return json;
}
I use System.Text.JsonSerializer. Use it like so:
string json = #"{
""Id"": ""1000000872920320"",
""TransactionDateUtc"": ""2021-09-06T14:42:17Z"",
""ProductId"": ""1"",
""AutoRenewing"": false,
""PurchaseToken"": ""ewoJInNpZ25hdHVyZSIgPSAic29tZSBsb25nIGJhc2U2NCBzdHJpbmciOwoJInB1cmNoYXNlLWluZm8iID0gImV3b0pJbTl5YVdkcGJtRnNMWEIxY21Ob1lYTmxMV1JoZEdVdGNITjBJaUE5SUNJeU1EQXdMVEF4TFRBeElEQXhPakF4T2pBeElFRnRaWEpwWTJFdlRHOXpYMEZ1WjJWc1pYTWlPd29KSW5WdWFYRjFaUzFwWkdWdWRHbG1hV1Z5SWlBOUlDSnpiMjFsVlVsRUlqc0tDU0p2Y21sbmFXNWhiQzEwY21GdWMyRmpkR2x2YmkxcFpDSWdQU0FpVTI5dFpWUkpSQ0k3Q2draVluWnljeUlnUFNBaU15NDJJanNLQ1NKMGNtRnVjMkZqZEdsdmJpMXBaQ0lnUFNBaVUyOXRaVlJKUkNJN0Nna2ljWFZoYm5ScGRIa2lJRDBnSWpFaU93b0pJbWx1TFdGd2NDMXZkMjVsY25Ob2FYQXRkSGx3WlNJZ1BTQWlVRlZTUTBoQlUwVkVJanNLQ1NKdmNtbG5hVzVoYkMxd2RYSmphR0Z6WlMxa1lYUmxMVzF6SWlBOUlDSnpiMjFsSUc1MWJXVnlhV01nZEdsdFpTQnpkR0Z0Y0NCcGJpQnRjeUk3Q2draWRXNXBjWFZsTFhabGJtUnZjaTFwWkdWdWRHbG1hV1Z5SWlBOUlDSnpiMjFsSUZWSlJDSTdDZ2tpY0hKdlpIVmpkQzFwWkNJZ1BTQWlNU0k3Q2draWFYUmxiUzFwWkNJZ1BTQWljMjl0WlNCVmJtbHhkV1VnU1hSbGJTQkpSQ0k3Q2draWRtVnljMmx2YmkxbGVIUmxjbTVoYkMxcFpHVnVkR2xtYVdWeUlpQTlJQ0l3SWpzS0NTSnBjeTFwYmkxcGJuUnlieTF2Wm1abGNpMXdaWEpwYjJRaUlEMGdJbVpoYkhObElqc0tDU0p3ZFhKamFHRnpaUzFrWVhSbExXMXpJaUE5SUNKemIyMWxJRzUxYldWeWFXTWdkR2x0WlNCemRHRnRjQ0JwYmlCdGN5STdDZ2tpY0hWeVkyaGhjMlV0WkdGMFpTSWdQU0FpTWpBd01DMHdNUzB3TVNBd01Ub3dNVG93TVNCRmRHTXZSMDFVSWpzS0NTSnBjeTEwY21saGJDMXdaWEpwYjJRaUlEMGdJbVpoYkhObElqc0tDU0p2Y21sbmFXNWhiQzF3ZFhKamFHRnpaUzFrWVhSbElpQTlJQ0l5TURBd0xUQXhMVEF4SURBeE9qQXhPakF4SUVWMFl5OUhUVlFpT3dvSkltSnBaQ0lnUFNBaWFYQmhJR0Z3Y0NCaWRXNWtiR1VnYm1GdFpTSTdDZ2tpY0hWeVkyaGhjMlV0WkdGMFpTMXdjM1FpSUQwZ0lqSXdNREF0TURFdE1ERWdNREU2TURFNk1ERWdRVzFsY21sallTOU1iM05mUVc1blpXeGxjeUk3Q24wPSI7CgkiZW52aXJvbm1lbnQiID0gIlNhbmRib3giOwoJInBvZCIgPSAiMTAwIjsKCSJzaWduaW5nLXN0YXR1cyIgPSAiMCI7Cn0="",
""State"": 0,
""ConsumptionState"": 0,
""IsAcknowledged"": false,
""Payload"": null
}";
var info = JsonSerializer.Deserialize<ResultFromApple>(json);
var purchaseTokenJson = MakeReadableJson(info.PurchaseToken);
PurchaseToken purchaseToken = JsonSerializer.Deserialize<PurchaseToken>(purchaseTokenJson, new JsonSerializerOptions()
{
AllowTrailingCommas = true
});
var purchaseInfoJson = MakeReadableJson(purchaseToken.PurchaseInfo);
PurchaseInfo purchaseInfo = JsonSerializer.Deserialize<PurchaseInfo>(purchaseInfoJson, new JsonSerializerOptions()
{
AllowTrailingCommas = true
});
And your models:
public class PurchaseToken
{
public string signature { get; set; }
[JsonPropertyName("purchase-info")]
public string PurchaseInfo { get; set; }
public string environment { get; set; }
public string pod { get; set; }
[JsonPropertyName("signing-status")]
public string SigningStatus { get; set; }
}
public class ResultFromApple
{
public string Id { get; set; }
public DateTime TransactionDateUtc { get; set; }
public string ProductId { get; set; }
public bool AutoRenewing { get; set; }
public string PurchaseToken { get; set; }
public int State { get; set; }
public int ConsumptionState { get; set; }
public bool IsAcknowledged { get; set; }
public object Payload { get; set; }
}
public class PurchaseInfo
{
[JsonPropertyName("original-purchase-date-pst")]
public string OriginalPurchaseDatePst { get; set; }
[JsonPropertyName("unique-identifier")]
public string UniqueIdentifier { get; set; }
[JsonPropertyName("original-transaction-id")]
public string OriginalTransactionId { get; set; }
public string bvrs { get; set; }
[JsonPropertyName("transaction-id")]
public string TransactionId { get; set; }
public string quantity { get; set; }
[JsonPropertyName("in-app-ownership-type")]
public string InAppOwnershipType { get; set; }
[JsonPropertyName("original-purchase-date-ms")]
public string OriginalPurchaseDateMs { get; set; }
[JsonPropertyName("unique-vendor-identifier")]
public string UniqueVendorIdentifier { get; set; }
[JsonPropertyName("product-id")]
public string ProductId { get; set; }
[JsonPropertyName("item-id")]
public string ItemId { get; set; }
[JsonPropertyName("version-external-identifier")]
public string VersionExternalIdentifier { get; set; }
[JsonPropertyName("is-in-intro-offer-period")]
public string IsInIntroOfferPeriod { get; set; }
[JsonPropertyName("purchase-date-ms")]
public string PurchaseDateMs { get; set; }
[JsonPropertyName("purchase-date")]
public string PurchaseDate { get; set; }
[JsonPropertyName("is-trial-period")]
public string IsTrialPeriod { get; set; }
[JsonPropertyName("original-purchase-date")]
public string OriginalPurchaseDate { get; set; }
public string bid { get; set; }
[JsonPropertyName("purchase-date-pst")]
public string PurchaseDatePst { get; set; }
}

I would use Pidgin to parse something like this.
Here's some code that parses your data into a Dictionary<string, string> and prints out the results:
using System;
using System.Collections.Generic;
using System.Linq;
using Pidgin;
using static Pidgin.Parser;
namespace NotQuiteJsonParsing
{
class Program
{
const string PurchaseInfo = #"{
""original-purchase-date-pst"" = ""2000-01-01 01:01:01 America/Los_Angeles"";
""unique-identifier"" = ""someUID"";
""original-transaction-id"" = ""SomeTID"";
""bvrs"" = ""3.6"";
""transaction-id"" = ""SomeTID"";
""quantity"" = ""1"";
""in-app-ownership-type"" = ""PURCHASED"";
""original-purchase-date-ms"" = ""some numeric time stamp in ms"";
""unique-vendor-identifier"" = ""some UID"";
""product-id"" = ""1"";
""item-id"" = ""some Unique Item ID"";
""version-external-identifier"" = ""0"";
""is-in-intro-offer-period"" = ""false"";
""purchase-date-ms"" = ""some numeric time stamp in ms"";
""purchase-date"" = ""2000-01-01 01:01:01 Etc/GMT"";
""is-trial-period"" = ""false"";
""original-purchase-date"" = ""2000-01-01 01:01:01 Etc/GMT"";
""bid"" = ""ipa app bundle name"";
""purchase-date-pst"" = ""2000-01-01 01:01:01 America/Los_Angeles"";
}";
private static string IEnumerableCharToString(IEnumerable<char> value)
{
// Feel free to swap this with an alternative answer to https://stackoverflow.com/questions/8108313
return new string(value.ToArray());
}
// Parsers can be slow to build, so minimise the
// number of times this setup method is called. See
// https://github.com/benjamin-hodgson/Pidgin#speed-tips
private static Parser<char, Dictionary<string, string>> BuildBlockParser()
{
var stringParser = Char('"').Then(AnyCharExcept('"').AtLeastOnceUntil(Char('"')));
var lineParser = Map(
(key, value) => new KeyValuePair<string, string>(IEnumerableCharToString(key), IEnumerableCharToString(value)),
stringParser.Before(SkipWhitespaces).Before(Char('=')).Before(SkipWhitespaces),
stringParser.Before(SkipWhitespaces).Before(Char(';')).Before(SkipWhitespaces));
var linesParser = lineParser.AtLeastOnce().Map(pairs => new Dictionary<string, string>(pairs));
var blockParser = SkipWhitespaces.Then(Char('{')).Then(SkipWhitespaces).Then(linesParser).Before(Char('}'));
return blockParser;
}
static void Main()
{
var blockParser = BuildBlockParser();
var result = blockParser.ParseOrThrow(PurchaseInfo);
foreach (var (key, value) in result)
{
Console.WriteLine($"{key} = {value}");
}
}
}
}
This parser has the following limitations:
Values in key-value pairs are always strings, not numbers, booleans or anything else.
There is no support for escaped " characters in keys or values. I don't know how they would be escaped.
There is no support for escape sequences such as \r, \n, \t or \u0123.

It's a bit embarrassing. I saw the problem as the opposite before. Try this, I think this can meet the needs:
Parse JavaScript string
static void Main(string[] args)
{
var jsStr = #"{
""signature"" = ""some long base64 string"";
""purchase-info"" = ""ewoJIm9yaWdpbmFsLXB1cmNoYXNlLWRhdGUtcHN0IiA9ICIyMDAwLTAxLTAxIDAxOjAxOjAxIEFtZXJpY2EvTG9zX0FuZ2VsZXMiOwoJInVuaXF1ZS1pZGVudGlmaWVyIiA9ICJzb21lVUlEIjsKCSJvcmlnaW5hbC10cmFuc2FjdGlvbi1pZCIgPSAiU29tZVRJRCI7CgkiYnZycyIgPSAiMy42IjsKCSJ0cmFuc2FjdGlvbi1pZCIgPSAiU29tZVRJRCI7CgkicXVhbnRpdHkiID0gIjEiOwoJImluLWFwcC1vd25lcnNoaXAtdHlwZSIgPSAiUFVSQ0hBU0VEIjsKCSJvcmlnaW5hbC1wdXJjaGFzZS1kYXRlLW1zIiA9ICJzb21lIG51bWVyaWMgdGltZSBzdGFtcCBpbiBtcyI7CgkidW5pcXVlLXZlbmRvci1pZGVudGlmaWVyIiA9ICJzb21lIFVJRCI7CgkicHJvZHVjdC1pZCIgPSAiMSI7CgkiaXRlbS1pZCIgPSAic29tZSBVbmlxdWUgSXRlbSBJRCI7CgkidmVyc2lvbi1leHRlcm5hbC1pZGVudGlmaWVyIiA9ICIwIjsKCSJpcy1pbi1pbnRyby1vZmZlci1wZXJpb2QiID0gImZhbHNlIjsKCSJwdXJjaGFzZS1kYXRlLW1zIiA9ICJzb21lIG51bWVyaWMgdGltZSBzdGFtcCBpbiBtcyI7CgkicHVyY2hhc2UtZGF0ZSIgPSAiMjAwMC0wMS0wMSAwMTowMTowMSBFdGMvR01UIjsKCSJpcy10cmlhbC1wZXJpb2QiID0gImZhbHNlIjsKCSJvcmlnaW5hbC1wdXJjaGFzZS1kYXRlIiA9ICIyMDAwLTAxLTAxIDAxOjAxOjAxIEV0Yy9HTVQiOwoJImJpZCIgPSAiaXBhIGFwcCBidW5kbGUgbmFtZSI7CgkicHVyY2hhc2UtZGF0ZS1wc3QiID0gIjIwMDAtMDEtMDEgMDE6MDE6MDEgQW1lcmljYS9Mb3NfQW5nZWxlcyI7Cn0="";
""environment"" = ""Sandbox"";
""pod"" = ""100"";
""signing-status"" = ""0"";
}";
var model = GetModelFromJsStr<PurchaseToken>(jsStr);
}
public class PurchaseToken
{
[JsonPropertyName("signature")]
public string Signature { get; set; }
[JsonPropertyName("purchase-info")]
public string PurchaseInfo { get; set; }
[JsonPropertyName("environment")]
public string Environment { get; set; }
[JsonPropertyName("pod")]
public int Pod { get; set; }
[JsonPropertyName("signing-status")]
public int SigningStatus { get; set; }
}
public static T GetModelFromJsStr<T>(string jsStr) where T : new()
{
var model = new T();
var rows = jsStr.Split("\n").Select(item => item.Trim()).ToList();
foreach (var prop in typeof(T).GetProperties())
{
var jsonProperty = prop.GetCustomAttribute<JsonPropertyNameAttribute>();
if (jsonProperty != null)
{
var matchRow = rows.FirstOrDefault(row => row.StartsWith(#$"""{jsonProperty.Name}"" ="));
if (matchRow != null)
{
var pattern = #$"^""{jsonProperty.Name}"" = ""(.*)"";$";
var match = Regex.Match(matchRow, pattern);
if (match?.Groups?.Count > 1)
{
// convertible types can be extended
if (prop.PropertyType == typeof(int))
{
int.TryParse(match.Groups[1].Value, out var propValue);
prop.SetValue(model, propValue);
}
else
{
prop.SetValue(model, match.Groups[1].Value);
}
}
}
}
};
return model;
}

Was able to handle this by with following code using Newtonsoft.Json.JsonConverter:
Create models for the data
public class InAppPurchaseToken
{
public string Signature { get; set; }
[JsonProperty("purchase-info")]
public string PurchaseInfo { get; set; }
public string Environment { get; set; }
public string Pod { get; set; }
public string SigningStatus { get; set; }
}
public class InAppPurchaseInfo
{
[JsonProperty("original-purchase-date-pst")]
public string originalpurchasedatepst { get; set; }
[JsonProperty("unique-identifier")]
public string uniqueidentifier { get; set; }
[JsonProperty("original-transaction-id")]
public string originaltransactionid { get; set; }
public string bvrs { get; set; }
[JsonProperty("transaction-id")]
public string transactionid { get; set; }
public string quantity { get; set; }
[JsonProperty("in-app-ownership-type")]
public string inappownershiptype { get; set; }
[JsonProperty("original-purchase-date-ms")]
public string originalpurchasedatems { get; set; }
[JsonProperty("unique-vendor-identifier")]
public string uniquevendoridentifier { get; set; }
[JsonProperty("product-id")]
public string productid { get; set; }
[JsonProperty("item-id")]
public string itemid { get; set; }
[JsonProperty("version-external-identifier")]
public string versionexternalidentifier { get; set; }
[JsonProperty("is-in-intro-offer-period")]
public string isinintroofferperiod { get; set; }
[JsonProperty("purchase-date-ms")]
public string purchasedatems { get; set; }
[JsonProperty("purchase-date")]
public string purchasedate { get; set; }
[JsonProperty("is-trial-period")]
public string istrialperiod { get; set; }
[JsonProperty("original-purchase-date")]
public string originalpurchasedate { get; set; }
public string bid { get; set; }
[JsonProperty("purchase-date-pst")]
public string purchasedatepst { get; set; }
}
And the handle the parsing using following code (assuming you have deserialized the initial json as inAppPayData C# object):
public string GetJsonFromBase64JsStr(string base64JsStr)
{
if (string.IsNullOrWhiteSpace(base64JsStr))
return "";
var decodedStr2Parse = Encoding.UTF8.GetString(Convert.FromBase64String(base64JsStr));
//var ConvertedJsonStrComplete = decodedStr2Parse.Replace("\" = \"", "\":\"").Replace(";\n", ",").Replace(",}", "}"); //more error prone
var ConvertedJsonStrPartial = decodedStr2Parse.Replace("\" = \"", "\":\"").Replace(";\n", ",");
var ConvertedJsonStrComplete = ConvertedJsonStrPartial.Remove(ConvertedJsonStrPartial.LastIndexOf(",}"), 1);
return ConvertedJsonStrComplete;
}
//call the above function with base64JsString
var purchaseJsonStr = GetJsonFromBase64JsStr(inAppPayData.PurchaseToken);
var purchaseTokenObj = JsonConvert.DeserializeObject<InAppPurchaseToken>(purchaseJsonStr);
var purchaseInfoJsonStr = GetJsonFromBase64JsStr(purchaseTokenObj.PurchaseInfo);
var purchaseInfoObj = JsonConvert.DeserializeObject<InAppPurchaseInfo>(purchaseInfoJsonStr);

C# SHA512 to Hex string

There's a javascript library called Crypto-js and i'm trying to convert some methods I use to c#.
For example in javascript:
var payload = JSON.stringify({ market: "BTC-ETH", order: { price: "0.02159338", side: "buy", size: "0.024" } });
var contentHash = cryptoJS.SHA512(payload).toString(cryptoJS.enc.Hex);
console.log(contentHash);
In C#
public class OrdersVM
{
public string Market { get; set; }
public Order Order { get; set; }
}
public class Order
{
public string Price { get; set; }
public string Side { get; set; }
public string Size { get; set; }
}
public async Task<IActionResult> Orders([FromBody] OrdersVM vm) {
var payload = JsonConvert.SerializeObject(vm);
var contentHash = sha512Hex(payload).ToLower();
Console.WriteLine(contentHash);
}
public string sha512Hex(string input)
{
var bytes = Encoding.UTF8.GetBytes(input);
using (var hash = SHA512.Create())
{
hash.ComputeHash(bytes);
return BitConverter.ToString(hash.Hash).Replace("-", "");
}
}
contentHash for javascript is
"99bb05af8aace509189e08625bb4e475a9daaafc92edf5c85fa1aefcc16c16e4533c23843c5806aef01c97e8cb4150b2dc129d04d3b6a50331833fe5cb8158fc"
and for c#
"731b92cf482ff90ffe759e356959ec005334062bdc3c2cc78b48c3041d21a45ecaa6b33f6df2971fa868f94f04b7596e818104cb1017ed1c436365beac3a01d1"
What am I doing wrong with c# conversion?

The issue is that JSON.Net, by default, will serialise your property names exactly as the appear, meaning they all start with a capital letter. There are two ways to fix this:
Using JsonProperty to explicitly control the property name serialisation. For example:
public class OrdersVM
{
[JsonProperty("market")]
public string Market { get; set; }
[JsonProperty("order")]
public Order Order { get; set; }
}
Use a contract resolver to tell JSON.Net how to process the names. Fortunately there is one provided for you that will do this:
var settings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
}
var payload = JsonConvert.SerializeObject(vm, settings);

Regex - Prevent pasting in an input [duplicate]

This question already has answers here:
Parse JSON array in JSON.NET
(3 answers)
Closed 5 years ago.
I have regex expression, which prevented me entering certain characters in the input field
onKeyMonitor ($event) {
const value = $event.key;
const pattern = /^[#$^<>[\]{}]*$/;
const regex = new RegExp(pattern);
if (regex.test(value)) {
$event.preventDefault();
}
}
But this is failing if I paste the same special characters, What needs to be done here.
I have written an onpaste event as well. with similar logic, but it is failing.

Initially you could define the following classes:
public class AnswerOption
{
[JsonProperty("parentQuestion")]
public string ParentQuestion { get; set; }
[JsonProperty("parentAnswer")]
public object ParentAnswer { get; set; }
[JsonProperty("answerOption")]
public object Option { get; set; }
}
public class Question
{
[JsonProperty("questionId")]
public int QuestionId { get; set; }
[JsonProperty("questionName")]
public string QuestionName { get; set; }
[JsonProperty("questionType")]
public string QuestionType { get; set; }
[JsonProperty("questionSequenceNumber")]
public int QuestionSequenceNumber { get; set; }
[JsonProperty("pageNo")]
public int PageNo { get; set; }
[JsonProperty("highlightedText")]
public string HighlightedText { get; set; }
[JsonProperty("isDynamicText")]
public string IsDynamicText { get; set; }
[JsonProperty("answerOptions")]
public IList<AnswerOption> AnswerOptions { get; set; }
}
public class DataObj
{
[JsonProperty("surveyId")]
public int SurveyId { get; set; }
[JsonProperty("questions")]
public IList<Question> Questions { get; set; }
}
public class Example
{
[JsonProperty("dataObj")]
public DataObj DataObj { get; set; }
[JsonProperty("errorCode")]
public int ErrorCode { get; set; }
[JsonProperty("errorMessage")]
public string ErrorMessage { get; set; }
}
Then you could deserialize your json as below:
var example = JsonConvert.DeserializeObject<Example>(json);
where json is a variable that holds the json you want to deserialize.
Last you could access the firsts question id as below:
labl1.text = example.DataObj?.Questions?.FirstOrDefault()?.QuestionId;

Use this website to help you generate C# classes from Json string http://json2csharp.com/
Note: Don't forget to double the data type.

Converting MVC Model List to JSLINQ Error

I have three dropdownlist BodyPart, ExamDetail and ExamView and I have complete data set for these lists. I don't need to call controller again and again whenever the dropdown change event call but I want to fetch list from my model's property list. I am using JS Library to apply LINQ Queryto avoid any loops in code.
My Class Architecture is given below:
public class BodyPart
{
public string ID { get; set; }
public string Text { get; set; }
public List<ExamDetail> examdetail { get; set; }
}
public class ExamDetail
{
public string ID { get; set; }
public string Text { get; set; }
public List<ExamView> examview { get; set; }
}
public class ExamView
{
public string ID { get; set; }
public string Text { get; set; }
}
I have proper data in all my list, here is my js code to fetch ExamDetail Record
var selectedBodyPart = $("#BodyPartDDL").val();
var examdetailList = JSLINQ(#Model.bodypart).Where(function (item) {
return item.ID == selectedBodyPart; });
But I am getting "Uncaught SyntaxError: Unterminated template literal(…)" error where I pass my model list. I need to pass this examdetailList to my ExamDetail partial view. Thank you.

Client side validations not working in Internet Explorer 9

I just created mvc 4 Application .In that application I have a form to insert department. for that form I'm using client side validations.
these client side validations properly working in Firefox and Chrome , but not in IE 9 (Internet Explorer)
This is my model class
public partial class tbl_hec_Department
{
public tbl_hec_Department()
{
this.tbl_hec_level_of_Study = new HashSet<tbl_hec_level_of_Study>();
this.tbl_hec_Programme = new HashSet<tbl_hec_Programme>();
}
[Required]
[DisplayName("Department ID")]
[Remote("doesDepartment_IDExist", "HEC", HttpMethod = "POST", ErrorMessage = "Department ID already exists.")]
public string Department_ID { get; set; }
[Required]
[DisplayName("Name of Department")]
public string Name_of_Department { get; set; }
[DataType(DataType.PhoneNumber)]
[DisplayName("Telephone Number")]
//[StringLength(50, ErrorMessage = "Please enter a minimum of {2} characters", MinimumLength = 6)]
public string Contact_General_Line { get; set; }
//[StringLength(50, ErrorMessage = "Please enter a minimum of {2} characters", MinimumLength = 6)]
[DisplayName("Fax Number")]
public string Contact_Fax_Number { get; set; }
[Required]
[EmailAddress(ErrorMessage = "Please enter a valid email address")]
[DisplayName("Email Address")]
public string Contact_Email { get; set; }
[Required(ErrorMessage = "Please select the {0}")]
[DisplayName("University / Institute")]
public string HEI_ID { get; set; }
[Required(ErrorMessage = "Please select the {0}")]
[DisplayName("College")]
public string College_ID { get; set; }
[DisplayName("Status")]
public Nullable<bool> Status { get; set; }
public string Create_By { get; set; }
public Nullable<System.DateTime> Create_Date { get; set; }
public string Update_By { get; set; }
public Nullable<System.DateTime> Update_Date { get; set; }
public virtual tbl_hec_College tbl_hec_College { get; set; }
public virtual tbl_hec_University tbl_hec_University { get; set; }
public virtual ICollection<tbl_hec_level_of_Study> tbl_hec_level_of_Study { get; set; }
public virtual ICollection<tbl_hec_Programme> tbl_hec_Programme { get; set; }
}
theses are the scripts files in Scripts Folder
this is how i render script files in layout files
#Scripts.Render("~/Scripts/jquery.min.js")
#Scripts.Render("~/Scripts/jquery-migrate-1.2.1.js")
#Scripts.Render("~/Scripts/bootstrap.min.js")
#Scripts.Render("~/Scripts/jquery-ui.min.js")
#Scripts.Render("~/Scripts/raphael-min.js")
#Scripts.Render("~/Scripts/morris.min.js")
#Scripts.Render("~/Scripts/jquery.sparkline.min.js")
#Scripts.Render("~/Scripts/jquery-jvectormap-1.2.2.min.js")
#Scripts.Render("~/Scripts/jquery-jvectormap-world-mill-en.js")
#Scripts.Render("~/Scripts/jquery.knob.js")
#Scripts.Render("~/Scripts/daterangepicker.js")
#Scripts.Render("~/Scripts/bootstrap-datepicker.js")
#Scripts.Render("~/Scripts/bootstrap3-wysihtml5.all.min.js")
#Scripts.Render("~/Scripts/icheck.min.js")
#Scripts.Render("~/Scripts/app.js")
#Scripts.Render("~/Scripts/dashboard.js")
#Scripts.Render("~/Scripts/demo.js")

I'm pretty sure that in client machine browser java script is disabled. First you check whether java script is enabled or disabled in client browser. You can check it programatically using this code.
public static bool IsJavascriptEnabled( )
{
bool retVal = true;
//get the registry key for Zone 3(Internet Zone)
Microsoft.Win32.RegistryKey key = Registry.CurrentUser.OpenSubKey(#"Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\3", true);
if (key != null)
{
Object value = key.GetValue(DWORD_FOR_ACTIVE_SCRIPTING, VALUE_FOR_ENABLED);
if( value.ToString().Equals(VALUE_FOR_DISABLED) )
{
retVal = false;
}
}
return retVal;
}
You can also visit this link How to check java scxript in client browser enabled or not

Develop Reference

JavaScript is the programming language of the Web.

HtmlAgilityPack how to get div that gets added by js? - javascript

Related

Deserialize javascript string rather then JSON using C#

C# SHA512 to Hex string

Regex - Prevent pasting in an input [duplicate]

Converting MVC Model List to JSLINQ Error

Client side validations not working in Internet Explorer 9

Categories

Resources