Develop Reference

How to Derive the Key and Initial Vector in Node.js

I have a shared key that I need to derive an iv from so I can decipher.
The apple business chat docs state:
Generate the Derived Key and Initial Vector
Run the shared key through the X9.63 Key Derivation Function with SHA256 hash function. This results in a 48-byte payload. Your results should be rV3qrszd0PMPgeRhNnlOYA==
Heres what I tried. I used scryptSync and pbkdf2Sync crypto functions with many 'salt' configurations. I'm unsure if these are the correct functions for this job.
const crypto = require('crypto');
const keyLength = 48;
// sharedKey is a base64 string
const sharedKey = "2lvSJsBO2keUHRfvPG6C1RMUmGpuDbdgNrZ9YD7RYnvAcfgq/fjeYr1p0hWABeif";
// publicKey is a base64 string
const publicKey = "BDiRKNnPiPUb5oala31nkmCaXMB0iyWy3Q93p6fN7vPxEQSUlFVsInkJzPBBqmW1FUIY1KBA3BQb3W3Qv4akZ8kblqbmvupE/EJzPKbROZFBNvxpvVOHHgO2qadmHAjHSg=="
const key1 = crypto.scryptSync(sharedKey, 'salt', keyLength);
console.log(key2.toString('base64'));
const key2 = crypto.pbkdf2Sync(sharedKey, 'salt', 10000, keyLength, 'sha256');
console.log(key2.toString('base64'));
// results should be:
// mAzkYatDlz4SzrCyM23NhgL/+mE3eGgfUz9h1CFPhZM=
// iv: rV3qrszd0PMPgeRhNnlOYA==
Below is the Apple sample code for Deriving the Key and Initial Vector with a X9.63 Key Derivation Function.
def ITOSP(self, longint, length):
"""ITOSP, short for Integer-to-Octet-String Primitive, converts a non-negative integer
to an octet string of a specified length. This particular function is defined in the
PKCS #1 v2.1: RSA Cryptography Standard (June 14, 2002)
https://www.cryptrec.go.jp/cryptrec_03_spec_cypherlist_files/PDF/pkcs-1v2-12.pdf"""
hex_string = "%X" % longint
assert len(hex_string) <= 2 * length, "ITOSP function: Insufficient length for encoding"
return binascii.a2b_hex(hex_string.zfill(2 * length))
def KDFX963(self, inbyte_x, shared_data, key_length, hashfunct=sha256, hash_len=32):
"""KDFX963 is a key derivation function (KDF) that takes as input byte sequence inbyte_x
and additional shared data shared_data and outputs a byte sequence key of length
key_length. This function is defined in ANSI-X9.63-KDF, and this particular flavor of
KDF is known as X9.63. You can read more about it from:
http://www.secg.org/sec1-v2.pdf"""
assert key_length >= 0, "KDFX963 function: key_length should be positive integer"
k = key_length / float(hash_len)
k = int(ceil(k))
acc_str = ""
for i in range(1, k+1):
h = hashfunct()
h.update(inbyte_x)
h.update(self.ITOSP(i, 4))
h.update(shared_data)
acc_str = acc_str + h.hexdigest()
return acc_str[:key_length * 2]

X9.63 KDF is a key derivation function, described e.g. here and here. scrypt and PBKDF2 are also KDFs, but different ones, so of course the expected result cannot be reproduced with them.
So you need a NodeJS library that supports X.963 KDF. If you can't find one, you could also implement your own.
X9.63 KDF expects a shared secret and a shared info and determines a keysize large key as follows:
Create a 4 byte counter ci which is incremented starting with 0x0000001.
Concatenate the data conci = shared secret | ci | shared info
Hash the result hashi = hash(conci)
Concatenate the hashes hash1 | hash2 | ... until an output of length keysize has been generated.
More formally, including various checks, the algorithm is described in the links above. The Python code posted later in the question also implements this logic.
One possible NodeJS implementation (omitting the checks from the specification) is:
var crypto = require('crypto');
var digest = 'sha256';
var digestLen = 32;
function X963KDF(sharedSecret, sharedInfo, keySize){
var maxCount = Math.ceil(keySize/digestLen);
var result = Buffer.allocUnsafe(0);
for (var count = 1; count < maxCount + 1; count++){
var counter = Buffer.allocUnsafe(4);
counter.writeUInt32BE(count, 0);
var current = Buffer.concat([sharedSecret, counter, sharedInfo]);
var hash = crypto.createHash(digest).update(current).digest();
result = Buffer.concat([result, hash]);
}
return result.slice(0, keySize);
}
Test:
In the question the shared secret is posted, but not the shared info. An internet search reveals that the posted problem is described e.g. here, so that the shared info can also be determined (nevertheless it would be better if you add this information to your question):
var sharedSecret = Buffer.from('2lvSJsBO2keUHRfvPG6C1RMUmGpuDbdgNrZ9YD7RYnvAcfgq/fjeYr1p0hWABeif', 'base64')
var sharedInfo = Buffer.from('04389128d9cf88f51be686a56b7d6792609a5cc0748b25b2dd0f77a7a7cdeef3f111049494556c227909ccf041aa65b5154218d4a040dc141bdd6dd0bf86a467c91b96a6e6beea44fc42733ca6d139914136fc69bd53871e03b6a9a7661c08c74a', 'hex');
var keyiv = X963KDF(sharedSecret, sharedInfo, 48);
var key = keyiv.slice(0,32).toString('base64');
var iv = keyiv.slice(32, 48).toString('base64');
console.log("Key: ", key); // Key: mAzkYatDlz4SzrCyM23NhgL/+mE3eGgfUz9h1CFPhZM=
console.log("IV: ", iv); // IV: rV3qrszd0PMPgeRhNnlOYA==
The generated key and IV are equal to the expected values.

How to get X509Certificate thumbprint in Javascript?

I need to write a function in javascript (forge) that get a thumbnail of a pfx certificate. I created a test certificate(mypfx.pfx). By using c# X509Certificate2 library, I can see thumbprint of input certificate in X509Certificate2 object by passing file bytes array and password. Here is c# code snippet:
X509Certificate2 certificate = new X509Certificate2(byteArrayCertData, password);
var thumbprint = certificate.Thumbprint;
//thumbprint is a hex encoding SHA-1 hash
But when I am trying to do the same thing in javascript (using forge). I can't get a correct thumbprint. Here is my Javascript code:
var certi = fs.readFileSync('c:/mypfx.pfx');
let p12b64 = Buffer(certi).toString('base64');
let p12Der = forge.util.decode64(p12b64);
var outAsn1 = forge.asn1.fromDer(p12Der);
var pkcs12 = forge.pkcs12.pkcs12FromAsn1(outAsn1, false, "1234");
var fp = null;
for (var sci = 0; sci < pkcs12.safeContents.length; ++sci) {
var safeContents = pkcs12.safeContents[sci];
for (var sbi = 0; sbi < safeContents.safeBags.length; ++sbi) {
var safeBag = safeContents.safeBags[sbi];
if (safeBag.cert != undefined && safeBag.cert.publicKey != undefined) {
fp = forge.pki.getPublicKeyFingerprint(safeBag.cert.publicKey, {type: 'RSAPublicKey'});
//Is this fingerprint value I am looking for??
break;
}
}
}
The result is different value compare to c# thumbprint which seems to be wrong. I tried different functions in pkcs12.js file. None of them works. It's acceptable using other JS library as long as correct thumbprint result is produced. Please help and correct any mistakes I made. Thanks!

You are comparing different data. The certificate thumbprint is not the same that the public key fingerprint.
The certificate thumbprint is a hash calculated on the entire certificate. Seems forge does not have a method, but you can calculate yourself
//SHA-1 on certificate binary data
var md = forge.md.sha1.create();
md.start();
md.update(certDer);
var digest = md.digest();
//print as HEX
var hex = digest.toHex();
console.log(hex);
To convert the forge certificate to DER (binary) you can use this
var certAsn1 = forge.pki.certificateToAsn1(cert);
var certDer = forge.asn1.toDer(certAsn1).getBytes();

Scrape webpage that requires md5 hash as a parameter

I am trying to scrape the data from the below link, in a c# console app:
https://www.eex-transparency.com/homepage/power/germany/production/availability/non-usability
Using the developer tools in chrome I can see that its possible to get a json response, the url to get this is:
https://www.eex-transparency.com/dsp/tem-12?country=de&expires=1454345128&md5=TRhtJei_go4ueLeekBc8yw
the website uses this js file (https://www.eex-transparency.com/assets/js/tpe-website.js) to generate the expires and md5 hash key. I think I've figured out that the expires value is a unix datetime. I have never used javascript before so finding it hard to figure out how they construct the md5.
The Javascript that generates these code is:
generateCryptedParams=function(url,clientIP)
{
var cryptedParams,md5,md5Encoded,md5WithoutSpeciaChars,parser,timePoint,urlPath;
return timePoint=moment().tz("Europe/Berlin").add(1,"minute").unix(),
parser=document.createElement("a"),
parser.href=url,
urlPath=parser.pathname,
"/"!==urlPath[0]&&(urlPath="/"+urlPath),
md5=CryptoJS.MD5(urlPath+timePoint+clientIP+" zYeHzBomGdgV"),
md5Encoded=md5.toString(CryptoJS.enc.Base64),
md5WithoutSpeciaChars=replaceSpecialChars(md5Encoded),
cryptedParams={"expires":timePoint,"md5":md5WithoutSpeciaChars}
}
replaceSpecialChars=function(str)
{
var key,specialChars,value;
specialChars={"=":"","\\+":"-","/":"_","%":"_"};
for(key in specialChars)
value=specialChars[key],
str=str.replace(new RegExp(key,"g"),value);
return str
}
As i said I think I'm comfortable with the timepoint part but the md5 is confusing me. Below is my C# code to replicate their but when I pass the md5 hash their site returns a 403 Forbidden error.
public Tuple<string, Int32> GenerateCrypto(string url, string ipAddress)
{
string cetId = "Central European Standard Time";
TimeZoneInfo cetZone = TimeZoneInfo.FindSystemTimeZoneById(cetId);
var CETDateTime = TimeZoneInfo.ConvertTimeFromUtc(DateTime.UtcNow, cetZone);
//Int32 unixTimestamp = (Int32)(CETDateTime.Subtract(new DateTime(1970, 1, 1))).TotalSeconds;
Int32 unixTimestamp = (Int32)(DateTime.UtcNow.AddMinutes(1).Subtract(new DateTime(1970, 1, 1))).TotalSeconds;
url = url.Split('/')[3];
var md5 = CipherUtility.GenerateMd5(url + unixTimestamp + ipAddress + " zYeHzBomGdgV");
var md5Encoded = Convert.ToBase64String(Encoding.UTF8.GetBytes(md5));
var md5withoutSpecialCharts = replaceSpecialChars(md5Encoded);
md5withoutSpecialCharts = md5withoutSpecialCharts.Substring(0, 22);
return new Tuple<string, Int32>(md5withoutSpecialCharts, unixTimestamp);
}

The solution was that I needed to concatenate a const string to all the elements before hashing it.

title casing and Abbreviations in javascript

I am trying to Titlecase some text which contains corporate names and their stock symbols.
Example (these strings are concatenated as corporate name, which gets title cased and the symbol in parens): AT&T (T)
John Deere Inc. (DE)
These corporate names come from our database which draws them from a stock pricing service. I have it working EXCEPT for when the name is an abbreviation like AT&T
That is return, and you guessed it right, like At&t. How can I preserve casing in abbreviations. I thought to use indexof to get the position of any &'s and uppercase the two characters on either side of it but that seems hackish.
Along the lines of(pseudo code)
var indexPos = myString.indexOf("&");
var fixedString = myString.charAt(indexPos - 1).toUpperCase().charAt(indexPos + 1).toUpperCase()
Oops, forgot to include my titlecase function
function toTitleCase(str) {
return str.replace(/([^\W_]+[^\s-]*) */g, function (txt) {
return txt.charAt(0).toUpperCase() + txt.substr(1).toLowerCase();
});
}
Any better suggestions?

A better title case function may be
function toTitleCase(str) {
return str.replace(
/(\b.)|(.)/g,
function ($0, $1, $2) {
return ($1 && $1.toUpperCase()) || $2.toLowerCase();
}
);
}
toTitleCase("foo bAR&bAz a.e.i."); // "Foo Bar&Baz A.E.I."
This will still transform AT&T to At&T, but there's no information in the way it's written to know what to do, so finally
// specific fixes
if (str === "At&T" ) str = "AT&T";
else if (str === "Iphone") str = "iPhone";
// etc
// or
var dict = {
"At&T": "AT&T",
"Iphone": "iPhone"
};
str = dict[str] || str;
Though of course if you can do it right when you enter the data in the first place it will save you a lot of trouble

This is a general solution for title case, without taking your extra requirements of "abbreviations" into account:
var fixedString = String(myString).toLowerCase().replace(/\b\w/g, String.toUpperCase);
Although I agree with other posters that it's better to start with the data in the correct format in the first place. Not all proper names conform to title case, with just a couple examples being "Werner von Braun" and "Ronald McDonald." There's really no algorithm you can program into a computer to handle the often arbitrary capitalization of proper names, just like you can't really program a computer to spell check proper names.
However, you can certainly program in some exception cases, although I'm still not sure that simply assuming that any word with an ampersand in it should be in all caps always appropriate either. But that can be accomplished like so:
var titleCase = String(myString).toLowerCase().replace(/\b\w/g, String.toUpperCase);
var fixedString = titleCase.replace(/\b\w*\&\w*\b/g, String.toUpperCase);
Note that your second example of "John Deere Inc. (DE)" still isn't handled properly, though. I suppose you could add some other logic to say, put anything word between parentheses in all caps, like so:
var titleCase = String(myString).toLowerCase().replace(/\b\w/g, String.toUpperCase);
var titleCaseCapAmps = titleCase.replace(/\b\w*\&\w*\b/g, String.toUpperCase);
var fixedString = titleCaseCapAmps.replace(/\(.*\)/g, String.toUpperCase);
Which will at least handle your two examples correctly.

How about this: Since the number of registered companies with the stock exchange is finite, and there's a well-defined mapping between stock symbols and company names, your best best is probably to program that mapping into your code, to look up the company name by the ticker abbreviation, something like this:
var TickerToName =
{
A: "Agilent Technologies",
AA: "Alcoa Inc.",
// etc., etc.
}
Then it's just a simple lookup to get the company name from the ticker symbol:
var symbol = "T";
var CompanyName = TickerToName[symbol] || "Unknown ticker symbol: " + symbol;
Of course, I would be very surprised if there was not already some kind of Web Service you could call to get back a company name from a stock ticker symbol, something like in this thread:
Stock ticker symbol lookup API
Or maybe there's some functionality like this in the stock pricing service you're using to get the data in the first place.

The last time I faced this situation, I decided that it was less trouble to simply include the few exceptions here and there as need.
var titleCaseFix = {
"At&t": "AT&T"
}
var fixit(str) {
foreach (var oldCase in titleCaseFix) {
var newCase = titleCaseFix[oldCase];
// Look here for various string replace options:
// http://stackoverflow.com/questions/542232/in-javascript-how-can-i-perform-a-global-replace-on-string-with-a-variable-insi
}
return str;
}

Read a file with latest Date from a folder using JQuery

I already have a folder, with the files in it with names below in a specific format i.e. MODEL_RELEASEDATE
File names in the folder named Smartphone
SmartphoneA_11122012
SmartphoneA_01022013
SmartphoneA_09102013
SmartphoneA_10072012
SmartphoneA_12042012
**SmartphoneB_08282013**
SmartphoneB_04152013
SmartphoneB_08282012
SmartphoneB_01062013
.
.
.
.
and so on
I want to write a jquery code where I can use a specific keyword from format, from above list, I will pass the value SmartphoneA and I should be able to read the file with the latest release date. Same as in case when I pass the keyword SmartphoneB.
If I pass k/w SmartphoneB, result should be served from file highlighted above, i.e. SmartphoneB_08282013
my current code reads the file name only with specific k/w. I have few alterations to be made.
<script type="text/javascript" src="http://code.jquery.com/jquery-1.10.2.min.js"></script>
<script type="text/javascript">
var metaKeywords=$('meta[name=keywords]').attr("content");//this utility works based upon the keywords on the pages.
var reqdKeyword = new Array();
reqdKeyword = metaKeywords.split(",");
var randKeyword = reqdKeyword[Math.floor(Math.random() * reqdKeyword.length)];
var cdnUrl = "http://abc.com/xyz/mobiles/";
var jsnUrl = ".json?callback=showDetail";
var finalUrl= cdnUrl.concat(randKeyword.trim()).concat(jsnUrl);
/*
Rest of the code goes here
*/
</script>

The cool thing about dates is that you can easily sort them if you have the date in "descending" order (i.e., year month day hour second). Using that, we can go through your files to grab just the ones that start with the right prefix, then easily grab the latest one:
var filenames = [
'SmartphoneA_11122012',
'SmartphoneA_01022013',
'SmartphoneA_09102013',
'SmartphoneA_10072012',
'SmartphoneA_12042012',
'SmartphoneB_08282013',
'SmartphoneB_04152013',
'SmartphoneB_08282012',
'SmartphoneB_01062013'
],
whichPhone = 'SmartphoneB', // Dummy value, this would come from user interaction or whatever
possibleFiles = [];
// This goes through your list of filenames and picks out just the ones that match `whichPhone`, then puts them into another array containing a "cleaned-up" date and some other metadata-esque stuff
for (var i = 0, j = filenames.length; i < j; i++) {
var filename = filenames[i];
if (filename.indexOf(whichPhone) > -1) {
possibleFiles.push({
index: i,
filename: filename,
date: parseInt(filename.split('_')[1].replace(/(\d{2})(\d{2})(\d{4})/, function(match, month, day, year) {
return year + month + day;
}), 10)
});
}
}
// Now we go through the `possibleFiles` and figure out which one has the latest date
var latestFileDate = 0,
theActualFilenameYouWantFinally;
for (var i = 0, j = possibleFiles.length; i < j; i++) {
var possibleFile = possibleFiles[i];
if (possibleFile.date > latestFileDate) {
latestFileDate = possibleFile.date;
theActualFilenameYouWantFinally = filenames[possibleFile.index];
}
}
// And, finally, your result
console.log(theActualFilenameYouWantFinally);
EDIT: I didn't use jQuery for this answer because meh, you don't really need jQuery for things like this. Don't get me wrong, John Resig is brilliant, and I use jQuery in almost everything, but for loops are damned fast and easy to work with, and stuff like this isn't really jQuery's strong suit anyhow.

You will need a server-side code to return you a list of URIs (file names), then you can write JavaScript code to parse it (but in this case it is probably better if your server-side code will return the right name right away based on query string). In the worst case scenario you can place a dir.txt file on the server which will be listing all the files in that folder and e.g. run cron job to update it as needed.
jQuery will have no way to list remote files on the server unless your server supports it in one way or another.
Update
Once you have the file you need:
a) tokenize it into an array, e.g. like this
var names = dir.split("\n");
b) leave only strings starting with keyword and cut keyword off
names = $(names).map(function(n,i) {
return (n.indexOf(keyword) == 0) ? n.split('_')[1] : null;
}).get();
now you have an array like this ['11122012', '01022013', ...]
c) find max date in this array
var dateNum = Math.max.apply( null,
$.map(names,function(n,i){ return parseInt(
n.replace(/(\d{2})(\d{2})(\d{4})/, function(match, month, day, year) {
return year + month + day;
}))
}) );
var maxDate = dateNum.toString().replace(/(\d{4})(\d{2})(\d{2})/,
function (match, year, month, day) { return month + day + year; }
);
var fileName = keyword + "_" + maxDate;
voila, your fileName contains the name with max date.
There are other ways of doing it, e.g. really parsing the date into Date object. Also, you can simply iterate your files once, without array mapping and Math.max() iterator. As the amount of code wins over speed here, to find the optimal one depends on where you could re-use its bits and pieces without compromising maintainability.
http://jsfiddle.net/Exceeder/VLB2E/