Develop Reference

Issues with identifying and encoding urls

I'm having issues with parsing/manipulating URI:
Problem Statement:
I want to encode f[users.comma] and return it.
[Case-1] I get an url from backend service, encode f[users.comma] and return it.
[Case-2] I get an url from backend service and f[users.comma] is already encoded. So don't double encode and return it.
Expected Output:
`/demo/bigquery/order_items?fields=users.email&f[users.comma]=%22Abbeville%2C+Georgia%22`
Code:
const encodedExample = `/demo/bigquery/order_items?fields=users.email&f[users.comma]=%22Abbeville%2C+Georgia%22` // the last param is encoded
const regularExample2 = `/demo/bigquery/order_items?fields=users.email&f[users.comma]="Abbeville, Georgia"` //
const specialEncode = (url) => {
for (let queryParam of urlObj) {
const [urlKey, urlValue] = queryParam
// Check to see if url contains f[users.comma]
if (urlKey.includes('f[')) {
urlObj.set(urlKey, encodeURI(urlValue))
}
}
return urlObj.toString() // doesn't seem to work
}
I feel like I am going offroad with my approach. I'd appreciate some help here.

Since the backend service returns an encoded or decode url
We can first decode the url from the backend service (this won't produce any exceptions if url is already encoded)
const encodedExample = `/demo/bigquery/order_items?fields=users.email&f[users.comma]=%22Abbeville%2C+Georgia%22` // the last param is encoded
const regularExample2 = `/demo/bigquery/order_items?fields=users.email&f[users.comma]="Abbeville, Georgia"`
const specialEncode = (url) => {
let decodedUrl = decodeURI(url);
let encodedUrl = encodeURI(decodedUrl);
// fix "f[users.comma]" because encodeURI will encode the [ and ] as well
encodedUrl = encodedUrl.replace("f%5Busers.comma%5D", "f[users.comma]")
console.log(encodedUrl);
return encodedUrl;
}
specialEncode(encodedExample); // logs and returns: /demo/bigquery/order_items?fields=users.email&f[users.comma]=%22Abbeville%252C+Georgia%22
specialEncode(regularExample2); // logs and returns: /demo/bigquery/order_items?fields=users.email&f[users.comma]=%22Abbeville%252C+Georgia%22
The code above works fine for both encoded and decoded urls

Kraken API private request authentication {"error":["EAPI:Invalid key"]} - Google Script

I have been trying to communicate with the private API on kraken. The error I get suggests {"error":["EAPI:Invalid key"]} that the encryption/decryption steps are correct. I have tried creating new keys, does not help. I'm wondering if the 'format' of the signature variable is wrong, even though correct in nature.
function balance () {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("API_read_only");
var key = sheet.getRange("B5").getValue()
var secret = sheet.getRange("B6").getValue()
// (API method, nonce, and POST data)
var path = "/0/private/TradeBalance"
var nonce = new Date () * 1000
var postdata = "nonce=" + nonce
//Algorithms
//Calculate the SHA256 of the nonce and the POST data
// using goolge script lib
// using more succint function from https://stackoverflow.com/questions/16216868/get-back-a-string-representation-from-computedigestalgorithm-value-byte
function SHA_256 (str) {
return Utilities.computeDigest(Utilities.DigestAlgorithm.SHA_256, str).reduce(function(str,chr){
chr = (chr < 0 ? chr + 256 : chr).toString(16);
return str + (chr.length==1?'0':'') + chr;
},'');
}
var api_sha256 = SHA_256(nonce + postdata)
//Decode the API secret (the private part of the API key) from base64 // need to stringyfy
var base64 = Utilities.base64Decode(secret)
var base64s = Utilities.newBlob(base64).getDataAsString()
//Calculate the HMAC of the URI path and the SHA256, using SHA512 as the HMAC hash and the decoded API secret as the HMAC key
var hamc512_uri = Utilities.computeHmacSha256Signature(path + api_sha256,base64s)
var hamc512_uris = Utilities.newBlob(hamc512_uri).getDataAsString()
//Encode the HMAC into base64
var signature = Utilities.base64Encode(hamc512_uris)
Logger.log(signature)
//An example of the algorithm using the variables shown above is as follows:
//Base64Encode(HMAC-SHA512 of ("/0/private/TradeBalance" + SHA256("1540973848000nonce=1540973848000&asset=xxbt")) using Base64Decode("FRs+gtq09rR7OFtKj9BGhyOGS3u5vtY/EdiIBO9kD8NFtRX7w7LeJDSrX6cq1D8zmQmGkWFjksuhBvKOAWJohQ==") as the HMAC key
//The result is the API-Sign value / signature.
// connect
var url = "https://api.kraken.com" + path;
var options = {
method: 'post',
headers: {
'API-Key': key,
'API-Sign': signature
},
payload: postdata
};
var response = UrlFetchApp.fetch (url, options);
json = response.getContentText ();
Logger.log(json)
}

While I cannot spot what's wrong with your code I faced the same problem as well (thinking I have everything correct but getting a EAPI:Invalid key) with different libraries.
The approach that helped me was:
Take some posted working solution, e.g. https://stackoverflow.com/a/43081507/672008 (in Java)
Check that it really works
Fix the nonce parameter to get a stable HMAC end results
Massage my code until I get then same intermediate & end results
In the end I was successful using this library: https://www.npmjs.com/package/jssha
The code:
import jssha from 'jssha';
const secret = '...';
const nonce = 1642383717038;
const message = '';
const path = '/0/private/Balance';
const data = 'nonce=' + nonce;
const dataHash = new jssha('SHA-256', 'TEXT');
dataHash.update(nonce + data + message);
let utf8Encode = new TextEncoder();
const hmacHash = new jssha('SHA-512', 'UINT8ARRAY', { hmacKey: { value: secret, format: 'B64' } });
hmacHash.update(utf8Encode.encode(path));
hmacHash.update(dataHash.getHash('UINT8ARRAY'));
console.log('hmac', hmacHash.getHash('B64'));

How do I convert this JS signature generator function into Ruby?

I'am trying to integrate the Zoom SDK into my application and I am having trouble figuring out how to convert their example code for generating the signature into ruby.
Example Code -
const base64JS = require('js-base64');
const hmacSha256 = require('crypto-js/hmac-sha256');
const encBase64 = require('crypto-js/enc-base64');
function generateSignature(data) {
let signature = '';
const ts = new Date().getTime();
const msg = base64JS.Base64.encode(data.apiKey + data.meetingNumber + ts + data.role);
const hash = hmacSha256(msg, data.apiSecret);
signature = base64JS.Base64.encodeURI(`${data.apiKey}.${data.meetingNumber}.${ts}.${data.role}.${encBase64.stringify(hash)}`);
return signature;
}
const data = {apiKey: "" ,
apiSecret: "",
meetingNumber: 888,
role: 0}
console.log(generateSignature(data));
How would the generateSignature function look like in ruby?
I've tried a few times but the outputted signature differed when I tried writing this in Ruby. I suspect that I'am encoding and decoding improperly.
This is the javascript code above that I modified slightly to cross reference
const base64JS = require('js-base64');
const hmacSha256 = require('crypto-js/hmac-sha256');
const encBase64 = require('crypto-js/enc-base64');
function generateSignature(data) {
let signature = '';
const ts = "1569600658561"
const msg = base64JS.Base64.encode(data.apiKey + data.meetingNumber + ts + data.role);
console.log(msg); // This matches the ruby
const hash = hmacSha256(msg, data.apiSecret);
signature = base64JS.Base64.encodeURI(`${data.apiKey}.${data.meetingNumber}.${ts}.${data.role}.${encBase64.stringify(hash)}`);
return signature;
}
data = {
apiKey: 'api_key',
apiSecret: 'secret',
meetingNumber: '1000',
role: '0'
}
console.log(generateSignature(data));
This is my attempt in ruby
class ZoomSignatureGenerator
def self.generate
data = {
api_key: 'api_key',
api_secret: 'secret',
meeting_number: '1000',
role: '0'
}
ts = "1569600658561"
msg = Base64.encode64(data[:api_key] + data[:meeting_number] + ts + data[:role]);
puts(msg)
hash = OpenSSL::HMAC.hexdigest(OpenSSL::Digest.new('sha256'), data[:api_secret], msg)
signature = Base64.urlsafe_encode64("#{data[:api_key]}.#{data[:meeting_number]}.#{ts}.#{data[:role]}.#{Base64.encode64(hash)}");
return signature
end
end
I expected them to be the same output. But they end up being different.
Hope someone can help me :)

the output of Base64.encode64(value) is separated by a newline character, use Base64.strict_encode64(value) to generates the same result as Node.js.
require "base64"
require "openssl"
def generate_signature(api_key, api_secrete, meeting_number, role)
timestamp = 1544683367752
message = Base64.strict_encode64("#{api_key}#{meeting_number}#{timestamp}#{role}")
hash = Base64.strict_encode64(OpenSSL::HMAC.digest('sha256', api_secrete, message))
signature = Base64.strict_encode64("#{api_key}.#{meeting_number}.#{timestamp}.#{role}.#{hash}")
end
generate_signature("key", "secret", 123456, 0)

Encountered the same problem, apparently Zoom uses base 64 encoded binary string, so instead of using HMAC.hexdigest try this
hmac = OpenSSL::HMAC.new(API_KEY, 'sha256')
hmac << data
hash = Base64.encode64(hmac.digest)
digest method returns the authentication code as a binary string,
hexdigest returns the authentication code as a hex-encoded string

How to convert this signature method from crypto (node) to crypto-js (browser)?

I have a signature method that is meant to be used in Node.js but I'd like to implement it client-side with crypto-js. It should work in latest Chrome versions.
I have tried to follow some answers like this one: Decode a Base64 String using CryptoJS
But I either get errors such as "Error: Malformed UTF-8 data", or a different result than the expected hmacDigest.
I am not sure how I could find an alternative to the "binary" digest although I found this question:
How to get digest representation of CryptoJS.HmacSHA256 in JS
The method is supposed to answer the following:
"Message signature using HMAC-SHA512 of (URI path + SHA256(nonce + POST data)) and base64 decoded secret API key"
This is the Nodejs version (with crypto):
const crypto = require('crypto')
function sign(path, params, secret) {
const message = querystring.stringify(params)
const secretBase64 = Buffer.from(secret, 'base64')
const hash = crypto.createHash('sha256')
const hmac = crypto.createHmac('sha512', secretBase64)
const hashDigest = hash.update(params.nonce + message).digest('binary')
const hmacDigest = hmac.update(path + hashDigest, 'binary').digest('base64')
return hmacDigest
}
note: querystring is just an helper module that can also run in browsers: https://nodejs.org/api/querystring.html
This is my attempt (wip) at implementing with crypto-js:
import cryptojs from 'crypto-js')
function sign (path, params, secret) {
const message = querystring.stringify(params)
const secretParsed = cryptojs.enc.Base64.parse(secret)
const secretDecoded = cryptojs.enc.Utf8.stringify(secretParsed) // -> this throws an error as "Error: Malformed UTF-8 data"
const hash = cryptojs.SHA256(params.nonce + message).toString(cryptojs.enc.hex)
const hmac = cryptojs.HmacSHA512(path + hash, secretDecoded).toString(cryptojs.enc.hex)
return hmac
}

Try this ! I think this is what you looking for !
const crypto = require("crypto")
const sign = (path, params, secret) => {
const message = querystring.stringify(params)
const secret = Buffer.from(secret, 'base64')
let sign = params.nonce + message
let hash = crypto.createHmac('sha256', secret)
.update(sign)
.digest("base64").toString()
let encoded = encodeURIComponent(hash)
return encoded
}

Using JavaScript to properly sign a string using HmacSHA256

In the Houndify API Docs for Authentication, you have the following block of content:
An Example of Authenticating a Request
Let's assume we have the following information:
UserID: ae06fcd3-6447-4356-afaa-813aa4f2ba41
RequestID: 70aa7c25-c74f-48be-8ca8-cbf73627c05f
Timestamp: 1418068667
ClientID: KFvH6Rpy3tUimL-pCUFpPg==
ClientKey: KgMLuq-k1oCUv5bzTlKAJf_mGo0T07jTogbi6apcqLa114CCPH3rlK4c0RktY30xLEQ49MZ-C2bMyFOVQO4PyA==
Concatenate the UserID string, RequestID string, and TimeStamp string in the following format: {user_id};{request_id}{timestamp}
With the values from the example, the expected output would be in this case: ae06fcd3-6447-4356-afaa-813aa4f2ba41;70aa7c25-c74f-48be-8ca8-cbf73627c05f1418068667
Sign the message with the decoded ClientKey. The result is a 32-byte binary string (which we can’t represent visually). After base-64 encoding, however, the signature is: myWdEfHJ7AV8OP23v8pCH1PILL_gxH4uDOAXMi06akk=
The client then generates two authentication headers Hound-Request-Authentication and Hound-Client-Authentication.
The Hound-Request-Authentication header is composed by concatenating the UserID and RequestID in the following format: {user-id};{request-id}. Continuing the example above, the value for this header would be:
Hound-Request-Authentication: ae06fcd3-6447-4356-afaa-813aa4f2ba41;70aa7c25-c74f-48be-8ca8-cbf73627c05f
The Hound-Client-Authentication header is composed by concatening the ClientID, the TimeStamp string and the signature in the following format: {client-id};{timestamp};{signature}. Continuing the example above, the value for this header would be: Hound-Client-Authentication: KFvH6Rpy3tUimL-pCUFpPg==;1418068667;myWdEfHJ7AV8OP23v8pCH1PILL_gxH4uDOAXMi06akk=
For Number 3, it says "Sign the message with the decoded ClientKey". The "message" and "ClientKey" are two distinct strings.
My question(s): How do you sign one string with another string i.e. what exactly does that mean? And how would you do that in JavaScript?
var message = 'my_message';
var key = 'signing_key';
//??what next??
I'm trying to figure all this out so I can create a pre-request script in Postman to do a proper HmacSHA256 hash.

According to the documentation, if you're using one of their SDKs, it will automatically authenticate your requests:
SDKs already handle authentication for you. You just have to provide
the SDK with the Client ID and Client Key that was generated for your
client when it was created. If you are not using an SDK, use the code
example to the right to generate your own HTTP headers to authenticate
your request.
However, if you want to do it manually, I believe you need to compute the HMAC value of the string they describe in the link in your question and then send it base64 encoded as part of the Hound-Client-Authentication header in your requests. They provide an example for node.js:
var uuid = require('node-uuid');
var crypto = require('crypto');
function generateAuthHeaders (clientId, clientKey, userId, requestId) {
if (!clientId || !clientKey) {
throw new Error('Must provide a Client ID and a Client Key');
}
// Generate a unique UserId and RequestId.
userId = userId || uuid.v1();
// keep track of this requestId, you will need it for the RequestInfo Object
requestId = requestId || uuid.v1();
var requestData = userId + ';' + requestId;
// keep track of this timestamp, you will need it for the RequestInfo Object
var timestamp = Math.floor(Date.now() / 1000),
unescapeBase64Url = function (key) {
return key.replace(/-/g, '+').replace(/_/g, '/');
},
escapeBase64Url = function (key) {
return key.replace(/\+/g, '-').replace(/\//g, '_');
},
signKey = function (clientKey, message) {
var key = new Buffer(unescapeBase64Url(clientKey), 'base64');
var hash = crypto.createHmac('sha256', key).update(message).digest('base64');
return escapeBase64Url(hash);
},
encodedData = signKey(clientKey, requestData + timestamp),
headers = {
'Hound-Request-Authentication': requestData,
'Hound-Client-Authentication': clientId + ';' + timestamp + ';' + encodedData
};
return headers;
};

So basically, signing [in this specific case] simply means to create a hash of the string using a hash algorithm in addition to a key, as opposed to a keyless hash [like MD5]. For example:
var message = 'my_message';
var key = 'signing_key';
var hashed_message = hash_func(message, key);
where hash_func is a hashing algorithm like HmacSHA256 (the hashing algorithm in question).
The reason I was trying to figure this out was to test authentication for the Houndify API using Postman. Fortunately, Postman has a nice feature called pre-request scripts [complete with hashing algorithms] that helps if you need to pre-generate values that need to be sent along with your request.
After much fiddling around, I managed to create a pre-request script that properly authenticates to this API (see code below).
var unescapeBase64Url = function (key) {
return key.replace(/-/g, '+').replace(/_/g, '/');
},
escapeBase64Url = function (key) {
return key.replace(/\+/g, '-').replace(/\//g, '_');
},
guid = function() {
function s4() {
return Math.floor((1 + Math.random()) * 0x10000)
.toString(16)
.substring(1);
}
return s4() + s4() + '-' + s4() + '-' + s4() + '-' +
s4() + '-' + s4() + s4() + s4();
};
var client_id_str = environment["client-id"];
var client_key_str = environment["client-key"];
var user_id_str = environment["user-id"];
var request_id_str = guid();
var timestamp_str = Math.floor(Date.now() / 1000);
var client_key_dec_str = CryptoJS.enc.Base64.parse(unescapeBase64Url(client_key_str));
var message_str = user_id_str+";"+request_id_str+timestamp_str;
var signature_hash_obj = CryptoJS.HmacSHA256(message_str, client_key_dec_str);
var signature_str = signature_hash_obj.toString(CryptoJS.enc.Base64);
var hound_request_str = user_id_str+";"+request_id_str;
var hound_client_str = client_id_str+";"+timestamp_str+";"+escapeBase64Url(signature_str);
postman.setEnvironmentVariable("hound-request-authentication", hound_request_str);
postman.setEnvironmentVariable("hound-client-authentication", hound_client_str);
Note that you are going to have to create environment variables in Postman for client-id, client-key, and user-id, as well as for the header variables hound-request-authentication and hound-client-authentication to hold the final values that will be referenced when defining headers.
Hope it helps.