Yes, there is 100 topics about it.
Yes most of them are snippets/parts of code that answer the specific problem and don't actually help.
So perhaps this topic would help provide a "complete" solution for symmetric and maybe if some1 would be willing to help with asymmetric private/public key example.
So here are pre-reqs
javascript:
npm intall crypto
c++
https://github.com/QuasarApp/Qt-AES/tree/master
and Qt
Now In order to do encryption the tutorial on this page works quite well >
Example 2 > https://www.geeksforgeeks.org/node-js-crypto-createdecipheriv-method/?ref=lbp
Now as far as I can tell, say we create our Key - password :
const password = 'mySuperFancyPassword';
// Defining key
export const key = crypto.scryptSync(password, 'salt', 32);
This password is not the same as the one we would made in C++ using >
QAESEncryption encryption(QAESEncryption::AES_256, QAESEncryption::CBC,QAESEncryption::PKCS7);
QString key("mySuperFancyPassword");
QByteArray hashKey = QCryptographicHash::hash(key.toLocal8Bit(), QCryptographicHash::Sha256);
QByteArray decodeText = encryption.decode(jsByteArray, hashKey , jsIv);
Because Qt-AES takes Hash rather than whatever crypto.scryptSync() produces.
I suppose the question is... how can I match these 2 passwords?
If I were to pass javascript key-hex to C++ and convert it to byte array (auto key = QByteArray::fromHex(hexByte)) C++ library will decompile the string properly and with PKCS7 padding it will match javascript.
Now I know that I should use OpenSSL as that is standard, but every time I look at it I want to cry.
So this library here seems to be very dummy friendly so far...
However, if any1 is interested in openSSL, there is this interesting "file" > https://github.com/soroush/qtz-security/blob/dev/lib/src/crypto.cpp
That shows how to do it OpenSSL but I get error 0 in
error_code = EVP_DecryptFinal_ex(ctx, plaintext + len, &len);
indecryptRawData(const QByteArray& input, const QByteArray& rawKey, const QByteArray& rawIV)
So same issue, black magic! I did match my EVP_aes_256_cbc settings between JS and C++ in second library.
Bottom line, can any1 help me convert the KEY to properly match between C++ and javascript?
Or help with second lib openSSL? But I take its the same issue of salt/key generation...
UPDATE!
Big thanks to #absolute.madness for his solution!
Also, I found another way of... "partially" solving the problem.
I found out that crypto has PKCS5_PBKDF2_HMAC support too! So here is a proposed workflow for that one, however even tho I can send from Javascript > C++, I can't send C++ > Javascript using the QAESEncryption library due to (I think) incorrect padding...? As I crash at
decrypted = Buffer.concat([decrypted, decipher.final()]); .final() statement I think.
Here is Javascript & C++ code that I got working up to 50%.
JS:
// Defining password
const password: string = process.env.KEY_LICENSE_GENERIC! as string
// Defining key
var key: Buffer
crypto.pbkdf2(password, 'salt_', 10000, 32,
'sha256', (err, derivedKey) => {
if (err) {
throw new Error();
}
key = derivedKey
})
const iv = crypto.randomBytes(16);
export function encrypt2(text: string) {
// Creating Cipheriv with its parameter
let cipher = crypto.createCipheriv('aes-256-cbc', Buffer.from(key), iv);
// Updating text
let encrypted = cipher.update(text);
// Using concatenation iv + encrypt + enging & padding?
encrypted = Buffer.concat([iv, encrypted, cipher.final()]);
return encrypted.toString('hex')
}
// A decrypt function
export function decrypt2(text: string) {
let rawData = Buffer.from(text, 'hex');
if (rawData.length > 16) {
let iv = rawData.subarray(0, 16) // We put IV as 1st 16 bytes.
let encr = rawData.subarray(16, rawData.length)
// Creating Decipher
let decipher = crypto.createDecipheriv(
'aes-256-cbc', Buffer.from(key), iv);
// Updating encrypted text
let decrypted = decipher.update(encr);
decrypted = Buffer.concat([decrypted, decipher.final()]);
return decrypted.toString()
}
return ""
}
c++
#include <openssl/rand.h>
#include <openssl/hmac.h>
#include <openssl/evp.h>
QByteArray generateKey(const QByteArray &phrase, bool encode, const int iterations) {
const int length = 32;
QByteArray salt("salt_");
unsigned char key[length];
PKCS5_PBKDF2_HMAC(
phrase.data(), phrase.size(),
(const unsigned char *) (salt.data()), salt.size(),
iterations, EVP_sha256(),
length, key
);
return encode ? QByteArray((const char *) (key), length).toBase64(QByteArray::Base64UrlEncoding) : QByteArray((const char *) (key), length);
}
QByteArray randomBytes(int size) {
QByteArray bytes(size, char(0));
if (RAND_bytes((unsigned char *) (bytes.data()), bytes.size()) != 1) {
QRandomGenerator::securelySeeded().fillRange((quint32 *) (bytes.data()), bytes.size() / sizeof(quint32));
}
return bytes;
}
void decrypt(){
QByteArray hexEnc = reply.readAll(); // QNetworkReply*
QByteArray enc = QByteArray::fromHex(hexEnc.toUtf8());
auto iv = enc.mid(0, 16);
enc = enc.mid(16, enc.size());
QAESEncryption encryption(QAESEncryption::AES_256,
QAESEncryption::CBC,QAESEncryption::PKCS7);
QByteArray decodeText = encryption.decode(enc, generateKey("Fancy
password", false, 10000), iv);
/// Remove padding, I think this is missing when we encrypt.
QString decodedString = QString(encryption.removePadding(decodeText ));
}
void encrypt(){
auto iv = randomBytes(16);
auto encrypted = encryption.encode("Hello test code",
generateKey("Fancy password", false, 10000), iv); // bad encrypt, js will crash.
}
You cannot just use SHA-256 to match scrypt key derivation algorithm, obviously. Scrypt is defined in RFC 7914 and it's not (as of yet) implemented in Qt via its interfaces. OpenSSL (used by Qt) supports it on the other hand. I added 2 implementations of the Node.js example1 which you reference: the first one uses OpenSSL & Qt-AES, the second uses pure OpenSSL. Initially, I got an error from EVP_DecryptFinal_ex similar to what you described. When I started to debug it turned out that EVP_DecodeBlock was returning incorrect size when decoding from base64. After using EVP_DecodeInit/EVP_DecodeUpdate/EVP_DecodeFinal to handle base64 instead of EVP_DecodeBlock as was suggested here the error was gone.
I include the c++ code which roughly translates js-code from example 1 to c++ (I used OpenSSL 1.1.1q for testing):
#include <QDebug>
#include <openssl/aes.h>
#include <openssl/evp.h>
#include <openssl/kdf.h>
#include "qaesencryption.h"
void error(const char *msg)
{
qCritical(msg);
}
#define ERROR(msg) \
{ \
qCritical(msg); \
return; \
}
// scrypt key derivation function/algorithm, see also
// https://www.openssl.org/docs/man1.1.1/man7/scrypt.html
// returns -1 on error and 1 on success
int scrypt_kdf(unsigned char *key, size_t *keylen,
const unsigned char *pass, size_t passlen,
const unsigned char *salt, size_t saltlen,
uint64_t N = 16384, uint64_t r = 8, uint64_t p = 1)
{
// Note, default values for N, r, p are taken from
// https://nodejs.org/api/crypto.html#cryptoscryptsyncpassword-salt-keylen-options
EVP_PKEY_CTX *kctx;
int ret = 1;
kctx = EVP_PKEY_CTX_new_id(EVP_PKEY_SCRYPT, NULL);
if(EVP_PKEY_derive_init(kctx) <= 0)
{
error("EVP_PKEY_derive_init failed");
ret = -1;
}
if(1 == ret && EVP_PKEY_CTX_set1_pbe_pass(kctx, pass, passlen) <= 0)
{
error("EVP_PKEY_CTX_set1_pbe_pass failed");
ret = -1;
}
if(1 == ret && EVP_PKEY_CTX_set1_scrypt_salt(kctx, salt, saltlen) <= 0)
{
error("EVP_PKEY_CTX_set1_scrypt_salt failed");
ret = -1;
}
if(1 == ret && EVP_PKEY_CTX_set_scrypt_N(kctx, N) <= 0)
{
error("EVP_PKEY_CTX_set_scrypt_N failed");
ret = -1;
}
if (1 == ret && EVP_PKEY_CTX_set_scrypt_r(kctx, 8) <= 0)
{
error("EVP_PKEY_CTX_set_scrypt_r failed");
ret = -1;
}
if (1 == ret && EVP_PKEY_CTX_set_scrypt_p(kctx, 1) <= 0)
{
error("EVP_PKEY_CTX_set_scrypt_p failed");
ret = -1;
}
if (1 == ret && EVP_PKEY_derive(kctx, key, keylen) <= 0)
{
error("EVP_PKEY_derive failed");
ret = -1;
}
EVP_PKEY_CTX_free(kctx);
return ret;
}
// we use OpenSSL for scrypt key derivation algorithm and Qt/Qt-AES for decryption
void example1_openssl_and_qt(void)
{
unsigned char key[24];
size_t sz_key = sizeof(key);
const char password[] = "bncaskdbvasbvlaslslasfhj";
const char salt[] = "GfG";
QByteArray iv(16, char(0));
QByteArray encrypted = QByteArray::fromBase64("MfHwhG/WPv+TIbG/qM78qA==");
// you can also try
// encrypted = QByteArray::fromBase64(
// "j9QsjAFxuIAK0zvi5Iq2Z2+mo44RRpR2VMnJTNS7Ey0IkPjsGSJ+A+OPuvAqGO77Ww"
// "S2rI0dnJVREkFz0v8hug==");
if(scrypt_kdf(
key, &sz_key, reinterpret_cast<const unsigned char*>(password),
sizeof(password)-1, reinterpret_cast<const unsigned char*>(salt),
sizeof(salt)-1) <= 0)
{
ERROR("Key derivation failed");
}
OPENSSL_assert(sz_key == sizeof(key));
QAESEncryption encryption(QAESEncryption::AES_192, QAESEncryption::CBC,
QAESEncryption::PKCS7);
QByteArray decrypted = encryption.decode(
encrypted, QByteArray(reinterpret_cast<char*>(key), sizeof(key)), iv);
qDebug() << decrypted;
}
// we use qt only for base64 decoding
void example1_pure_openssl(void)
{
int len; // general purpose length variable, used in EVP_*Update/EVP_*Final
EVP_ENCODE_CTX *b64ctx;
unsigned char key[24];
size_t sz_key = sizeof(key);
EVP_CIPHER_CTX *dctx;
const char password[] = "bncaskdbvasbvlaslslasfhj";
const char salt[] = "GfG";
unsigned char iv[16] = { 0 }; // 16 zero bytes
char encrypted_b64[] = "MfHwhG/WPv+TIbG/qM78qA==";
// you can also try
// char encrypted_b64[] = "j9QsjAFxuIAK0zvi5Iq2Z2+mo44RRpR2VMnJTNS7Ey0IkPjsG"
// "SJ+A+OPuvAqGO77WwS2rI0dnJVREkFz0v8hug==";
// Note, base64 encoding is supposed to be b64size = (size + 2) / 3 * 4
// characters long, where size is the size of the encoded string, therefore
// the following assert checks that the size is correct and thus the size
// of the maximum decoded string size can be calculated as
// max_size = 3 * b64size / 4
// https://stackoverflow.com/questions/13378815/base64-length-calculation
OPENSSL_assert((sizeof(encrypted_b64) - 1) % 4 == 0);
unsigned char encrypted[3 * (sizeof(encrypted_b64) - 1) / 4];
unsigned char decrypted[sizeof(encrypted) + 1]; // +1 for terminating 0
int sz_decoded, sz_decrypted;
// Note, do not use EVP_DecodeBlock for decoding from base64 as it returns
// wrong decoded length and ignores padding, see
// https://github.com/openssl/openssl/issues/17197
b64ctx = EVP_ENCODE_CTX_new();
EVP_DecodeInit(b64ctx);
if(EVP_DecodeUpdate(b64ctx, encrypted, &sz_decoded,
(const unsigned char*)encrypted_b64,
sizeof (encrypted_b64) - 1) < 0)
{
EVP_ENCODE_CTX_free(b64ctx);
ERROR("EVP_DecodeUpdate failed");
}
if(EVP_DecodeFinal(b64ctx, encrypted + sz_decoded, &len) <= 0)
{
EVP_ENCODE_CTX_free(b64ctx);
ERROR("EVP_DecodeFinal failed");
}
sz_decoded += len;
EVP_ENCODE_CTX_free(b64ctx);
OPENSSL_assert(sz_decoded <= sizeof(encrypted));
if(scrypt_kdf(
key, &sz_key, (const unsigned char*)password, sizeof(password)-1,
(const unsigned char*)salt, sizeof(salt)-1) <= 0)
{
ERROR("Key derivation failed");
}
OPENSSL_assert(sz_key == sizeof(key));
dctx = EVP_CIPHER_CTX_new();
if (EVP_DecryptInit_ex(dctx, EVP_aes_192_cbc(), NULL, key, iv) <= 0)
{
EVP_CIPHER_CTX_free(dctx);
ERROR("EVP_DecryptInit_ex failed");
}
if(EVP_CIPHER_CTX_set_key_length(dctx, 24) <= 0)
{
EVP_CIPHER_CTX_free(dctx);
ERROR("EVP_CIPHER_CTX_set_key_length failed");
}
if(EVP_DecryptUpdate(dctx, decrypted, &sz_decrypted,
encrypted, sz_decoded) <= 0)
{
EVP_CIPHER_CTX_free(dctx);
ERROR("EVP_DecryptUpdate failed");
}
if(EVP_DecryptFinal_ex(dctx, decrypted + sz_decrypted, &len) <= 0)
{
EVP_CIPHER_CTX_free(dctx);
ERROR("EVP_DecryptFinal_ex failed");
}
EVP_CIPHER_CTX_free(dctx);
sz_decrypted += len;
// do not forget the null terminator
decrypted[sz_decrypted] = 0;
qDebug() << (const char*)decrypted;
}
int main(void)
{
qDebug() << "example1_openssl_and_qt decryption:";
example1_openssl_and_qt();
qDebug() << "example1_pure_openssl decryption:";
example1_pure_openssl();
return 0;
}
I also attach the code I used to generate the additional encrypted data:
const crypto = require('crypto');
const algorithm = 'aes-192-cbc';
const password = 'bncaskdbvasbvlaslslasfhj';
const plaintext = 'Lorem ipsum dolor sit amet, consectetur adipiscing';
const key = crypto.scryptSync(password, 'GfG', 24);
const iv = Buffer.alloc(16, 0);
const cipher = crypto.createCipheriv(algorithm, key, iv);
const encrypted = Buffer.concat([cipher.update(plaintext), cipher.final()]);
console.log(encrypted.toString('base64'));
UPD
C++
void pbkdf2withsha256_pure_openssl(void)
{
int len; // general purpose length variable, used in EVP_*Update/EVP_*Final
EVP_ENCODE_CTX *b64ctx;
const int sz_key = 32;
unsigned char key[sz_key];
// Note, base64 encoding size is supposed to be b64size = (size + 2) / 3 * 4
// characters long, where size is the size of the source string
// https://stackoverflow.com/questions/13378815/base64-length-calculation
unsigned char key_b64[(sz_key + 2) / 3 * 4 + 1];
int sz_key_b64;
const char password[] = "myPassw0rd";
const unsigned char salt[] = "mySalt";
if(PKCS5_PBKDF2_HMAC(password, sizeof(password) - 1, salt, sizeof(salt) - 1,
10000, EVP_sha256(), sz_key, key) < 1)
{
ERROR("PKCS5_PBKDF2_HMAC failed");
}
b64ctx = EVP_ENCODE_CTX_new();
EVP_EncodeInit(b64ctx);
if(EVP_EncodeUpdate(b64ctx, key_b64, &sz_key_b64, key, sz_key) < 0)
{
EVP_ENCODE_CTX_free(b64ctx);
ERROR("EVP_DecodeUpdate failed");
}
EVP_EncodeFinal(b64ctx, key_b64 + sz_key_b64, &len);
sz_key_b64 += len;
EVP_ENCODE_CTX_free(b64ctx);
qDebug() << (const char*)key_b64;
}
JS
crypto = require ('crypto');
crypto.pbkdf2('myPassw0rd', 'mySalt', 10000, 32,
'sha256', (err, key) => {
if (err) throw new Error();
console.log(key.toString('base64'))
})
I'm writing code to read/write to NTAG424 DNA NFC tags. I'm doing this completely in javascript, because I want to be able to use it in a react native app.
In NTAG 424 DNA and NTAG 424 DNA TagTamper features and hints they show the results you should be getting for every step. But they use a python solution.
The input message is A55A0001008013C56268A548D8FBBF237CCCAA20EC7E6E48C3DEF9A4C675360F and the output (according to the manual) is 1309C877509E5A215007FF0ED19CA564. Whereas I get 7CCEF6FEB32F34CA48CB685ECAA0F32C.
Because I need to be able to use this code commercially, I cannot just use any library.
function generateSubkeys(key) {
const cipher = crypto.createCipheriv("aes-128-ecb", key, "");
const cryptedKey = cipher.update(iv);
let subkey1 = bt.bitShiftLeft(cryptedKey);
if (msb(cryptedKey[0]) & 0x80) {
subkey1 = xor(subkey1, 0x87);
}
let subkey2 = bt.bitShiftLeft(subkey1);
if (msb(subkey1[0]) & 0x80) {
subkey2 = xor(subkey2, 0x87);
}
return { subkey1: subkey1, subkey2: subkey2 };
}
function msb(bytes) {
return bytes >>> 31;
}
function aes(key, message) {
const cipher = crypto.createCipheriv(
"aes-" + key.length * 8 + "-cbc",
key,
iv
);
var result = cipher.update(message);
cipher.final();
return result;
}
function aesCmac(key, message) {
const { subkey1, subkey2 } = generateSubkeys(Buffer.from(key, "hex"));
let numBlocks = Math.ceil(message.length / blockSize);
var lastBlockRemainder = message.length % blockSize;
if (numBlocks === 0) {
numBlocks = 1;
}
var messageArray = getMessageArray(message, numBlocks, lastBlockRemainder);
if (lastBlockRemainder === 0) {
messageArray[numBlocks - 1] = xor(messageArray[numBlocks - 1], subkey1);
} else {
messageArray[numBlocks - 1] = xor(messageArray[numBlocks - 1], subkey2);
}
var c = aes(
key,
Buffer.concat(messageArray.slice(0, messageArray.length - 1))
);
let c_xor_m = xor(c, messageArray[messageArray.length - 1]);
c = aes(key, c_xor_m);
return c;
}
function getMessageArray(message, numBlocks, lastBlockRemainder) {
var index = 0;
var messageArray = [];
if (lastBlockRemainder !== 0) {
let padding = "80" + "00".repeat(16 - lastBlockRemainder - 1);
let appendToMessage = Buffer.from(padding, "hex");
message = Buffer.concat([message, appendToMessage]);
}
for (index = 0; index < numBlocks; index++) {
let messageBlock = message.slice(
index * blockSize,
(index + 1) * blockSize
);
messageArray.push(messageBlock);
}
return messageArray;
}
I already tried the one mentioned here AES-CMAC module for Node.js? and completely rewriting the code to my own version of an AES-CMAC algorithm. In both the one I tried and the one I made (with the help of NIST Special Publication 800-38B), I get the same results.
Now I'm stuck between thinking either my code is wrong, or the python crypto library (where I don't completely understand the code) is wrong.
Can anyone help me figure out which of the two is true? And in case my code is wrong, help me fix it.
I found the answer: The Crypto library in javascript has an aes-cbc cipher, that says (and it does) accepts buffers and arrays. But the outcomes of both are different.
When I used a UInt8Array I got the right outcome. I completely rewrote an aes-cmac algorithm, just to figure out this what all I needed.
I need to convert strings to some form of hash. Is this possible in JavaScript?
I'm not utilizing a server-side language so I can't do it that way.
String.prototype.hashCode = function() {
var hash = 0,
i, chr;
if (this.length === 0) return hash;
for (i = 0; i < this.length; i++) {
chr = this.charCodeAt(i);
hash = ((hash << 5) - hash) + chr;
hash |= 0; // Convert to 32bit integer
}
return hash;
}
const str = 'revenue'
console.log(str, str.hashCode())
Source
Many of the answers here are the same String.hashCode hash function taken from Java. It dates back to 1981 from Gosling Emacs, is extremely weak, and makes zero sense performance-wise in modern JavaScript. In fact, implementations could be significantly faster by using ES6 Math.imul, but no one took notice. We can do much better than this, at essentially identical performance.
Here's one I did—cyrb53, a simple but high quality 53-bit hash. It's quite fast, provides very good* hash distribution, and because it outputs 53 bits, has significantly lower collision rates compared to any 32-bit hash. Also, you can ignore SA's CC license as it's public domain on my GitHub.
const cyrb53 = (str, seed = 0) => {
let h1 = 0xdeadbeef ^ seed,
h2 = 0x41c6ce57 ^ seed;
for (let i = 0, ch; i < str.length; i++) {
ch = str.charCodeAt(i);
h1 = Math.imul(h1 ^ ch, 2654435761);
h2 = Math.imul(h2 ^ ch, 1597334677);
}
h1 = Math.imul(h1 ^ (h1 >>> 16), 2246822507) ^ Math.imul(h2 ^ (h2 >>> 13), 3266489909);
h2 = Math.imul(h2 ^ (h2 >>> 16), 2246822507) ^ Math.imul(h1 ^ (h1 >>> 13), 3266489909);
return 4294967296 * (2097151 & h2) + (h1 >>> 0);
};
console.log(`cyrb53('a') -> ${cyrb53('a')}`)
console.log(`cyrb53('b') -> ${cyrb53('b')}`)
console.log(`cyrb53('revenge') -> ${cyrb53('revenge')}`)
console.log(`cyrb53('revenue') -> ${cyrb53('revenue')}`)
console.log(`cyrb53('revenue', 1) -> ${cyrb53('revenue', 1)}`)
console.log(`cyrb53('revenue', 2) -> ${cyrb53('revenue', 2)}`)
console.log(`cyrb53('revenue', 3) -> ${cyrb53('revenue', 3)}`)
*It is roughly similar to the well-known MurmurHash/xxHash algorithms. It uses a combination of multiplication and Xorshift to generate the hash, but not as thorough. As a result it's faster than either would be in JavaScript and significantly simpler to implement, but may not pass all tests in SMHasher. This is not a cryptographic hash function, so don't use this for security purposes.
Like any proper hash, it has an avalanche effect, which basically means small changes in the input have big changes in the output making the resulting hash appear more 'random':
"501c2ba782c97901" = cyrb53("a")
"459eda5bc254d2bf" = cyrb53("b")
"fbce64cc3b748385" = cyrb53("revenge")
"fb1d85148d13f93a" = cyrb53("revenue")
You can optionally supply a seed (unsigned integer, 32-bit max) for alternate streams of the same input:
"76fee5e6598ccd5c" = cyrb53("revenue", 1)
"1f672e2831253862" = cyrb53("revenue", 2)
"2b10de31708e6ab7" = cyrb53("revenue", 3)
Technically, it is a 64-bit hash, that is, two uncorrelated 32-bit hashes computed in parallel, but JavaScript is limited to 53-bit integers. If convenient, the full 64-bit output can be used by altering the return statement with a hex string or array.
return [h2>>>0, h1>>>0];
// or
return (h2>>>0).toString(16).padStart(8,0)+(h1>>>0).toString(16).padStart(8,0);
// or
return 4294967296n * BigInt(h2) + BigInt(h1);
Be aware that constructing hex strings drastically slows down batch processing. The array is much more efficient, but obviously requires two checks instead of one. I also included BigInt, which should be slightly faster than String, but still much slower than Array or Number.
Just for fun, here's TinySimpleHash, the smallest hash I could come up with that's still decent. It's a 32-bit hash in 89 chars with better quality randomness than even FNV or DJB2:
TSH=s=>{for(var i=0,h=9;i<s.length;)h=Math.imul(h^s.charCodeAt(i++),9**9);return h^h>>>9}
EDIT
based on my jsperf tests, the accepted answer is actually faster: http://jsperf.com/hashcodelordvlad
ORIGINAL
if anyone is interested, here is an improved ( faster ) version, which will fail on older browsers who lack the reduce array function.
hashCode = function(s) {
return s.split("").reduce(function(a, b) {
a = ((a << 5) - a) + b.charCodeAt(0);
return a & a;
}, 0);
}
// testing
console.log(hashCode("hello."));
console.log(hashCode("this is a text."));
console.log(hashCode("Despacito by Luis Fonsi"));
one-liner arrow function version :
hashCode = s => s.split('').reduce((a,b)=>{a=((a<<5)-a)+b.charCodeAt(0);return a&a},0)
// testing
console.log(hashCode("hello."));
console.log(hashCode("this is a text."));
console.log(hashCode("Despacito by Luis Fonsi"));
Note: Even with the best 32-bit hash, collisions will occur sooner or later.
The hash collision probability can be calculated as
,
approximated as
(see here).
This may be higher than intuition suggests:
Assuming a 32-bit hash and k=10,000 items, a collision will occur with a probability of 1.2%.
For 77,163 samples the probability becomes 50%!
(calculator).
I suggest a workaround at the bottom.
In an answer to this question
Which hashing algorithm is best for uniqueness and speed?,
Ian Boyd posted a good in depth analysis.
In short (as I interpret it), he comes to the conclusion that MurmurHash is best, followed by FNV-1a.
Java’s String.hashCode() algorithm that esmiralha proposed seems to be a variant of DJB2.
FNV-1a has a a better distribution than DJB2, but is slower
DJB2 is faster than FNV-1a, but tends to yield more collisions
MurmurHash3 is better and faster than DJB2 and FNV-1a (but the optimized implementation requires more lines of code than FNV and DJB2)
Some benchmarks with large input strings here: http://jsperf.com/32-bit-hash
When short input strings are hashed, murmur's performance drops, relative to DJ2B and FNV-1a: http://jsperf.com/32-bit-hash/3
So in general I would recommend murmur3.
See here for a JavaScript implementation:
https://github.com/garycourt/murmurhash-js
If input strings are short and performance is more important than distribution quality, use DJB2 (as proposed by the accepted answer by esmiralha).
If quality and small code size are more important than speed, I use this implementation of FNV-1a (based on this code).
/**
* Calculate a 32 bit FNV-1a hash
* Found here: https://gist.github.com/vaiorabbit/5657561
* Ref.: http://isthe.com/chongo/tech/comp/fnv/
*
* #param {string} str the input value
* #param {boolean} [asString=false] set to true to return the hash value as
* 8-digit hex string instead of an integer
* #param {integer} [seed] optionally pass the hash of the previous chunk
* #returns {integer | string}
*/
function hashFnv32a(str, asString, seed) {
/*jshint bitwise:false */
var i, l,
hval = (seed === undefined) ? 0x811c9dc5 : seed;
for (i = 0, l = str.length; i < l; i++) {
hval ^= str.charCodeAt(i);
hval += (hval << 1) + (hval << 4) + (hval << 7) + (hval << 8) + (hval << 24);
}
if( asString ){
// Convert to 8 digit hex string
return ("0000000" + (hval >>> 0).toString(16)).substr(-8);
}
return hval >>> 0;
}
Improve Collision Probability
As explained here, we can extend the hash bit size using this trick:
function hash64(str) {
var h1 = hash32(str); // returns 32 bit (as 8 byte hex string)
return h1 + hash32(h1 + str); // 64 bit (as 16 byte hex string)
}
Use it with care and don't expect too much though.
Based on accepted answer in ES6. Smaller, maintainable and works in modern browsers.
function hashCode(str) {
return str.split('').reduce((prevHash, currVal) =>
(((prevHash << 5) - prevHash) + currVal.charCodeAt(0))|0, 0);
}
// Test
console.log("hashCode(\"Hello!\"): ", hashCode('Hello!'));
EDIT (2019-11-04):
one-liner arrow function version :
const hashCode = s => s.split('').reduce((a,b) => (((a << 5) - a) + b.charCodeAt(0))|0, 0)
// test
console.log(hashCode('Hello!'))
I'm a bit surprised nobody has talked about the new SubtleCrypto API yet.
To get an hash from a string, you can use the subtle.digest method :
function getHash(str, algo = "SHA-256") {
let strBuf = new TextEncoder().encode(str);
return crypto.subtle.digest(algo, strBuf)
.then(hash => {
window.hash = hash;
// here hash is an arrayBuffer,
// so we'll connvert it to its hex version
let result = '';
const view = new DataView(hash);
for (let i = 0; i < hash.byteLength; i += 4) {
result += ('00000000' + view.getUint32(i).toString(16)).slice(-8);
}
return result;
});
}
getHash('hello world')
.then(hash => {
console.log(hash);
});
This is a refined and better performing variant, and matches Java's implementation of the standard object.hashCode() for CharSequence.
String.prototype.hashCode = function() {
var hash = 0, i = 0, len = this.length;
while ( i < len ) {
hash = ((hash << 5) - hash + this.charCodeAt(i++)) << 0;
}
return hash;
};
Here is also one that returns only positive hashcodes:
String.prototype.hashcode = function() {
return this.hashCode()+ 2147483647 + 1;
};
And here is a matching one for Java that only returns positive hashcodes:
public static long hashcode(Object obj) {
return ((long) obj.hashCode()) + Integer.MAX_VALUE + 1l;
}
Without prototype for those that do not want to attach it to String :
function hashCode(str) {
var hash = 0, i = 0, len = str.length;
while ( i < len ) {
hash = ((hash << 5) - hash + str.charCodeAt(i++)) << 0;
}
return hash;
}
function hashcode(str) {
hashCode(str) + 2147483647 + 1;
}
Enjoy!
If it helps anyone, I combined the top two answers into an older-browser-tolerant version, which uses the fast version if reduce is available and falls back to esmiralha's solution if it's not.
/**
* #see http://stackoverflow.com/q/7616461/940217
* #return {number}
*/
String.prototype.hashCode = function(){
if (Array.prototype.reduce){
return this.split("").reduce(function(a,b){a=((a<<5)-a)+b.charCodeAt(0);return a&a},0);
}
var hash = 0;
if (this.length === 0) return hash;
for (var i = 0; i < this.length; i++) {
var character = this.charCodeAt(i);
hash = ((hash<<5)-hash)+character;
hash = hash & hash; // Convert to 32bit integer
}
return hash;
}
Usage is like:
var hash = "some string to be hashed".hashCode();
UUID v3 and UUID v5 actually are hashes for a given input string.
UUID v3 is based on MD5,
UUID v5 is based on SHA-1.
So, the most obvious choice would be to go for UUID v5.
Fortunately, there is a popular npm package, which includes all UUID algorithms.
npm install uuid
To actually generate a UUID v5, you need a unique namespace. This namespace acts like a seed, and should be a constant, to assure that for a given input the output will always be the same. Ironically, you should generate a UUID v4 as a namespace. And the easiest way to do so, is using some online tool.
Once you've got a namespace, you're all set.
import { v5 as uuidv5 } from 'uuid';
const MY_NAMESPACE = '1b671a64-40d5-491e-99b0-da01ff1f3341';
const hash = uuidv5('input', MY_NAMESPACE);
If your input string will always be an URL for instance, then there are some default namespaces which you can use.
const hashForURL = uuidv5('https://www.w3.org/', uuidv5.URL);
Here is a compact ES6 friendly readable snippet
const stringHashCode = str => {
let hash = 0
for (let i = 0; i < str.length; ++i)
hash = Math.imul(31, hash) + str.charCodeAt(i)
return hash | 0
}
I'm kinda late to the party, but you can use this module: crypto:
const crypto = require('crypto');
const SALT = '$ome$alt';
function generateHash(pass) {
return crypto.createHmac('sha256', SALT)
.update(pass)
.digest('hex');
}
The result of this function is always is 64 characters string; something like this: "aa54e7563b1964037849528e7ba068eb7767b1fab74a8d80fe300828b996714a"
My quick (very long) one liner based on FNV's Multiply+Xor method:
my_string.split('').map(v=>v.charCodeAt(0)).reduce((a,v)=>a+((a<<7)+(a<<3))^v).toString(16);
Thanks to the example by mar10, I found a way to get the same results in C# AND Javascript for an FNV-1a. If unicode chars are present, the upper portion is discarded for the sake of performance. Don't know why it would be helpful to maintain those when hashing, as am only hashing url paths for now.
C# Version
private static readonly UInt32 FNV_OFFSET_32 = 0x811c9dc5; // 2166136261
private static readonly UInt32 FNV_PRIME_32 = 0x1000193; // 16777619
// Unsigned 32bit integer FNV-1a
public static UInt32 HashFnv32u(this string s)
{
// byte[] arr = Encoding.UTF8.GetBytes(s); // 8 bit expanded unicode array
char[] arr = s.ToCharArray(); // 16 bit unicode is native .net
UInt32 hash = FNV_OFFSET_32;
for (var i = 0; i < s.Length; i++)
{
// Strips unicode bits, only the lower 8 bits of the values are used
hash = hash ^ unchecked((byte)(arr[i] & 0xFF));
hash = hash * FNV_PRIME_32;
}
return hash;
}
// Signed hash for storing in SQL Server
public static Int32 HashFnv32s(this string s)
{
return unchecked((int)s.HashFnv32u());
}
JavaScript Version
var utils = utils || {};
utils.FNV_OFFSET_32 = 0x811c9dc5;
utils.hashFnv32a = function (input) {
var hval = utils.FNV_OFFSET_32;
// Strips unicode bits, only the lower 8 bits of the values are used
for (var i = 0; i < input.length; i++) {
hval = hval ^ (input.charCodeAt(i) & 0xFF);
hval += (hval << 1) + (hval << 4) + (hval << 7) + (hval << 8) + (hval << 24);
}
return hval >>> 0;
}
utils.toHex = function (val) {
return ("0000000" + (val >>> 0).toString(16)).substr(-8);
}
A fast and concise one which was adapted from here:
String.prototype.hashCode = function() {
var hash = 5381, i = this.length
while(i)
hash = (hash * 33) ^ this.charCodeAt(--i)
return hash >>> 0;
}
SubtleCrypto.digest
I’m not utilizing a server-side language so I can’t do it that way.
Are you sure you can’t do it that way?
Did you forget you’re using Javascript, the language ever-evolving?
Try SubtleCrypto. It supports SHA-1, SHA-128, SHA-256, and SHA-512 hash functions.
async function hash(message/*: string */) {
const text_encoder = new TextEncoder;
const data = text_encoder.encode(message);
const message_digest = await window.crypto.subtle.digest("SHA-512", data);
return message_digest;
} // -> ArrayBuffer
function in_hex(data/*: ArrayBuffer */) {
const octets = new Uint8Array(data);
const hex = [].map.call(octets, octet => octet.toString(16).padStart(2, "0")).join("");
return hex;
} // -> string
(async function demo() {
console.log(in_hex(await hash("Thanks for the magic.")));
})();
The Jenkins One at a Time Hash is quite nice:
//Credits (modified code): Bob Jenkins (http://www.burtleburtle.net/bob/hash/doobs.html)
//See also: https://en.wikipedia.org/wiki/Jenkins_hash_function
//Takes a string of any size and returns an avalanching hash string of 8 hex characters.
function jenkinsOneAtATimeHash(keyString)
{
let hash = 0;
for (charIndex = 0; charIndex < keyString.length; ++charIndex)
{
hash += keyString.charCodeAt(charIndex);
hash += hash << 10;
hash ^= hash >> 6;
}
hash += hash << 3;
hash ^= hash >> 11;
//4,294,967,295 is FFFFFFFF, the maximum 32 bit unsigned integer value, used here as a mask.
return (((hash + (hash << 15)) & 4294967295) >>> 0).toString(16)
};
Examples:
jenkinsOneAtATimeHash('test')
"31c25ec1"
jenkinsOneAtATimeHash('a')
"ca2e9442"
jenkinsOneAtATimeHash('0')
"6e3c5c6b"
You can also remove the .toString(16) part at the end to generate numbers:
jenkinsOneAtATimeHash2('test')
834821825
jenkinsOneAtATimeHash2('a')
3392050242
jenkinsOneAtATimeHash2('0')
1849449579
Note that if you do not need to hash a string or key specifically, but just need a hash generated out of thin air, you can use:
window.crypto.getRandomValues(new Uint32Array(1))[0].toString(16)
Examples:
window.crypto.getRandomValues(new Uint32Array(1))[0].toString(16)
"6ba9ea7"
window.crypto.getRandomValues(new Uint32Array(1))[0].toString(16)
"13fe7edf"
window.crypto.getRandomValues(new Uint32Array(1))[0].toString(16)
"971ffed4"
and the same as above, you can remove the `.toString(16) part at the end to generate numbers:
window.crypto.getRandomValues(new Uint32Array(1))[0]
1154752776
window.crypto.getRandomValues(new Uint32Array(1))[0]
3420298692
window.crypto.getRandomValues(new Uint32Array(1))[0]
1781389127
Note: You can also generate multiple values at once with this method, e.g.:
window.crypto.getRandomValues(new Uint32Array(3))
Uint32Array(3) [ 2050530949, 3280127172, 3001752815 ]
I needed a similar function (but different) to generate a unique-ish ID based on the username and current time. So:
window.newId = ->
# create a number based on the username
unless window.userNumber?
window.userNumber = 0
for c,i in window.MyNamespace.userName
char = window.MyNamespace.userName.charCodeAt(i)
window.MyNamespace.userNumber+=char
((window.MyNamespace.userNumber + Math.floor(Math.random() * 1e15) + new Date().getMilliseconds()).toString(36)).toUpperCase()
Produces:
2DVFXJGEKL
6IZPAKFQFL
ORGOENVMG
... etc
edit Jul 2022: As #canRau points out the authors of shortid prefer nanoid now https://github.com/ai/nanoid/
If you want to avoid collisions you may want to use a secure hash like SHA-256.
There are several JavaScript SHA-256 implementations.
I wrote tests to compare several hash implementations, see https://github.com/brillout/test-javascript-hash-implementations.
Or go to http://brillout.github.io/test-javascript-hash-implementations/, to run the tests.
I do not see any reason to use this overcomplicated crypto code instead of ready-to-use solutions, like object-hash library, or etc. relying on vendor is more productive, saves time and reduces maintenance cost.
Just use https://github.com/puleos/object-hash
var hash = require('object-hash');
hash({foo: 'bar'}) // => '67b69634f9880a282c14a0f0cb7ba20cf5d677e9'
hash([1, 2, 2.718, 3.14159]) // => '136b9b88375971dff9f1af09d7356e3e04281951'
I have combined the two solutions (users esmiralha and lordvlad) to get a function that should be faster for browsers that support the js function reduce() and still compatible with old browsers:
String.prototype.hashCode = function() {
if (Array.prototype.reduce) {
return this.split("").reduce(function(a,b){a=((a<<5)-a)+b.charCodeAt(0);return a&a},0);
} else {
var hash = 0, i, chr, len;
if (this.length == 0) return hash;
for (i = 0, len = this.length; i < len; i++) {
chr = this.charCodeAt(i);
hash = ((hash << 5) - hash) + chr;
hash |= 0; // Convert to 32bit integer
}
return hash;
}
};
Example:
my_string = 'xyz';
my_string.hashCode();
I went for a simple concatenation of char codes converted to hex strings. This serves a relatively narrow purpose, namely just needing a hash representation of a SHORT string (e.g. titles, tags) to be exchanged with a server side that for not relevant reasons can't easily implement the accepted hashCode Java port. Obviously no security application here.
String.prototype.hash = function() {
var self = this, range = Array(this.length);
for(var i = 0; i < this.length; i++) {
range[i] = i;
}
return Array.prototype.map.call(range, function(i) {
return self.charCodeAt(i).toString(16);
}).join('');
}
This can be made more terse and browser-tolerant with Underscore. Example:
"Lorem Ipsum".hash()
"4c6f72656d20497073756d"
I suppose if you wanted to hash larger strings in similar fashion you could just reduce the char codes and hexify the resulting sum rather than concatenate the individual characters together:
String.prototype.hashLarge = function() {
var self = this, range = Array(this.length);
for(var i = 0; i < this.length; i++) {
range[i] = i;
}
return Array.prototype.reduce.call(range, function(sum, i) {
return sum + self.charCodeAt(i);
}, 0).toString(16);
}
'One time, I hired a monkey to take notes for me in class. I would just sit back with my mind completely blank while the monkey scribbled on little pieces of paper. At the end of the week, the teacher said, "Class, I want you to write a paper using your notes." So I wrote a paper that said, "Hello! My name is Bingo! I like to climb on things! Can I have a banana? Eek, eek!" I got an F. When I told my mom about it, she said, "I told you, never trust a monkey!"'.hashLarge()
"9ce7"
Naturally more risk of collision with this method, though you could fiddle with the arithmetic in the reduce however you wanted to diversify and lengthen the hash.
Adding this because nobody did yet, and this seems to be asked for and implemented a lot with hashes, but it's always done very poorly...
This takes a string input, and a maximum number you want the hash to equal, and produces a unique number based on the string input.
You can use this to produce a unique index into an array of images (If you want to return a specific avatar for a user, chosen at random, but also chosen based on their name, so it will always be assigned to someone with that name).
You can also use this, of course, to return an index into an array of colors, like for generating unique avatar background colors based on someone's name.
function hashInt (str, max = 1000) {
var hash = 0;
for (var i = 0; i < str.length; i++) {
hash = ((hash << 5) - hash) + str.charCodeAt(i);
hash = hash & hash;
}
return Math.round(max * Math.abs(hash) / 2147483648);
}
This generates a consistent hash based on any number of params passed in:
/**
* Generates a hash from params passed in
* #returns {string} hash based on params
*/
function fastHashParams() {
var args = Array.prototype.slice.call(arguments).join('|');
var hash = 0;
if (args.length == 0) {
return hash;
}
for (var i = 0; i < args.length; i++) {
var char = args.charCodeAt(i);
hash = ((hash << 5) - hash) + char;
hash = hash & hash; // Convert to 32bit integer
}
return String(hash);
}
fastHashParams('hello world') outputs "990433808"
fastHashParams('this',1,'has','lots','of','params',true) outputs "1465480334"
Slightly simplified version of #esmiralha's answer.
I don't override String in this version, since that could result in some undesired behaviour.
function hashCode(str) {
var hash = 0;
for (var i = 0; i < str.length; i++) {
hash = ~~(((hash << 5) - hash) + str.charCodeAt(i));
}
return hash;
}
function hashCode(str) {
return str.split('').reduce((prevHash, currVal) =>
(((prevHash << 5) - prevHash) + currVal.charCodeAt(0))|0, 0);
}
// Test
console.log("hashCode(\"Hello!\"): ", hashCode('Hello!'));