I'm trying to assist a mobile developer with encrypting an image and uploading it to Azure blob storage. The mobile app uses expo and crypto js. I'm not super familiar with mobile dev or expo but it looks like expo gives you access to a base64 encoded version of the image.
My goal is to encrypt that image data, using crypto js, and upload it to Azure blob storage.
The specifics of expo or Azure aren't really that important to my question, but I figure they're worth mentioning. What is important, I think, is that I'm using crypto js to AES encrypt that image data.
I'm starting with a base64 string of image data and so I use crypto js to parse that like follows ...
const words = CryptoES.enc.Base64.parse(data);
This gives me a WordArray representing the image data, I think (from the base64 string which the mobile API gives me).
Next I can encrypt that image data like so ...
const encrypted = CryptoES.AES.encrypt(words, AES_KEY, { iv: AES_IV });
Now that I have the encrypted data, I would like to write it out to a file in just a binary hex format or whatever. I don't want base64 text in the file and I don't want a hex-string in the file - I'd like it to contain the literal encrypted data byte for byte.
I'm not sure how to get this data.
I guess it's just "toString" but when I do that it says invalid utf8. This is a JPG file that is being dealt with.
How can I get just the actual byte data and write that to a file with crypto js?
CryptoJS.AES.encrypt() returns a CipherParams object that encapsulates, among others, the ciphertext as WordArray. One possibility is to convert this WordArray to a Uint8Array with a custom method. In the following code this conversion is done by convertWordArrayToUint8Array():
function convertWordArrayToUint8Array(wordArray) {
var arrayOfWords = wordArray.hasOwnProperty("words") ? wordArray.words : [];
var length = wordArray.hasOwnProperty("sigBytes") ? wordArray.sigBytes : arrayOfWords.length * 4;
var uInt8Array = new Uint8Array(length), index=0, word, i;
for (i=0; i<length; i++) {
word = arrayOfWords[i];
uInt8Array[index++] = word >> 24;
uInt8Array[index++] = (word >> 16) & 0xff;
uInt8Array[index++] = (word >> 8) & 0xff;
uInt8Array[index++] = word & 0xff;
}
return uInt8Array;
}
var AES_KEY = CryptoJS.enc.Utf8.parse('0123456789012345');
var AES_IV = CryptoJS.enc.Utf8.parse('5432109876543210');
var plaintext = 'The quick brown fox jumps over the lazy dog';
var ciphertextCP = CryptoJS.AES.encrypt(plaintext, AES_KEY, { iv: AES_IV }); // CipherParams object
var ciphertextWA = ciphertextCP.ciphertext; // WordArray
var ciphertextArr = convertWordArrayToUint8Array(ciphertextWA); // Uint8Array
This Uint8Array can now be stored in a file, e.g. with:
var fileName = "encdata.bin";
saveByteArray([ciphertextArr], fileName);
using saveByteArray() from here.
Another approach is to convert the WordArray to a binary string using the Latin1 encoder:
var ciphertextBinStr = ciphertextWA.toString(CryptoJS.enc.Latin1);
which can then easily be converted to a Uint8Array, e.g. with:
function str2Uint8Array(str) {
const arr = new Uint8Array(new ArrayBuffer(str.length));
for (let i = 0, strLen = str.length; i < strLen; i++)
arr[i] = str.charCodeAt(i);
return arr;
}
var ciphertextArr = str2Uint8Array(ciphertextBinStr);
Why whenever I run encodeURIComponent('my_url'), I get a weird looking URL like https%3A%2F%2Fr[...]... Why is that and how do I solve it?
Btw, I'm using Windows.
function createSharedAccessToken(uri, saName, saKey) {
uri = // '...'
saName = // '...'
saKey = // '...'
if (!uri || !saName || !saKey) {
throw "Missing required parameter";
}
var encoded = encodeURIComponent(uri);
var now = new Date();
var week = 60 * 60 * 24 * 7;
var ttl = Math.round(now.getTime() / 1000) + week;
var signature = encoded + '\n' + ttl;
var signatureUTF8 = utf8.encode(signature);
var hash = crypto.createHmac('sha256', saKey).update(signatureUTF8).digest('base64');
return 'SharedAccessSignature sr=' + encoded + '&sig=' +
encodeURIComponent(hash) + '&se=' + ttl + '&skn=' + saName;
}
The type of 'wierd looking url' you shared is eactly what encodeURIComponent is designed to return.
all character except
A-Z a-z 0-9 - _ . ! ~ * ' ( ) get replaced by escape sequences.
For example, any spaces become %20 and / becomes %2F.
A typical URL, processed by encodeURIComponent, might look like this:
https%3A%2F%2Fstackoverflow.com
When you are ready to display the original string, you simply use decodeURIComponent to reverse the process:
decodeYRIComponent('https%3A%2F%2Fstackoverflow.com');
// becomes: https://stackoverflow.com
It is well explained here, why we need to encode the URL
if a user writes Jack & Jill, the text may get encoded as Jack & Jill. Without encodeURIComponent() the ampersand could be interpreted on the server as the start of a new field and jeopardize the integrity of the data.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent
If we need it in human readable form, we always have https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURI
I am using Java to encrypt a text payload with Triple DES. First I create an ephemeral key that I will use for encrypting the payload:
private byte[] createEphemeralKey() throws Exception {
KeyGenerator keygen = KeyGenerator.getInstance("DESede");
keygen.init(168);
return keygen.generateKey().getEncoded();
}
Then I encrypt my payload with said key:
private String encryptTripleDES(byte[] ephemeralKey, String payload) throws Exception {
Cipher cipher = Cipher.getInstance("DESede/ECB/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, new SecretKeySpec(ephemeralKey, "DESede"));
byte[] plainTextBytes = payload.getBytes();
byte[] cipherText = cipher.doFinal(plainTextBytes);
return Base64.getEncoder().encodeToString(cipherText);
}
Also need a padding function to ensure the data length is divisable by 8:
private String adjustPadding(String input, int blockSize) {
int len = input.length() % blockSize;
int paddingLength = (len == 0) ? 0 : (blockSize - len);
while (paddingLength > 0) {
input += "F";
paddingLength--;
}
return input;
}
And here is my process end to end:
String data = "Marnus"
byte[] = ephemeralKey = createEphemeralKey();
String adjustedData = adjustPadding (data,8);
String encryptedPayload = encryptTripleDES(ephemeralKey, adjustedData);
String encodedKey = Base64.getEncoder().encodeToString(ephemeralKey)
So I take the 2 variables encryptedPayload and encodedKey, that are both Base64 encoded string, and send it off via HTTP to node express app.
In the Javascript side of things, I use node-forge - Here is the part of my express app that does the decryption:
let nodeBuffer = Buffer.from(data, 'base64')
let input = forge.util.createBuffer(nodeBuffer.toString('binary'))
// 3DES key and IV sizes
let keySize = 24;
let ivSize = 8;
let derivedBytes = forge.pbe.opensslDeriveBytes(ephemeralKey, null, keySize + ivSize);
let buffer = forge.util.createBuffer(derivedBytes);
let key = buffer.getBytes(keySize)
let iv = buffer.getBytes(ivSize)
let decipher = forge.cipher.createDecipher('3DES-ECB', key)
decipher.start({iv: iv})
decipher.update(input)
console.log('decipher result', decipher.finish())
let decryptedResult = decipher.output.data;
Here is an Triples DES example in the node-forge docs:
A few notes:
I create a node-forge buffer from a regular buffer since I don't have a input file like the examples gives. Here is how the docs states one should create one buffer from the other:
*I use base64 as that is what I used in the java side to encode the data that was sent.
Then, I dont have a salt so I left the 2'nd param null in opensslDeriveBytes as specified in the docs I should do.
Thirdly, I am also not sure if my keysize of 24 is correct?
My results
So doing an end to end test yields the following:
In my Java app, the test data was "Marnus", the encryptedPayload was ez+RweSAd+4= and the encodedKey was vCD9mBnWHPEBiQ0BGv7gc6GUCOoBgLCu.
Then in my javascript code data was obviously ez+RweSAd+4=(encryptedPayload) and the ephemeralKey was vCD9mBnWHPEBiQ0BGv7gc6GUCOoBgLCu(encodedKey).
After the decryption ran, the value of decryptedResult was ©ýÕ?µ{', which is obviously just garbage since it was not encoded yet, but I cant figure out which encoding to use?
I tried using forge.util.encode64(decipher.output.data), but that just gave me qf3VP7UYeyc=, which is not right.
For what it's worth, here is the type that decipher.output
With a lot more tweaking and testing different options, I got it working - and the good news is I managed to get it all working with the built in crypto library in nodejs (v12.18.4).
First things first, the JAVA side just needs a change to the key size (from 168 to 112), the rest remains the same - see below example as one single method (should be split up in final implementation of course for testability and usability):
//Some data:
String payload = "{\"data\":\"somedata\"}";
// Create Key
KeyGenerator keygen = KeyGenerator.getInstance("DESede");
keygen.init(112);
byte[] ephemeralKey = keygen.generateKey().getEncoded();
// Adjust the data, see adjustPadding method in the question for details.
String data = adjustPadding (payload,8);
// Wil now be "{"data":"somedata"}FFFFF", can just chop off extra in JS if need be. When sending JSON one knows the end of the object will always be "}"
// Do Encrypt
Cipher cipher = Cipher.getInstance("DESede/ECB/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, new SecretKeySpec(ephemeralKey, "DESede"));
byte[] plainTextBytes = data.getBytes();
byte[] cipherText = cipher.doFinal(plainTextBytes);
String encryptedPayload = Base64.getEncoder().encodeToString(cipherText);
//Lastly, Base64 the key so you can transport it too
String encodedKey = Base64.getEncoder().encodeToString(ephemeralKey)
on the Javascript side of things we keep it simple:
// I'm using TS, so change the import if you do plain JS
import crypto = require('crypto')
//need bytes from the base64 payload
let buff = Buffer.from(ephemeralKey, 'base64')
const decipher = crypto.createDecipheriv('des-ede3', buff, null)
decipher.setAutoPadding(false)
let decrypted = decipher.update(data, 'base64', 'utf8')
decrypted += decipher.final('utf8')
console.log(decrypted)
//{"data":"somedata"}FFFFF"
In a web form the answers (packed in a jsonstring) are encrypted in several steps. First a random key is generated. Second the random key is used for AES encryption of the jsonstring. The random key is encrypted as well. Both are send in the body of a mail.
// Generate Random key
var rand_key = ('0000' + Math.random().toString(36).replace('.', '')).substr(-10);
console.log('rand_key', rand_key)
//var pubkey = "MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALDjeFwFNhMCjMwcRVVKG1VvfsntEVPR3lNTujJnNk1+iSqZ4Tl5Lwq9GbwO+qlYVwXHNmeqG7rkEhL9uyDIZVECAwEAAQ=="
// rsa_key_public07012016.bin
//var pubkey = "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCv8FVei4Q2ehmYsSCv/uODSojIOGHwfQe686S1cEH5i/1mGME5ZzNqyy0d+lhMRD0tr7Sje7JoCEC/XRIZaiKJjpl1+3RXotf/Cx3bd9H7WtitshZB1m38ZZFsrX4oigMpUPFbCefMeBS4hvvNnmtl08lQGhfIXdXeflZsgWRHtQIDAQAB";
// my_pub_key.pem
var pubkey ="MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA38gtENP9/hpirSCIsPh6CAVm0UmME4XBlPyK8yhwk079EUJpNzlEhu9HKcA/B7Fxo2lNoY9Tb9e+PYtJ6+VOB4+Y6zgGMX7cchYmumKRTbbQ6FNfBE5Q8XnOAUlgC7gNrs0e5lW7JH1kWlK+eTT4TANT7F3US09aXmym+fZaRInbXmJujGnDIbRIIbzr5FE82EeMpw2TqRWV466wz5EeFWSSQ8EqV1pSox8B1ywb6cnB/Vofs2qR9Zf2efi9TMcSGm/ij/p9IZcbLeep9qfGsv29lbLNMfwNwQyH0JU27eAM4tPdirceZPxfD6iiILmKzN253BMoAeQCp6us53CnGQIDAQAB"
// Make form_data a JSON string
var jsonstring = JSON.stringify(form_data);
// Create AES encrypted object
var aes_encrypted_json = CryptoJS.AES.encrypt(jsonstring, rand_key);
// Encrypt rand_key
var encrypt = new JSEncrypt();
//console.log('encrypt obj', encrypt);
encrypt.setPublicKey(pubkey);
var encrypted_rand_key = encrypt.encrypt(rand_key);
//var encrypted = encrypt.encrypt(jsonstring);
console.log('encypted', encrypted_rand_key);
var mail_body = encrypted_rand_key + aes_encrypted_json
console.log('body', mail_body)
var mailto_string = "mailto:info#xyz.com?subject=FORM&body=" + encodeURIComponent(mail_body);
$('#mailtosend').attr('href', mailto_string);
At the recipient mail server side I want to decrypt the random generated key and the jsonstring using a private key using the pycryptodome package.
from Crypto.PublicKey import RSA
from Crypto.Cipher import PKCS1_v1_5
from base64 import *
def decrypt(key, text):
if type(key) == str:
key = key.encode()
if type(text) == str:
text = text.encode()
rsakey = RSA.importKey(key)
rsakey = PKCS1_v1_5.new(rsakey)
d = rsakey.decrypt(text, 'bolloux')
return d
# rand_key am2mhiwwmi
text = "ZvcrluUmZLY3lRRw01W9mQnhMn7zzpIWn1Bo3csM/ZZ0pWY/8H2dCB9fZDi9/cmp0UtIqDXhLd7SIwyxqrFgPcHUuEHlZl0WQcjSty8PjadG2Abulk1XqEQV4u0Gb/bFGDBMcf5tV1G0d4FFcBPE8r8inrxUjSj2CSffVL8gIGq3ZfY5g7t5FOZV8npBCEONgOLKYnzIiHrHUuXWsOaMAqxMFOLd5DTDLKAkyMybDClsLW9ka+CvWd5fnZBCvO2ziehFp7b9PG4QPSnQpdC8jNLGZB2h0FI8YQD6IyUwmVluUbAlPMqwd6A2CBdGCbfbMChaA5R7bJgKkYhPOQTjaQ=="
text = b64decode(text.encode())
with open('my_priv_key.pem', 'rb') as f:
key = f.read()
decrypt(key, text)
I run into a encoding problem. "UnicodeDecodeError: 'ascii' codec can't decode byte 0xf7 in position 1: ordinal not in range(128)" The encoding is complicating the issue beyond my capabilities.
My questions:
1. How can I resolve the encoding problem ?
2. How can I make the decryption work ?
Thanks
The issue is more than likely caused by b64decode(text) returning a str that contains values such as \xf7 and then attempting to .encode() those values within your decrypt function. encode will use the default encoding which in this case is ascii. I would personally remove the calls to encode unless you specifically have a reason you are doing so.
I have a C# function as below:
string stringvalue = "530500480530490480530480480520570480520510500490";
var encodedvalue= Encoding.Unicode.GetBytes(stringvalue);
using (HashAlgorithm ssp = System.Security.Cryptography.HashAlgorithm.Create("SHA256"))
{
var digest = ssp.ComputeHash(encodedvalue);
return BitConverter.ToString(digest);
}
I need to create a javascript function that match the code above so that the end result for both C# and JS is the same.
Currently in my JS code, I'm using this:
var hash = CryptoJS.SHA256("530500480530490480530480480520570480520510500490");
var hexhash = hash.toString(CryptoJS.enc.hex);
This is the result of my hexhash:
d956678c8f12c65299daf35019a9a1eb3e6eb9855fd850aeb5aafe46057d179e
But in my C# code, this line of var digest = ssp.ComputeHash(bPass); return the following array:
I don't know much about encoding. Please tell me what type of result is being populated in the c# code above? If I'm not mistaken, the ComputeHash is returning bytes but I need lots of reading to confirm that which is another long hour of studying
I tried many different ways of converting the JS Sha256 code but no luck. I'm stuck at this particular line for almost a day.
Please help. Thanks
EDIT:
Sorry for the code error. I had updated the C# code. ComputeHash accept an array
In my example I am using System.Security.Cryptography.SHA256Managed to get SHA256 in C#.
The method SHA256Managed.ComputeHash takes a byte array as a parameter and return another byte array. Now we need to convert back your byte array to a string.
The following code return the same result a Javascript SHA-256.
byte[] bytes = Encoding.UTF8.GetBytes("530500480530490480530480480520570480520510500490");
SHA256Managed hashstring = new SHA256Managed();
byte[] hash = hashstring.ComputeHash(bytes);
string hashString = string.Empty;
foreach (byte x in hash)
{
hashString += String.Format("{0:x2}", x);
}
return(hashString);
Just to explain : String.Format("{0:x2}", x)
X means Hexadecimal format.
2 means 2 characters.
I finally found the answer after uncountable hours of trial and error.
The C# code var digest = ssp.ComputeHash(encodedvalue) is returning byte array from the result of var encodedvalue= Encoding.Unicode.GetBytes(stringvalue); as Jean replied. In order to create the function in Javascript, I need to ensure that the encodedvalue is producing the correct encoding format and size just like the one in C#.
Using only CryptoJS, I manage to get the matching result from below
function GetHexFromString() {
var stringVal = '8563A578-7402-4567-A6CE-4DE4E0825B021234';
// Convert the string to UTF 16 little-endian
// Result: 560530540510650530550560450550520480500450520530540550450650540670690450520680690520690480560500530660480500490500510520
var utf16le = CryptoJS.enc.Utf16LE.parse(stringVal);
// Convert to Sha256 format and get the word array
var utf16Sha256 = CryptoJS.SHA256(utf16le);
// Convert the Sha256 word array to Uint8Array to get the 32 byte array just to see the result to ensure it match with the C# function
// Result: 94,203,69,29,35,202,209,149,121,144,44,6,98,250,141,161,102,7,238,35,228,117,111,236,118,115,51,113,134,72,52,69
var utf16sha256Array = convertWordArrayToUint8Array(utf16Sha256);
// Convert the Sha256 to hex (if i'm not mistaken, it's base 16) format
var hexSha256 = utf16Sha256.toString(CryptoJS.enc.hex);
// Insert a dash in between 2 characters in the string
hexSha256 = hexSha256.replace(/(\S{2})/g, "$1-");
// Remove the last dash in the string
hexSha256 = hexSha256.replace(/-$/, "");
// Final Result: 5E-CB-45-1D-23-CA-D1-95-79-90-2C-06-62-FA-8D-A1-66-07-EE-23-E4-75-6F-EC-76-73-33-71-86-48-34-45
return hexSha256.toUpperCase();
}
function convertWordArrayToUint8Array(wordArray) {
var len = wordArray.words.length,
u8_array = new Uint8Array(len << 2),
offset = 0, word, i
;
for (i = 0; i < len; i++) {
var word = wordArray.words[i];
u8_array[offset++] = word >> 24;
u8_array[offset++] = (word >> 16) & 0xff;
u8_array[offset++] = (word >> 8) & 0xff;
u8_array[offset++] = word & 0xff;
}
return u8_array;
}
Hope it help whoever that need such method
An alternative to Koo SengSeng's answer (if you don't want to use CryptoJS library).
SHA256 function is from here, the arrToUintArr function is from Koo SengSeng's answer.
var SHA256=function a(b){function c(a,b){return a>>>b|a<<32-b}for(var d,e,f=Math.pow,g=f(2,32),h="length",i="",j=[],k=8*b[h],l=a.h=a.h||[],m=a.k=a.k||[],n=m[h],o={},p=2;64>n;p++)if(!o[p]){for(d=0;313>d;d+=p)o[d]=p;l[n]=f(p,.5)*g|0,m[n++]=f(p,1/3)*g|0}for(b+="\x80";b[h]%64-56;)b+="\x00";for(d=0;d<b[h];d++){if(e=b.charCodeAt(d),e>>8)return;j[d>>2]|=e<<(3-d)%4*8}for(j[j[h]]=k/g|0,j[j[h]]=k,e=0;e<j[h];){var q=j.slice(e,e+=16),r=l;for(l=l.slice(0,8),d=0;64>d;d++){var s=q[d-15],t=q[d-2],u=l[0],v=l[4],w=l[7]+(c(v,6)^c(v,11)^c(v,25))+(v&l[5]^~v&l[6])+m[d]+(q[d]=16>d?q[d]:q[d-16]+(c(s,7)^c(s,18)^s>>>3)+q[d-7]+(c(t,17)^c(t,19)^t>>>10)|0),x=(c(u,2)^c(u,13)^c(u,22))+(u&l[1]^u&l[2]^l[1]&l[2]);l=[w+x|0].concat(l),l[4]=l[4]+w|0}for(d=0;8>d;d++)l[d]=l[d]+r[d]|0}for(d=0;8>d;d++)for(e=3;e+1;e--){var y=l[d]>>8*e&255;i+=(16>y?0:"")+y.toString(16)}return i};
var arrToUintArr=function(a){for(var l=a.length,b=new Uint8Array(l<<2),o=0,w,i=0;i<l;i++) w=a[i],b[o++]=w>>24,b[o++]=(w>>16)&0xff,b[o++]=(w>>8)&0xff,b[o++]=w&0xff;return b;}
var computeHash=function(k){for(var a=[],s=SHA256(k),i=0;i<8;i++) a.push(parseInt(s.substr(i*8,8),16));return arrToUintArr(a);}
computeHash(k) will return an array of numbers representing bytes.
This is equal to below code in C#:
new System.Security.Cryptography.SHA256CryptoServiceProvider().ComputeHash(Encoding.UTF8.GetBytes(k));
Try
var digest = ssp.ComputeHash(Encoding.UTF8.GetBytes(stringvalue))
return BitConverter.ToString(digest)
.Replace("-", string.Empty)
.ToLowerInvariant();
That js library is converting the string to UTF8 before calculating its hash.
typescript code:
private computeHash(text: string): string {
return CryptoJS.SHA256(text).toString();
}
c# equivalent:
private string ComputeHash(string text)
{
using (var sha256 = SHA256.Create())
{
var bytes = Encoding.UTF8.GetBytes(text);
var hash = sha256.ComputeHash(bytes);
return hash.Aggregate(string.Empty, (current, x) => current + $"{x:x2}");
}
}
after two days of research it works perfectly! Two different codes give the same result.
js
const sha1 = require('sha1');
const getHash = str =>{
const hashingBytes = Buffer.from(sha1(str), "hex");
const base64Value = Buffer.from(hashingBytes).toString('base64');
return base64Value;
}
c#
System.Security.Cryptography.SHA1 sha = new System.Security.Cryptography.SHA1CryptoServiceProvider();
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(str);
byte[] hashingbytes = sha.ComputeHash(bytes);
var hash = Convert.ToBase64String(hashingbytes);