Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 10 months ago.
The community reviewed whether to reopen this question 9 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
I am writing a login for a forum, and need to hash the password client side in javascript before sending it on to the server. I'm having trouble figuring out which SHA-256 implementation I can actually trust. I was expecting there to be some kind of authoritative script that everyone used, but I'm finding loads of different projects all with their own implementations.
I realize using other people's crypto is always a leap of faith unless you're qualified to review it yourself, and that there is no universal definition of "trustworthy", but this seems like something common and important enough that there ought to be some kind of consensus on what to use. Am I just naive?
Edit since it comes up a lot in the comments: Yes, we do a more stringent hash again on the server side. The client side hashing is not the final result that we save in the database. The client side hashing is because the human client requests it. They have not given a specific reason why, probably they just like overkill.
On https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/digest I found this snippet that uses internal js module:
async function sha256(message) {
// encode as UTF-8
const msgBuffer = new TextEncoder().encode(message);
// hash the message
const hashBuffer = await crypto.subtle.digest('SHA-256', msgBuffer);
// convert ArrayBuffer to Array
const hashArray = Array.from(new Uint8Array(hashBuffer));
// convert bytes to hex string
const hashHex = hashArray.map(b => b.toString(16).padStart(2, '0')).join('');
return hashHex;
}
Note that crypto.subtle in only available on https or localhost - for example for your local development with python3 -m http.server you need to add this line to your /etc/hosts:
0.0.0.0 localhost
Reboot - and you can open localhost:8000 with working crypto.subtle.
OUTDATED: Many modern browsers now have first-class support for crypto operations. See Vitaly Zdanevich's answer below.
The Stanford JS Crypto Library contains an implementation of SHA-256. While crypto in JS isn't really as well-vetted an endeavor as other implementation platforms, this one is at least partially developed by, and to a certain extent sponsored by, Dan Boneh, who is a well-established and trusted name in cryptography, and means that the project has some oversight by someone who actually knows what he's doing. The project is also supported by the NSF.
It's worth pointing out, however...
... that if you hash the password client-side before submitting it, then the hash is the password, and the original password becomes irrelevant. An attacker needs only to intercept the hash in order to impersonate the user, and if that hash is stored unmodified on the server, then the server is storing the true password (the hash) in plain-text.
So your security is now worse because you decided add your own improvements to what was previously a trusted scheme.
For those interested, this is code for creating SHA-256 hash using sjcl:
import sjcl from 'sjcl'
const myString = 'Hello'
const myBitArray = sjcl.hash.sha256.hash(myString)
const myHash = sjcl.codec.hex.fromBits(myBitArray)
Forge's SHA-256 implementation is fast and reliable.
To run tests on several SHA-256 JavaScript implementations, go to http://brillout.github.io/test-javascript-hash-implementations/.
The results on my machine suggests forge to be the fastest implementation and also considerably faster than the Stanford Javascript Crypto Library (sjcl) mentioned in the accepted answer.
Forge is 256 KB big, but extracting the SHA-256 related code reduces the size to 4.5 KB, see https://github.com/brillout/forge-sha256
No, there's no way to use browser JavaScript to improve password security. I highly recommend you read this article. In your case, the biggest problem is the chicken-egg problem:
What's the "chicken-egg problem" with delivering Javascript cryptography?
If you don't trust the network to deliver a password, or, worse, don't trust the server not to keep user secrets, you can't trust them to deliver security code. The same attacker who was sniffing passwords or reading diaries before you introduce crypto is simply hijacking crypto code after you do.
[...]
Why can't I use TLS/SSL to deliver the Javascript crypto code?
You can. It's harder than it sounds, but you safely transmit Javascript crypto to a browser using SSL. The problem is, having established a secure channel with SSL, you no longer need Javascript cryptography; you have "real" cryptography.
Which leads to this:
The problem with running crypto code in Javascript is that practically any function that the crypto depends on could be overridden silently by any piece of content used to build the hosting page. Crypto security could be undone early in the process (by generating bogus random numbers, or by tampering with constants and parameters used by algorithms), or later (by spiriting key material back to an attacker), or --- in the most likely scenario --- by bypassing the crypto entirely.
There is no reliable way for any piece of Javascript code to verify its execution environment. Javascript crypto code can't ask, "am I really dealing with a random number generator, or with some facsimile of one provided by an attacker?" And it certainly can't assert "nobody is allowed to do anything with this crypto secret except in ways that I, the author, approve of". These are two properties that often are provided in other environments that use crypto, and they're impossible in Javascript.
Basically the problem is this:
Your clients don't trust your servers, so they want to add extra security code.
That security code is delivered by your servers (the ones they don't trust).
Or alternatively,
Your clients don't trust SSL, so they want you use extra security code.
That security code is delivered via SSL.
Note: Also, SHA-256 isn't suitable for this, since it's so easy to brute force unsalted non-iterated passwords. If you decide to do this anyway, look for an implementation of bcrypt, scrypt or PBKDF2.
I found this implementation very easy to use. Also has a generous BSD-style license:
jsSHA: https://github.com/Caligatio/jsSHA
I needed a quick way to get the hex-string representation of a SHA-256 hash. It only took 3 lines:
var sha256 = new jsSHA('SHA-256', 'TEXT');
sha256.update(some_string_variable_to_hash);
var hash = sha256.getHash("HEX");
It is possible to use CryptoJS - https://www.npmjs.com/package/crypto-js
import sha256 from 'crypto-js/sha256'
const hash = sha256('Text')
Besides the Stanford lib that tylerl mentioned. I found jsrsasign very useful (Github repo here:https://github.com/kjur/jsrsasign). I don't know how exactly trustworthy it is, but i've used its API of SHA256, Base64, RSA, x509 etc. and it works pretty well. In fact, it includes the Stanford lib as well.
If all you want to do is SHA256, jsrsasign might be a overkill. But if you have other needs in the related area, I feel it's a good fit.
js-sha256 is an npm package you can use, and unlike the popular crypto.subtle which only works on secure connections(localhost/https) it can work regardless. Of course having a secure connection is still the best. I was using crypto.subtle and it always worked because I run my webapp using localhost and it failed the instant I tried it on a server. I had to switch to the js-sha256 npm package as a temporary solution until a secure connection can be configured.
ethers.js has a SHA256 (https://docs.ethers.io/v5/api/utils/hashing/)
const { ethers } = require('ethers');
ethers.utils.sha256(ethers.utils.toUtf8Bytes('txt'));
Related
We have a site where we're using zxcvbn by Dropbox to inform users of their password strength, however, we've been getting occasional reports that it doesn't work.
Turns out that these users (reasonably rare) are accessing our website from their workplace which has a strict corporate firewall policy, because the js file contains swearwords and NSFW words (to mark the password as insecure if it contains these commonly used words), the whole JS file is being blocked from loading.
The rest of our site loads fine, including other JS files.
How could we encrypt or minify this js file to a point where it didn't get blocked for having "bad" words in the request, but be successfully decrypted at the client side to actually do it's job and detect unsafe passwords?
This JS Fiddle will (sort of) demonstrate the problem: https://jsfiddle.net/0cgap96m/3/
<script src="https://cdnjs.cloudflare.com/ajax/libs/zxcvbn/4.4.2/zxcvbn.js" integrity="sha512-TZlMGFY9xKj38t/5m2FzJ+RM/aD5alMHDe26p0mYUMoCF5G7ibfHUQILq0qQPV3wlsnCwL+TPRNK4vIWGLOkUQ==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
<div id="test">
</div>
window.onload = function(){
var name = prompt("Put in a fake password to test?");
var passwordStrength = zxcvbn(name);
document.getElementById('test').innerHTML = JSON.stringify(passwordStrength);
};
That should work fine normally - now try blocking https://cdnjs.cloudflare.com/ajax/libs/zxcvbn/4.4.2/zxcvbn.js using an adblocker or something, and it'll obviously start failing. This is essentially what's happening for the users, but it's blocked by their corporate firewall rather than a local adblocker.
To confound the filter, you can try substituting the literal characters with JavaScript's syntax for the unicode representation of those characters.
This works even with identifiers!
var f\u006F\u006F = 'b\u0061\u0072';
console.log(foo); // outputs: bar
You could download the built js file and alter the passwords list, to split up the string in between the NSFW words. Then your copy of the library instead.
In zxcvbn.js the insecure words are defined like this (shortened here for this example)
var frequency_lists;frequency_lists=
{passwords:"123456,password,eatshit,goodluck,starcraft"}
So, by doing this:
var frequency_lists;frequency_lists=
{passwords:"123456,password,eatsh" + "it,goodluck,starcraft"}
a firewall scanning for swear words shouldn't recognize that as a swear anymore.
EDIT:
I might suggest a PR to their repo to have their code build with this format might be a better solution with the additional benefit of solving the issue for anyone else using this library, as well as allowing you to update to newer versions. But from quickly looking at the github, I see you'd need to be familiar with coffeescript + python. The original solution is much quicker and doesn't require knowledge in other languages.
how about a simple error handling client-side and a proper validation server-side?
Actually, you don't even need the validation, but if the typed/submitted password is sent to evaluation on server-side when client-side is not available may cover all the bases you need.
And if you need validation, well you should have it server-side, too, anyway, right?
I am struggling at this one part of code where I need to decrypt AES on my server side (Ruby) from a client. I know all the information, but I am struggling with re-producing the encryption.
I am currently using CryptoJS from https://github.com/brix/crypto-js. I am not sure which version of AES it is using which might be my first problem.
An example of how I currently encrypt my data in Javascript is:
encodeURIComponent(CryptoJS.AES.encrypt("Message","Key").toString())
Right now I am currently using openssl and cgi in Ruby to try to decrypt. This is wrong and not working, but I wanted to show what I am trying as I believe it is close. I don't understand how the key is used in the encryption, but I am following the example I found here
require "openssl"
require "cgi"
cipher = OpenSSL::Cipher.new('AES-128-CBC')
cipher.encrypt
key = "Key"
iv = cipher.random_iv
encrypted = cipher.update("Message") + cipher.final
puts CGI::escape(encrypted.to_s)
I have just put "Message" and "Key" to not share my information, I am an amateur when it comes to security and cryptography, but I have done these things in lower level languages without problems. I believe the problem happens in two main areas
My lack of knowledge of how these high level languages work, and the libraries I am using
The strings are sometimes UTF-8 vs UTF-16 in these langauges, so passing the "Message" as a string might be causing problems
FULL EXAMPLE OF ENCRYPTION AND DECRYPTION IN JAVASCRIPT:
Encrypting and URL encoding with input 1:
encodeURIComponent(CryptoJS.AES.encrypt("1","Key").toString())
Result:
"U2FsdGVkX19Lp8ItQaO5h6Lj68sheHeYrIkJAfqt1Tw%3D"
Decoding URL and Decryption:
CryptoJS.AES.decrypt(decodeURIComponent("U2FsdGVkX19Lp8ItQaO5h6Lj68sheHeYrIkJAfqt1Tw%3D"), "Key").toString(CryptoJS.enc.Utf8)
Result:
"1"
At least one of your problems is noted by Artjom B noted in the comment above, and this is a frequent problem with trying to get crypto-js to interoperate with other libraries: crypto-js is not taking in a "Key" the way you are using it but instead a password. Passwords are not keys!!!
Internally, crypto-js uses a very poor algorithm for converting the password into a key. The algorithm is poor for two reasons: (1) it is based upon the insecure MD5, and (2) converting passwords to keys should be a slow process to deter brute force guessing of passwords. In crypto-js, the process is not slow.
To get you headed the right direction, do not call:
CryptoJS.AES.encrypt("Message","Password")
Instead, call
CryptoJS.AES.encrypt("Message", key, { iv: iv });
You might also need to explore padding to get it to interoperate with Ruby.
I am using kafka-node (https://github.com/SOHU-Co/kafka-node) consumer to retrieve data. I think the data which I get is compressed with SNAPPY. How do I decompress the data after I get it. I tried using node-snappy (https://github.com/kesla/node-snappy) to decompress the data, but it didn't work.
Is there any option in the library to set the compression to none?
Anyone used kafka-node library to get data from kafka..??
Thanks,
chandu
I also encountered these exact problems. I found a solution, at last! You can use kafkacat ('like netcat for kafka') download here, which requires librdkafka. This enables you to interact with Kafka from the command line using the librdkafka C/C++ library. (no JVM required =D )
Now that we have those dependencies taken care of we can use this fun node.js repo: node-kafkacat
You'll find that there's likely enough documentation between those three libraries to get you started and unlike some of the other kafka-node modules on github, they seem to have been updated fairly recently.
I have only successfully installed on Linux and Mac so far but things are working great with our Apache/Java environment. I'm not the author of any of these packages, btw - just some guy who kept hoping your question would be answered over the last couple weeks.
By default, compression is determined by the producer through the configuration property 'compression.type'. Currently gzip, snappy and lz4 are supported. kafka-node should automatically uncompress both gzip and snappy. I just tried it with snappy and this works out of the box. lz4 appears not to be implemented by kafka-node at this point.
Compression can be configured using the configuration property 'compression.type':
Per broker: https://kafka.apache.org/documentation/#brokerconfigs. Default: producer determines compression
Per topic: https://kafka.apache.org/documentation/#brokerconfigs (scroll further down). Default: producer determines compression
Per producer: https://kafka.apache.org/documentation/#producerconfigs. Default: none
If you can't influence the producer, you could therefore consider overriding the topic configuration.
Also, note that kafka-node may not directly return the data as you expect. For instance, my (key, value) pair is of type (integer, integer), and I have to call both message.key.readIntBE() and message.value.readIntBE() to extract the integer values.
Hope this helps!
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
Is there a standard library or tool out there for computing and applying differences to JSON documents? Basically I have a bunch of largish documents that I want to keep synchronized across a network, and I would prefer to avoid having to resend their entire state each time that I want to synchronize them (since many of these variables aren't going to change). In other words, I only want to transmit the fields which changed, not retransmit the entire object. I would think that it would be convenient to have something like the following set of methods:
//Start with two distinct objects on the server
// prev represents a copy of the state of the object on the client
// next represents a copy of the state of the object on the server
//
//1. Compute a patch
patch = computePatch(prev, next);
//2. Send patch over the network
//3. Apply the patch on the client
applyPatch(prev, patch);
//Final invariant:
// prev represents an equivalent object to JSON.parse(JSON.stringify(next))
I could certainly implement one myself, but there are quite a few edge cases that need to be considered. Here are some of the straightforward (though somewhat unsatisfactory) methods that I can think of such as:
Roll my own JSON patcher. Asymptotically, this is probably the best way to go, since it would be possible to support all the relevant features of JSON documents, along with supporting some specialized methods for doing stuff like diffing ints, doubles and strings (using relative encoding/edit distance). However, JSON has a lot of special cases and I am a bit leery of trying to do this without a lot of testing, and so I would much prefer to find something that already solves this problem for me so that I can trust it, and not have to worry about network Heisenbugs showing up due to mistakes in my JSON patching
Just compute the edit distance directly between the JSON strings using dynamic programming. Unfortunately, this doesn't work if the client and server have different JSON implementations (ie the order of their fields could be serialized differently), and it is also pretty expensive being a quadratic time operation.
Use protocol buffers. Protocol buffers have a built in diff method which does exactly what I want, and they are a nice binary-serializable network friendly format. Unfortunately, because they are also strictly typed, they lack many of the advantages of using JSON such as the ability to dynamically add and remove fields. Right now this is the approach I am currently leaning towards, but it could make future maintenance really horrible as I would need to continually update each of my objects.
Do something really nasty, like make a custom protocol for each type of object, and hope that I get it right in both places (yeah right!).
Of course what I am really hoping for is for someone here on stackoverflow to come through and save the day with a reference to a space efficient javascript object differ/patcher that has been well tested in production environments and across multiple browsers.
*Update*
I started writing my own patcher, an early version of it is available at github here:
https://github.com/mikolalysenko/patcher.js
I guess since there doesn't seem to be much out here, I will instead accept as an alternative answer a list of interesting test cases for a JSON patcher.
I've been mantaining a json diff & patch library at github (yes, shameless plug):
https://github.com/benjamine/JsonDiffPatch
it handles long strings automatically using Neil Fraser's diff_match_patch lib.
it works both on browsers and server (unit tests running on both env).
(full feature list is on project page)
The only thing you probably would need, that's not implemented is the option to inject custom diff/patch functions for specific objects, but that doesn't sound hard to add, you're welcome to fork it, and even better send a pull request.
Regards,
The JSON-patch standard has been updated.
https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-patch-10
You can find an implementation for applying patches and generating patches at https://github.com/Starcounter-Jack/Fast-JSON-Patch
I came across this question searching for implementations of json-patch. If you are rolling your own you might want to base it on this draft.
https://datatracker.ietf.org/doc/html/draft-pbryan-json-patch-00
Use JSON Patch which is the standard way to do this.
JSON Patch is a format for describing changes to a
JSON document. It can be used to avoid sending a whole document when
only a part has changed. When used in combination with the HTTP PATCH
method it allows partial updates for HTTP APIs in a standards
compliant way.
The patch documents are themselves JSON documents.
JSON Patch is specified in RFC 6902 from the IETF.
Libraries exist for most platforms and programming languages.
At the time of writing, Javascript, Python, PHP, Ruby, Perl, C, Java, C#, Go, Haskell and Erlang are supported (full list and libraries here).
Here is a list for javascript
Fast-JSON-Patch both diffs and patches, 509,361 weekly downloads on NPM
jiff both diffs and patches, 5,075 weekly downloads on npm
jsonpatch-js only applies patches, 2,014 weekly downloads in npm
jsonpatch.js only applies patches, 1,470 weekly downloads in npm
JSON8 Patch both diffs and patches, 400 weekly downloads on npm
By far, everybody (myself included), is using the Fast-JSON-Patch library. It works in NodeJS and in the browser.
im building a web app in html5.. basically a form with a time counter and questions and answers.
im looking for a way that the user cannot change the score (that is calculated from the time took to answer the question) via browser debugger or etc.
encrypting the raw data sounds like an options.. but when the data is at dom, the user can change it.
i added some "time checking" in server side.. but still i would prefer some client side protection as well.
any suggestions? thanks
I'm no web pro, but I'd say just stick all the validation on the server side. From what I know about people exploiting MMORPGs, there is always a way to access/change client side data.
What you're asking for is impossible. No matter how you implement it, the user can use debugging tools to alter how the code runs in their browser - or, ultimately, just generate the HTTP POST request themselves, independent of your code.
Well, since you're saying you're using html5, why don't you just use the storage support?
e.g:
var store = sessionStorage.question= new Array();
store[0]="10s";
store[1]="5s";
Now just set that programmatically! It will last for the whole session
Put that in a file and import it and the better-than-average user wont know where to look!
You can also check This Link for a more robust solution
As Nick says, a determined user will be able to get round any encryption scheme you use on the client machine. At most you can make it difficult for them to break. You need to do two things, 1) encrypt so as to make tampering difficult and 2) try to detect any tampering that does occur.
I don't know what is available off the shelf for Javascript, if available then use AES for encryption and HMAC to detect tampering. If you have to write your own then use RC4 for encryption (not as strong as AES but much simpler to code) and a checksum to detect tampering.
One thing you can do to make it more difficult for an attacker to find your encryption key and HMAC key is not to store them in one place. Have two arrays such that the real key is array1 XOR array2. That way the actual key is not explicitly in code anywhere.