Convert Uint16Array containing 8-bit integers to regular integer - javascript

How do you convert following
var data = new Uint16Array([131, 220]);
to integer? Expected value is somewhere around 970

Basically you need to reduce the values by multiplying with 28 and cut off the first bit.
(This is not a general converting, but rather for this special purpose.)
var data = new Uint16Array([131, 220]),
value = [].reduce.call(data, function (r, a) {
return (r << 8) + a;
}, 0) & ((1 << 15) - 1);
console.log(value);

Related

What's the JS equivalent of Pythons Tensorflow tf.random.set_seed(123)? [duplicate]

Is it possible to seed the random number generator (Math.random) in JavaScript?
No, it is not possible to seed Math.random(). The ECMAScript specification is intentionally vague on the subject, providing no means for seeding nor require that browsers even use the same algorithm. So such a function must be externally provided, which thankfully isn't too difficult.
I've implemented a number of good, short and fast Pseudorandom number generator (PRNG) functions in plain JavaScript. All of them can be seeded and provide high quality numbers. These are not intended for security purposes--if you need a seedable CSPRNG, look into ISAAC.
First of all, take care to initialize your PRNGs properly. To keep things simple, the generators below have no built-in seed generating procedure, but accept one or more 32-bit numbers as the initial seed state of the PRNG. Similar or sparse seeds (e.g. a simple seed of 1 and 2) have low entropy, and can cause correlations or other randomness quality issues, sometimes resulting in the output having similar properties (such as randomly generated levels being similar). To avoid this, it is best practice to initialize PRNGs with a well-distributed, high entropy seed and/or advancing past the first 15 or so numbers.
There are many ways to do this, but here are two methods. Firstly, hash functions are very good at generating seeds from short strings. A good hash function will generate very different results even when two strings are similar, so you don't have to put much thought into the string. Here's an example hash function:
function cyrb128(str) {
let h1 = 1779033703, h2 = 3144134277,
h3 = 1013904242, h4 = 2773480762;
for (let i = 0, k; i < str.length; i++) {
k = str.charCodeAt(i);
h1 = h2 ^ Math.imul(h1 ^ k, 597399067);
h2 = h3 ^ Math.imul(h2 ^ k, 2869860233);
h3 = h4 ^ Math.imul(h3 ^ k, 951274213);
h4 = h1 ^ Math.imul(h4 ^ k, 2716044179);
}
h1 = Math.imul(h3 ^ (h1 >>> 18), 597399067);
h2 = Math.imul(h4 ^ (h2 >>> 22), 2869860233);
h3 = Math.imul(h1 ^ (h3 >>> 17), 951274213);
h4 = Math.imul(h2 ^ (h4 >>> 19), 2716044179);
return [(h1^h2^h3^h4)>>>0, (h2^h1)>>>0, (h3^h1)>>>0, (h4^h1)>>>0];
}
Calling cyrb128 will produce a 128-bit hash value from a string which can be used to seed a PRNG. Here's how you might use it:
// Create cyrb128 state:
var seed = cyrb128("apples");
// Four 32-bit component hashes provide the seed for sfc32.
var rand = sfc32(seed[0], seed[1], seed[2], seed[3]);
// Only one 32-bit component hash is needed for mulberry32.
var rand = mulberry32(seed[0]);
// Obtain sequential random numbers like so:
rand();
rand();
Note: If you want a slightly more robust 128-bit hash, consider MurmurHash3_x86_128, it's more thorough, but intended for use with large arrays.
Alternatively, simply choose some dummy data to pad the seed with, and advance the generator beforehand a few times (12-20 iterations) to mix the initial state thoroughly. This has the benefit of being simpler, and is often used in reference implementations of PRNGs, but it does limit the number of initial states:
var seed = 1337 ^ 0xDEADBEEF; // 32-bit seed with optional XOR value
// Pad seed with Phi, Pi and E.
// https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number
var rand = sfc32(0x9E3779B9, 0x243F6A88, 0xB7E15162, seed);
for (var i = 0; i < 15; i++) rand();
Note: the output of these PRNG functions produce a positive 32-bit number (0 to 232-1) which is then converted to a floating-point number between 0-1 (0 inclusive, 1 exclusive) equivalent to Math.random(), if you want random numbers of a specific range, read this article on MDN. If you only want the raw bits, simply remove the final division operation.
JavaScript numbers can only represent whole integers up to 53-bit resolution. And when using bitwise operations, this is reduced to 32. Modern PRNGs in other languages often use 64-bit operations, which require shims when porting to JS that can drastically reduce performance. The algorithms here only use 32-bit operations, as it is directly compatible with JS.
Now, onward to the the generators. (I maintain the full list with references and license info here)
sfc32 (Simple Fast Counter)
sfc32 is part of the PractRand random number testing suite (which it passes of course). sfc32 has a 128-bit state and is very fast in JS.
function sfc32(a, b, c, d) {
return function() {
a >>>= 0; b >>>= 0; c >>>= 0; d >>>= 0;
var t = (a + b) | 0;
a = b ^ b >>> 9;
b = c + (c << 3) | 0;
c = (c << 21 | c >>> 11);
d = d + 1 | 0;
t = t + d | 0;
c = c + t | 0;
return (t >>> 0) / 4294967296;
}
}
You may wonder what the | 0 and >>>= 0 are for. These are essentially 32-bit integer casts, used for performance optimizations. Number in JS are basically floats, but during bitwise operations, they switch into a 32-bit integer mode. This mode is processed faster by JS interpreters, but any multiplication or addition will cause it to switch back to a float, resulting in a performance hit.
Mulberry32
Mulberry32 is a simple generator with a 32-bit state, but is extremely fast and has good quality randomness (author states it passes all tests of gjrand testing suite and has a full 232 period, but I haven't verified).
function mulberry32(a) {
return function() {
var t = a += 0x6D2B79F5;
t = Math.imul(t ^ t >>> 15, t | 1);
t ^= t + Math.imul(t ^ t >>> 7, t | 61);
return ((t ^ t >>> 14) >>> 0) / 4294967296;
}
}
I would recommend this if you just need a simple but decent PRNG and don't need billions of random numbers (see Birthday problem).
xoshiro128**
As of May 2018, xoshiro128** is the new member of the Xorshift family, by Vigna & Blackman (professor Vigna was also responsible for the Xorshift128+ algorithm powering most Math.random implementations under the hood). It is the fastest generator that offers a 128-bit state.
function xoshiro128ss(a, b, c, d) {
return function() {
var t = b << 9, r = a * 5; r = (r << 7 | r >>> 25) * 9;
c ^= a; d ^= b;
b ^= c; a ^= d; c ^= t;
d = d << 11 | d >>> 21;
return (r >>> 0) / 4294967296;
}
}
The authors claim it passes randomness tests well (albeit with caveats). Other researchers have pointed out that it fails some tests in TestU01 (particularly LinearComp and BinaryRank). In practice, it should not cause issues when floats are used (such as in these implementations), but may cause issues if relying on the raw lowest order bit.
JSF (Jenkins' Small Fast)
This is JSF or 'smallprng' by Bob Jenkins (2007), who also made ISAAC and SpookyHash. It passes PractRand tests and should be quite fast, although not as fast as sfc32.
function jsf32(a, b, c, d) {
return function() {
a |= 0; b |= 0; c |= 0; d |= 0;
var t = a - (b << 27 | b >>> 5) | 0;
a = b ^ (c << 17 | c >>> 15);
b = c + d | 0;
c = d + t | 0;
d = a + t | 0;
return (d >>> 0) / 4294967296;
}
}
No, it is not possible to seed Math.random(), but it's fairly easy to write your own generator, or better yet, use an existing one.
Check out: this related question.
Also, see David Bau's blog for more information on seeding.
NOTE: Despite (or rather, because of) succinctness and apparent elegance, this algorithm is by no means a high-quality one in terms of randomness. Look for e.g. those listed in this answer for better results.
(Originally adapted from a clever idea presented in a comment to another answer.)
var seed = 1;
function random() {
var x = Math.sin(seed++) * 10000;
return x - Math.floor(x);
}
You can set seed to be any number, just avoid zero (or any multiple of Math.PI).
The elegance of this solution, in my opinion, comes from the lack of any "magic" numbers (besides 10000, which represents about the minimum amount of digits you must throw away to avoid odd patterns - see results with values 10, 100, 1000). Brevity is also nice.
It's a bit slower than Math.random() (by a factor of 2 or 3), but I believe it's about as fast as any other solution written in JavaScript.
No, but here's a simple pseudorandom generator, an implementation of Multiply-with-carry I adapted from Wikipedia (has been removed since):
var m_w = 123456789;
var m_z = 987654321;
var mask = 0xffffffff;
// Takes any integer
function seed(i) {
m_w = (123456789 + i) & mask;
m_z = (987654321 - i) & mask;
}
// Returns number between 0 (inclusive) and 1.0 (exclusive),
// just like Math.random().
function random()
{
m_z = (36969 * (m_z & 65535) + (m_z >> 16)) & mask;
m_w = (18000 * (m_w & 65535) + (m_w >> 16)) & mask;
var result = ((m_z << 16) + (m_w & 65535)) >>> 0;
result /= 4294967296;
return result;
}
Antti Sykäri's algorithm is nice and short. I initially made a variation that replaced JavaScript's Math.random when you call Math.seed(s), but then Jason commented that returning the function would be better:
Math.seed = function(s) {
return function() {
s = Math.sin(s) * 10000; return s - Math.floor(s);
};
};
// usage:
var random1 = Math.seed(42);
var random2 = Math.seed(random1());
Math.random = Math.seed(random2());
This gives you another functionality that JavaScript doesn't have: multiple independent random generators. That is especially important if you want to have multiple repeatable simulations running at the same time.
Please see Pierre L'Ecuyer's work going back to the late 1980s and early 1990s. There are others as well. Creating a (pseudo) random number generator on your own, if you are not an expert, is pretty dangerous, because there is a high likelihood of either the results not being statistically random or in having a small period. Pierre (and others) have put together some good (pseudo) random number generators that are easy to implement. I use one of his LFSR generators.
https://www.iro.umontreal.ca/~lecuyer/myftp/papers/handstat.pdf
Combining some of the previous answers, this is the seedable random function you are looking for:
Math.seed = function(s) {
var mask = 0xffffffff;
var m_w = (123456789 + s) & mask;
var m_z = (987654321 - s) & mask;
return function() {
m_z = (36969 * (m_z & 65535) + (m_z >>> 16)) & mask;
m_w = (18000 * (m_w & 65535) + (m_w >>> 16)) & mask;
var result = ((m_z << 16) + (m_w & 65535)) >>> 0;
result /= 4294967296;
return result;
}
}
var myRandomFunction = Math.seed(1234);
var randomNumber = myRandomFunction();
It's not possible to seed the builtin Math.random function, but it is possible to implement a high quality RNG in Javascript with very little code.
Javascript numbers are 64-bit floating point precision, which can represent all positive integers less than 2^53. This puts a hard limit to our arithmetic, but within these limits you can still pick parameters for a high quality Lehmer / LCG random number generator.
function RNG(seed) {
var m = 2**35 - 31
var a = 185852
var s = seed % m
return function () {
return (s = s * a % m) / m
}
}
Math.random = RNG(Date.now())
If you want even higher quality random numbers, at the cost of being ~10 times slower, you can use BigInt for the arithmetic and pick parameters where m is just able to fit in a double.
function RNG(seed) {
var m_as_number = 2**53 - 111
var m = 2n**53n - 111n
var a = 5667072534355537n
var s = BigInt(seed) % m
return function () {
return Number(s = s * a % m) / m_as_number
}
}
See this paper by Pierre l'Ecuyer for the parameters used in the above implementations:
https://www.ams.org/journals/mcom/1999-68-225/S0025-5718-99-00996-5/S0025-5718-99-00996-5.pdf
And whatever you do, avoid all the other answers here that use Math.sin!
To write your own pseudo random generator is quite simple.
The suggestion of Dave Scotese is useful but, as pointed out by others, it is not quite uniformly distributed.
However, it is not because of the integer arguments of sin. It's simply because of the range of sin, which happens to be a one dimensional projection of a circle. If you would take the angle of the circle instead it would be uniform.
So instead of sin(x) use arg(exp(i * x)) / (2 * PI).
If you don't like the linear order, mix it a bit up with xor. The actual factor doesn't matter that much either.
To generate n pseudo random numbers one could use the code:
function psora(k, n) {
var r = Math.PI * (k ^ n)
return r - Math.floor(r)
}
n = 42; for(k = 0; k < n; k++) console.log(psora(k, n))
Please also note that you cannot use pseudo random sequences when real entropy is needed.
Many people who need a seedable random-number generator in Javascript these days are using David Bau's seedrandom module.
Math.random no, but the ran library solves this. It has almost all distributions you can imagine and supports seeded random number generation. Example:
ran.core.seed(0)
myDist = new ran.Dist.Uniform(0, 1)
samples = myDist.sample(1000)
Here's the adopted version of Jenkins hash, borrowed from here
export function createDeterministicRandom(): () => number {
let seed = 0x2F6E2B1;
return function() {
// Robert Jenkins’ 32 bit integer hash function
seed = ((seed + 0x7ED55D16) + (seed << 12)) & 0xFFFFFFFF;
seed = ((seed ^ 0xC761C23C) ^ (seed >>> 19)) & 0xFFFFFFFF;
seed = ((seed + 0x165667B1) + (seed << 5)) & 0xFFFFFFFF;
seed = ((seed + 0xD3A2646C) ^ (seed << 9)) & 0xFFFFFFFF;
seed = ((seed + 0xFD7046C5) + (seed << 3)) & 0xFFFFFFFF;
seed = ((seed ^ 0xB55A4F09) ^ (seed >>> 16)) & 0xFFFFFFFF;
return (seed & 0xFFFFFFF) / 0x10000000;
};
}
You can use it like this:
const deterministicRandom = createDeterministicRandom()
deterministicRandom()
// => 0.9872818551957607
deterministicRandom()
// => 0.34880331158638
No, like they said it is not possible to seed Math.random()
but you can install external package which make provision for that. i used these package which can be install using these command
npm i random-seed
the example is gotten from the package documentation.
var seed = 'Hello World',
rand1 = require('random-seed').create(seed),
rand2 = require('random-seed').create(seed);
console.log(rand1(100), rand2(100));
follow the link for documentation https://www.npmjs.com/package/random-seed
SIN(id + seed) is a very interesting replacement for RANDOM functions that cannot be seeded like SQLite:
https://stackoverflow.com/a/75089040/7776828
Most of the answers here produce biased results. So here's a tested function based on seedrandom library from github:
!function(f,a,c){var s,l=256,p="random",d=c.pow(l,6),g=c.pow(2,52),y=2*g,h=l-1;function n(n,t,r){function e(){for(var n=u.g(6),t=d,r=0;n<g;)n=(n+r)*l,t*=l,r=u.g(1);for(;y<=n;)n/=2,t/=2,r>>>=1;return(n+r)/t}var o=[],i=j(function n(t,r){var e,o=[],i=typeof t;if(r&&"object"==i)for(e in t)try{o.push(n(t[e],r-1))}catch(n){}return o.length?o:"string"==i?t:t+"\0"}((t=1==t?{entropy:!0}:t||{}).entropy?[n,S(a)]:null==n?function(){try{var n;return s&&(n=s.randomBytes)?n=n(l):(n=new Uint8Array(l),(f.crypto||f.msCrypto).getRandomValues(n)),S(n)}catch(n){var t=f.navigator,r=t&&t.plugins;return[+new Date,f,r,f.screen,S(a)]}}():n,3),o),u=new m(o);return e.int32=function(){return 0|u.g(4)},e.quick=function(){return u.g(4)/4294967296},e.double=e,j(S(u.S),a),(t.pass||r||function(n,t,r,e){return e&&(e.S&&v(e,u),n.state=function(){return v(u,{})}),r?(c[p]=n,t):n})(e,i,"global"in t?t.global:this==c,t.state)}function m(n){var t,r=n.length,u=this,e=0,o=u.i=u.j=0,i=u.S=[];for(r||(n=[r++]);e<l;)i[e]=e++;for(e=0;e<l;e++)i[e]=i[o=h&o+n[e%r]+(t=i[e])],i[o]=t;(u.g=function(n){for(var t,r=0,e=u.i,o=u.j,i=u.S;n--;)t=i[e=h&e+1],r=r*l+i[h&(i[e]=i[o=h&o+t])+(i[o]=t)];return u.i=e,u.j=o,r})(l)}function v(n,t){return t.i=n.i,t.j=n.j,t.S=n.S.slice(),t}function j(n,t){for(var r,e=n+"",o=0;o<e.length;)t[h&o]=h&(r^=19*t[h&o])+e.charCodeAt(o++);return S(t)}function S(n){return String.fromCharCode.apply(0,n)}if(j(c.random(),a),"object"==typeof module&&module.exports){module.exports=n;try{s=require("crypto")}catch(n){}}else"function"==typeof define&&define.amd?define(function(){return n}):c["seed"+p]=n}("undefined"!=typeof self?self:this,[],Math);
function randIntWithSeed(seed, max=1) {
/* returns a random number between [0,max] including zero and max
seed can be either string or integer */
return Math.round(new Math.seedrandom('seed' + seed)()) * max
}
test for true randomness of this code: https://es6console.com/kkjkgur2/
There are plenty of good answers here but I had a similar issue with the additional requirement that I would like portability between Java's random number generator and whatever I ended up using in JavaScript.
I found the java-random package
These two pieces of code had identical output assuming the seed is the same:
Java:
Random randomGenerator = new Random(seed);
int randomInt;
for (int i=0; i<10; i++) {
randomInt = randomGenerator.nextInt(50);
System.out.println(randomInt);
}
JavaScript:
let Random = require('java-random');
let rng = new Random(seed);
for (let i=0; i<10; i++) {
let val = rng.nextInt(50);
console.log(val);
}
Do what bryc suggests ... but before you use his cyrb128 hash function to initialise, note that the return statement throws away 32 bits of entropy. Exclusive-or the four values together = 0. You should probably make the first element (h2^h3^h4) >>> 0.
I have written a function that returns a seeded random number, it uses Math.sin to have a long random number and uses the seed to pick numbers from that.
Use :
seedRandom("k9]:2#", 15)
it will return your seeded number
the first parameter is any string value ; your seed.
the second parameter is how many digits will return.
function seedRandom(inputSeed, lengthOfNumber){
var output = "";
var seed = inputSeed.toString();
var newSeed = 0;
var characterArray = ['0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','y','x','z','A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','U','R','S','T','U','V','W','X','Y','Z','!','#','#','$','%','^','&','*','(',')',' ','[','{',']','}','|',';',':',"'",',','<','.','>','/','?','`','~','-','_','=','+'];
var longNum = "";
var counter = 0;
var accumulator = 0;
for(var i = 0; i < seed.length; i++){
var a = seed.length - (i+1);
for(var x = 0; x < characterArray.length; x++){
var tempX = x.toString();
var lastDigit = tempX.charAt(tempX.length-1);
var xOutput = parseInt(lastDigit);
addToSeed(characterArray[x], xOutput, a, i);
}
}
function addToSeed(character, value, a, i){
if(seed.charAt(i) === character){newSeed = newSeed + value * Math.pow(10, a)}
}
newSeed = newSeed.toString();
var copy = newSeed;
for(var i=0; i<lengthOfNumber*9; i++){
newSeed = newSeed + copy;
var x = Math.sin(20982+(i)) * 10000;
var y = Math.floor((x - Math.floor(x))*10);
longNum = longNum + y.toString()
}
for(var i=0; i<lengthOfNumber; i++){
output = output + longNum.charAt(accumulator);
counter++;
accumulator = accumulator + parseInt(newSeed.charAt(counter));
}
return(output)
}
A simple approach for a fixed seed:
function fixedrandom(p){
const seed = 43758.5453123;
return (Math.abs(Math.sin(p)) * seed)%1;
}
In PHP, there is function srand(seed) which generate fixed random value for particular seed.
But, in JS, there is no such inbuilt function.
However, we can write simple and short function.
Step 1: Choose some Seed (Fix Number).
var seed = 100;
Number should be Positive Integer and greater than 1, further explanation in Step 2.
Step 2: Perform Math.sin() function on Seed, it will give sin value of that number. Store this value in variable x.
var x;
x = Math.sin(seed); // Will Return Fractional Value between -1 & 1 (ex. 0.4059..)
sin() method returns a Fractional value between -1 and 1.And we don't need Negative value, therefore, in first step choose number greater than 1.
Step 3: Returned Value is a Fractional value between -1 and 1. So mulitply this value with 10 for making it more than 1.
x = x * 10; // 10 for Single Digit Number
Step 4: Multiply the value with 10 for additional digits
x = x * 10; // Will Give value between 10 and 99 OR
x = x * 100; // Will Give value between 100 and 999
Multiply as per requirement of digits.
The result will be in decimal.
Step 5: Remove value after Decimal Point by Math's Round (Math.round()) Method.
x = Math.round(x); // This will give Integer Value.
Step 6: Turn Negative Values into Positive (if any) by Math.abs method
x = Math.abs(x); // Convert Negative Values into Positive(if any)
Explanation End.Final Code
var seed = 111; // Any Number greater than 1
var digit = 10 // 1 => single digit, 10 => 2 Digits, 100 => 3 Digits and so. (Multiple of 10)
var x; // Initialize the Value to store the result
x = Math.sin(seed); // Perform Mathematical Sin Method on Seed.
x = x * 10; // Convert that number into integer
x = x * digit; // Number of Digits to be included
x = Math.round(x); // Remove Decimals
x = Math.abs(x); // Convert Negative Number into Positive
Clean and Optimized Functional Code
function random_seed(seed, digit = 1) {
var x = Math.abs(Math.round(Math.sin(seed++) * 10 * digit));
return x;
}
Then Call this function using
random_seed(any_number, number_of_digits)any_number is must and should be greater than 1.number_of_digits is optional parameter and if nothing passed, 1 Digit will return.
random_seed(555); // 1 Digit
random_seed(234, 1); // 1 Digit
random_seed(7895656, 1000); // 4 Digit
For a number between 0 and 100.
Number.parseInt(Math.floor(Math.random() * 100))

How to split large integer into an array of 8-bit integers

Wondering how to convert the output of arbitrarily sized integers like 1 or 12345 or 5324617851000199519157 to an array of integers.
[1] // for the first one
// [probably just a few values for the second 12345...]
[1, 123, 255, 32, ...] // not sure here...
I am not sure what the resulting value would look like or how to compute it, but somehow it would be something like:
A bunch of 8-bit numbers that can be used to reconstruct (somehow) the original arbitrary integer. I am not sure what calculations would be required to do this either. But all I do know is that each unique arbitrarily-sized integer should result in a unique array of 8-bit values. That is, no two different date integers should result in the same array.
It doesn't matter the language much how this is implemented, but probably an imperative language like JavaScript or C.
I am pretty sure the arrays should all be the same length as well, but if that's not possible then knowing how to do it a different way would be okay.
I'm not sure if this is too brute-forcey for what you want, but you can take an arbitrary string and just do the long division into a unit8Array.
Here's a function (borrowed liberally from here) that will convert back and forth from an arbitrarily long string:
function eightBit(str){
let dec = [...str], sum = []
while(dec.length){
let s = 1 * dec.shift()
for(let i = 0; s || i < sum.length; i++){
s += (sum[i] || 0) * 10
sum[i] = s % 256
s = (s - sum[i]) / 256
}
}
return Uint8Array.from(sum.reverse())
}
function eightBit2String(arr){
var dec = [...arr], sum = []
while(dec.length){
let s = 1 * dec.shift()
for(let i = 0; s || i < sum.length; i++){
s += (sum[i] || 0) * 256
sum[i] = s % 10
s = (s - sum[i]) / 10
}
}
return sum.reverse().join('')
}
// sanity check
console.log("256 = ", eightBit('256'), "258 = ", eightBit('258'))
let n = '47171857151875817758571875815815782572758275672576575677'
let a = eightBit(n)
console.log("to convert:", n)
console.log("converted:", a.toString())
let s = eightBit2String(a)
console.log("converted back:", s)
No doubt, there are some efficiencies to be found (maybe you can avoid the interim arrays).
Most languages, including C and Javascript, have bit-shifting and bit-masking operations as part of their basic math operations. But beware Javascript: numbers are 64 bits, but only 32-bit masking operations are allowed. So:
let bignum = Date.now();
let hi = Math.floor(bignum / 0x100000000),
lo = bignum & 0xFFFFFFFF,
bytes = [
(hi >> 24) & 0xFF,
(hi >> 16) & 0xFF,
(hi >> 8) & 0xFF,
hi & 0xFF,
(lo >> 24) & 0xFF,
(lo >> 16) & 0xFF,
(lo >> 8) & 0xFF,
lo & 0xFF
];

Create a bit mask for Javascript typed arrays

I am working with Javascript typed arrays, and I need to compress them as much as possible for networking purposes.
The smallest built in array Javascript has is 8 bits per entry. This will store numbers between 0 and 255.
However the data I'm working with will only contain numbers between 0 and 3. This can can be stored using 2 bits.
So my question is, if I have an 8 bit array that is populated with data only using numbers between 0 and 3, how can I "convert" it into a 2 bit array?
I know I'll need to use a bit operator, but I'm not sure how to make a mask that will only focus on 2 bits at a time.
A longer example is hard to fit into a comment :)
Up front, please note that very often, network data is compressed already - e.g. with gzip (especially when there is concern about data volume and the network libraries are setup properly). However, this is not always the case and would still not be as compact as doing it manually.
You need to keep track of two things, the current array index and the current slot inside the 8-Bit that is being read or written. For writing, | is useful, for reading &. Shifts (<< or >>) are used to select the position.
const randomTwoBitData = () => Math.floor(Math.random() * 4);
//Array of random 2-Bit data
const sampleData = Array(256).fill().map(e => randomTwoBitData());
//four entries per 8-Bit
let buffer = new Uint8Array(sampleData.length / 4);
//Writing data, i made my life easy
//because the data is divisible by four and fits perfectly.
for (let i = 0; i < sampleData.length; i += 4) {
buffer[i / 4] =
sampleData[i] |
(sampleData[i + 1] << 2) |
(sampleData[i + 2] << 4) |
(sampleData[i + 3] << 6);
}
//padding for console logging
const pad = (s, n) => "0".repeat(Math.max(0, n - s.length)) + s;
//Some output to see results at the middle
console.log(`buffer: ${pad(buffer[31].toString(2), 8)}, ` +
`original data: ${pad(sampleData[127].toString(2), 2)}, ` +
`${pad(sampleData[126].toString(2), 2)}, ` +
`${pad(sampleData[125].toString(2), 2)}, ` +
`${pad(sampleData[124].toString(2), 2)}`);
console.log("(order of original data inverted for readability)");
console.log("");
//Reading back:
let readData = [];
buffer.forEach(byte => {
readData.push(byte & 3); // 3 is 00000011 binary
readData.push((byte & 12) >> 2); // 12 is 00001100 binary
readData.push((byte & 48) >> 4); // 48 is 00110000 binary
readData.push((byte & 192) >> 6); // 192 is 11000000 binary
});
//Check if data read from compacted buffer is the same
//as the original
console.log(`original data and re-read data are identical: ` +
readData.every((e, i) => e === sampleData[i]));
Here is a function to do 8 bits number to 2 bits array of length 4 with & and >>:
function convert8to2(val){
var arr = [];
arr.push((val&parseInt('11000000', 2))>>6);
arr.push((val&parseInt('00110000', 2))>>4);
arr.push((val&parseInt('00001100', 2))>>2);
arr.push((val&parseInt('00000011', 2)));
return arr;
}
function convert2to8(arr){
if(arr.length != 4)
throw 'erorr';
return (arr[0]<<6)+(arr[1]<<4)+(arr[2]<<2)+arr[3];
}
// 228 = 11100100
var arr = convert8to2(228);
console.log(arr);
console.log(convert2to8(arr));
Edited
Change the example value and format the binary number with leading 0
Edited
Add convert2to8 and create an example usage:
function convert8to2(val){
var arr = [];
arr.push((val&parseInt('11000000', 2))>>6);
arr.push((val&parseInt('00110000', 2))>>4);
arr.push((val&parseInt('00001100', 2))>>2);
arr.push((val&parseInt('00000011', 2)));
return arr;
}
function convert2to8(arr){
if(arr.length != 4)
throw 'erorr';
return (arr[0]<<6)+(arr[1]<<4)+(arr[2]<<2)+arr[3];
}
var randomData = [];
for(var i=0;i<10;i++){
randomData.push(Math.floor(Math.random() * 255));
}
console.log(randomData);
var arrayOf2 = []
for(var i=0;i<randomData.length;i++){
arrayOf2.push(convert8to2(randomData[i]));
}
console.log(arrayOf2);
var arrayOf8 = [];
for(var i=0;i<arrayOf2.length;i++){
arrayOf8.push(convert2to8(arrayOf2[i]));
}
console.log(arrayOf8);

JavaScript calculate hashcode from real number and integer number

Hi there I need function to calculate unique integer number from number (real number double precision) and integer.
Try explain I am developing GIS application in javascript and I am working with complex vector object like polygon (array of points object with two coordinate in ring) and lines array of points. I need fast algorithm to recognize that element has been changed it must be really fast because my vector object is collection of thousand points . In C# I am calculating hash code from coordinate using bitwise operation XOR.
But javascript convert all operands in bitwise operation to integer but i need convert double precision to integer before apply bitwise in c# way (binnary). In reflector i see this that c# calculate hash code fro double like this and I need this function in javascript as fast as can be.
public override unsafe int GetHashCode() //from System.Double
{
double num = this;
if (num == 0.0)
{
return 0;
}
long num2 = *((long*) &num);
return (((int) num2) ^ ((int) (num2 >> 32)));
}
Example:
var rotation = function (n) {
n = (n >> 1) | ((n & 0x001) << 31);
return n;
}
var x: number = 1;
var y: number = 5;
var hash = x ^ rotation(y); // result is -2147483645
var x1: number = 1.1;
var y1: number = 5;
var hash1 = x1 ^ rotation(y1); // result is -2147483645
Example result is not correct hash == hash1
Example 2: Using to string there is correct result but calculate Hash from string is to complicate and I thing is not fast enough.
var rotation = function (n) {
n = (n >> 1) | ((n & 0x001) << 31);
return n;
}
var GetHashCodeString = function(str: string): number {
var hash = 0, i, l, ch;
if (str.length == 0) return hash;
for (i = 0, l = str.length; i < l; i++) {
ch = str.charCodeAt(i);
hash = ((hash << 5) - hash) + ch;
hash |= 0; // Convert to 32bit integer
}
return hash;
}
var x: number = 1;
var y: number = 5;
var hash = GetHashCodeString(x.toString()) ^ rotation(GetHashCodeString(y.toString()));
//result is -2147483605
var x1: number = 1.1;
var y1: number = 5;
var hash1 = GetHashCodeString(x1.toString()) ^ rotation(GetHashCodeString(y1.toString()));
//result is -2147435090
Example2 result is correct hash != hash1
Is there some faster way than converting number to string than calculate hash from each character? Because my object is very large and it will take lot of time and operation in this way ...
I try do it using TypedArrays but yet I am not successful.
Thanks very much for your help
Hi there I tried use TypedArrays to calculate Hash code from number and the result is interesting. In IE the performance 4x better in Chrome 2x in FireFox this approach is equal to string version ...
var GetHashCodeNumber = function (n: number): number {
//create 8 byte array buffer number in js is 64bit
var arr = new ArrayBuffer(8);
//create view to array buffer
var dv = new DataView(arr);
//set number to buffer as 64 bit float
dv.setFloat64(0, n);
//now get first 32 bit from array and convert it to integer
// from offset 0
var c = dv.getInt32(0);
//now get next 32 bit from array and convert it to integer
//from offset 4
var d = dv.getInt32(4);
//XOR first end second integer numbers
return c ^ d;
}
I think this can be useful for someone
EDIT: using one buffer and DataView is faster !
Here is a faster way to do this in JavaScript.
const kBuf = new ArrayBuffer(8);
const kBufAsF64 = new Float64Array(kBuf);
const kBufAsI32 = new Int32Array(kBuf);
function hashNumber(n) {
// Remove this `if` if you want 0 and -0 to hash to different values.
if (~~n === n) {
return ~~n;
}
kBufAsF64[0] = n;
return kBufAsI32[0] ^ kBufAsI32[1];
}
It's 250x faster than the DataView approach: see benchmark.
I looked up some hashing libraries to see how they did it: xxhashjs, jshashes, etc.
Most seem to take a string or an ArrayBuffer, and also depend on UINT32-like functionality. This is equivalent to you needing a binary representation of the double (from your C# example). Notably I did not find any solution that included more-strange types, other than in another (unanswered) question.
His solution uses a method proposed here, which converts it to various typed arrays. This is most likely what you want, and the fastest accurate solution (I think).
I highly recommend that you structure your code to traverse objects/arrays as desired, and also benchmark the solution to see how comparable it is to your existing methods (the non-working one and the string one).

Convert byte array to numbers in JavaScript

I have JavaScript code that retrieves numerical vectors from a web-service. The original data is an array of doubles that is converted to a byte array and then base64 encoded. I decode from base64 in JavaScript, but then I don't know how to transform the resulting bytes into an array of numbers.
This was the only way I could think of off the top of my head to do it.
function bytesToDouble(str,start) {
start *= 8;
var data = [str.charCodeAt(start+7),
str.charCodeAt(start+6),
str.charCodeAt(start+5),
str.charCodeAt(start+4),
str.charCodeAt(start+3),
str.charCodeAt(start+2),
str.charCodeAt(start+1),
str.charCodeAt(start+0)];
var sign = (data[0] & 1<<7)>>7;
var exponent = (((data[0] & 127) << 4) | (data[1]&(15<<4))>>4);
if(exponent == 0) return 0;
if(exponent == 0x7ff) return (sign) ? Number.POSITIVE_INFINITY : Number.NEGATIVE_INFINITY;
var mul = Math.pow(2,exponent - 1023 - 52);
var mantissa = data[7]+
data[6]*Math.pow(2,8*1)+
data[5]*Math.pow(2,8*2)+
data[4]*Math.pow(2,8*3)+
data[3]*Math.pow(2,8*4)+
data[2]*Math.pow(2,8*5)+
(data[1]&15)*Math.pow(2,8*6)+
Math.pow(2,52);
return Math.pow(-1,sign)*mantissa*mul;
}
var data = atob("AAAAAABsskAAAAAAAPmxQAAAAAAAKrF");
alert(bytesToDouble(data,0)); // 4716.0
alert(bytesToDouble(data,1)); // 4601.0
This should give you a push in the right direction, though it took me a while to remember how to deal with doubles.
One big caveats to note though:
This relies on the atob to do the base64 decoding, which is not supported everywhere, and aside from that probably isn't a great idea anyway. What you really want to do is unroll the base64 encoded string to an array of numbers (bytes would be the easiest to work with although not the most efficient thing on the planet). The reason is that when atob does its magic, it returns a string, which is far from ideal. Depending on the encoding the code points it maps to (especially for code points between 128 and 255) the resulting .charCodeAt() may not return what you expect.
And there may be some accuracy issues, because after all I am using a double to calculate a double, but I think it might be okay.
Base64 is fairly trivial to work with, so you should be able to figure that part out.
If you did switch to an array (rather than the str string now), then you would obviously drop the .charCodeAt() reference and just get the indices you want directly.
There is a functioning fiddle here
I assume we have used this function in web service (c#) to encode the double array data as string:
//Input: [552.4, 539.8]
//Output: IOz0gCAsscA=
private string ConvertToSerializableString(double[] input)
{
byte[] data = new byte[input.Length * 4];
for (int i = 0; i < input.Length; i++)
{
int source = (int)(input[i] * 1E6);
int dataIndex = i * 4;
data[dataIndex] = (byte)((source >> 24) & 0xFF);
data[dataIndex + 1] = (byte)((source >> 16) & 0xFF);
data[dataIndex + 2] = (byte)((source >> 8) & 0xFF);
data[dataIndex + 3] = (byte)(source & 0xFF);
}
return Convert.ToBase64String(data);
}
Then we can use the following client script (javascript) to decode it:
var base64EncodedDoubleArrayData = "IOz0gCAsscA=";
var byteData = window.atob(base64EncodedDoubleArrayData);
var doubleArray = [];
for (var iColumn = 0; iColumn < byteData.length; iColumn = iColumn + 4)
{
var item = (byteData.charCodeAt(iColumn) << 24) + (byteData.charCodeAt(iColumn + 1) << 16) + (byteData.charCodeAt(iColumn + 2) << 8) + byteData.charCodeAt(iColumn + 3);
var doubleResult = parseFloat(item/1e6);
doubleArray.push(doubleResult);
}
//it should return something like doubleArray = [552.4, 539.8]

Categories

Resources