Does the JavaScript code
var n = 8; // or some arbitrary integer literal
n >> 1;
always denote "integer devision by 2 without remainer"? My concern is the endianess if the integer literal is larger than one byte.
The background of my question is the following:
I have an integer variable in the range from 0 to 2^32-1 that would fit into an uint32 if I had a typed programming language different than JS. I need to convert this into an Uint4Array with four elements in little endian order.
My current JavaScript approach is:
function uInt32ToLEByteArray( n ) {
var byteArray = new Uint8Array(4);
for( var i = 0; i < 4; i++ ) {
byteArray[i] = n & 255;
n >> 8;
}
return byteArray;
}
This code works in my browser, but I wonder if this would do everywhere. The principal idea is the fill the array by taking the LSB and divdiding by 256. But a real divions "/" would convert the variable into a floating point variable. So I use ">>8" but this actually assumes big endianness.
The code you have given has absolutely no relevancy to endianess.
However, if you were to reinterpret the byte array in say uint32 array, then the result would be different depending on the endianess of the machine the browser runs on.
First, fix the bug in the code:
function uInt32ToLEByteArray(n) {
var byteArray = new Uint8Array(4);
for (var i = 0; i < 4; i++) {
byteArray[i] = n & 255;
n >>>= 8; //simply doing n >> 8 has no effect actually
}
return byteArray;
}
Then
var a = uInt32ToLEByteArray(0xFF)
console.log(a);
//always [255, 0, 0, 0]
var b = new Uint32Array(a.buffer);
console.log(b);
//[255] on little endian machines
//[4278190080] on big endian machines
Related
I need to add compression to my project and I decided to use the LZJB algorithm that is fast and the code is small. Found this library https://github.com/copy/jslzjb-k
But the API is not very nice because to decompress the file you need input buffer length (because Uint8Array is not dynamic you need to allocate some data). So I want to save the length of the input buffer as the first few bytes of Uint8Array so I can extract that value and create output Uint8Array based on that integer value.
I want the function that returns Uint8Array from integer to be generic, maybe save the length of the bytes into the first byte so you know how much data you need to extract to read the integer. I guess I need to extract those bytes and use some bit shifting to get the original number. But I'm not exactly sure how to do this.
So how can I write a generic function that converts an integer into Uint8Array that can be embedded into a bigger array and then extract that number?
Here are working functions (based on Converting javascript Integer to byte array and back)
function numberToBytes(number) {
// you can use constant number of bytes by using 8 or 4
const len = Math.ceil(Math.log2(number) / 8);
const byteArray = new Uint8Array(len);
for (let index = 0; index < byteArray.length; index++) {
const byte = number & 0xff;
byteArray[index] = byte;
number = (number - byte) / 256;
}
return byteArray;
}
function bytesToNumber(byteArray) {
let result = 0;
for (let i = byteArray.length - 1; i >= 0; i--) {
result = (result * 256) + byteArray[i];
}
return result;
}
by using const len = Math.ceil(Math.log2(number) / 8); the array have only bytes needed. If you want a fixed size you can use a constant 8 or 4.
In my case, I just saved the length of the bytes in the first byte.
General answer
These functions allow any integer (it uses BigInts internally, but can accept Number arguments) to be encoded into, and decoded from, any part of a Uint8Array. It is somewhat overkill, but I wanted to learn how to work with arbitrary-sized integers in JS.
// n can be a bigint or a number
// bs is an optional Uint8Array of sufficient size
// if unspecified, a large-enough Uint8Array will be allocated
// start (optional) is the offset
// where the length-prefixed number will be written
// returns the resulting Uint8Array
function writePrefixedNum(n, bs, start) {
start = start || 0;
let len = start+2; // start, length, and 1 byte min
for (let i=0x100n; i<n; i<<=8n, len ++) /* increment length */;
if (bs === undefined) {
bs = new Uint8Array(len);
} else if (bs.length < len) {
throw `byte array too small; ${bs.length} < ${len}`;
}
let r = BigInt(n);
for (let pos = start+1; pos < len; pos++) {
bs[pos] = Number(r & 0xffn);
r >>= 8n;
}
bs[start] = len-start-1; // write byte-count to start byte
return bs;
}
// bs must be a Uint8Array from where the number will be read
// start (optional, defaults to 0)
// is where the length-prefixed number can be found
// returns a bigint, which can be coerced to int using Number()
function readPrefixedNum(bs, start) {
start = start || 0;
let size = bs[start]; // read byte-count from start byte
let n = 0n;
if (bs.length < start+size) {
throw `byte array too small; ${bs.length} < ${start+size}`;
}
for (let pos = start+size; pos >= start+1; pos --) {
n <<= 8n;
n |= BigInt(bs[pos])
}
return n;
}
function test(n) {
const array = undefined;
const offset = 2;
let bs = writePrefixedNum(n, undefined, offset);
console.log(bs);
let result = readPrefixedNum(bs, offset);
console.log(n, result, "correct?", n == result)
}
test(0)
test(0x1020304050607080n)
test(0x0807060504030201n)
Simple 4-byte answer
This answer encodes 4-byte integers to and from Uint8Arrays.
function intToArray(i) {
return Uint8Array.of(
(i&0xff000000)>>24,
(i&0x00ff0000)>>16,
(i&0x0000ff00)>> 8,
(i&0x000000ff)>> 0);
}
function arrayToInt(bs, start) {
start = start || 0;
const bytes = bs.subarray(start, start+4);
let n = 0;
for (const byte of bytes.values()) {
n = (n<<8)|byte;
}
return n;
}
for (let v of [123, 123<<8, 123<<16, 123<<24]) {
let a = intToArray(v);
let r = arrayToInt(a, 0);
console.log(v, a, r);
}
Posting this one-liner in case it is useful to anyone who is looking to work with numbers below 2^53. This strictly uses bitwise operations and has no need for constants or values other than the input to be defined.
export const encodeUvarint = (n: number): Uint8Array => n >= 0x80
? Uint8Array.from([(n & 0x7f) | 0x80, ...encodeUvarint(n >> 7)])
: Uint8Array.from([n & 0xff]);
I have a working script in python doing string to integer conversion based on specified radix using long(16):
modulus=public_key["n"]
modulusDecoded = long(public_key["n"], 16)
which prints:
8079d7ae567dd2c02dadd1068843136314fa3893fa1fb1ab331682c6a85cad62b208d66c9974bbbb15d52676fd9907efb158c284e96f5c7a4914fd927b7326c40efa14922c68402d05ff53b0e4ccda90bbee5e6c473613e836e2c79da1072e366d0d50933327e77651b6984ddbac1fdecf1fd8fa17e0f0646af662a8065bd873
and
90218878289834622370514047239437874345637539049004160177768047103383444023879266805615186962965710608753937825108429415800005684101842952518531920633990402573136677611127418094912644368840442620417414685225340199872975797295511475162170060618806831021437109054760851445152320452665575790602072479287289305203
respectively.
This looks like a Hex to decimal conversion.
I tried to have the same result in JS but parseInt() and parseFloat() produce something completely different. On top of that JavaScript seems not to like chars in input string and sometimes returns NaN.
Could anyone please provide a function / guidance how to get the same functionality as in Python script?
Numbers in JavaScript are floating point so they always lose precision after a certain digit. To have unlimited numbers one could rather use an array of numbers from 0 to 9, which has an unlimited range. To do so based on the hex string input, i do a hex to int array conversion, then I use the double dabble algorithm to convert the array to BCD. That can be printed easily:
const hexToArray = arr => arr.split("").map(n => parseInt(n,16));
const doubleDabble = arr => {
var l = arr.length;
for( var b = l * 4; b--;){
//add && leftshift
const overflow = arr.reduceRight((carry,n,i) => {
//apply the >4 +3, then leftshift
var shifted = ((i < (arr.length - l ) && n>4)?n+3:n ) << 1;
//just take the right four bits and add the eventual carry value
arr[i] = (shifted & 0b1111) | carry;
//carry on
return shifted > 0b1111;
}, 0);
// we've exceeded the current array, lets extend it:
if(overflow) arr.unshift(overflow);
}
return arr.slice(0,-l);
};
const arr = hexToArray("8079d7");
const result = doubleDabble(arr);
console.log(result.join(""));
Try it
Using the built in api parseInt, you can get upto 100 digts of accuracy on Firefox and 20 digits of accuracy on Chrome.
a = parseInt('8079d7ae567dd2c02dadd1068843136314fa3893fa1fb1ab331682c6a85cad62b208d66c9974bbbb15d52676fd9907efb158c284e96f5c7a4914fd927b7326c40efa14922c68402d05ff53b0e4ccda90bbee5e6c473613e836e2c79da1072e366d0d50933327e77651b6984ddbac1fdecf1fd8fa17e0f0646af662a8065bd873', 16)
a.toPrecision(110)
> Uncaught RangeError: toPrecision() argument must be between 1 and 21
# Chrome
a.toPrecision(20)
"9.0218878289834615508e+307"
# Firefox
a.toPrecision(100)
"9.021887828983461550807409292694387726882781812072572899692574101215517323445643340153182035092932819e+307"
From the ECMAScript Spec,
Let p be ? ToInteger(precision).
...
If p < 1 or p > 100, throw a RangeError exception.
As described in this answer, JavaScript numbers cannot represent integers larger than 9.007199254740991e+15 without loss of precision.
Working with larger integers in JavaScript requires a BigInt library or other special-purpose code, and large integers will then usually be represented as strings or arrays.
Re-using code from this answer helps to convert the hexadecimal number representation
8079d7ae567dd2c02dadd1068843136314fa3893fa1fb1ab331682c6a85cad62b208d66c9974bbbb15d52676fd9907efb158c284e96f5c7a4914fd927b7326c40efa14922c68402d05ff53b0e4ccda90bbee5e6c473613e836e2c79da1072e366d0d50933327e77651b6984ddbac1fdecf1fd8fa17e0f0646af662a8065bd873
to its decimal representation
90218878289834622370514047239437874345637539049004160177768047103383444023879266805615186962965710608753937825108429415800005684101842952518531920633990402573136677611127418094912644368840442620417414685225340199872975797295511475162170060618806831021437109054760851445152320452665575790602072479287289305203
as demonstrated in the following snippet:
function parseBigInt(bigint, base) {
//convert bigint string to array of digit values
for (var values = [], i = 0; i < bigint.length; i++) {
values[i] = parseInt(bigint.charAt(i), base);
}
return values;
}
function formatBigInt(values, base) {
//convert array of digit values to bigint string
for (var bigint = '', i = 0; i < values.length; i++) {
bigint += values[i].toString(base);
}
return bigint;
}
function convertBase(bigint, inputBase, outputBase) {
//takes a bigint string and converts to different base
var inputValues = parseBigInt(bigint, inputBase),
outputValues = [], //output array, little-endian/lsd order
remainder,
len = inputValues.length,
pos = 0,
i;
while (pos < len) { //while digits left in input array
remainder = 0; //set remainder to 0
for (i = pos; i < len; i++) {
//long integer division of input values divided by output base
//remainder is added to output array
remainder = inputValues[i] + remainder * inputBase;
inputValues[i] = Math.floor(remainder / outputBase);
remainder -= inputValues[i] * outputBase;
if (inputValues[i] == 0 && i == pos) {
pos++;
}
}
outputValues.push(remainder);
}
outputValues.reverse(); //transform to big-endian/msd order
return formatBigInt(outputValues, outputBase);
}
var largeNumber =
'8079d7ae567dd2c02dadd1068843136314fa389'+
'3fa1fb1ab331682c6a85cad62b208d66c9974bb'+
'bb15d52676fd9907efb158c284e96f5c7a4914f'+
'd927b7326c40efa14922c68402d05ff53b0e4cc'+
'da90bbee5e6c473613e836e2c79da1072e366d0'+
'd50933327e77651b6984ddbac1fdecf1fd8fa17'+
'e0f0646af662a8065bd873';
//convert largeNumber from base 16 to base 10
var largeIntDecimal = convertBase(largeNumber, 16, 10);
//show decimal result in console:
console.log(largeIntDecimal);
//check that it matches the expected output:
console.log('Matches expected:',
largeIntDecimal === '90218878289834622370514047239437874345637539049'+
'0041601777680471033834440238792668056151869629657106087539378251084294158000056'+
'8410184295251853192063399040257313667761112741809491264436884044262041741468522'+
'5340199872975797295511475162170060618806831021437109054760851445152320452665575'+
'790602072479287289305203'
);
//check that conversion and back-conversion results in the original number
console.log('Converts back:',
convertBase(convertBase(largeNumber, 16, 10), 10, 16) === largeNumber
);
I am new to coding and javascript and was asked, for an assignment, to convert base 10 numbers to a binary base without using specific Javascript built in methods (like alert(a.toString(16))), and I am only allowed to use loops,arrays and functions. This is what i have so far:
var number = prompt("Enter an unsigned base 10 number");
if (number>=0) {
var base = prompt("Enter b for binary, o for octal, or h for hexadecimal");
if (base=="h"||base=="H") {
;
}
So as you can see, I don't have much to go on. I was curious as to what equation or formula I would use to convert the base 10 number, as well as how i'm supposed to show A=10, B=11, C=12 and so forth for a hexadecimal base. Any help would be greatly appreciated!
edit: This is a rather complicated way to do it,
as Alnitak showed me (see discussion below).
It is more a scibble, or the long way by foot.
Short explanation:
If we want to get the binary of the decimal number 10,
we have to try 2^n so that 2^n is still smaller than 10.
For example 2^3 = 8 (that is OK). But 2^4 = 16 (thats too big).
So we have 2^3 and store a 1 for that in an array at index 3.
Now we have to get the rest of 10-2^3, which is 2, and have to
make the same calculation again until we get a difference of zero.
At last we have to reverse the array because its the other way arround.
var a = prompt("Enter an unsigned base 10 number");
var arr = [];
var i = 0;
function decToBin(x) {
y = Math.pow(2, i);
if (y < x) {
arr[i] = 0;
i++;
decToBin(x);
} else if (y > x) {
i--;
newX = (x - Math.pow(2, i));
arr[i] = 1;
i = 0;
decToBin(newX)
} else if (y == x) {
arr[i] = 1;
result = arr.reverse().join();
}
return result;
}
var b = decToBin(a); // var b holds the result
document.write(b);
Hi there I need function to calculate unique integer number from number (real number double precision) and integer.
Try explain I am developing GIS application in javascript and I am working with complex vector object like polygon (array of points object with two coordinate in ring) and lines array of points. I need fast algorithm to recognize that element has been changed it must be really fast because my vector object is collection of thousand points . In C# I am calculating hash code from coordinate using bitwise operation XOR.
But javascript convert all operands in bitwise operation to integer but i need convert double precision to integer before apply bitwise in c# way (binnary). In reflector i see this that c# calculate hash code fro double like this and I need this function in javascript as fast as can be.
public override unsafe int GetHashCode() //from System.Double
{
double num = this;
if (num == 0.0)
{
return 0;
}
long num2 = *((long*) &num);
return (((int) num2) ^ ((int) (num2 >> 32)));
}
Example:
var rotation = function (n) {
n = (n >> 1) | ((n & 0x001) << 31);
return n;
}
var x: number = 1;
var y: number = 5;
var hash = x ^ rotation(y); // result is -2147483645
var x1: number = 1.1;
var y1: number = 5;
var hash1 = x1 ^ rotation(y1); // result is -2147483645
Example result is not correct hash == hash1
Example 2: Using to string there is correct result but calculate Hash from string is to complicate and I thing is not fast enough.
var rotation = function (n) {
n = (n >> 1) | ((n & 0x001) << 31);
return n;
}
var GetHashCodeString = function(str: string): number {
var hash = 0, i, l, ch;
if (str.length == 0) return hash;
for (i = 0, l = str.length; i < l; i++) {
ch = str.charCodeAt(i);
hash = ((hash << 5) - hash) + ch;
hash |= 0; // Convert to 32bit integer
}
return hash;
}
var x: number = 1;
var y: number = 5;
var hash = GetHashCodeString(x.toString()) ^ rotation(GetHashCodeString(y.toString()));
//result is -2147483605
var x1: number = 1.1;
var y1: number = 5;
var hash1 = GetHashCodeString(x1.toString()) ^ rotation(GetHashCodeString(y1.toString()));
//result is -2147435090
Example2 result is correct hash != hash1
Is there some faster way than converting number to string than calculate hash from each character? Because my object is very large and it will take lot of time and operation in this way ...
I try do it using TypedArrays but yet I am not successful.
Thanks very much for your help
Hi there I tried use TypedArrays to calculate Hash code from number and the result is interesting. In IE the performance 4x better in Chrome 2x in FireFox this approach is equal to string version ...
var GetHashCodeNumber = function (n: number): number {
//create 8 byte array buffer number in js is 64bit
var arr = new ArrayBuffer(8);
//create view to array buffer
var dv = new DataView(arr);
//set number to buffer as 64 bit float
dv.setFloat64(0, n);
//now get first 32 bit from array and convert it to integer
// from offset 0
var c = dv.getInt32(0);
//now get next 32 bit from array and convert it to integer
//from offset 4
var d = dv.getInt32(4);
//XOR first end second integer numbers
return c ^ d;
}
I think this can be useful for someone
EDIT: using one buffer and DataView is faster !
Here is a faster way to do this in JavaScript.
const kBuf = new ArrayBuffer(8);
const kBufAsF64 = new Float64Array(kBuf);
const kBufAsI32 = new Int32Array(kBuf);
function hashNumber(n) {
// Remove this `if` if you want 0 and -0 to hash to different values.
if (~~n === n) {
return ~~n;
}
kBufAsF64[0] = n;
return kBufAsI32[0] ^ kBufAsI32[1];
}
It's 250x faster than the DataView approach: see benchmark.
I looked up some hashing libraries to see how they did it: xxhashjs, jshashes, etc.
Most seem to take a string or an ArrayBuffer, and also depend on UINT32-like functionality. This is equivalent to you needing a binary representation of the double (from your C# example). Notably I did not find any solution that included more-strange types, other than in another (unanswered) question.
His solution uses a method proposed here, which converts it to various typed arrays. This is most likely what you want, and the fastest accurate solution (I think).
I highly recommend that you structure your code to traverse objects/arrays as desired, and also benchmark the solution to see how comparable it is to your existing methods (the non-working one and the string one).
How can I take an octet from the buffer and turn it into a binary sequence?
I want to decode protocol rfc1035 through node.js but find it difficult to work with bits.
Here is a code, but it does not suit me - because it is a blackbox for me:
var sliceBits = function(b, off, len) {
var s = 7 - (off + len - 1);
b = b >>> s;
return b & ~(0xff << len);
};
Use a bitmask, an octet is 8 bits so you can do something like the following:
for (var i = 7; i >= 0; i--) {
var bit = octet & (1 << i) ? 1 : 0;
// do something with the bit (push to an array if you want a sequence)
}
Example: http://jsfiddle.net/3NUVq/
You could probably make this more efficient, but the approach is pretty straightforward. This loops over an offset i, from 7 down to 0, and finds the ith bit using the bitmask 1 << i. If the ith bit is set then bit becomes 1, otherwise it will be 0.