output INT64 from js UDF - javascript

I'm trying to use BigQuery's INT64 type to hold bit encoded information. I have to use a javascript udf function and I'd like to use all the 64 bits.
My issue is that javascript only deals with int32 so 1 << 32 == 1 and I'm not sure how to use the full 64 range that BigQuery supports in the udf.

It’s not possible to directly convert Big Query’s INT64 type to JavaScript UDF, neither as input nor output, as JavaScript does not support 64-bit integer type [1]. You could use FLOAT64 instead, as far as the values are less than 2^53 - 1, since it follows the IEEE 754-2008 standard for double precision [2]. You can also use a string containing the number value. Here is the documentation for supported external UDF data types [3].

Related

Javascript bitwise to c#

I have some issue to convert a javascript code into c# the issue is with bitwise operator:
Javascript function:
return (s - (s | 0x0)) * 0x100000000 | 0x0;
C# function;
return (long)((s - ((long)s)) * 0x100000000);
If s = 1.7320508075688772
on Javascript report -1150833019
on c# report 3144134277
other example can be Javascript: (1779033703 << 0x1e) = -1073741824
c# (1779033703 << 0x1e) = 1910222893216694272
What i need is translate Javascript function into c# with same number result.
Thanks for help.
So, there are a few things going on here.
You have a type mismatch in your JavaScript. In Hex, 3144134277 is BB67AE85, and -1150833019 is FFFFFFFFBB67AE85. So, we can see that the JavaScript int32 is being implicitly converted to an unsigned int64.
You can't bitshift by 0. Bitshifting is dividing by 2^n, where n is, in this case, 0. That returns the same number, as 2^0 = 1.
(long)((ulong)(…) That's a double cast, and is considered bad form. Your number literal will be cast to an unsigned long, then cast again to a long. This just wastes cycles.
Your cast is a C style cast, in C# casting is more often done as object.ToInt()
So, in review, you have a bug in your JavaScript.
You can't expect the same behavior on C# by default. Because:
In JavaScript, a number is stored as a 64-bit floating point number
but the bit-wise operation is performed on a 32-bit binary number
So to perform a bit-operation JavaScript converts the number into a
32-bit binary number, perform the operation and convert back the
result to a 64-bit number.
So in your case you might be trying to cast a 64-bit number to 32-bit one and get a faulty result from there. Which in C# it wouldn't be a good thing to have in my opinion.

Multiplication of same number in Java and Javascript giving different values

I have an expression which I am evaluating both in Java and JavaScript.
Java
System.out.println(582344008L * 719476260);
Output: 418982688909250080
Javascript
document.write(582344008 * 719476260)
Output: 418982688909250050
Why is there a difference in both values?
Javascript numbers are all IEEE 754 doubles, so it looks like there is some floating point error going on here. see Difference between floats and ints in Javascript? for some more details
Preamble
As #Charlie Wallace says
582344008 * 719476260 is greater than Number.MAX_SAFE_INTEGER
What is JavaScript's highest integer value that a number can go to without losing precision?
As **#baseballlover723 and #RealSkeptic say it's a floating point error in precision/different storage type size rounding error.
See javascript types:
What are the 7 primitive data types in JavaScriptThere are 7 basic types in JavaScript.
number for numbers of any kind: integer or floating-point. Integer/Decimal values up to 16 digits of precision. JavaScript numbers are all floating point, stored according to the IEEE 754 standard. That standard has several formats. JavaScript uses binary64 or double precision. As the former name indicates, numbers are stored in a binary format, in 64 bits.
string for strings. A string may have one or more characters, there’s no separate single-character type.
boolean for true/false.
null for unknown values – a standalone type that has a single value null.
undefined for unassigned values – a standalone type that has a single value undefined.
object for more complex data structures.
symbol for unique identifiers.
The typeof operator allows us to see which type is stored in a variable.
Two forms: typeof x or typeof(x).
Returns a string with the name of the type, like "string".
For null returns "object" – this is an error in the language, it’s not actually an object.
What are the 8 primitive data types in Java?
byte for 8-bit signed integer
short for 16-bit signed integer
int for 32-bit signed integer
long for 64-bit signed integer
char for two bytes in Unicode
float for *decimal* values up to 7 digits of precision
double for *decimal* values up to 16 digits of precision (64 bits)
boolean for true/false
Decimal values with a fractional component is called floating point. They can be expressed in either standard or scientific notation.
Testing
A quick way to test is use an online java compiler (Compile and Execute Java Online (JDK 1.8.0))
public class HelloWorld{
public static void main(String []args){
System.out.println("Long Output: "+582344008L * 719476260L);
System.out.println("Float Output: "+582344008.0 * 719476260.0);
}
}
The command:
$javac HelloWorld.java
$java -Xmx128M -Xms16M HelloWorld
The Output:
Long Output: 418982688909250080
Float Output: 4.1898268890925005E17
My desktop calculator goes to this:
4.189826889E17
and for JavaScript online compiler:
//JavaScript-C24.2.0 (SpiderMonkey)
print("Output: "+582344008 * 719476260)
The Output:
Output: 418982688909250050

MongoDB int64 and JavaScript

I write a Long value from Java to MongoDB which stores it as an int64.
Browsing the data via RoboMongo I can see the following value:
nanoTimestamp: 1467100788819818000
I then fetch the values in JS (using meteor) and I end up with the following object:
Object {_bsontype: "Long", low_: 932437528, high_: 341586032}
How can I work with this type on the client side?
The problem here is that JavaScript's number type is IEEE-754 double-precision binary floating point, which has roughly 15 digits of decimal precision. So although you can get a JS number from that BSON Long:
// May not be precise!
var num = l.high_ * Math.pow(2,32) + l.low_;
...it won't be exactly the same number (in your example case, it'll come out 1467100837142847000).
If it's okay that it's imprecise (we are talking about nanoseconds here), you're all set.
If not, and you need to deal with these in JavaScript, you might consider recording them as a string rather than Long:
nanoTimestamp: "1467100788819818000"
...and then using one of the several JavaScript "big number" libraries that can do operations on arbitrarily-large integers or floating-point numbers.

Reassembling negative Python marshal int's into Javascript numbers

I'm writing a client-side Python bytecode interpreter in Javascript (specifically Typescript) for a class project. Parsing the bytecode was going fine until I tried out a negative number.
In Python, marshal.dumps(2) gives 'i\x02\x00\x00\x00' and marshal.dumps(-2) gives 'i\xfe\xff\xff\xff'. This makes sense as Python represents integers using two's complement with at least 32 bits of precision.
In my Typescript code, I use the equivalent of Node.js's Buffer class (via a library called BrowserFS, instead of ArrayBuffers and etc.) to read the data. When I see the character 'i' (i.e. buffer.readUInt8(offset) == 105, signalling that the next thing is an int), I then call readInt32LE on the next offset to read a little-endian signed long (4 bytes). This works fine for positive numbers but not for negative numbers: for 1 I get '1', but for '-1' I get something like '-272777233'.
I guess that Javascript represents numbers in 64-bit (floating point?). So, it seems like the following should work:
var longval = buffer.readInt32LE(offset); // reads a 4-byte long, gives -272777233
var low32Bits = longval & 0xffff0000; //take the little endian 'most significant' 32 bits
var newval = ~low32Bits + 1; //invert the bits and add 1 to negate the original value
//but now newval = 272826368 instead of -2
I've tried a lot of different things and I've been stuck on this for days. I can't figure out how to recover the original value of the Python integer from the binary marshal string using Javascript/Typescript. Also I think I deeply misunderstand how bits work. Any thoughts would be appreciated here.
Some more specific questions might be:
Why would buffer.readInt32LE work for positive ints but not negative?
Am I using the correct method to get the 'most significant' or 'lowest' 32 bits (i.e. does & 0xffff0000 work how I think it does?)
Separate but related: in an actual 'long' number (i.e. longer than '-2'), I think there is a sign bit and a magnitude, and I think this information is stored in the 'highest' 2 bits of the number (i.e. at number & 0x000000ff?) -- is this the correct way of thinking about this?
The sequence ef bf bd is the UTF-8 sequence for the "Unicode replacement character", which Unicode encoders use to represent invalid encodings.
It sounds like whatever method you're using to download the data is getting accidentally run through a UTF-8 decoder and corrupting the raw datastream. Be sure you're using blob instead of text, or whatever the equivalent is for the way you're downloading the bytecode.
This got messed up only for negative values because positive values are within the normal mapping space of UTF-8 and thus get translated 1:1 from the original byte stream.

Unpack BinaryString sent from JavaScript FileReader API to Python

I'm trying to unpack a binary string sent via Javascript's FileReader readAsBinaryString method in my python app. It seems I could use the struct module for this. I'm unsure what to provide as as the format for the unpack exactly.
Can someone confirm this is the right approach, and if so, what format I should specify?
According to the JS documentation:
The result will contain the file's data
as a binary string. Every byte is
represented by an integer in the range
[0..255].
It sounds as if you just have an ordinary string (or bytes object in Python 3), so I'm not sure what you need to unpack.
One method of accessing the byte data is to use a bytearray; this lets you index the byte data easily:
>>> your_data = b'\x00\x12abc'
>>> b = bytearray(your_data)
>>> b[0]
0
>>> b[1]
18
If you have it as a string and don't want to use a bytearray (which need Python 2.6 or later) then use ord to convert the character to an integer.
>>> ord(your_data[1])
18
If your binary data has a particular interpretation in terms of groups of bytes representing integers or floats with particular endianness then the struct module is certainly your friend, but you don't need it just to examine the byte data.

Categories

Resources