I'm looking to decode an encoded google polyline:
`~oia#
However to reverse one of the steps requires reversing the bitwise OR operation, which is destructive.
I see it's done here: How to decode Google's Polyline Algorithm? but I can't see how to do that in Javascript.
Here is what I have so far:
const partialDecodedPolyline = "`~oia#".split('').map(char => (char.codePointAt()-63).toString(2))
console.log(partialDecodedPolyline)
The next step is to reverse the bitwise OR... how is that possible?
There is a libriary for that https://github.com/mapbox/polyline/blob/master/src/polyline.js
/*
https://github.com/mapbox/polyline/blob/master/src/polyline.js
*/
const decode = function(str, precision) {
var index = 0,
lat = 0,
lng = 0,
coordinates = [],
shift = 0,
result = 0,
byte = null,
latitude_change,
longitude_change,
factor = Math.pow(10, Number.isInteger(precision) ? precision : 5);
// Coordinates have variable length when encoded, so just keep
// track of whether we've hit the end of the string. In each
// loop iteration, a single coordinate is decoded.
while (index < str.length) {
// Reset shift, result, and byte
byte = null;
shift = 0;
result = 0;
do {
byte = str.charCodeAt(index++) - 63;
result |= (byte & 0x1f) << shift;
shift += 5;
} while (byte >= 0x20);
latitude_change = ((result & 1) ? ~(result >> 1) : (result >> 1));
shift = result = 0;
do {
byte = str.charCodeAt(index++) - 63;
result |= (byte & 0x1f) << shift;
shift += 5;
} while (byte >= 0x20);
longitude_change = ((result & 1) ? ~(result >> 1) : (result >> 1));
lat += latitude_change;
lng += longitude_change;
coordinates.push([lat / factor, lng / factor]);
}
return coordinates;
};
console.log(decode("`~oia#"));
Related
This question already has answers here:
Bitshift in javascript
(4 answers)
Closed 3 years ago.
I am writing a Javascript version of this Microsoft string decoding algorithm and its failing on large numbers. This seems to be because of sizing (int / long) issues. If I step through the code in C# I see that the JS implementation fails on this line
n |= (b & 31) << k;
This happens when the values are (and the C# result is 240518168576)
(39 & 31) << 35
If I play around with these values in C# I can replicate the JS issue if b is an int. And If I set b to be long it works correctly.
So then I checked the max size of a JS number, and compared it to the C# long result
240518168576 < Number.MAX_SAFE_INTEGER = true
So.. I can see that there is some kind of number size issue happening but do not know how to force JS to treat this number as a long.
Full JS code:
private getPointsFromEncodedString(encodedLine: string): number[][] {
const EncodingString = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_-";
var points: number[][] = [];
if (!encodedLine) {
return points;
}
var index = 0;
var xsum = 0;
var ysum = 0;
while (index < encodedLine.length) {
var n = 0;
var k = 0;
debugger;
while (true) {
if (index >= encodedLine.length) {
return points;
}
var b = EncodingString.indexOf(encodedLine[index++]);
if (b == -1) {
return points;
}
n |= (b & 31) << k;
k += 5;
if (b < 32) {
break;
}
}
var diagonal = ((Math.sqrt(8 * n + 5) - 1) / 2);
n -= diagonal * (diagonal + 1) / 2;
var ny = n;
var nx = diagonal - ny;
nx = (nx >> 1) ^ -(nx & 1);
ny = (ny >> 1) ^ -(ny & 1);
xsum += nx;
ysum += ny;
points.push([ysum * 0.000001, xsum * 0.000001]);
}
console.log(points);
return points;
}
Expected input output:
Encoded string
qkoo7v4q-lmB0471BiuuNmo30B
Decoded points:
35.89431, -110.72522
35.89393, -110.72578
35.89374, -110.72606
35.89337, -110.72662
Bitwise operators treat their operands as a sequence of 32 bits
(zeroes and ones), rather than as decimal, hexadecimal, or octal
numbers. For example, the decimal number nine has a binary
representation of 1001. Bitwise operators perform their operations on
such binary representations, but they return standard JavaScript
numerical values.
(39 & 31) << 35 tries to shift 35 bits when there only 32
Bitwise Operators
To solve this problem you could use BigInt to perform those operations and then downcast it back to Number
Number((39n & 31n) << 35n)
You can try this:
function getPointsFromEncodedString(encodedLine) {
const EncodingString = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_-";
var points = [];
if (!encodedLine) {
return points;
}
var index = 0;
var xsum = 0;
var ysum = 0;
while (index < encodedLine.length) {
var n = 0n;
var k = 0n;
while (true) {
if (index >= encodedLine.length) {
return points;
}
var b = EncodingString.indexOf(encodedLine[index++]);
if (b === -1) {
return points;
}
n |= (b & 31n) << k;
k += 5n;
if (b < 32n) {
break;
}
}
var diagonal = ((Math.sqrt(8 * Number(n) + 5) - 1) / 2);
n -= diagonal * (diagonal + 1) / 2;
var ny = n;
var nx = diagonal - ny;
nx = (nx >> 1) ^ -(nx & 1);
ny = (ny >> 1) ^ -(ny & 1);
xsum += Number(nx);
ysum += Number(ny);
points.push([ysum * 0.000001, xsum * 0.000001]);
}
console.log(points);
return points;
}
From this answer I adapted the code below:
function _makeCRCTable() {
const CRCTable = new Uint32Array(256);
for (let i = 256; i--;) {
let char = i;
for (let j = 8; j--;) {
char = char & 1 ? 3988292384 ^ char >>> 1 : char >>> 1;
}
CRCTable[i] = char;
}
return CRCTable;
}
This code generates table as here, but for Ogg I need another table - as here.
From Ogg documentation:
32 bit CRC value (direct algorithm, initial val and final XOR = 0,
generator polynomial=0x04c11db7)
parseInt('04c11db7', 16)
return 79764919 - I tried this polynomial but resulting table is not correct.
I am new to the CRC field, as I found there are a few variations of CRC32 algorithm.
I'm not sure of javascript precedence, but the xor needs to occur after the shift:
char = char & 1 ? 3988292384 ^ (char >>> 1) : char >>> 1;
However the first table you show seems correct, as table[128] = table[0x80] = 3988292384 = 0xEDB88320 which is 0x104c11db7 bit reversed, then shifted right one bit.
The second table you have is for a left shifting CRC, where table[1] = x04c11db7. In this case the inner loop would include something like this:
let char = i << 24;
for (let j = 8; j--;) {
char = char & 0x80000000 ? 0x04c11db7 ^ char << 1 : char << 1;
}
Example C code for comparison, generates crc for the patterns {0x01}, {0x01,0x00}, {0x01,0x00,0x00}, {0x01,0x00,0x00,0x00}.
#include <stdio.h>
typedef unsigned char uint8_t;
typedef unsigned int uint32_t;
uint32_t crctbl[256];
void gentbl(void)
{
uint32_t crc;
uint32_t b;
uint32_t c;
uint32_t i;
for(c = 0; c < 0x100; c++){
crc = c<<24;
for(i = 0; i < 8; i++){
b = crc>>31;
crc <<= 1;
crc ^= (0 - b) & 0x04c11db7;
}
crctbl[c] = crc;
}
}
uint32_t crc32(uint8_t * bfr, size_t size)
{
uint32_t crc = 0;
while(size--)
crc = (crc << 8) ^ crctbl[(crc >> 24)^*bfr++];
return(crc);
}
int main(int argc, char** argv)
{
uint32_t crc;
uint8_t bfr[4] = {0x01,0x00,0x00,0x00};
gentbl();
crc = crc32(bfr, 1); /* 0x04c11db7 */
printf("%08x\n", crc);
crc = crc32(bfr, 2); /* 0xd219c1dc */
printf("%08x\n", crc);
crc = crc32(bfr, 3); /* 0x01d8ac87 */
printf("%08x\n", crc);
crc = crc32(bfr, 4); /* 0xdc6d9ab7 */
printf("%08x\n", crc);
return(0);
}
For JS:
function _makeCRC32Table() {
const polynomial = 79764919;
const mask = 2147483648;
const CRCTable = new Uint32Array(256);
for (let i = 256; i--;) {
let char = i << 24;
for (let j = 8; j--;) {
char = char & mask ? polynomial ^ char << 1 : char << 1;
}
CRCTable[i] = char;
}
return CRCTable;
}
How to use this table:
[1, 0].reduce((crc, byte) => crc << 8 >>> 0 ^ CRCTable[crc >>> 24 ^ byte], 0) >>> 0
Here we added >>> 0 that takes the module of the number - because there is no unsigned int in JS - JavaScript doesn't have integers. It only has double precision floating-point numbers.
Note that for Ogg you must set generated CRC in the reverse order.
In an array of bytes, such as [40, 0, 156, 243], I need to make 1 byte from the last two bits in each byte of the array. This is for decoding and encoding PICO-8 cartridges (see here), which stores the program data in PNG images. Here is the information that I've been given on how the data is stored:
For png files, the lower 2 bits of each pixel component makes up one byte, so 1 byte per pixel. The last 32 bytes are metadata for the PNG (since 160*205 = 0x8020, 32 extra bytes)
byte = (a << 6) | (r << 4) | (g << 2) | b;
Here's what I have so far, but the output is all gibberish:
var imgWidth = 160;
var imgHeight = 205;
var canvas = document.createElement('canvas');
var ctx = canvas.getContext('2d');
var img = document.createElement('img');
img.src = 'Invasion.png';
canvas.width = imgWidth;
canvas.height = imgHeight;
img.onload = function() {
var imgData = [];
var str = '';
var i = 0, n = 0, dataLen = 0;
ctx.drawImage(img, 0, 0);
imgData = ctx.getImageData(0, 0, imgWidth, imgHeight).data;
dataLen = imgData.length;
for(i; i < dataLen; i+=4) {
var r = imgData[i];
var g = imgData[i + 1];
var b = imgData[i + 2];
var a = imgData[i + 3];
r = (r & 3) << 4;
g = (g & 3) << 2;
b = (b & 3);
a = (a & 3) << 6;
byte = a | r | g | b; // assembled byte
str += String.fromCharCode(byte);
}
console.log(str);
};
Fiddle: https://jsfiddle.net/24o3fzg3/
Greatly appreciate any help on this, thanks!
To get the last two bits, try value & 3 rather than value % 100. (value % 100 takes the remainder of value after dividing by 100 in decimal, not binary.) You then need to shift those two bits into the appropriate position in the final byte. Something like this:
r = (pixels[n][0] & 3) << 4;
g = (pixels[n][1] & 3) << 2;
b = (pixels[n][2] & 3);
a = (pixels[n][3] & 3) << 6;
val = a | r | g | b; // assembled byte
(I'm not addressing whether this is the correct order for the bits in the assembled value or the correct pixel array order.)
I'm a not an js guy, but I think code should be like this:
...
r = (pixels[n][0] & 3);
g = (pixels[n][1] & 3);
b = (pixels[n][2] & 3);
a = (pixels[n][3] & 3);
byte = (a << 6) | (r << 4) | (g << 2) | b;
str += String.fromCharCode(byte);
...
I'm using JSBN to encrypt/decrypt data using public/private keypairs. It works great for text data, including hex strings.
My problem is now I have binary data, specifically Crypto-JS Word Arrays, that I need to encrypt with a public key and send along to another platform.
So consider this:
var key = CryptoJS.lib.WordArray.random(256/8);
var rsa = new RSAKey();
rsa.setPublic(modulus, exponent);
var encrypted_key = rsa.encrypt(key.toString());
This works but it means 'encrypted_key' is infact a hex string that's been encrypted, not the actual key. I need to encrypt the actual key.
So I see two challenges here:
1) I'm not 100% sure how to get the actual bytes out of a CryptoJS.lib.WordArray -- though that doesn't seem totally insurmountable.
2) I have no idea if it's even possible to encrypt binary data using JSBN. I'd love pointers to figure out how to do it.
Any thoughts?
The JSBN library contains a function, namely pkcs1pad2(), wherein it converts the text to numeric values using JavaScript's charCodeAt() function. You'll see that conversion code in the first while() loop:
function pkcs1pad2(s,n) {
if(n < s.length + 11) { // TODO: fix for utf-8
alert("Message too long for RSA");
return null;
}
var ba = new Array();
var i = s.length - 1;
while(i >= 0 && n > 0) {
var c = s.charCodeAt(i--);
if(c < 128) { // encode using utf-8
ba[--n] = c;
}
else if((c > 127) && (c < 2048)) {
ba[--n] = (c & 63) | 128;
ba[--n] = (c >> 6) | 192;
}
else {
ba[--n] = (c & 63) | 128;
ba[--n] = ((c >> 6) & 63) | 128;
ba[--n] = (c >> 12) | 224;
}
}
ba[--n] = 0;
var rng = new SecureRandom();
var x = new Array();
while(n > 2) { // random non-zero pad
x[0] = 0;
while(x[0] == 0) rng.nextBytes(x);
ba[--n] = x[0];
}
ba[--n] = 2;
ba[--n] = 0;
return new BigInteger(ba);
}
If you wish to encrypt binary data then you'll likely have to modify this function so it converts the input in the way you want it.
Below is an example of pkcs1pad2() modified to accept binary data in the form of a hex string. If you use this version of pkcs1pad2() then you can convert your CryptoJS.lib.WordArray into hex and pass that hex string to rsa.encrypt().
function pkcs1pad2(hexPlaintext,n) {
if(n < hexPlaintext.length/2 + 11) {
alert("Message too long for RSA");
return null;
}
var ba = new Array();
var i = hexPlaintext.length;
while(i >= 2 && n > 0) {
ba[--n] = parseInt(hexPlaintext.slice(i-2,i),16);
i-=2;
}
ba[--n] = 0;
var rng = new SecureRandom();
var x = new Array();
while(n > 2) { // random non-zero pad
x[0] = 0;
while(x[0] == 0) rng.nextBytes(x);
ba[--n] = x[0];
}
ba[--n] = 2;
ba[--n] = 0;
return new BigInteger(ba);
}
Alternatively, you could modify it to take the WordArray directly and convert that to the array format that is used by JSBN, but I'll leave that as an exercise for the reader.
the pkcs1pad2 function converted from javascript to java:
public BigInteger pkcs1pad2(String data,int keysize){
byte[] buffer=new byte[keysize];
Random rg=new Random();
if(keysize < data.length()+11)
return null;
int i = data.length() - 1;
while(i >= 0 && keysize > 0){
--keysize;
buffer[keysize] = (byte) data.charAt(i);
i--;
}
--keysize;
buffer[keysize] = 0;
while(keysize > 2){
--keysize;
buffer[keysize] = (byte) (rg.nextInt(254)+1);
}
--keysize;
buffer[keysize] = 2;
--keysize;
buffer[keysize] = 0;
return new BigInteger(buffer);
}
the rsa encription:
http://hc.apache.org/downloads.cgi
//you need httpcomponents-client-4.3.1-bin.zip from apache.org
//this contains working Base64 encoder!
import org.apache.commons.codec.binary.Base64;
public String encrypt(String data,String modulus,String exponent) throws UnsupportedEncodingException{
byte[] exp=Helper.hexToBytes(exponent.toCharArray());
byte[] mod=Helper.hexToBytes(modulus.toCharArray());
BigInteger expB=new BigInteger(exp);
BigInteger modB=new BigInteger(mod);
BigInteger data2=this.pkcs1pad2(data, (modB.bitLength()+7)>>3);
BigInteger data3=data2.modPow(expB, modB);
byte[] encoding = (new Base64()).encode(Helper.hexToBytes(data3.toString(16).toCharArray()));
return new String(encoding, "US-ASCII");
}
and the Helper.HexToBytes:
public static byte[] hexToBytes(char[] hex)throws IllegalArgumentException{
byte[] data = new byte[hex.length / 2];
for (int i = 0, j = 0; j < data.length; ++j){
int hi = Character.digit(hex[i++], 16);
int lo = Character.digit(hex[i++], 16);
if ((hi < 0) || (lo < 0))
throw new IllegalArgumentException();
data[j] = (byte) (hi << 4 | lo);
}
return data;
}
I'm writing a chat-server in node.js, and I want to store connected users IP-addresses in a mysql database as (unsigned) integers.
I have written a javascript method to convert an ip-address as string to an integer. I get some strange results however.
Here is my code:
function ipToInt(ip) {
var parts = ip.split(".");
var res = 0;
res += parseInt(parts[0], 10) << 24;
res += parseInt(parts[1], 10) << 16;
res += parseInt(parts[2], 10) << 8;
res += parseInt(parts[3], 10);
return res;
}
When I run call the method as ipToInt("192.168.2.44"); the result I get is -1062731220.
It seems like an overflow has occurred, which is strange, because the expected output (3232236076) is inside the number range in javascript (2^52).
When I inspect -1062731220 in binary form, I can see the 3232236076 is preserved, but filled with leading 1's.
I'm not sure, but I think the problem is with signed vs. unsigned integers.
Can any of you explain what is going on?
And possibly how to parse -1062731220 back to an string ip?
Why is the converted IP negative?
It's NOT an overflow. The first part of your IP address is 192 which converts to 11000000 in binary. You then shift that all the way to the left. When there is a 1 in the leftmost position of a 32 bit number, it's negative.
How do you convert back to a string?
Do the same thing you did to convert from a string but in reverse. Shift right (and mask)!
function intToIP(int) {
var part1 = int & 255;
var part2 = ((int >> 8) & 255);
var part3 = ((int >> 16) & 255);
var part4 = ((int >> 24) & 255);
return part4 + "." + part3 + "." + part2 + "." + part1;
}
Why reinvent the wheel? From Google:
OR, you can use what I found here:
http://javascript.about.com/library/blipconvert.htm
function dot2num(dot)
{
var d = dot.split('.');
return ((((((+d[0])*256)+(+d[1]))*256)+(+d[2]))*256)+(+d[3]);
}
function num2dot(num)
{
var d = num%256;
for (var i = 3; i > 0; i--)
{
num = Math.floor(num/256);
d = num%256 + '.' + d;
}
return d;
}
The result of the "<<" operator is always a signed, 32-bit integer, as per the spec.
When you shift back, use ">>>" to do an unsigned right shift.
You might also find this pattern useful:
ip.toLong = function toInt(ip){
var ipl=0;
ip.split('.').forEach(function( octet ) {
ipl<<=8;
ipl+=parseInt(octet);
});
return(ipl >>>0);
};
ip.fromLong = function fromInt(ipl){
return ( (ipl>>>24) +'.' +
(ipl>>16 & 255) +'.' +
(ipl>>8 & 255) +'.' +
(ipl & 255) );
};
If you're using something like node.js where you can add functionality through something like Npm then you can simply do:
npm install ip
To get that functionality from the source which is here:
https://github.com/indutny/node-ip/blob/master/lib/ip.js
You will also get a bunch of other IP utility functions with that.
You shifted left to get the original number - which is just 4 sets of bits regardless of the sign.
Shift right to get back to the IP. Doesn't matter what the sign is.
const ip2int = (x) => (x.split('.').reduce((a, v) => ((a << 8) + (+v)), 0) >>> 0);
One-Liner:
const ipToLong = ip => ip.split('.').map(parseFloat).reduce((total, part) => total * 256 + part);
Use this
function num2string(ip) {
return [24,16,8,0].map(n => (ip >> n) & 0xff).join(".")
}
function string2num(ip) {
return ip.split(".").reduce((sum,x,i) => sum + (x << 8*(3-i)), 0)
}
IP Addresses in the V4 space are unsigned 32 bit numbers, hence the IP address of FF.FF.FF.FF is 2^32 and cannot be greater then that number. Please see:
This stack overflow article on the same subject
To turn that number back into an IP address you must break the number down into its 4 parts since each byte is one octet of the address so convert the number to hex and then parse out each pair. You may or may not have to add a leading zero for the first octet.
Additionally you may have to deal with byte order of the integer ( endien issues ) but since most systems are intel based these days you might not have to deal with that.
var aaa = Number("0b"+ "192.168.2.44".split(".").map(
function(dec){
return ("00000000" + Number(dec).toString(2)).slice(-8);
}).join(""));
aaa.toString(2).match(/.{1,8}/g).map(
function(bin){
return Number("0b"+bin);
}).join(".");
I revised Evan's final answer a bit, particularly dot2num. It functions the same but might be more readable and is marginally slower.
function ip2num(ip) {
var parts = ip.split('.');
var num = 0;
num += d[0] * Math.pow(2, 24);
num += d[1] * Math.pow(2, 16);
num += d[2] * Math.pow(2, 8);
num += d[3];
return num;
}
function num2ip(num) {
var ip = num % 256;
for (var i=3; i > 0; i--) {
num = Math.floor(num / 256);
ip = num % 256 + '.' + ip;
}
return ip;
}
Try this solution, it might help:
function IpToInteger(ipAddr)
{
var parts = ipAddr.split('.');
return (((parts[0] ? parts[0] << 24 : 0) |
(parts[1] ? parts[1] << 16 : 0) |
(parts[2] ? parts[2] << 8 : 0) |
(parts[3])) >>> 0);
}
function IpAddressToLong(ip){
return ip.split('.').map((octet, index, array) => {
return parseInt(octet) * Math.pow(256, (array.length - index - 1));
}).reduce((prev, curr) => {
return prev + curr;
});
}
Taken from repo
function ip2num(ip) {
var d = ip.split(".");
var num = 0;
num += Number(d[0]) * Math.pow(256, 3);
num += Number(d[1]) * Math.pow(256, 2);
num += Number(d[2]) * Math.pow(256, 1);
num += Number(d[3]);
return num;
}
function num2ip(num) {
var ip = num % 256;
for (var i = 3; i > 0; i--) {
num = Math.floor(num / 256);
ip = (num % 256) + "." + ip;
}
return ip;
}
console.log(ip2num("192.168.0.1"));
console.log(num2ip(3232235521))
<h1>YOU IS WELCOME</h1>