Develop Reference

Extract optional arguments / parameters from a string

Lets say I have the following string, which I may receive and require my program to act up on:
ACTION -F filter string -L location string -M message string
How can I reliably extract the 'arguments' from this string, all of which are optional to the user?
I spent a lot of time instead writing ACTION, filter, location, message and using .split(", ") to put the args to an array but found this too difficult to work reliably.
var content = req.body.Body.split(", "); // [ Type, Filter, Location, Message]
var msgType = content[0].trim().toUpperCase();
if (msgType === 'INFO') {
// return info based on remainder of parameters
var filterGroup = content[1].trim();
var destination = content[2].trim();
var message = '';
// The message may be split further if it is written
// with a ',' so concat them back together.
if (content.length > 2) { // Optional message exists
// Message may be written in content[3] > content[n]
// depending how many ', ' were written in the message.
for (var i = 3; i < content.length; i++) {
message += content[i] + ", ";
}
}
}
Is the -a argument format even the best way? Much googling returns information on extracting the arguments used to run a nodeJS program, but that isn't applicable here, as the program is already running and the string not used at runtime. I'm at the design stage so have complete control over the format of the user input, but they're sending a SMS with these commands so the simpler the better!
I'm writing in javascript and would appreciate your thoughts

You can use a regex to match the -X pattern and then use a loop to extract the strings between each pattern match.
The regex here is /(?<=\s)\-[A-Za-z](?=\s)/g, which looks for a dash followed by a letter with a white space character on either side.
const input = "ACTION -F filter string -L location string -M message string";
let regex = /(?<=\s)\-[A-Za-z](?=\s)/g;
let args = [];
while(match = regex.exec(input))
{
args.push(match);
}
let solved = [];
for(let i = 0; i < args.length; i++)
{
let cmd = args[i][0];
let idx = args[i].index;
let val;
if(i + 1 == args.length)
val = input.substr(idx + cmd.length + 1);
else
val = input.substring(idx + cmd.length + 1, args[i + 1].index - 1);
solved.push({
cmd: cmd,
val: val
});
}
console.log(solved);

Howto detect wide characters in javascript?

I write a small parser for a custom query language which contains Chinese characters. When detecting syntax error, it outputs error message as following:
語法錯誤：應為數，但為字串。
索引 = '3213茂訊'"
^
The last line has only one '^' character to indicate the position of error token. For Chinese characters' visual length occupy two other characters, I need to detect wide character to calculate the '^' position to indicate the right token. Does anyone knows some function can detect the wide character in javascript?

I’m not sure if I understand you correct. But you might want to try the https://www.npmjs.com/package/wcwidth package. Which can be implemented as follows:
import wcwidth from 'wcwidth';
const getCharAtPosition = (str, position) => {
let currPos = 0;
return [...str].find(char => {
const charWidth = wcwidth(char);
const isPosition =
currPos === position || (charWidth === 2 && currPos === position - 1);
currPos += charWidth;
return isPosition;
});
};
const indicatorPos = ' ^'.indexOf('^');
console.log(getCharAtPosition(`索引 = '3213茂訊'"`, indicatorPos));
// will log: '
I didn’t test it, but something like this might work.

Convert Google Contact ID to Hex to use in URL

Google Contacts now (Jan 2019) issues a long (19 digit) decimal number id for each contact that you create.
Unfortunately, as discussed in this question the ID cannot be put into a URL to view the contact easily, however if you convert this decimal number to Hex it can be put into the URL.
So the question is, how to convert
c2913347583522826972
to
286E4A310F1EEADC
When I use the Decimal to Hex converter here it gives me
286E4A310F1EEADC if I drop the c (2nd function below is a version of the sites code, but it does use PHP too maybe)
However trying the following functions in Javascript give me mixed results
The first one is from this stack question which is the closest, just 2 digits off
function decimalToHexString(number)
{
number = parseFloat(number);
if (number < 0)
{
number = 0xFFFFFFFF + number + 1;
}
return number.toString(16);
}
console.log(decimalToHexString('2913347583522826972'));
//output 286e4a310f1eea00
function convertDec(inp,outp) {
var pd = '';
var output ;
var input = inp;
for (i=0; i < input.length; i++) {
var e=input[i].charCodeAt(0);var s = "";
output+= e + pd;
}
return output;
}
//return 50574951515255535651535050565054575550
Love to know your thoughts on improving this process

It seems like the limit of digit size. You have to use arrays if you need to convert bigger digits.
You can use hex2dec npm package to convert between hex and dec.
>> converter.decToHex("2913347583522826972", { prefix: false }
//286e4a310f1eeadc
Js example
On python side, you can simply do
dec = 2913347583522826972
// Python implicitly handles prefix
hexa = hex(dec)
print dec == int(hexa, 16)
// True
Python example
For more take a look at the following gist
https://gist.github.com/agirorn/0e740d012b620968225de58859ccef5c

How to generate short uid like "aX4j9Z" (in JS)

For my web application (in JavaScript) I want to generate short guids (for different objects - that are actually different types - strings and arrays of strings)
I want something like "aX4j9Z" for my uids (guids).
So these uids should be lightweight enough for web transfer and js string processing and quite unique for not a huge structure (not more than 10k elements). By saying "quite unique" I mean that after the generation of the uid I could check whether this uid does already exist in the structure and regenerate it if it does.

See #Mohamed's answer for a pre-packaged solution (the shortid package). Prefer that instead of any other solutions on this page if you don't have special requirements.
A 6-character alphanumeric sequence is pretty enough to randomly index a 10k collection (366 = 2.2 billion and 363 = 46656).
function generateUID() {
// I generate the UID from two parts here
// to ensure the random number provide enough bits.
var firstPart = (Math.random() * 46656) | 0;
var secondPart = (Math.random() * 46656) | 0;
firstPart = ("000" + firstPart.toString(36)).slice(-3);
secondPart = ("000" + secondPart.toString(36)).slice(-3);
return firstPart + secondPart;
}
UIDs generated randomly will have collision after generating ~ √N numbers (birthday paradox), thus 6 digits are needed for safe generation without checking (the old version only generates 4 digits which would have a collision after 1300 IDs if you don't check).
If you do collision checking, the number of digits can be reduced 3 or 4, but note that the performance will reduce linearly when you generate more and more UIDs.
var _generatedUIDs = {};
function generateUIDWithCollisionChecking() {
while (true) {
var uid = ("0000" + ((Math.random() * Math.pow(36, 4)) | 0).toString(36)).slice(-4);
if (!_generatedUIDs.hasOwnProperty(uid)) {
_generatedUIDs[uid] = true;
return uid;
}
}
}
Consider using a sequential generator (e.g. user134_item1, user134_item2, …) if you require uniqueness and not unpredictability. You could "Hash" the sequentially generated string to recover unpredictability.
UIDs generated using Math.random is not secure (and you shouldn't trust the client anyway). Do not rely on its uniqueness or unpredictability in mission critical tasks.

Update 08/2020:
shortid has been deprecated in favor of nanoid which is smaller and faster:
Small. 108 bytes (minified and gzipped). No dependencies. Size Limit controls the size.
Fast. It is 40% faster than UUID.
Safe. It uses cryptographically strong random APIs. Can be used in clusters.
Compact. It uses a larger alphabet than UUID (A-Za-z0-9_-). So ID size was reduced from 36 to 21 symbols.
Portable. Nano ID was ported to 14 programming languages.
import { nanoid } from 'nanoid'
// 21 characters (default)
// ~149 billion years needed, in order to have a 1% probability of at least one collision.
console.log(nanoid()) //=> "V1StGXR8_Z5jdHi6B-myT"
// 11 characters
// ~139 years needed, in order to have a 1% probability of at least one collision.
console.log(nanoid(11)) //=> "bdkjNOkq9PO"
More info here : https://zelark.github.io/nano-id-cc/
Old answer
There is also an awesome npm package for this : shortid
Amazingly short non-sequential url-friendly unique id generator.
ShortId creates amazingly short non-sequential url-friendly unique ids. Perfect for url shorteners, MongoDB and Redis ids, and any other id users might see.
By default 7-14 url-friendly characters: A-Z, a-z, 0-9, _-
Non-sequential so they are not predictable.
Supports cluster (automatically), custom seeds, custom alphabet.
Can generate any number of ids without duplicates, even millions per day.
Perfect for games, especially if you are concerned about cheating so you don't want an easily guessable id.
Apps can be restarted any number of times without any chance of repeating an id.
Popular replacement for Mongo ID/Mongoose ID.
Works in Node, io.js, and web browsers.
Includes Mocha tests.
Usage
var shortid = require('shortid');
console.log(shortid.generate()); //PPBqWA9

Here is a one liner, but it gives only lowercase letters and numbers:
var uuid = Math.random().toString(36).slice(-6);
console.log(uuid);

Get a simple counter to start from 100000000, convert the number into radix 36.
(100000000).toString(36); //1njchs
(2100000000).toString(36); //yqaadc
You can comfortably have 2 billion elegant unique ids, just like YouTube

The following generates 62^3 (238,328) unique values of 3 characters provided case sensitivity is unique and digits are allowed in all positions. If case insensitivity is required, remove either upper or lower case characters from chars string and it will generate 35^3 (42,875) unique values.
Can be easily adapted so that first char is always a letter, or all letters.
No dobut it can be optimised, and could also refuse to return an id when the limit is reached.
var nextId = (function() {
var nextIndex = [0,0,0];
var chars = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'.split('');
var num = chars.length;
return function() {
var a = nextIndex[0];
var b = nextIndex[1];
var c = nextIndex[2];
var id = chars[a] + chars[b] + chars[c];
a = ++a % num;
if (!a) {
b = ++b % num;
if (!b) {
c = ++c % num;
}
}
nextIndex = [a, b, c];
return id;
}
}());

var letters = 'abcdefghijklmnopqrstuvwxyz';
var numbers = '1234567890';
var charset = letters + letters.toUpperCase() + numbers;
function randomElement(array) {
with (Math)
return array[floor(random()*array.length)];
}
function randomString(length) {
var R = '';
for(var i=0; i<length; i++)
R += randomElement(charset);
return R;
}

This is an old question and there are some good answers, however I notice that we are in 2022 and we can use ES6 and if you don't like to depend on 3rd party libs. Here is a solution for you.
I implemented a very simple generator using the build-in functions that JavaScript offers to us these days. We will use Crypto.getRandomValues() and Uint8Array() so check the code below
const hashID = size => {
const MASK = 0x3d
const LETTERS = 'abcdefghijklmnopqrstuvwxyz'
const NUMBERS = '1234567890'
const charset = `${NUMBERS}${LETTERS}${LETTERS.toUpperCase()}`.split('')
const bytes = new Uint8Array(size)
crypto.getRandomValues(bytes)
return bytes.reduce((acc, byte) => `${acc}${charset[byte & MASK]}`, '')
}
console.log({id: hashID(6)})
This implementation uses these characters: [A-Z], [a-z], [0-9] in total they are 62 characters, if we add _ and - it will complete up to 64 characters like this:
const hashID = size => {
const MASK = 0x3d
const LETTERS = 'abcdefghijklmnopqrstuvwxyz'
const NUMBERS = '1234567890'
const charset = `${NUMBERS}${LETTERS}${LETTERS.toUpperCase()}_-`.split('')
const bytes = new Uint8Array(size)
crypto.getRandomValues(bytes)
return bytes.reduce((acc, byte) => `${acc}${charset[byte & MASK]}`, '')
}
console.log(`id: ${hashID(6)}`)
Note:
It will take around 2 days in order to have a 1% probability of at least one collision for 1000 IDs generated per hour with ID length of 6 characters. Keep this in mind when it is implemented into your project.

This will generate a sequence of unique values. It improves on RobG's answer by growing the string length when all values have been exhaused.
var IdGenerator = (function () {
var defaultCharset = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890!##$%^&*()_-+=[]{};:?/.>,<|".split("");
var IdGenerator = function IdGenerator(charset) {
this._charset = (typeof charset === "undefined") ? defaultCharset : charset;
this.reset();
};
IdGenerator.prototype._str = function () {
var str = "",
perm = this._perm,
chars = this._charset,
len = perm.length,
i;
for (i = 0; i < len; i++) {
str += chars[perm[i]];
}
return str;
};
IdGenerator.prototype._inc = function () {
var perm = this._perm,
max = this._charset.length - 1,
i;
for (i = 0; true; i++) {
if (i > perm.length - 1) {
perm.push(0);
return;
} else {
perm[i]++;
if (perm[i] > max) {
perm[i] = 0;
} else {
return;
}
}
}
};
IdGenerator.prototype.reset = function () {
this._perm = [];
};
IdGenerator.prototype.current = function () {
return this._str();
};
IdGenerator.prototype.next = function () {
this._inc();
return this._str();
};
return IdGenerator;
}).call(null);
Usage:
var g = new IdGenerator(),
i;
for (i = 0; i < 100; i++) {
console.log(g.next());
}
This gist contains the above implementation and a recursive version.

just randomly generate some strings:
function getUID(len){
var chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789',
out = '';
for(var i=0, clen=chars.length; i<len; i++){
out += chars.substr(0|Math.random() * clen, 1);
}
// ensure that the uid is unique for this page
return getUID.uids[out] ? getUID(len) : (getUID.uids[out] = out);
}
getUID.uids = {};

You can shorten a GUID to 20 printable ASCII characters without losing information or the uniqueness of the GUID.
Jeff Atwood blogged about that years ago:
Equipping our ASCII Armor

This solution combines Math.random() with a counter.
Math.random() should give about 53 bits of entropy (compared with UUIDv4's 128), but when combined with a counter should give plenty enough uniqueness for a temporary ID.
let _id_counter = 0
function id() {
return '_' + (_id_counter++).toString(36) + '_' + Math.floor(Math.random() * Number.MAX_SAFE_INTEGER).toString(36)
}
console.log(Array.from({length: 100}).map(() => id()))
Features:
Simple implementation
Output of about 13 chars
Case-insensitive
Safe for use as HTML id and React key
Not suitable for database storage

You can use the md5 algorithm for generating a random string. md5 is the node package
var randomChars = Math.random().toString(36).replace(/[^a-z]+/g, '').substr(0, 2);
var shortUrl = md5(originalUrl + randomChars + new Date()).substring(0, 5).toString();
console.log(shortUrl);
This will generate unique string every time.

JavaScript: Is IP In One Of These Subnets?

So I have ~12600 subnets:
eg. 123.123.208.0/20
and an IP.
I can use a SQLite Database or an array or whatever
There was a similar question asked about a month ago, however I am not looking for checking one IP against one subnet but a bunch of subnets (obviously the most efficient way, hopefully not O(total subnets)) :)
How can I check that the IP is one of in one of these subnets, I need true or false not the subnet if that helps optimisation.
There are similar subnets in the current list eg.:
(actual extract)
123.123.48.0/22 <-- not a typo
123.123.48.0/24 <-- not a typo
123.123.90.0/24
123.123.91.0/24
123.123.217.0/24
In total they range from 4.x.y.z to 222.x.y.z

The best approach is IMO making use of bitwise operators. For example, 123.123.48.0/22 represents (123<<24)+(123<<16)+(48<<8)+0 (=2071670784; this might be a negative number) as a 32 bit numeric IP address, and -1<<(32-22) = -1024 as a mask. With this, and likewise, your test IP address converted to a number, you can do:
(inputIP & testMask) == testIP
For example, 123.123.49.123 is in that range, as 2071671163 & -1024 is 2071670784
So, here are some tool functions:
function IPnumber(IPaddress) {
var ip = IPaddress.match(/^(\d+)\.(\d+)\.(\d+)\.(\d+)$/);
if(ip) {
return (+ip[1]<<24) + (+ip[2]<<16) + (+ip[3]<<8) + (+ip[4]);
}
// else ... ?
return null;
}
function IPmask(maskSize) {
return -1<<(32-maskSize)
}
test:
(IPnumber('123.123.49.123') & IPmask('22')) == IPnumber('123.123.48.0')
yields true.
In case your mask is in the format '255.255.252.0', then you can use the IPnumber function for the mask, too.

Try this:
var ip2long = function(ip){
var components;
if(components = ip.match(/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/))
{
var iplong = 0;
var power = 1;
for(var i=4; i>=1; i-=1)
{
iplong += power * parseInt(components[i]);
power *= 256;
}
return iplong;
}
else return -1;
};
var inSubNet = function(ip, subnet)
{
var mask, base_ip, long_ip = ip2long(ip);
if( (mask = subnet.match(/^(.*?)\/(\d{1,2})$/)) && ((base_ip=ip2long(mask[1])) >= 0) )
{
var freedom = Math.pow(2, 32 - parseInt(mask[2]));
return (long_ip > base_ip) && (long_ip < base_ip + freedom - 1);
}
else return false;
};
Usage:
inSubNet('192.30.252.63', '192.30.252.0/22') => true
inSubNet('192.31.252.63', '192.30.252.0/22') => false

I managed to solve this by using the node netmask module.
You can check if an IP belongs to a subnet by making something like this:
import { Netmask } from 'netmask'
const block = new Netmask('123.123.208.0/20')
const ip = '123.123.208.0'
console.log(block.contains(ip))
Will here print true.
You can install it by using:
npm i --save netmask

Convert the lower ip and the upper ip in the range to integers and store the range in the db then make sure both columns are indexed.
Off the top of my head (pseudo code):
function ipmap(w,x,y,z) {
return 16777216*w + 65536*x + 256*y + z;
}
var masks = array[ipmap(128,0,0,0), ipmap(196,0,0,0), ..., ipmap(255,255,255,255)]
function lowrange(w, x, y, z, rangelength) {
return ipmap(w, x, y, z) & masks[rangelength]
}
function hirange(w, x, y, z, rangelength) {
return lowrange(w, x, y, z, ,rangelength) + ipmap(255,255,255,255) - masks[rangelength];
}
That ought to do it.
To find whether a particular ip falls in any of the ranges, convert it to an integer and do:
SELECT COUNT(*) FROM ipranges WHERE lowrange <= 1234567 AND 1234567 <= highrange
The query optimizer should be able to speed this up greatly.

Functions IPnumber and IPmask are nice, however I would rather test like:
(IPnumber('123.123.49.123') & IPmask('22')) == (IPnumber('123.123.48.0') & IPmask('22'))
Because for each address, you only need to take into account the network part of the address. Hence doing IPmask('22') will zero-out the computer part of the address and you should do the same with the network address.

Keywords: Binary searching, preprocessing, sorting
I had a similar problem and binary search appears to be very efficient if you can pre-process your subnet list and sort it. Then you can achieve an asymptotic time complexity of O(log n).
Here's my code (MIT License, original location: https://github.com/iBug/pac/blob/854289a674578d096f60241804f5893a3fa17523/code.js):
function belongsToSubnet(host, list) {
var ip = host.split(".").map(Number);
ip = 0x1000000 * ip[0] + 0x10000 * ip[1] + 0x100 * ip[2] + ip[3];
if (ip < list[0][0])
return false;
// Binary search
var x = 0, y = list.length, middle;
while (y - x > 1) {
middle = Math.floor((x + y) / 2);
if (list[middle][0] < ip)
x = middle;
else
y = middle;
}
// Match
var masked = ip & list[x][1];
return (masked ^ list[x][0]) == 0;
}
And an example usage:
function isLan(host) {
return belongsToSubnet(host, LAN);
}
var LAN = [
[0x0A000000, 0xFF000000], // 10.0.0.0/8
[0x64400000, 0xFFC00000], // 100.64.0.0/10
[0x7F000000, 0xFF000000], // 127.0.0.0/8
[0xA9FE0000, 0xFFFF0000], // 169.254.0.0/16
[0xAC100000, 0xFFF00000], // 172.16.0.0/12
[0xC0A80000, 0xFFFF0000] // 192.168.0.0/16
];
isLan("127.12.34.56"); // => true
isLan("8.8.8.8"); // => false (Google's Public DNS)
You can get a PAC script* and see how it performs (it loads a China IP list from somewhere else, sorts them and formats them appropriately) against 5000s of subnets. In practice its speed is surprisingly satisfactory.
The preprocessing code can be inspected using F12 Dev Tools on the above page. In short, you need to convert 1.2.3.4/16 to [0x01020304, 0xFFFF0000], i.e. 32-bit unsigned integer for the IP address and network mask.
* Link goes to my personal website.