Cryptographically secure string from 36-character alphabet? - javascript

I'm trying to generate some tokens. They have to be 26 characters along from the alphabet [a-z0-9].
The closest solution I've found is part 2 from this answer, but the string won't be uniformly distributed.
If my alphabet was a power of 2 in length, this wouldn't be so hard, but as it stands, I'm not sure how to do this properly.
Specifically, this is what I've got so far:
export function createSessionId() {
const len = 26;
let bytes = new Crypto.randomBytes(len);
let result = new Array(len);
const alphabet = 'abcdefghijklmnopqrstuvwxyz0123456789';
for(let i=0; i<len; ++i) {
result[i] = alphabet[bytes[i]%alphabet.length];
}
return result.join('');
}
But I'm pretty sure that won't be distributed properly because of the modulo.

Let's see what you got here:
An alphabet of 36 characters, where each character is randomly selected at once by using modulo 36, is indeed not evenly distributed.
You have several options:
Select a whole series of bytes to represent the total string. That means, you need to select enough bytes to represent a number in the range of 0 - 36^26. That way, your value will be evenly distributed (as far as the crypto provider allows for).
If you insist on choosing one digit per time, you want to make sure its value will be distributed evenly, using modulo 36 does not do the job, as you correctly assumed. In this case, you can either
a) interpret 8 bytes as a float and multiply the result with 36
b) modulo by a power of 2 greater than 36. Then, search for the greatest multiple of 36 lower than that power of two, discard any value outside of that value space and modulo 36 the valid values.
For 2a), the distribution is not perfectly even, but very close to chance, assumed that the crypto provider is fair.
For 2b), the distribution is even. But this will be paid by a higher (and especially unpredictable) runtime when throwing away unneeded results. Of course you can statistically calculate the runtime, but the worst case is an infinite one, if your RNG keeps producing invalid results forever (which is very, very unlikely, but theoretically possible).
My advice is 2a). Take a series of bytes, interpret them as a float value and multiply the result with 36.

Here's my implementation of Psi's answer, 2b.
export function createSessionId() {
const len = 26;
let bytes = Crypto.randomBytes(30); // a few extra in case we're unlucky
let result = new Array(len);
const alphabet = 'abcdefghijklmnopqrstuvwxyz0123456789';
let i =0;
let j = 0;
for(;;) {
if(i >= bytes.length) {
bytes = Crypto.randomBytes(((len-j)*1.2)|0); // we got unlucky, gather up some more entropy
i = 0;
}
let value = bytes[i++];
if(value >= 252) { // we need a multiple of 36 for an even distribution
continue;
}
result[j++] = alphabet[value % alphabet.length];
if(j >= len) {
break;
}
}
return result.join('');
}
There's less than a 2% chance of needing to reroll (4/255), so I think this ought to be efficient enough.
Hard to unit test something like this, but this passes:
test(createSessionId.name, () => {
let ids = new Set();
let dist = Array.apply(null,new Array(26)).map(() => ({}));
for(let i=0; i<1500; ++i) {
let id = createSessionId();
if(ids.has(id)) {
throw new Error(`Not unique`);
}
ids.add(id);
for(let j=0; j<id.length; ++j) {
dist[j][id[j]] = true;
}
}
for(let i=0; i<26; ++i) {
expect(Object.keys(dist[i]).length).toEqual(36);
}
});

Related

Is there a way to avoid number to string conversion & nested loops for performance?

I just took a coding test online and this one question really bothered me. My solution was correct but was rejected for being unoptimized. The question is as following:
Write a function combineTheGivenNumber taking two arguments:
numArray: number[]
num: a number
The function should check all the concatenation pairs that can result in making a number equal to num and return their count.
E.g. if numArray = [1, 212, 12, 12] & num = 1212 then we will have return value of 3 from combineTheGivenNumber
The pairs are as following:
numArray[0]+numArray[1]
numArray[2]+numArray[3]
numArray[3]+numArray[2]
The function I wrote for this purpose is as following:
function combineTheGivenNumber(numArray, num) {
//convert all numbers to strings for easy concatenation
numArray = numArray.map(e => e+'');
//also convert the `hay` to string for easy comparison
num = num+'';
let pairCounts = 0;
// itereate over the array to get pairs
numArray.forEach((e,i) => {
numArray.forEach((f,j) => {
if(i!==j && num === (e+f)) {
pairCounts++;
}
});
});
return pairCounts;
}
console.log('Test 1: ', combineTheGivenNumber([1,212,12,12],1212));
console.log('Test 2: ', combineTheGivenNumber([4,21,42,1],421));
From my experience, I know conversion of number to string is slow in JS, but I am not sure whether my approach is wrong/lack of knowledge or does the tester is ignorant of this fact. Can anyone suggest further optimization of the code snipped?
Elimination of string to number to string will be a significant speed boost but I am not sure how to check for concatenated numbers otherwise.
Elimination of string to number to string will be a significant speed boost
No, it won't.
Firstly, you're not converting strings to numbers anywhere, but more importantly the exercise asks for concatenation so working with strings is exactly what you should do. No idea why they're even passing numbers. You're doing fine already by doing the conversion only once for each number input, not every time your form a pair. And last but not least, avoiding the conversion will not be a significant improvement.
To get a significant improvement, you should use a better algorithm. #derpirscher is correct in his comment: "[It's] the nested loop checking every possible combination which hits the time limit. For instance for your example, when the outer loop points at 212 you don't need to do any checks, because regardless, whatever you concatenate to 212, it can never result in 1212".
So use
let pairCounts = 0;
numArray.forEach((e,i) => {
if (num.startsWith(e)) {
//^^^^^^^^^^^^^^^^^^^^^^
numArray.forEach((f,j) => {
if (i !== j && num === e+f) {
pairCounts++;
}
});
}
});
You might do the same with suffixes, but it becomes more complicated to rule out concatenation to oneself there.
Optimising further, you can even achieve a linear complexity solution by putting the strings in a lookup structure, then when finding a viable prefix just checking whether the missing part is an available suffix:
function combineTheGivenNumber(numArray, num) {
const strings = new Map();
for (const num of numArray) {
const str = String(num);
strings.set(str, 1 + (strings.get(str) ?? 0));
}
const whole = String(num);
let pairCounts = 0;
for (const [prefix, pCount] of strings) {
if (!whole.startsWith(prefix))
continue;
const suffix = whole.slice(prefix.length);
if (strings.has(suffix)) {
let sCount = strings.get(suffix);
if (suffix == prefix) sCount--; // no self-concatenation
pairCounts += pCount*sCount;
}
}
return pairCounts;
}
(the proper handling of duplicates is a bit difficile)
I like your approach of going to strings early. I can suggest a couple of simple optimizations.
You only need the numbers that are valid "first parts" and those that are valid "second parts"
You can use the javascript .startsWith and .endsWith to test for those conditions. All other strings can be thrown away.
The lengths of the strings must add up to the length of the desired answer
Suppose your target string is 8 digits long. If you have 2 valid 3-digit "first parts", then you only need to know how many valid 5-digit "second parts" you have. Suppose you have 9 of them. Those first parts can only combine with those second parts, and give you 2 * 9 = 18 valid pairs.
You don't actually need to keep the strings!
It struck me that if you know you have 2 valid 3-digit "first parts", you don't need to keep those actual strings. Knowing that they are valid 2-digit first parts is all you need to know.
So let's build an array containing:
How many valid 1-digit first parts do we have?,
How many valid 2-digit first parts do we have?,
How many valid 3-digit first parts do we have?,
etc.
And similarly an array containing the number of valid 1-digit second parts, etc.
X first parts and Y second parts can be combined in X * Y ways
Except if the parts are the same length, in which case we are reusing the same list, and so it is just X * (Y-1).
So not only do we not need to keep the strings, but we only need to do the multiplication of the appropriate elements of the arrays.
5 1-char first parts & 7 3-char second parts = 5 * 7 = 35 pairs
6 2-char first part & 4 2-char second parts = 6 * (4-1) = 18 pairs
etc
So this becomes extremely easy. One pass over the strings, tallying the "first part" and "second part" matches of each length. This can be done with an if and a ++ of the relevant array element.
Then one pass over the lengths, which will be very quick as the array of lengths will be very much shorter than the array of actual strings.
function combineTheGivenNumber(numArray, num) {
const sElements = numArray.map(e => "" + e);
const sTarget = "" + num;
const targetLength = sTarget.length
const startsByLen = (new Array(targetLength)).fill(0);
const endsByLen = (new Array(targetLength)).fill(0);
sElements.forEach(sElement => {
if (sTarget.startsWith(sElement)) {
startsByLen[sElement.length]++
}
if (sTarget.endsWith(sElement)) {
endsByLen[sElement.length]++
}
})
// We can now throw away the strings. We have two separate arrays:
// startsByLen[1] is the count of strings (without attempting to remove duplicates) which are the first character of the required answer
// startsByLen[2] similarly the count of strings which are the first 2 characters of the required answer
// etc.
// and endsByLen[1] is the count of strings which are the last character ...
// and endsByLen[2] is the count of strings which are the last 2 characters, etc.
let pairCounts = 0;
for (let firstElementLength = 1; firstElementLength < targetLength; firstElementLength++) {
const secondElementLength = targetLength - firstElementLength;
if (firstElementLength === secondElementLength) {
pairCounts += startsByLen[firstElementLength] * (endsByLen[secondElementLength] - 1)
} else {
pairCounts += startsByLen[firstElementLength] * endsByLen[secondElementLength]
}
}
return pairCounts;
}
console.log('Test 1: ', combineTheGivenNumber([1, 212, 12, 12], 1212));
console.log('Test 2: ', combineTheGivenNumber([4, 21, 42, 1], 421));
Depending on a setup, the integer slicing can be marginally faster
Although in the end it falls short
Also, when tested on higher N values, the previous answer exploded in jsfiddle. Possibly a memory error.
As far as I have tested with both random and hand-crafted values, my solution holds. It is based on an observation, that if X, Y concantenated == Z, then following must be true:
Z - Y == X * 10^(floor(log10(Y)) + 1)
an example of this:
1212 - 12 = 1200
12 * 10^(floor((log10(12)) + 1) = 12 * 10^(1+1) = 12 * 100 = 1200
Now in theory, this should be faster then manipulating strings. And in many other languages it most likely would be. However in Javascript as I just learned, the situation is a bit more complicated. Javascript does some weird things with casting that I haven't figured out yet. In short - when I tried storing the numbers(and their counts) in a map, the code got significantly slower making any possible gains from this logarithm shenanigans evaporate. Furthermore, storing them in a custom-crafted data structure isn't guaranteed to be faster since you have to build it etc. Also it would be quite a lot of work.
As it stands this log comparison is ~ 8 times faster in a case without(or with just a few) matches since the quadratic factor is yet to kick in. As long as the possible postfix count isn't too high, it will outperform the linear solution. Unfortunately it is still quadratic in nature with the breaking point depending on a total number of strings as well as their length.
So if you are searching for a needle in a haystack - for example you are looking for a few pairs in a huge heap of numbers, this can help. In the other case of searching for many matches, this won't help. Similarly, if the input array was sorted, you could use binary search to push the breaking point further up.
In the end, unless you manage to figure out how to store ints in a map(or some custom implementation of it) in a way that doesn't completely kill the performance, the linear solution of the previous answer will be faster. It can still be useful even with the performance hit if your computation is going to be memory heavy. Storing numbers takes less space then storing strings.
var log10 = Math.log(10)
function log10floored(num) {
return Math.floor(Math.log(num) / log10)
}
function combineTheGivenNumber(numArray, num) {
count = 0
for (var i=0; i!=numArray.length; i++) {
let portion = num - numArray[i]
let removedPart = Math.pow(10, log10floored(numArray[i]))
if (portion % (removedPart * 10) == 0) {
for (var j=0; j!=numArray.length; j++) {
if (j != i && portion / (removedPart * 10) == numArray[j] ) {
count += 1
}
}
}
}
return count
}
//The previous solution, that I used for timing, comparison and check purposes
function combineTheGivenNumber2(numArray, num) {
const strings = new Map();
for (const num of numArray) {
const str = String(num);
strings.set(str, 1 + (strings.get(str) ?? 0));
}
const whole = String(num);
let pairCounts = 0;
for (const [prefix, pCount] of strings) {
if (!whole.startsWith(prefix))
continue;
const suffix = whole.slice(prefix.length);
if (strings.has(suffix)) {
let sCount = strings.get(suffix);
if (suffix == prefix) sCount--; // no self-concatenation
pairCounts += pCount*sCount;
}
}
return pairCounts;
}
var myArray = []
for (let i =0; i!= 10000000; i++) {
myArray.push(Math.floor(Math.random() * 1000000))
}
var a = new Date()
t1 = a.getTime()
console.log('Test 1: ', combineTheGivenNumber(myArray,15285656));
var b = new Date()
t2 = b.getTime()
console.log('Test 2: ', combineTheGivenNumber2(myArray,15285656));
var c = new Date()
t3 = c.getTime()
console.log('Test1 time: ', t2 - t1)
console.log('test2 time: ', t3 - t2)
Small update
As long as you are willing to take a performance hit with the setup and settle for the ~2 times performance, using a simple "hashing" table can help.(Hashing tables are nice and tidy, this is a simple modulo lookup table. The principle is similar though.)
Technically this isn't linear, practicaly it is enough for the most cases - unless you are extremely unlucky and all your numbers fall in the same bucket.
function combineTheGivenNumber(numArray, num) {
count = 0
let size = 1000000
numTable = new Array(size)
for (var i=0; i!=numArray.length; i++) {
let idx = numArray[i] % size
if (numTable[idx] == undefined) {
numTable[idx] = [numArray[i]]
} else {
numTable[idx].push(numArray[i])
}
}
for (var i=0; i!=numArray.length; i++) {
let portion = num - numArray[i]
let removedPart = Math.pow(10, log10floored(numArray[i]))
if (portion % (removedPart * 10) == 0) {
if (numTable[portion / (removedPart * 10) % size] != undefined) {
let a = numTable[portion / (removedPart * 10) % size]
for (var j=0; j!=a.length; j++) {
if (j != i && portion / (removedPart * 10) == a[j] ) {
count += 1
}
}
}
}
}
return count
}
Here's a simplified, and partially optimised approach with 2 loops:
// let's optimise 'combineTheGivenNumber', where
// a=array of numbers AND n=number to match
const ctgn = (a, n) => {
// convert our given number to a string using `toString` for clarity
// this isn't entirely necessary but means we can use strict equality later
const ns = n.toString();
// reduce is an efficient mechanism to return a value based on an array, giving us
// _=[accumulator], na=[array number] and i=[index]
return a.reduce((_, na, i) => {
// convert our 'array number' to an 'array number string' for later concatenation
const nas = na.toString();
// iterate back over our array of numbers ... we're using an optimised/reverse loop
for (let ii = a.length - 1; ii >= 0; ii--) {
// skip the current array number
if (i === ii) continue;
// string + number === string, which lets us strictly compare our 'number to match'
// if there's a match we increment the accumulator
if (a[ii] + nas === ns) ++_;
}
// we're done
return _;
}, 0);
}

Generate random & unique 4 digit codes without brute force

I'm building an app and in one of my functions I need to generate random & unique 4 digit codes. Obviously there is a finite range from 0000 to 9999 but each day the entire list will be wiped and each day I will not need more than the available amount of codes which means it's possible to have unique codes for each day. Realistically I will probably only need a few hundred codes a day.
The way I've coded it for now is the simple brute force way which would be to generate a random 4 digit number, check if the number exists in an array and if it does, generate another number while if it doesn't, return the generated number.
Since it's 4 digits, the runtime isn't anything too crazy and I'm mostly generating a few hundred codes a day so there won't be some scenario where I've generated 9999 codes and I keep randomly generating numbers to find the last remaining one.
It would also be fine to have letters in there as well instead of just numbers if it would make the problem easier.
Other than my brute force method, what would be a more efficient way of doing this?
Thank you!
Since you have a constrained number of values that will easily fit in memory, the simplest way I know of is to create a list of the possible values and select one randomly, then remove it from the list so it can't be selected again. This will never have a collision with a previously used number:
function initValues(numValues) {
const values = new Array(numValues);
// fill the array with each value
for (let i = 0; i < values.length; i++) {
values[i] = i;
}
return values;
}
function getValue(array) {
if (!array.length) {
throw new Error("array is empty, no more random values");
}
const i = Math.floor(Math.random() * array.length);
const returnVal = array[i];
array.splice(i, 1);
return returnVal;
}
// sample code to use it
const rands = initValues(10000);
console.log(getValue(rands));
console.log(getValue(rands));
console.log(getValue(rands));
console.log(getValue(rands));
This works by doing the following:
Generate an array of all possible values.
When you need a value, select one from the array with a random index.
After selecting the value, remove it from the array.
Return the selected value.
Items are never repeated because they are removed from the array when used.
There are no collisions with used values because you're always just selecting a random value from the remaining unused values.
This relies on the fact that an array of integers is pretty well optimized in Javascript so doing a .splice() on a 10,000 element array is still pretty fast (as it can probably just be memmove instructions).
FYI, this could be made more memory efficient by using a typed array since your numbers can be represented in 16-bit values (instead of the default 64 bits for doubles). But, you'd have to implement your own version of .splice() and keep track of the length yourself since typed arrays don't have these capabilities built in.
For even larger problems like this where memory usage becomes a problem, I've used a BitArray to keep track of previous usage of values.
Here's a class implementation of the same functionality:
class Randoms {
constructor(numValues) {
this.values = new Array(numValues);
for (let i = 0; i < this.values.length; i++) {
this.values[i] = i;
}
}
getRandomValue() {
if (!this.values.length) {
throw new Error("no more random values");
}
const i = Math.floor(Math.random() * this.values.length);
const returnVal = this.values[i];
this.values.splice(i, 1);
return returnVal;
}
}
const rands = new Randoms(10000);
console.log(rands.getRandomValue());
console.log(rands.getRandomValue());
console.log(rands.getRandomValue());
console.log(rands.getRandomValue());
Knuth's multiplicative method looks to work pretty well: it'll map numbers 0 to 9999 to a random-looking other number 0 to 9999, with no overlap:
const hash = i => i*2654435761 % (10000);
const s = new Set();
for (let i = 0; i < 10000; i++) {
const n = hash(i);
if (s.has(n)) { console.log(i, n); break; }
s.add(n);
}
To implement it, simply keep track of an index that gets incremented each time a new one is generated:
const hash = i => i*2654435761 % (10000);
let i = 1;
console.log(
hash(i++),
hash(i++),
hash(i++),
hash(i++),
hash(i++),
);
These results aren't actually random, but they probably do the job well enough for most purposes.
Disclaimer:
This is copy-paste from my answer to another question here. The code was in turn ported from yet another question here.
Utilities:
function isPrime(n) {
if (n <= 1) return false;
if (n <= 3) return true;
if (n % 2 == 0 || n % 3 == 0) return false;
for (let i = 5; i * i <= n; i = i + 6) {
if (n % i == 0 || n % (i + 2) == 0) return false;
}
return true;
}
function findNextPrime(n) {
if (n <= 1) return 2;
let prime = n;
while (true) {
prime++;
if (isPrime(prime)) return prime;
}
}
function getIndexGeneratorParams(spaceSize) {
const N = spaceSize;
const Q = findNextPrime(Math.floor(2 * N / (1 + Math.sqrt(5))))
const firstIndex = Math.floor(Math.random() * spaceSize);
return [firstIndex, N, Q]
}
function getNextIndex(prevIndex, N, Q) {
return (prevIndex + Q) % N
}
Usage
// Each day you bootstrap to get a tuple of these parameters and persist them throughout the day.
const [firstIndex, N, Q] = getIndexGeneratorParams(10000)
// need to keep track of previous index generated.
// it’s a seed to generate next one.
let prevIndex = firstIndex
// calling this function gives you the unique code
function getHashCode() {
prevIndex = getNextIndex(prevIndex, N, Q)
return prevIndex.toString().padStart(4, "0")
}
console.log(getHashCode());
Explanation
For simplicity let’s say you want generate non-repeat numbers from 0 to 35 in random order. We get pseudo-randomness by polling a "full cycle iterator"†. The idea is simple:
have the indexes 0..35 layout in a circle, denote upperbound as N=36
decide a step size, denoted as Q (Q=23 in this case) given by this formula‡
Q = findNextPrime(Math.floor(2 * N / (1 + Math.sqrt(5))))
randomly decide a starting point, e.g. number 5
start generating seemingly random nextIndex from prevIndex, by
nextIndex = (prevIndex + Q) % N
So if we put 5 in we get (5 + 23) % 36 == 28. Put 28 in we get (28 + 23) % 36 == 15.
This process will go through every number in circle (jump back and forth among points on the circle), it will pick each number only once, without repeating. When we get back to our starting point 5, we know we've reach the end.
†: I'm not sure about this term, just quoting from this answer
‡: This formula only gives a nice step size that will make things look more "random", the only requirement for Q is it must be coprime to N
This problem is so small I think a simple solution is best. Build an ordered array of the 10k possible values & permute it at the start of each day. Give the k'th value to the k'th request that day.
It avoids the possible problem with your solution of having multiple collisions.

Shuffling a poker deck in JavaScript with window.crypto.getRandomValues

A poker deck has 52 cards and thus 52! or roughly 2^226 possible permutations.
Now I want to shuffle such a deck of cards perfectly, with truly random results and a uniform distribution, so that you can reach every single one of those possible permutations and each is equally likely to appear.
Why is this actually necessary?
For games, perhaps, you don't really need perfect randomness, unless there's money to be won. Apart from that, humans probably won't even perceive the "differences" in randomness.
But if I'm not mistaken, if you use shuffling functions and RNG components commonly built into popular programming languages, you will often get no more than 32 bits of entropy and 2^32 states. Thus, you will never be able to reach all 52! possible permutations of the deck when shuffling, but only about ...
0.000000000000000000000000000000000000000000000000000000005324900157 %
... of the possible permutations. That means a whole lot of all the possible games that could be played or simulated in theory will never actually be seen in practice.
By the way, you can further improve the results if you don't reset to the default order every time before shuffling but instead start with the order from the last shuffle or keep the "mess" after a game has been played and shuffle from there.
Requirements:
So in order to do what is described above, one needs all of the following three components, as far as I have understood:
A good shuffling algorithm that ensures a uniform distribution.
A proper RNG with at least 226 bits of internal state. Since we're on deterministic machines, a PRNG will be all we'll get, and perhaps this should be a CSPRNG.
A random seed with at least 226 bits of entropy.
Solutions:
Now is this achievable? What do we have?
Fisher-Yates shuffle will be fine, as far as I can see.
The xorshift7 RNG has more than the required 226 bits of internal state and should suffice.
Using window.crypto.getRandomValues we can generate the required 226 bits of entropy to be used as our seed. If that still isn't enough, we can add some more entropy from other sources.
Question:
Are the solutions (and also the requirements) mentioned above correct? How can you implement shuffling using these solutions in JavaScript in practice then? How do you combine the three components to a working solution?
I guess I have to replace the usage of Math.random in the example of the Fisher-Yates shuffle with a call to xorshift7. But that RNG outputs a value in the [0, 1) float range and I need the [1, n] integer range instead. When scaling that range, I don't want to lose the uniform distribution. Moreover, I wanted about 226 bits of randomness. If my RNG outputs just a single Number, isn't that randomness effectively reduced to 2^53 (or 2^64) bits because there are no more possibilities for the output?
In order to generate the seed for the RNG, I wanted to do something like this:
var randomBytes = generateRandomBytes(226);
function generateRandomBytes(n) {
var data = new Uint8Array(
Math.ceil(n / 8)
);
window.crypto.getRandomValues(data);
return data;
}
Is this correct? I don't see how I could pass randomBytes to the RNG as a seed in any way, and I don't know how I could modify it to accep this.
Here's a function I wrote that uses Fisher-Yates shuffling based on random bytes sourced from window.crypto. Since Fisher-Yates requires that random numbers are generated over varying ranges, it starts out with a 6-bit mask (mask=0x3f), but gradually reduces the number of bits in this mask as the required range gets smaller (i.e., whenever i is a power of 2).
function shuffledeck() {
var cards = Array("A♣️","2♣️","3♣️","4♣️","5♣️","6♣️","7♣️","8♣️","9♣️","10♣️","J♣️","Q♣️","K♣️",
"A♦️","2♦️","3♦️","4♦️","5♦️","6♦️","7♦️","8♦️","9♦️","10♦️","J♦️","Q♦️","K♦️",
"A♥️","2♥️","3♥️","4♥️","5♥️","6♥️","7♥️","8♥️","9♥️","10♥️","J♥️","Q♥️","K♥️",
"A♠️","2♠️","3♠️","4♠️","5♠️","6♠️","7♠️","8♠️","9♠️","10♠️","J♠️","Q♠️","K♠️");
var rndbytes = new Uint8Array(100);
var i, j, r=100, tmp, mask=0x3f;
/* Fisher-Yates shuffle, using uniform random values from window.crypto */
for (i=51; i>0; i--) {
if ((i & (i+1)) == 0) mask >>= 1;
do {
/* Fetch random values in 100-byte blocks. (We probably only need to do */
/* this once.) The `mask` variable extracts the required number of bits */
/* for efficient discarding of random numbers that are too large. */
if (r == 100) {
window.crypto.getRandomValues(rndbytes);
r = 0;
}
j = rndbytes[r++] & mask;
} while (j > i);
/* Swap cards[i] and cards[j] */
tmp = cards[i];
cards[i] = cards[j];
cards[j] = tmp;
}
return cards;
}
An assessment of window.crypto libraries really deserves its own question, but anyway...
The pseudorandom stream provided by window.crypto.getRandomValues() should be sufficiently random for any purpose, but is generated by different mechanisms in different browsers. According to a 2013 survey:
Firefox (v. 21+) uses NIST SP 800-90 with a 440-bit seed. Note: This standard was updated in 2015 to remove the (possibly backdoored) Dual_EC_DRBG elliptic curve PRNG algorithm.
Internet Explorer (v. 11+) uses one of the algorithms supported by BCryptGenRandom (seed length = ?)
Safari, Chrome and Opera use an ARC4 stream cipher with a 1024-bit seed.
Edit:
A cleaner solution would be to add a generic shuffle() method to Javascript's array prototype:
// Add Fisher-Yates shuffle method to Javascript's Array type, using
// window.crypto.getRandomValues as a source of randomness.
if (Uint8Array && window.crypto && window.crypto.getRandomValues) {
Array.prototype.shuffle = function() {
var n = this.length;
// If array has <2 items, there is nothing to do
if (n < 2) return this;
// Reject arrays with >= 2**31 items
if (n > 0x7fffffff) throw "ArrayTooLong";
var i, j, r=n*2, tmp, mask;
// Fetch (2*length) random values
var rnd_words = new Uint32Array(r);
// Create a mask to filter these values
for (i=n, mask=0; i; i>>=1) mask = (mask << 1) | 1;
// Perform Fisher-Yates shuffle
for (i=n-1; i>0; i--) {
if ((i & (i+1)) == 0) mask >>= 1;
do {
if (r == n*2) {
// Refresh random values if all used up
window.crypto.getRandomValues(rnd_words);
r = 0;
}
j = rnd_words[r++] & mask;
} while (j > i);
tmp = this[i];
this[i] = this[j];
this[j] = tmp;
}
return this;
}
} else throw "Unsupported";
// Example:
deck = [ "A♣️","2♣️","3♣️","4♣️","5♣️","6♣️","7♣️","8♣️","9♣️","10♣️","J♣️","Q♣️","K♣️",
"A♦️","2♦️","3♦️","4♦️","5♦️","6♦️","7♦️","8♦️","9♦️","10♦️","J♦️","Q♦️","K♦️",
"A♥️","2♥️","3♥️","4♥️","5♥️","6♥️","7♥️","8♥️","9♥️","10♥️","J♥️","Q♥️","K♥️",
"A♠️","2♠️","3♠️","4♠️","5♠️","6♠️","7♠️","8♠️","9♠️","10♠️","J♠️","Q♠️","K♠️"];
deck.shuffle();
Combining this answer from here with this answer from another question, it seems the following could be a more general and modular (though less optimized) version:
// Fisher-Yates
function shuffle(array) {
var i, j;
for (i = array.length - 1; i > 0; i--) {
j = randomInt(0, i + 1);
swap(array, i, j);
}
}
// replacement for:
// Math.floor(Math.random() * (max - min)) + min
function randomInt(min, max) {
var range = max - min;
var bytesNeeded = Math.ceil(Math.log2(range) / 8);
var randomBytes = new Uint8Array(bytesNeeded);
var maximumRange = Math.pow(Math.pow(2, 8), bytesNeeded);
var extendedRange = Math.floor(maximumRange / range) * range;
var i, randomInteger;
while (true) {
window.crypto.getRandomValues(randomBytes);
randomInteger = 0;
for (i = 0; i < bytesNeeded; i++) {
randomInteger <<= 8;
randomInteger += randomBytes[i];
}
if (randomInteger < extendedRange) {
randomInteger %= range;
return min + randomInteger;
}
}
}
function swap(array, first, second) {
var temp;
temp = array[first];
array[first] = array[second];
array[second] = temp;
}
I personally think you could move outside the box a little bit. If you're that worried about randomness, you could look into an API key from random.org ( https://api.random.org/json-rpc/1/ ) or parse it out of a link like this: https://www.random.org/integer-sets/?sets=1&num=52&min=1&max=52&seqnos=on&commas=on&order=index&format=html&rnd=new .
Sure, your datasets could be intercepted, but if you get a few hundred thousand sets of them then shuffle those sets you would be fine.

Calculate maximum available rows and columns to fill with N amount of items

By reviewing this and this, I've come up with a function, that's probably more complex than it should be, but, man, my math sux:
function tablize(elements)
{
var root = Math.floor(Math.sqrt(elements));
var factors = [];
for (var i = 1; i <= root; i++)
{
if (elements % i === 0)
{
factors.push([i, elements / i]);
}
}
var smallest = null;
for (var f = 0; f < factors.length; f++)
{
var factor = factors[f];
var current = Math.abs(factor[0] - factor[1]);
if (!smallest || factors[smallest] > factor)
{
smallest = f;
}
}
return factors[smallest];
}
While this does work, it provides results I'm not satisfied with.
For instance - 7, it's divided in 1x7, where I'd like it to be 3x3. That's the minimum, optimal, grid size needed to fill with 7 elements.
Also - 3, it's divided in 1x3, where I'd like it to be 2x2.
I need this for a live camera feed frame distribution on a monitor, but I'm totally lost. The only way I can think of is building an extra function to feed with previously generated number and divide again, but that seems wrong.
What is the optimal solution to solve this?
For squares:
function squareNeeded(num) {
return Math.ceil(Math.sqrt(num));
}
http://jsfiddle.net/aKNVq/
(I think you mean the smallest square of a whole number that is bigger than the given amount, because if you meant a rectangle, then your example for seven would be 2*4 instead of 3*3.)

reordering objects without impacting other objects

I have a list of items (think, files in a directory), where the order of these items is arbitrarily managed by a user. The user can insert an item between other items, delete items, and move them around.
What is the best way to store the ordering as a property of each item so that when a specific item is inserted or moved, the ordering property of the other items is not affected? These objects will be stored in a database.
An ideal implementation would be able to support inifinite number of insertions/reorders.
The test I'm using to identify the limitations of the approach are as follows:
With 3 items x,y,z, repeatedly take the item on the left and put it between the other two; then take the object on the right and put it between the other two; keep going until some constraint is violated.
For others' reference, I have included some algorithms I have tried.
1.1. Decimals, double-precision
Store the order as a decimal. To insert an between two items with orders x and y, calculate its order as x/2+y/2.
Limitations:
Precision, or performance. Using doubles, when the denominator becomes too big, we end up with x/2+y/2==x . In Javascript, it can only handle 25 shuffles.
function doubles(x,y,z) {
for (var i = 0; i < 10000000; i++) {
//x,y,z
//x->v1: y,v1,z
//z->v2: y,v2,v1
var v1 = y/2 + z/2
var v2 = y/2 + v1/2
x = y
y = v2
z = v1
if (x == y) {
console.log(i)
break
}
}
}
>doubles(1, 1.5, 2)
>25
1.2. Decimals, BigDecimal
The same as above, but using BigDecimal from https://github.com/iriscouch/bigdecimal.js. In my test, the performance degraded unusably quickly. It might be a good choice for other frameworks, but not for client-side javascript.
I threw that implementation away and don't have it anymore.
2.1. Fractions
Store the order as a (numerator, denominator) integer tuple. To insert an item between items xN/xD and yN/yD, give it a value of (xN+yN)/(xD+yD) (which can easily be shown to be between the other two numbers).
Limitations:
precision or overflow.
function fractions(xN, xD, yN, yD, zN, zD){
for (var i = 0; i < 10000000; i++) {
//x,y,z
//x->v1: y,v1,z
//z->v2: y,v2,v1
var v1N = yN + zN, v1D = yD + zD
var v2N = yN + v1N, v2D = yD + v1D
xN = yN, xD=yD
yN = v2N, yD=v2D
zN = v1N, zd=v1D
if (!isFinite(xN) || !isFinite(xD)) { // overflow
console.log(i)
break
}
if (xN/xD == yN/yD) { //precision
console.log(i)
break
}
}
}
>fractions(1,1,3,2,2,1)
>737
2.2. Fractions with GCD reduction
The same as above, but reduce fractions using a Greatest Common Denomenator algorithm:
function gcd(x, y) {
if(!isFinite(x) || !isFinite(y)) {
return NaN
}
while (y != 0) {
var z = x % y;
x = y;
y = z;
}
return x;
}
function fractionsGCD(xN, xD, yN, yD, zN, zD) {
for (var i = 0; i < 10000000; i++) {
//x,y,z
//x->v1: y,v1,z
//z->v2: y,v2,v1
var v1N = yN + zN, v1D = yD + zD
var v2N = yN + v1N, v2D = yD + v1D
var v1gcd=gcd(v1N, v1D)
var v2gcd=gcd(v2N, v2D)
xN = yN, xD = yD
yN = v2N/v2gcd, yD=v2D/v2gcd
zN = v1N/v1gcd, zd=v1D/v1gcd
if (!isFinite(xN) || !isFinite(xD)) { // overflow
console.log(i)
break
}
if (xN/xD == yN/yD) { //precision
console.log(i)
break
}
}
}
>fractionsGCD(1,1,3,2,2,1)
>6795
3. Alphabetic
Use alphabetic ordering. The idea is to start with an alphabet (say, ascii printable range of [32..126]), and grow the strings. So, ('O' being the middle of our range), to insert between "a" and "c", use "b", to insert between "a" and "b", use "aO", and so forth.
Limitations:
The strings would get so long as to not fit in a database.
function middle(low, high) {
for(var i = 0; i < high.length; i++) {
if (i == low.length) {
//aa vs aaf
lowcode=32
hicode = high.charCodeAt(i)
return low + String.fromCharCode( (hicode - lowcode) / 2)
}
lowcode = low.charCodeAt(i)
hicode = high.charCodeAt(i)
if(lowcode==hicode) {
continue
}
else if(hicode - lowcode == 1) {
// aa vs ab
return low + 'O';
} else {
// aa vs aq
return low.slice(0,i) + String.fromCharCode(lowcode + (hicode - lowcode) / 2)
}
}
}
function alpha(x,y,z, N) {
for (var i = 0; i < 10000; i++) {
//x,y,z
//x->v1: y,v1,z
//z->v2: y,v2,v1
var v1 = middle(y, z)
var v2 = middle(y, v1)
x = y
y = v2
z = v1
if(x.length > N) {
console.log(i)
break
}
}
}
>alpha('?', 'O', '_', 256)
1023
>alpha('?', 'O', '_', 512)
2047
Perhaps I have missed something fundamental and I will admit I know little enough about javascript, but surely you can just implement a doubly-linked list to deal with this? Then re-ordering a,b,c,d,e,f,g,h to insert X between d and e you just unlink d->e, link d->X and then link X->e and so on.
Because in any of the scenarios above, either you will run out of precision (and your infinite ordering is lost) or you'll end up with very long sort identifiers and no memory :)
Software axiom #1: KEEP IT SIMPLE until you have found a compelling, real and proven reason to make it more complicated.
So, I'd argue that it's extra and unnecessary code and maintenance to maintain your own order property when the DOM is already doing it for you. Why not just let the DOM maintain the order and you can dynamically generate a set of brain-dead simple sequence numbers for the current ordering any time you need it? CPUs are plenty fast to generate new sequence numbers for all items anytime you need it or anytime it changes. And, if you want to save this new ordering on the server, just send the whole sequence to the server.
Implementing one of these splitting sequences so you can always insert more objects without ever renumbering anything is going to be a lot of code and a lot of opportunities for bugs. You should not go there until it's been proven that you really need that level of complication.
Store the items in an array, and use splice() to insert and delete elements.
Or is this not acceptable because of the comment you made in response to the linked list answer?
The problem you are trying to solve is potentially insertion sort which has a simple implementation of O(n^2). But there are ways to improve it.
Suppose there is an order variable associated to each element. You can assign these orders smartly with large gaps between variables and get an amortized O(n*log(n)) mechanism. Look at (Insertion sort is nlogn)

Categories

Resources