Develop Reference

Weighted random sample of array items *without replacement*

Javascript/ECMAScript 6 specific solution desired.
I want to generate a random sample from an array of objects using an array of weighted values for each object. The population list contains the actual members of the population - not the types of members. Once one is selected for a sample, it can't be selected again.
An analogous problem to the one I'm working on would be simulating a probable outcome for a chess tournament. Each player's rating would be their weight. A player can only place once (1st, 2nd, or 3rd place) per tournament.
To pick a likely list of the top 3 winners could look like:
let winners = wsample(chessPlayers, // population
playerRatings, // weights
3); // sample size
The weighted list may, or may not, be integer values. It could be floats like [0.2, 0.1, 0.7, 0.3], or it could be integers like [20, 10, 70, 30]. The weights do not have to add up to a value that represents 100%.
Peter below gave me a good reference on a general algorithm, however It's not specific to JS: https://stackoverflow.com/a/62459274/7915759 it may be a good point of reference.
Solutions to the problem that rely on generating a second population list with each member copied weight number of times may not be a practical solution. Each weight in the weights array could be very high numbers, or they could be fractional; basically, any non-negative value.
Some additional questions:
Is there already an accumulate() function available in JS?
Is there a bisect() type function in JS that does a binary search of sorted lists?
Are there any efficient and low memory footprint JS modules available with statistical functions that include solutions for the above?

The following implementation selects k out of n elements, without replacement, with weighted probabilities, in O(n + k log n) by keeping the accumulated weights of the remaining elements in a sum heap:
function sample_without_replacement<T>(population: T[], weights: number[], sampleSize: number) {
let size = 1;
while (size < weights.length) {
size = size << 1;
}
// construct a sum heap for the weights
const root = 1;
const w = [...new Array(size) as number[], ...weights, 0];
for (let index = size - 1; index >= 1; index--) {
const leftChild = index << 1;
const rightChild = leftChild + 1;
w[index] = (w[leftChild] || 0) + (w[rightChild] || 0);
}
// retrieves an element with weight-index r
// from the part of the heap rooted at index
const retrieve = (r: number, index: number): T => {
if (index >= size) {
w[index] = 0;
return population[index - size];
}
const leftChild = index << 1;
const rightChild = leftChild + 1;
try {
if (r <= w[leftChild]) {
return retrieve(r, leftChild);
} else {
return retrieve(r - w[leftChild], rightChild);
}
} finally {
w[index] = w[leftChild] + w[rightChild];
}
}
// and now retrieve sampleSize random elements without replacement
const result: T[] = [];
for (let k = 0; k < sampleSize; k++) {
result.push(retrieve(Math.random() * w[root], root));
}
return result;
}
The code is written in TypeScript. You can transpile it to whatever version of EcmaScript you need in the TypeScript playground.
Test code:
const n = 1E7;
const k = n / 2;
const population: number[] = [];
const weight: number[] = [];
for (let i = 0; i < n; i++) {
population[i] = i;
weight[i] = i;
}
console.log(`sampling ${k} of ${n} elments without replacement`);
const sample = sample_without_replacement(population, weight, k);
console.log(sample.slice(0, 100)); // logging everything takes forever on some consoles
console.log("Done")
Executed in Chrome, this samples 5 000 000 out of 10 000 000 entries in about 10 seconds.

This is one approach, but not the most efficient.
Its efficiency can be improved by using a binary indexed tree as its prefix sum.
The highest level function. It iterates k times, calling wchoice() each time. To remove the currently selected member from the population, I just set its weight to 0.
/**
* Produces a weighted sample from `population` of size `k` without replacement.
*
* #param {Object[]} population The population to select from.
* #param {number[]} weights The weighted values of the population.
* #param {number} k The size of the sample to return.
* #returns {[number[], Object[]]} An array of two arrays. The first holds the
* indices of the members in the sample, and
* the second holds the sample members.
*/
function wsample(population, weights, k) {
let sample = [];
let indices = [];
let index = 0;
let choice = null;
let acmwts = accumulate(weights);
for (let i=0; i < k; i++) {
[index, choice] = wchoice(population, acmwts, true);
sample.push(choice);
indices.push(index);
// The below updates the accumulated weights as if the member
// at `index` has a weight of 0, eliminating it from future draws.
// This portion could be optimized. See note below.
let ndecr = weights[index];
for (; index < acmwts.length; index++) {
acmwts[index] -= ndecr;
}
}
return [indices, sample];
}
The section of code above that updates the accumulated weights array is the point of inefficiency in the algorithm. Worst case it's O(n - ?) to update on every pass. Another solution here follows a similar algorithm to this, but uses a binary indexed tree to reduce the cost of updating the prefix sum to an O(log n) operation.
wsample() calls wchoice() which selects one member from the weighted list. wchoice() generates an array of cumulative weights, generates a random number from 0 to the total sum of the weights (last item in the cumulative weights list). Then finds its insertion point in the cumulative weights; which is the winner:
/**
* Randomly selects a member of `population` weighting the probability each
* will be selected using `weights`. `accumulated` indicates whether `weights`
* is pre-accumulated, in which case it will skip its accumulation step.
*
* #param {Object[]} population The population to select from.
* #param {number[]} weights The weights of the population.
* #param {boolean} [accumulated] true if weights are pre-accumulated.
* Treated as false if not provided.
* #returns {[number, Object]} An array with the selected member's index and
* the member itself.
*/
function wchoice(population, weights, accumulated) {
let acm = (accumulated) ? weights : accumulate(weights);
let rnd = Math.random() * acm[acm.length - 1];
let idx = bisect_left(acm, rnd);
return [idx, population[idx]];
}
Here's a JS implementation I adapted from the binary search algorithm from https://en.wikipedia.org/wiki/Binary_search_algorithm
/**
* Finds the left insertion point for `target` in array `arr`. Uses a binary
* search algorithm.
*
* #param {number[]} arr A sorted ascending array.
* #param {number} target The target value.
* #returns {number} The index in `arr` where `target` can be inserted to
* preserve the order of the array.
*/
function bisect_left(arr, target) {
let n = arr.length;
let l = 0;
let r = n - 1;
while (l <= r) {
let m = Math.floor((l + r) / 2);
if (arr[m] < target) {
l = m + 1;
} else if (arr[m] >= target) {
r = m - 1;
}
}
return l;
}
I wasn't able to find an accumulator function ready-made for JS, so I wrote a simple one myself.
/**
* Generates an array of accumulated values for `numbers`.
* e.g.: [1, 5, 2, 1, 5] --> [1, 6, 8, 9, 14]
*
* #param {number[]} numbers The numbers to accumulate.
* #returns {number[]} An array of accumulated values.
*/
function accumulate(numbers) {
let accm = [];
let total = 0;
for (let n of numbers) {
total += n;
accm.push(total)
}
return accm;
}

Am I using the proper predicate for my lambda?

I've been working on this program for a few hours, and I finally got it to output - NaN. I dont know how this could be, I'm pushing a product of real numbers into the array... Somebody help! What did I miss? The problem is to find the largest product produced by 13 adjacent digits within the 1000 digit number assigned to _1000digits.
// what is the largest product of 13 adjacent digits within this 1000 digit number
function largestProduct() {
_1000digits = 7316717653133062491922511967442657474235534919493496983520312774506326239578318016984801869478851843858615607891129494954595017379583319528532088055111254069874715852386305071569329096329522744304355766896648950445244523161731856403098711121722383113622298934233803081353362766142828064444866452387493035890729629049156044077239071381051585930796086670172427121883998797908792274921901699720888093776657273330010533678812202354218097512545405947522435258490771167055601360483958644670632441572215539753697817977846174064955149290862569321978468622482839722413756570560574902614079729686524145351004748216637048440319989000889524345065854122758866688116427171479924442928230863465674813919123162824586178664583591245665294765456828489128831426076900422421902267105562632111110937054421750694165896040807198403850962455444362981230987879927244284909188845801561660979191338754992005240636899125607176060588611646710940507754100225698315520005593572972571636269561882670428252483600823257530420752963450;
separateDigits = _1000digits.toString().split("");
products = [];
var a = 0;
var b = 1;
var c = 2;
var d = 3;
var e = 4;
var f = 5;
var g = 6;
var h = 7;
var i = 8;
var j = 9;
var k = 10;
var l = 11;
var m = 12;
while (m <= 999) {
products.push(
separateDigits[a] *
separateDigits[b] *
separateDigits[c] *
separateDigits[d] *
separateDigits[e] *
separateDigits[f] *
separateDigits[g] *
separateDigits[h] *
separateDigits[i] *
separateDigits[j] *
separateDigits[k] *
separateDigits[l] *
separateDigits[m]
);
a++;
b++;
c++;
d++;
e++;
f++;
g++;
h++;
i++;
j++;
k++;
l++;
m++;
}
products.sort((a, b) => a - b);
console.log(products.pop());
}
largestProduct();

Short:
To work with such huge numbers you'll want to use a special data structure, like BigInt.
Long:
There are a few issues with your code, the first one is trying to store such a huge number in a variable without any treatment. A JavaScript number can only store values up to 25^3 - 1, your number is a lot bigger than that.
If you run:
_1000digits = 7316717653133062491922511967442657474235534919493496983520312774506326239578318016984801869478851843858615607891129494954595017379583319528532088055111254069874715852386305071569329096329522744304355766896648950445244523161731856403098711121722383113622298934233803081353362766142828064444866452387493035890729629049156044077239071381051585930796086670172427121883998797908792274921901699720888093776657273330010533678812202354218097512545405947522435258490771167055601360483958644670632441572215539753697817977846174064955149290862569321978468622482839722413756570560574902614079729686524145351004748216637048440319989000889524345065854122758866688116427171479924442928230863465674813919123162824586178664583591245665294765456828489128831426076900422421902267105562632111110937054421750694165896040807198403850962455444362981230987879927244284909188845801561660979191338754992005240636899125607176060588611646710940507754100225698315520005593572972571636269561882670428252483600823257530420752963450
console.log(_1000digits)
You'll see the output is "Infinity" because that's such a huge number JavaScript doesn't know how to store it entirely.
You're also not checking if the numbers you're accessing actually exist, so if you put a smaller number in _1000digits you'll end up multiplying by undefined, which will result in NaN:
_1000digits = 700
separateDigits = _1000digits.toString().split("")
var f = 5
console.log(separateDigits[f])

Perceptron Javascript Inconsistencies

Building a basic Perceptron. My results after training are very inconsistent, even after 1000's of epochs. The weights seem to adjust properly, however the model fails to accurately predict. A second pairs of eyes on the structure would be greatly appreciated, struggling to find where I went wrong. The accuracy consistently tops out at 60%.
// Perceptron
class Perceptron {
constructor (x_train, y_train, learn_rate= 0.1, epochs=10) {
this.epochs = epochs
this.x_train = x_train
this.y_train = y_train
this.learn_rate = learn_rate
this.weights = new Array(x_train[0].length)
// initialize random weights
for ( let n = 0; n < x_train[0].length; n++ ) {
this.weights[n] = this.random()
}
}
// generate random float between -1 and 1 (for generating weights)
random () {
return Math.random() * 2 - 1
}
// activation function
activation (n) {
return n < 0 ? 0 : 1
}
// y-hat output given an input tensor
predict (input) {
let total = 0
this.weights.forEach((w, index) => { total += input[index] * w }) // multiply each weight by each input vector value
return this.activation(total)
}
// training perceptron on data
fit () {
for ( let e = 0; e < this.epochs; e++) { // epochs loop
for ( let i = 0; i < this.x_train.length; i++ ) { // iterate over each training sample
let prediction = this.predict(this.x_train[i]) // predict sample output
console.log('Expected: ' + this.y_train[i] + ' Model Output: ' + prediction) // log expected vs predicted
let loss = this.y_train[i] - prediction // calculate loss
for ( let w = 0; w < this.weights.length; w++ ) { // loop weights for update
this.weights[w] += loss * this.x_train[i][w] * this.learn_rate // update all weights to reduce loss
}
}
}
}
}
x = [[1, 1, 1], [0, 0, 0], [0, 0, 1], [1, 1, 0], [0, 0, 1]]
y = [1, 0, 0, 1, 0]
p = new Perceptron(x, y, epochs=5000, learn_rate=.1)
Updated:
// Perceptron
module.exports = class Perceptron {
constructor (x_train, y_train, epochs=1000, learn_rate= 0.1) {
// used to generate percent accuracy
this.accuracy = 0
this.samples = 0
this.x_train = x_train
this.y_train = y_train
this.epochs = epochs
this.learn_rate = learn_rate
this.weights = new Array(x_train[0].length)
this.bias = 0
// initialize random weights
for ( let n = 0; n < x_train[0].length; n++ ) {
this.weights[n] = this.random()
}
}
// returns percent accuracy
current_accuracy () {
return this.accuracy/this.samples
}
// generate random float between -1 and 1 (for generating weights)
random () {
return Math.random() * 2 - 1
}
// activation function
activation (n) {
return n < 0 ? 0 : 1
}
// y-hat output given an input tensor
predict (input) {
let total = this.bias
this.weights.forEach((w, index) => { total += input[index] * w }) // multiply each weight by each input vector value
return this.activation(total)
}
// training perceptron on data
fit () {
// epochs loop
for ( let e = 0; e < this.epochs; e++) {
// for each training sample
for ( let i = 0; i < this.x_train.length; i++ ) {
// get prediction
let prediction = this.predict(this.x_train[i])
console.log('Expected: ' + this.y_train[i] + ' Model Output: ' + prediction)
// update accuracy measures
this.y_train[i] === prediction ? this.accuracy += 1 : this.accuracy -= 1
this.samples++
// calculate loss
let loss = this.y_train[i] - prediction
// update all weights
for ( let w = 0; w < this.weights.length; w++ ) {
this.weights[w] += loss * this.x_train[i][w] * this.learn_rate
}
this.bias += loss * this.learn_rate
}
// accuracy post epoch
console.log(this.current_accuracy())
}
}
}

It's just a syntactic error :)
Switch the order of the last two parameters, like this:
p = new Perceptron(x, y, learn_rate=.1, epochs=5000)
And now everything should work fine.
However, a more serious problem lies in your implementation:
You forgot the bias
With a perceptron you're trying to learn a linear function, something of the form of
y = wx + b
but what you're currently computing is just
y = wx
This is fine if what you're trying to learn is just the identity function of a single input, like in your case. But it fails to work as soon as you start doing something slightly more complex like trying to learn the AND function, which can be represented like this:
y = x1 + x2 - 1.5
How to fix?
Really easy, just initialise this.bias = 0 in the constructor. Then, in predict(), you initialise let total = this.bias and, in fit(), add this.bias += loss * this.learn_rate right after the inner-most loop.

Javascript function to generate random integers with nonuniform probabilities

In javascript (or jquery) is there a simple function to have four integers with their probability values: 1|0.41, 2|0.29, 3|0.25, 4|0.05
how can I generate these four numbers taking into account their probabilities ?
This question is very similar to the one posted here: generate random integers with probabilities
HOWEVER the solution posted there:
function randomWithProbability() {
var notRandomNumbers = [1, 1, 1, 1, 2, 2, 2, 3, 3, 4];
var idx = Math.floor(Math.random() * notRandomNumbers.length);
return notRandomNumbers[idx];
}
states in the comment "create notRandomNumbers dynamically (given the numbers and their weight/probability)"
This is insufficient for my needs. That works well when the probabilities are say 10%,20%, 60%,10%.
In that case constructing notRandomNumbers with the required distribution is easy and the array size is small. But in the general case where probabilities can be something like 20.354%,30.254% etc , the array size would be huge to correctly model the situation.
Is there a clean solution to this more general problem?
EDIT: Thanks Georg, solution accepted, here is my final version, which may be useful for others. I have split the calculation of the cumulative into a separate function in order to avoid extra additions at each call to get a new random number.
function getRandomBinFromCumulative(cumulative) {
var r = Math.random();
for (var i = 0; i < cumulative.length; i++) {
if (r <= cumulative[i])
return i;
}
}
function getCummulativeDistribution(probs) {
var cumulative = [];
var sum = probs[0];
probs.forEach(function (p) {
cumulative.push(sum);
sum += p;
});
// the next 2 lines are optional
cumulative[cumulative.length - 1] = 1; //force to 1 (if input total was <>1)
cumulative.shift(); //remove the first 0
return cumulative;
}
function testRand() {
var probs = [0.1, 0.3, 0.3, 0.3];
var c = getCummulativeDistribution(probs);
console.log(c);
for (var i = 0; i < 100; i++) {
console.log(getRandomBinFromCumulative(c));
}
}

Just accumulate the probabilities and return an item for which current_sum >= random_number:
probs = [0.41, 0.29, 0.25, 0.05];
function item() {
var r = Math.random(), s = 0;
for(var i = 0; i < probs.length; i++) {
s += probs[i];
if(r <= s)
return i;
}
}
// generate 100000 randoms
a = [];
c = 0;
while(c++ < 100000) {
a.push(item());
}
// test actual distibution
c = {}
a.forEach(function(x) {
c[x] = (c[x] || 0) + 1;
});
probs.forEach(function(_, x) {
document.write(x + "=" + c[x] / a.length + "<br>")
});

Create a second parallel array with corresponding weights and use a "wheel" algorithm to get an index.
function randomWithProbability()
{
var notRandomNumbers = [1,2,3,4];
var w = [0.41, 0.29, 0.25, 0.05];
var placehldr = 0;
var maxProb = 0.41;
var index = Math.floor(Math.random() * w.length);
var i = 0;
placehldr = Math.random() * (maxProb * 2);
while(placehldr > index )
{
placehldr -= w[index];
index = (index + 1) % w.length
}
return (notRandomNumbers[index]);
}
This video has a good explanation as to why it works, it's easier to understand with the visual representation.
https://www.youtube.com/watch?v=wNQVo6uOgYA

There is an elegant solution only requiring a single comparison due to A. J. Walker (Electronics Letters 10, 8 (1974), 127-128; ACM Trans. Math Software 3 (1977), 253-256) and described in Knuth, TAOCP Vol. 2, 120-121.
You can also find a description here, generate random numbers within a range with different probabilities.

javascript - generate a new random number

I have a variable that has a number between 1-3.
I need to randomly generate a new number between 1-3 but it must not be the same as the last one.
It happens in a loop hundreds of times.
What is the most efficient way of doing this?

May the powers of modular arithmetic help you!!
This function does what you want using the modulo operator:
/**
* generate(1) will produce 2 or 3 with probablity .5
* generate(2) will produce 1 or 3 with probablity .5
* ... you get the idea.
*/
function generate(nb) {
rnd = Math.round(Math.random())
return 1 + (nb + rnd) % 3
}
if you want to avoid a function call, you can inline the code.

Here is a jsFiddle that solves your problem : http://jsfiddle.net/AsMWG/
I've created an array containing 1,2,3 and first I select any number and swap it with the last element. Then I only pick elements from position 0 and 1, and swap them with last element.

var x = 1; // or 2 or 3
// this generates a new x out of [1,2,3] which is != x
x = (Math.floor(2*Math.random())+x) % 3 + 1;

You can randomly generate numbers with the random number generator built in to javascript. You need to use Math.random().
If you're push()-ing into an array, you can always check if the previously inserted one is the same number, thus you regenerate the number. Here is an example:
var randomArr = [];
var count = 100;
var max = 3;
var min = 1;
while (randomArr.length < count) {
var r = Math.floor(Math.random() * (max - min) + min);
if (randomArr.length == 0) {
// start condition
randomArr.push(r);
} else if (randomArr[randomArr.length-1] !== r) {
// if the previous value is not the same
// then push that value into the array
randomArr.push(r);
}
}

As Widor commented generating such a number is equivalent to generating a number with probability 0.5. So you can try something like this (not tested):
var x; /* your starting number: 1,2 or 3 */
var y = Math.round(Math.random()); /* generates 0 or 1 */
var i = 0;
var res = i+1;
while (i < y) {
res = i+1;
i++;
if (i+1 == x) i++;
}

The code is tested and it does for what you are after.
var RandomNumber = {
lastSelected: 0,
generate: function() {
var random = Math.floor(Math.random()*3)+1;
if(random == this.lastSelected) {
generateNumber();
}
else {
this.lastSelected = random;
return random;
}
}
}
RandomNumber.generate();