Develop Reference

Unique identifier with smallest base possible, without collisions on old/new set of data

I receive one or more trees of nodes.
Nodes may, or may not have ID properties on them.
Currently I am iterating through the tree and adding random 8 digit numbers on nodes which do not have ID property. As I do not expect more than 10k nodes in the trees chance of having collisions is very small.
Still I am considering how best to reduce the length of the IDs to maybe 4 digits while making sure there are no collisions within one tree. What comes to my mind is to iterate once through the tree gathering existing IDs into a Set and than again adding new IDs while checking against the Set that there are no collisions. Set would have to be reset for each tree.
I would appreciate your opinion on this matter and advice if there are more performant ways of achieving this.
Appendix A:
I am considering following (simplified 0-9) issue. If I have a Set of existing IDs [0, 1, 2, 5, 8, 9] I would have to generate random numbers until I get e.g. 4 (no collision) which I am concerned would be a bit slow on a larger Set and surely not the optimal route.

Here you have a really simple approach which will generate for you an array of not used numbers with a given MAX range.
const MAX = 30;
const usedNumbers = [3, 4, 12, 13, 14, 16, 23, 27];
// https://stackoverflow.com/questions/3746725/how-to-create-an-array-containing-1-n
const notUsedNumbers = Array.from(Array(MAX), (_, i) => i+1).filter(i => !usedNumbers.includes(i));
console.log(notUsedNumbers);
And link to fiddle: https://jsfiddle.net/L9r6anq1/

10^8 possibilities with random selection means you have a 50% chance of collision with only 10^4 objects (see Birthday Paradox); that is not "very small" odds. Reducing that to only 10^4 possibilities with 10^4 objects means collisions will approach 100% as you get toward the end and, if you ever have 10k+1 objects one day, will never terminate.
In general, if you want to use a relatively short ID space, you're going to need a very efficient conflict detection system, e.g. keeping all assigned (or not assigned) values in a scratchpad, or just give up on randomly assigning values and go sequentially.

How to make random mutations

I'm doing a project on natural selection for cells for fun. Each code has "dna" which is just a set of instructions. The dna can either have REMOVE WASTE, DIGEST FOOD, or REPAIR WALL. I won't really go into detail what they do, because that would take too long. But the only reason evolution really happens is through genetic mutations. I'm wondering if this is possible in javascript, and how to do it. For example, the starting cell has 5 dna strands. But if it reproduces, the child can have 4, or 6. And some of the dna strands can be altered. This is my code so far:
var strands = ["DIGEST FOOD", "REPAIR WALL", "REMOVE WASTE"];
var dna = [];
for (let i = 0; i < 5; i++) {
if (parent) {
// something about the parents dna, and the mutation chance
}
else {
dna.push(strands[Math.floor(Math.random() * 3)]); // if cell doesn't have parent
}
}
I'm just wondering if this is possible in javascript, and how to succesfully do it. Sorry if the question isn't too clear.
Edit: Let me rephrase a little. What I'm trying to achieve is a genetic mutation in the new cell. Like:
if (parent) {
dna.push(parent);
if (Math.random() < 0.5) {
changeStrand(num);
}
if (Math.random() < 0.5) {
addStrand(num);
}
if (Math.random() < 0.5) {
removeStrand(num);
}
}
function changeStrand() {
// change the strand
}
function newStrand(num) {
// add random strands
}
function removeStrand(num) {
// remove random strands
}
or something like that

For a genetic algorithm, you basically want to take two slices from each parent and stitch them together, whilst ensuring the end result is still a valid dna strand.
For a fixed sized DNA sequence (such a N queens positions), the technique would be to pick a random slice point (1-3 | 4-8) and then combine these slices from the parents to create a child.
For your usecase, you need two random slices who sum of sizes adds upto 4-6. So possibly two slices of size 2-3. You could potentially take one from the front, and the other from the back. Else you could first pick a random output size, and then fill it will two random sequences for either parent.
Array.slice() and Array.splice() are probably the functions you want to use.
You can also add in a random mutation to the end result. Viruses at the speed limit of viable genetic evolution have an average of 1 mutation per transcription. Which means some transcriptions won’t have mutations, which is equivalent to allowing some of the parents in the parent generation to survive.
You can also experiment with different variations. Implement these as feature flags, and see what works best in practice.
Also compare with Beam Search, which essentially keeps a copy of the N best results from each generation. You may or may not want to keep the best from the parent generation to survive unmutated.
Another idea is to compute a distance metric between individuals, and add a cost for being too close to an existing member of the population, and this will select for genetic diversity.
In the standard model, variation occurs both by point mutations in the letter sequence and by “crossover” (in which the DNA of an offspring is generated by combining long sections of DNA from each parent).
The analogy to local search algorithms has already been described; the principal difference between stochastic beam search and evolution is the use of sexual reproduction, wherein successors are generated from multiple organisms rather than just one. The actual mechanisms of evolution are, however, far richer than most genetic algorithms allow. For example, mutations can involve reversals, duplications, and movement of large chunks of DNA; some viruses borrow DNA from one organism and insert it in another, and there are transposable genes that do nothing but copy themselves many thousands of times within the genome. There are even genes that poison cells from potential mates that do not carry the gene, thereby increasing their own chances of replication. Most important is the fact that the genes themselves encode the mechanisms whereby the genome is reproduced and translated into an organism. In genetic algorithms, those mechanisms are a separate program that is not represented within the strings being manipulated.
Artificial Intelligence: A Modern Approach. (Third edition) by Stuart Russell and Peter Norvig.

If you want to have random numbers you can use Math.random() for that. In the linked page there are also some examples for getting values between x and y for example:
// Source https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/random
function getRandomInt(min, max) {
min = Math.ceil(min);
max = Math.floor(max);
return Math.floor(Math.random() * (max - min)) + min;
}
I am not sure this is what you try to achieve - since you already make use of the Math.random() function.

.forEach only moving half of array elements

Firstly, I'm new to JavaScript and in an effort to gain a better understanding of OOP I've decided to build solitaire (the three-card version mind you). In solitaire--as I'm sure is well known-- you deal out the cards in the whatever pattern and the remaining cards are placed off to the side; I'm calling these the 'reserve' deck. Then you move the cards you dealt around following the rules of the game (can only place a red 6 on a black 7, etc.) and when you run out of options you can pull three cards off the top of 'reserve' and see if you can use the third card to continue playing. If not, you pull three more cards, so on and so forth until you run out of cards in the 'reserve' deck. At this point, all the cards you pulled off the top of 'reserve' deck can then get placed back into the 'reserve' deck and then reused in the same manner.
And this is where my javascript is breaking. So I've created a Table object with the properties playSlots, winSlots, reserve and pick. playSlots is an array of arrays in which cards get dealt into, reserve is an array of the cards existent within the 'reserve' deck, winSlots isn't relative to this question and pick is an array that becomes populated with cards from reserve as it becomes necessary throughout game play. Pretty self-explanatory. When reserve becomes empty and you want to put the pick cards back into it, my code for this is currently:
if(!Array.isArray(newGame.reserve) || !newGame.reserve.length){
newGame.pick.forEach(function(cur, i, arr){
newGame.reserve.push(arr.shift());
});
}
The problem with this is that this only iterates through half of the cards in the pick array. I went through it in the debugger and it is, in fact adding the cards back into the reserve deck one at a time but once it gets to the 13th card it breaks out of the loop (28 cards cards get dealt out in the solitaire pattern, leaving 24 in the reserve deck...).
I was pretty sure this issue has something to do with the !newGame.reserve.length condition. That condition is something I found here:
Check if array is empty / does not exist. JS
and although the answer-er is pretty clear in his his explanation of this, I'm still kinda fuzzy on what it means. Thing is, I also tried this for a condition:
if(newGame.reserve[0] === undefined)
and had the exact same result. The .forEach loop is only iterating over half of the array elements.... What am I missing here?

You can achieve this easier using Array splice
Note: this reverses pick to the end of reserve, whereas your code does not
If you do not need it reversed, simply remove the .reverse()
It's not clear if you want pick reversed into reserve or not
var newGame = {
pick: [1,2,3,4],
reserve: [10,11,12]
};
newGame.reserve.push(...newGame.pick.splice(0, newGame.pick.length).reverse());
console.log(newGame);
in "old school" javascript, this can be written using .push.apply
var newGame = {
pick: [1,2,3,4],
reserve: [10,11,12]
};
newGame.reserve.push.apply(newGame.reserve, newGame.pick.splice(0, newGame.pick.length).reverse());
console.log(newGame);

Leaderboard ranking with Firebase

I have project that I need to display a leaderboard of the top 20, and if the user not in the leaderboard they will appear in the 21st place with their current ranking.
Is there efficient way to this?
I am using Cloud Firestore as a database. I believe it was mistake to choose it instead of MongoDB but I am in the middle of the project so I must do it with Cloud Firestore.
The app will be use by 30K users. Is there any way to do it without getting all the 30k users?
this.authProvider.afs.collection('profiles', ref => ref.where('status', '==', 1)
.where('point', '>', 0)
.orderBy('point', 'desc').limit(20))
This is code I did to get the top 20 but what will be the best practice for getting current logged in user rank if they are not in the top 20?

Finding an arbitrary player's rank in leaderboard, in a manner that scales is a common hard problem with databases.
There are a few factors that will drive the solution you'll need to pick, such as:
Total Number players
Rate that individual players add scores
Rate that new scores are added (concurrent players * above)
Score range: Bounded or Unbounded
Score distribution (uniform, or are their 'hot scores')
Simplistic approach
The typical simplistic approach is to count all players with a higher score, eg SELECT count(id) FROM players WHERE score > {playerScore}.
This method works at low scale, but as your player base grows, it quickly becomes both slow and resource expensive (both in MongoDB and Cloud Firestore).
Cloud Firestore doesn't natively support count as it's a non-scalable operation. You'll need to implement it on the client-side by simply counting the returned documents. Alternatively, you could use Cloud Functions for Firebase to do the aggregation on the server-side to avoid the extra bandwidth of returning documents.
Periodic Update
Rather than giving them a live ranking, change it to only updating every so often, such as every hour. For example, if you look at Stack Overflow's rankings, they are only updated daily.
For this approach, you could schedule a function, or schedule App Engine if it takes longer than 540 seconds to run. The function would write out the player list as in a ladder collection with a new rank field populated with the players rank. When a player views the ladder now, you can easily get the top X + the players own rank in O(X) time.
Better yet, you could further optimize and explicitly write out the top X as a single document as well, so to retrieve the ladder you only need to read 2 documents, top-X & player, saving on money and making it faster.
This approach would really work for any number of players and any write rate since it's done out of band. You might need to adjust the frequency though as you grow depending on your willingness to pay. 30K players each hour would be $0.072 per hour($1.73 per day) unless you did optimizations (e.g, ignore all 0 score players since you know they are tied last).
Inverted Index
In this method, we'll create somewhat of an inverted index. This method works if there is a bounded score range that is significantly smaller want the number of players (e.g, 0-999 scores vs 30K players). It could also work for an unbounded score range where the number of unique scores was still significantly smaller than the number of players.
Using a separate collection called 'scores', you have a document for each individual score (non-existent if no-one has that score) with a field called player_count.
When a player gets a new total score, you'll do 1-2 writes in the scores collection. One write is to +1 to player_count for their new score and if it isn't their first time -1 to their old score. This approach works for both "Your latest score is your current score" and "Your highest score is your current score" style ladders.
Finding out a player's exact rank is as easy as something like SELECT sum(player_count)+1 FROM scores WHERE score > {playerScore}.
Since Cloud Firestore doesn't support sum(), you'd do the above but sum on the client side. The +1 is because the sum is the number of players above you, so adding 1 gives you that player's rank.
Using this approach, you'll need to read a maximum of 999 documents, averaging 500ish to get a players rank, although in practice this will be less if you delete scores that have zero players.
Write rate of new scores is important to understand as you'll only be able to update an individual score once every 2 seconds* on average, which for a perfectly distributed score range from 0-999 would mean 500 new scores/second**. You can increase this by using distributed counters for each score.
* Only 1 new score per 2 seconds since each score generates 2 writes
** Assuming average game time of 2 minute, 500 new scores/second could support 60000 concurrent players without distributed counters. If you're using a "Highest score is your current score" this will be much higher in practice.
Sharded N-ary Tree
This is by far the hardest approach, but could allow you to have both faster and real-time ranking positions for all players. It can be thought of as a read-optimized version of of the Inverted Index approach above, whereas the Inverted Index approach above is a write optimized version of this.
You can follow this related article for 'Fast and Reliable Ranking in Datastore' on a general approach that is applicable. For this approach, you'll want to have a bounded score (it's possible with unbounded, but will require changes from the below).
I wouldn't recommend this approach as you'll need to do distributed counters for the top level nodes for any ladder with semi-frequent updates, which would likely negate the read-time benefits.
Final thoughts
Depending on how often you display the leaderboard for players, you could combine approaches to optimize this a lot more.
Combining 'Inverted Index' with 'Periodic Update' at a shorter time frame can give you O(1) ranking access for all players.
As long as over all players the leaderboard is viewed > 4 times over the duration of the 'Periodic Update' you'll save money and have a faster leaderboard.
Essentially each period, say 5-15 minutes you read all documents from scores in descending order. Using this, keep a running total of players_count. Re-write each score into a new collection called scores_ranking with a new field players_above. This new field contains the running total excluding the current scores player_count.
To get a player's rank, all you need to do now is read the document of the player's score from score_ranking -> Their rank is players_above + 1.

One solution not mentioned here which I'm about to implement in my online game and may be usable in your use case, is to estimate the user's rank if they're not in any visible leaderboard because frankly the user isn't going to know (or care?) whether they're ranked 22,882nd or 22,838th.
If 20th place has a score of 250 points and there are 32,000 players total, then each point below 250 is worth on average 127 places, though you may want to use some sort of curve so as they move up a point toward bottom of the visible leaderboard they don't jump exactly 127 places each time - most of the jumps in rank should be closer to zero points.
It's up to you whether you want to identify this estimated ranking as an estimation or not, and you could add some a random salt to the number so it looks authentic:
// Real rank: 22,838
// Display to user:
player rank: ~22.8k // rounded
player rank: 22,882nd // rounded with random salt of 44
I'll be doing the latter.

Alternative perspective - NoSQL and document stores make this type of task overly complex. If you used Postgres this is pretty simple using a count function. Firebase is tempting because it's easy to get going with but use cases like this are when relational databases shine. Supabase is worth a look https://supabase.io/ similar to firebase so you can get going quickly with a backend but its opensource and built on Postgres so you get a relational database.

A solution that hasn't been mentioned by Dan is the use of security rules combined with Google Cloud Functions.
Create the highscore's map. Example:
highScores (top20)
Then:
Give the users write/read access to highScores.
Give the document/map highScores the smallest score in a property.
Let the users only write to highScores if his score > smallest score.
Create a write trigger in Google Cloud Functions that will activate when a new highScore is written. In that function, delete the smallest score.
This looks to me the easiest option. It is realtime as well.

You could do something with cloud storage. So manually have a file that has all the users' scores (in order), and then you just read that file and find the position of the score in that file.
Then to write to the file, you could set up a CRON job to periodically add all documents with a flag isWrittenToFile false, add them all to the file (and mark them as true). That way you won't eat up your writes. And reading a file every time the user wants to view their position is probably not that intensive. It could be done from a cloud function.

2022 Updated and Working Answer
To solve the problem of having a leaderboards with user and points, and to know your position in this leaderboards in an less problematic way, I have this solution:
1) You should create your Firestorage Document like this
In my case, I have a document perMission that has for each user a field, with the userId as property and the respective leaderboard points as value.
It will be easier to update the values inside my Javascript code.
For example, whenever an user completed a mission (update it's points):
import { doc, setDoc, increment } from "firebase/firestore";
const docRef = doc(db, 'leaderboards', 'perMission');
setDoc(docRef, { [userId]: increment(1) }, { merge: true });
The increment value can be as you want. In my case I run this code every time the user completes a mission, increasing the value.
2) To get the position inside the leaderboards
So here in your client side, to get your position, you have to order the values and then loop through them to get your position inside the leaderboards.
Here you can also use the object to get all the users and its respective points, ordered. But here I am not doing this, I am only interested in my position.
The code is commented explaining each block.
// Values coming from the database.
const leaderboards = {
userId1: 1,
userId2: 20,
userId3: 30,
userId4: 12,
userId5: 40,
userId6: 2
};
// Values coming from your user.
const myUser = "userId4";
const myPoints = leaderboards[myUser];
// Sort the values in decrescent mode.
const sortedLeaderboardsPoints = Object.values(leaderboards).sort(
(a, b) => b - a
);
// To get your specific position
const myPosition = sortedLeaderboardsPoints.reduce(
(previous, current, index) => {
if (myPoints === current) {
return index + 1 + previous;
}
return previous;
},
0
);
// Will print: [40, 30, 20, 12, 2, 1]
console.log(sortedLeaderboardsPoints);
// Will print: 4
console.log(myPosition);
You can now use your position, even if the array is super big, the logic is running in the client side. So be careful with that. You can also improve the client side code, to reduce the array, limit it, etc.
But be aware that you should do the rest of the code in your client side, and not Firebase side.
This answer is mainly to show you how to store and use the database in a "good way".

Connecting Rooms

I've created a simple algorithm for a game I'm working on that creates a cave like structure. The algorithm outputs a 2 dimensional array of bits that represent the open area's. Example:
000000000000000000000000
010010000000000111100000
011110000000011111111000
011111110000011111111100
011111111001111111111110
011000000000001111000000
000000000000000000000000
(0's represent wall, 1's represent open areas)
The problem is that the algorithm can sometimes create a cave that has 2 non connected sections (as in the above example). I've written a function that gives me an array of arrays that contain all the x, y positions of the open spots for each area
My question is, given a number of lists that contain all of the x,y coordinates for each open area what is the fastest way to "connect" these area's be a corridor that is a minimum of 2 thickness wide.
(I'm writing this in javascript but even just pseudo code will help me out)
I've tried comparing the distances from every point in one area to every other area in another area, finding the two points that have the closest distance then cutting out a path from those 2 two points but this approach is way to slow I'm hoping there is another way.

Given two caves A and B, choose a point x in A and y in B (at random will do, the two closest or locally closest is better). Drill a corridor of thickness 2 between A and B (use Bresenham's algorithm). If you have multiple disconnected caves, do the above for each edge (A,B) of the minimal spanning tree of the graph of all the caves (edge weight is the length of the corridor you'll drill if you choose this edge).
Edit for the edit: to approximate the distance between two caves, you can use hill climbing. It will return the global minimum for convex caves in O(n) rather than the naive O(n2). For non-convex caves, do multiple iterations of hill climbing with initial guess chosen in random.

If you need the exactly minimal solution, you can consider first building the frontiers of your caves and then applying O(nm) algorithm. This will eliminate the need to compare distances between interior points of your caves. Then as soon as you know the distances between each pair of caves, you build the minimal spanning tree, then you drill your tunnels.

Since I don't know too much from your description, here are some hints I would consider:
How do you look for the pair of nearest points? Do you use a naive brute-force approach and thus obtain a run time of O(n*n)? Or are you using a more efficient variant taking O(n log n) time?
If you have obtained the closest points, I'd use a simple line-drawing algorithm.
Another approach might be that you generate a structure that definitely has only one single connected area. Therefore you could do the following: First you take a random cell (x,y) and set it to 1. Then, you traverse all it's neighbours and for each of them you randomly set it to 1 or leave it at 0. For each cell set to 1, you do the same, i.e. you traverse it's neighbours and set them randomly to 1 or 0. This guarantees that you won't have two separate areas.
An algorithm to ensure this could be the following (in python):
def setCell(x,y,A):
if x>=len(A) or y>=len(A[0]) or x<0 or y<0:
return
A[x][y] = 1
def getCell(x,y,A):
if x>=len(A) or y>=len(A[0]) or x<0 or y<0:
return 1
return A[x][y]
def generate(height, width):
A = [[0 for _ in xrange(width)] for _ in xrange(height)]
from random import randint
import Queue
(x,y) = (randint(0, height-1), randint(0, width-1))
setCell (x,y,A)
q = Queue.Queue()
q.put((x,y))
while not q.empty():
(x,y) = q.get()
for (nx, ny) in [(x+1,y), (x-1,y), (x,y+1), (x,y-1)]:
if randint(0,8)<=6:
if getCell(nx,ny,A)==0:
setCell(nx,ny,A)
if randint(0,2)<=1:
q.put((nx,ny))
return A
def printField(A):
for l in A:
for c in l:
print (" " if c==1 else "X"),
print ""
Then printField(generate(20,30)) does the job. Probably you'll have to adjust the parameters for random stuff so it fits your needs.