I am trying to find out the best way to implement nextId() in a Javascript app that issues unique IDs for its objects. If it matters, it's a program that I am doing as a learning experience that's pure JS, HTML, and CSS (no libraries, frameworks, DBMS, etc.). I saw a similar question on here on SO (although I wasn't able to find it again for the link) with answers that included not only storing a list of possible ids paired with a boolean value to determine if the id is used, but also storing deleted ids in a recycling list to use for future objects that need it. I think the latter option sounds better, but I'm sure there are even more ways to do it. Does anyone know if there is a pattern, algorithm, or otherwise best practice for this task?
EDIT:
I would like to allow users to share data at some point soon in the application's life, so IDs that already exist would likely become an issue at some point. I would like the IDs to be permanent as I will be persisting data with LocalStorage. A simple integer will work which I will prefix with a letter or two to identify the type of object. It would also be nice to fill in the holes, so the integer doesn't get too high when users use it long-term (wishful thinking).
Also, all objects are constructed from strings at the beginning of the program (I know it's insane).
If you just need an id that is unique per the lifetime of a page, you can use a simple monotomically increasing counter in the page:
var getUniqueID = (function() {
var cntr = 0;
return function() {
return cntr++;
};
})();
var idA = getUniqueID();
var idB = getUniqueID();
To make sure your ids are unique among all users is a taller task. Without involving a central server that coins unique ids for you, the general concept here is to create an id that is a combination of three things:
A token that is unique to the user (like a userID)
A token that is guaranteed to be unique for the session (like what we have above)
A random value.
Done right, there can never be a collision between two different users (because the userID is in the id) and the counter makes it so no user ever generates the same id twice in the same session and the random value makes the odds of a user generating the same id themselves in the same session extremely small.
var getGUID = (function() {
var cntr = 0;
return function(userID) {
var rand = Math.random().toString().replace(".", "");
return userID + "_" + rand + "_" + cntr++;
};
})();
var idA = getGUID(myUserID);
var idB = getGUID(myUserID);
Note: this is the simpler approach on GUID generation that assumes you already have a userID. There is a whole lot of research and literature on various strategies for generating a GUID which you can certainly read a lot more about if you want something beyond this. Some references on the topic:
http://en.wikipedia.org/wiki/Globally_unique_identifier
http://betterexplained.com/articles/the-quick-guide-to-guids/
http://www.uddi.org/pubs/draft-leach-uuids-guids-01.txt
Depending on the use case, I like to create a complex unique id:
function newGUID(){
var result = '';
var hexcodes = "0123456789abcdef".split("");
for (var index = 0; index < 32; index++) {
var value = Math.floor(Math.random() * 16);
switch (index) {
case 8:
result += '-';
break;
case 12:
value = 4;
break;
}
result += hexcodes[value];
}
return result;
}
You could use UUIDs as IDs. There's one answer here in SO where you can generate UUIDs in JS. UUIDs are usually enough to be used as IDs. Just to be sure that the id isn't a dupe, you can have an object whose keys are the used IDs. As IDs are generated, you can keep track of them by adding them in the object. You can then look them up by doing obj.hasOwnProperty(id).
You can do it like
var idStorage = {};
var id;
// generate ID that's unique and hasn't been used
do{
id = guid();
} while (idStorage.hasOwnProperty(id));
idStorage[id] = true;
// ID is usable
Also, IDs are supposed to be unique. They should never be reused at all.
AngularJS has a very simply approach to generating unique IDs: just increment a global counter. From src/Angular.js:
var uid = 0;
// ...
/**
* A consistent way of creating unique IDs in angular.
*
* Using simple numbers allows us to generate 28.6 million unique ids per second for 10 years before
* we hit number precision issues in JavaScript.
*
* Math.pow(2,53) / 60 / 60 / 24 / 365 / 10 = 28.6M
*
* #returns {number} an unique alpha-numeric string
*/
function nextUid() {
return ++uid;
}
Of course, you should choose a solution depending on what time frame the generated IDs should be unique. The above solution will generate IDs which are unique for one session of one web page. For a simple single-page application (which is Angular's use case), this will do just fine. If you need them to be unique across multiple page loads, or unique for the same user, you'll need to persist the uid for a bit longer (in a cookie or in a database). If they need to be unique for a longer time, you might also need to look into longer IDs with more than 2^53 possible values.
I would use a UUID v4 since these IDs would always be unique in any circumstance (no additional logic to check if these ids are in use or to recicle old ones), just issue a new id whenever you need one.
Very simple implementation in JS as follows:
function generateUUID(){
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
var r = Math.random()*16|0, v = c == 'x' ? r : (r&0x3|0x8);
return v.toString(16);
});
}
$(function() {
$('button').bind('click', function() {
$('input').val(generateUUID());
});
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<input type="text" style="width: 350px;"/><button>GENERATE NEW ID</button>
Well, it's a bit unclear what you expect from these ids, however if you only want a unique id per type of entity in your system and your data must only live in memory, then you could use the following approach:
Note: I saw from the comments you wanted a prefix which represents the entity.
function identitySequence(base) {
var id = base || 0;
return function () {
return ++id;
};
}
//Lets say you had some sort of repository for every entity
function InMemoryUserRepository(nextId) {
this._nextId = nextId;
}
InMemoryUserRepository.prototype = {
constructor: InMemoryUserRepository,
get nextId() { return 'user-' + this._nextId(); }
//other operations, like save...
};
var userRepository = new InMemoryUserRepository(identitySequence());
userRepository.nextId; //user-1
Now, lets say you wanted your id sequence to be persistent in the localStorage, here's what you could do (extending the above code):
var userRepository = new InMemoryUserRepository(localStorageIdentitySequence('userIdSeq'));
userRepository.nextId; //user-1
//Reload page
userRepository.nextId; //user-2
function localStorageIdentitySequence(storageKey) {
var next = identitySequence(+localStorage.getItem(storageKey));
return function () {
return +(localStorage[storageKey] = next());
};
}
This works fine for a single machine, however if you want a unique id generator that will generate unique IDs across machines, then this approach will not work. You would have to generate the ID from a server that can be accessed by all clients, or you can generate a GUID on the client instead. There would be no way to know if the GUID was already generated by another client, but that would be very unlikely.
Related
Say you want to have a set of 1- to 2-digit hexadecimal numbers, so 256 numbers. Just using a small set to get at the problem, but it would work with any sized string.
So you have a potential N or 256 numbers in this case. You are going to "generate" a new ID for every new data record that comes your way. So it starts of and randomly gives you af, then 1d, then 8a, etc.
The straightforward naïve way to do this is to just simply generate all the numbers in order, then shuffle them, and just pop from the set. This works fine when you only have 256 numbers. But if you have millions or billions of numbers it is impractical as you might have a lot of waisted generated IDs not being used for long periods of time. I would like to avoid this.
So my question is what is the or a fastest way to create a unique key string like this, without generating all of them in advance, and without going in order just incrementing by 1 or whatnot. That is, the key should be seemingly random.
One way I can imagine is using a trie to store the already used/generated values. Then when you are to get a new value, you generate a random one, then check the trie to see if it's already used. I have no idea how to tell how efficient this is though, but it seems like it would be very bad performing once you start running out of IDs and are down to the last few ones in the set. You would generate lots of already generated IDs, and traverse the trie for each, so it would be slow.
I am wondering if there is a more efficient way of doing this, without generating them all in advance. Also, the data records won't be used in figuring out the ID, as the records might be extremely large and complex.
Maybe there is a way to sort of randomly traverse (and generate) a trie at once, and in that way generate the ID since you end up at a unique random place in the trie. Something along those lines perhaps, I don't know.
Also, I am not sophisticated with hashing so I don't know if there would be any good methods with that.
I assume that you could generate sequential IDs; that is, that you have a reliable way of knowing exactly how many IDs have been generated to date. Then it is sufficient to encrypt this count with any reasonably fast encryption algorithm.
The encryption would be done on the count as a binary number, and the encrypted result with most algorithms would be the same size, also binary. If desired, you could base-64 or hex encode the result to make it easier to use as a character string.
Since encryption must be a bijection (that is, a one-to-one mapping) in order for decryption to be possible, this is guaranteed to produce a different result each time until the total ID count overflows. If it is a reasonable encryption function, then the result will appear random (otherwise the cipher would be vulnerable).
I am not sure how performant it will be but my idea is use a object or Map and Math.random()
let obj = {}
function generateRandomId(){
let id = Math.abs( 0.5 - Math.random()) * 1000
if(obj[id]){
generateRandomId()
} else {
obj[id] = true
}
return id
}
console.log(generateRandomId())
console.log(generateRandomId())
console.log(generateRandomId())
console.log(generateRandomId())
But if you are ok with using a modules i find this one is most useful
uuid this generates RFC4122 UUIDS.
I think that there should be some tradeoff between speed, flexibility and efficiency.
On one had pseudo random generators will give you that even distribution of keys and will be reasonably fast to generate. However checking for an existing id would be slow. You can use bloom filters (saving memory) or tries but then as you said at some point you should increase the space.
Another option is to use Gray code which will produce every key (but not at random order). You need to keep track of the last issued code.
I think a mixing function is what you want. It will move bits around in your input to produce an output of the same length. It's reversible so each input corresponds to a unique output.
Since you want the input data to not take part in the id generation, you'll need a surrogate id. You can assign an incrementing id to each record and use the mix function to scramble the id.
You will get something like:
Record A => id == 1 => mixed id == 0x7ed55d16
Record B => id == 2 => mixed id == 0xc761c23c
etc.
See here for some inspiration:
https://crypto.stackexchange.com/questions/12145/need-32-bit-mixing-function-that-has-perfect-avalanche-between-octets
https://gist.github.com/badboy/6267743
I am considering something like this:
var trie = buildTrie()
var id1 = genId(trie)
var id2 = genId(trie)
console.log(id1,id2)
function buildTrie() {
var trie = buildNode(0)
return trie
function buildNode(level) {
if (level == 7) { // 8 bits
var node = {
available: true,
leaf: true
}
return node
} else {
var a = buildNode(level + 1)
var b = buildNode(level + 1)
var node = {
availableLeft: true,
availableRight: true,
left: a,
right: b
}
a.parent = node
b.parent = node
return node
}
}
}
function genId(node) {
var bytes = []
step(node, bytes)
var id = parseInt(bytes.join(''), 2).toString(16)
return id
function step(node, bytes) {
if (node.leaf) {
node.available = false
var c = node
var p = c.parent
while (p) {
if (p.left == c) {
p.availableLeft = false
} else if (p.right == c) {
p.availableRight = false
}
if (!p.availableLeft && !p.availableRight) {
c = p
p = p.parent
} else {
p = false
}
}
}
var randomDirection = Math.random() >= 0.5
if (randomDirection) {
if (node.availableLeft) {
bytes.push(0)
step(node.left, bytes)
} else if (node.availableRight) {
bytes.push(1)
step(node.right, bytes)
}
} else {
if (node.availableRight) {
bytes.push(1)
step(node.right, bytes)
} else if (node.availableLeft) {
bytes.push(0)
step(node.left, bytes)
}
}
}
}
Maybe there is a better way.
So recently I built a search and replace program with Java, now I am working on translating/rebuilding that program with JavaScript. However, I am having trouble finding JS method alternatives for next() and hasNext(). I am new to JS so I don't know what JS methods would work similarly to the Java methods I am used to.
This is my program, I commented through it to show exactly what I am doing with the previously mentioned methods. Basic set up, 2 text areas, one for the search box (search criteria, box 2), and one for the main document (the field of search, box 1). It basically boils down to a cross-reference. It will highlight all the similarities between the documents.
function search() {
//define an array to store the search criteria.
var array = [];
// define a counter.
var n = 0;
// define a constant for the first box, the search field.
const box1 = document.getElementById("box1");
// define a constant for the second box, the search criteria.
const box2 = document.getElementById("box2");
// loop through the search criteria, storing each word as a seperate element in the array.
// this uses non js terms, this is where I need the help.
while (box2.hasNext()) {
array[n] = box2.next();
n = n + 1;
}
// resets the counter.
n = 0;
// loops through each search item, finding and replacing each item with itself, surrounded by mark tags.
while (n <= array.length) {
box1.replace(array[n], "<mark>" + array[n] + "</mark>");
}
}
</script>
There is bound to be other issues, bugs and syntax, feel free to point them out but lets try and keep the focus on the methodology (i.e. method alternatives for next() and hasNext()).
Thanks.
-EDIT- I'd prefer to use native alternative (no jquery) becuase I know even less about jquery than I do js.
I am creating localstorage dynamically like below
window.localStorage["DBname"+count]=JSON.stringify(values)
at the same time i am also creating some other local storage like
window.localStorage["login_detail"]=login_string
window.localStorage["page_config"]=page_config
so i can not use window.localStorage.clear(),because it will clear all my localstorage which i dont want, and also i can not use window.localStorage.removeItem["DBname"+count] because i count know how many "count" will come while execution.
so is there any way to delete localstorage like group delete kind of? help me out.
Simple solution
var prefix = "DBname",
continue = true,
on = 0;
while (continue) {
continue = localStorage.getItem(prefix + on)?true:false;
try { localStorage.removeItem(prefix + on); } catch (e) { continue = false; }
on += 1;
}
This will remove all local storage items beginning with DBname
Tests
Start:
DBname1
DBblah
DBnameCats
DBname 2
DBname 3
End:
DBblah
DBnameCats
An even better solution
var keys = Object.keys(localStorage),
prefix = "DBname";
for (var i = 0; i < keys.length; i += 1) {
if (keys[i].indexOf(prefix) === 0) {
localStorage.removeItem(keys[i]);
}
}
Tests
Start:
DBname1
DBblah
foobar
DBnameCats
DBname 2
DBname 3
End:
DBblah
foobar
Dealing with support
Add this one line:
Object.keys||(Object.keys=function(r){var e=[];for(var n in r)r.hasOwnProperty(n)&&e.push(n);return e});
That will let you use An even better solution on practically every browser.
I suppose you could iterate over all elements in the LocalStorage and find the keys you want to delete each individually.
A quick example would be:
function clearLocalStorageByKeyMatch(word) {
// Find items in the array where he key contains the word 'word'
var filtered = localStorage.filter(function(item, key) {
return key.indexOf(word) > -1;
});
Object.keys(filtered).forEach(function(item) {
localStorage[item] = null;
});
}
// clear localStorage where config word appear
// clearLocalStorageByKeyMatch('config');
You can also look at a library that handles that for you.
Iterating though localStorage keys/items - NO
Iterating over all the localStorage keys is not a good solution. Since localStorage could be used by other applications as well, it would be very inefficient to iterate through all items in localStorage and then remove them.
What is the best Approach
Since you are the owner of you application, you better can maintain the counts. On a rough level you can keep the "counts" you want to delete in future, may be the start and end position of count in the localStorage itself.
Once you know the start & end limit of "Count" - iterate and delete all the "DbName + Count" found in localStorage.
Another Workaround
You can have one single entity as JSON object for your application. Think of it as you are name-spacing all the localStorage items and encapsulating all of them inside one single variable.
var myApp={
"Dbname1":{},
"Dbname2":{},
"login_detail":{},
"page_config":{}
};
for saving this into localStorage:
localStorage["myApp"]=JSON.stringify(myApp);
for retrieving it back:
var myApp=JSON.parse(localStorage["myApp"]);
//now you can delete whatever db you want from `myApp` and save it back to localStorage.
This will keep other application out of danger from your code.
I'm considering generating unique identifiers for data in javascript using one of the uuid methods discussed here. Most likely something along the lines of this one since it uses window.crypto if it's available.
These id's don't need to be globally unique, only unique per user. Will this generate sufficiently unique ids for a large scale application? Is there any reason to think this will result in id collisions? Can javascript generate a sufficiently random uuid for this to work? It looks like window.crypto is fairly widely available and this particular project already requires reasonably modern browsers.
MORE INFO: some background about the problem can be found here
From the comments on this answer:
... (cont'd) The odds of two IDs generated by this function colliding
are, literally, astronomically small. All but 6 of the 128 bits of the
ID are randomly generated, which means that for any two ids, there's a
1 in 2^^122 (or 5.3x10^^36) chance they'll collide. – broofa
That's a 1 in 5,316,911,983,139,663,491,615,228,241,121,378,304 (5.3 undecillion) chance of collision.
Also make sure to validate that when a user attempts to create a new record with this uuid, that the uuid is not already in use. This falls under the larger strategy of never trusting user input.
You can also test this generator if you aren't convinced:
function getUUID() {
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
var r = Math.random()*16|0, v = c == 'x' ? r : (r&0x3|0x8);
return v.toString(16);
});
}
var uuids = [];
var last;
var numGenerated = 0;
do {
last = getUUID();
if (uuids.indexOf(last) === -1) {
uuids.push(last);
numGenerated++;
console.log(numGenerated);
} else {
break;
}
} while (true);
console.log('Got collision after ' + numGenerated + ' generated UUIDS.');
I'm running it in Node.js (V8) right now and am still collision-free after 170,000 ids. Edit: 240,000.
I have some JavaScript that I wrote in a pinch, but I think it could be optimized greatly by someone smarter than me. This code runs on relatively small objects, but it runs a fair amount of times, so its worth getting right:
/**
* Determine the maximum quantity we can show (ever) for these size/color combos
*
* #return int=settings.limitedStockThreshold
*/
function getMaxDefaultQuantity() {
var max_default_quantity = 1;
if (inventory && inventory.sizes) {
sizecolor_combo_loop:
for (var key in inventory.sizes) {
if (inventory.sizes[key].combos) {
for (var key2 in inventory.sizes[key].combos) {
var sizecolor_combo = inventory.sizes[key].combos[key2];
if (isBackorderable(sizecolor_combo)) {
//if even one is backorderable, we can break out
max_default_quantity = settings.limitedStockThreshold;
break sizecolor_combo_loop;
} else {
//not backorderable, get largest quantity (sizecolor_combo or max_default_quantity)
var qoh = parseInt(sizecolor_combo.quantityOnHand || 1);
if (qoh > max_default_quantity) {
max_default_quantity = qoh;
};
};
};
};
};
};
return Math.min(max_default_quantity, settings.limitedStockThreshold);
};
First, inventory is a object returned via JSON. It has a property inventory.sizes that contain all of the available sizes for a product. Each size has a property inventory.sizes.combos which maps to all of the available colors for a size. Each combo also has a property quantityOnHand that tells the quantity available for that specific combo. (the JSON structure returned cannot be modified)
What the code does is loop through each size, then each size's combos. It then checks if the size-color combo is backorderable (via another method). If it any combo is backorderable, we can stop because the default quantity is defined elsewhere. If the combo isn't backorderable, the max_default_quantity is the largest quantityOnHand we find (with a maximum of settings.limitedStockThreshold).
I really don't like the nested for loops and my handling of the math and default values feels overly complicated.
Also, this whole function is wrapped in a much larger jQuery object if that helps clean it up.
Have you considered using map-reduce? See a live example of a functional approach.
This particular example uses underscore.js so we can keep it on a elegant level without having to implement the details.
function doStuff(inventory) {
var max = settings.limitedStockThreshold;
if (!(inventory && inventory.sizes)) return;
var quantity = _(inventory.sizes).chain()
.filter(function(value) {
return value.combos;
})
.map(function(value) {
return _(value.combos).chain()
.map(function(value) {
return isBackorderable(value) ? max : value.quantityOnHand;
})
.max().value();
})
.max().value();
return Math.min(quantity, max);
}
As for an explanation:
We take the inventory.sizes set and remove any that don't contain combos. We then map each size to the maximum quantity of it's colour. We do this mapping each combo to either its quantity or the maximum quantity if backordable. We then take a max of that set.
Finally we take a max of set of maxQuantities per size.
We're still effectily doing a double for loop since we take two .max on the set but it doesn't look as dirty.
There are also a couple of if checks that you had in place that are still there.
[Edit]
I'm pretty sure the above code can be optimized a lot more. but it's a different way of looking at it.
Unfortunately, JavaScript doesn't have much in the way of elegant collection processing capabilities if you have to support older browsers, so without the help of additional libraries, a nested loop like the one you've written is the way to go. You could consider having the values precomputed server-side instead, perhaps cached, and including it in the JSON to avoid having to run the same computations again and again.