I am trying to sort sets of associated key value pairs. They look like this:
{"word":"a","votes":9326,"userMade":"FALSE","limiter":"FALSE"},
But organized into labeled subsets of preferably a string or perhaps an index if necessary.
The data-set is a vote-per-use table of most used english words being parsed into pages.
I will be appending them as text to other html elements due to the constraints my use case, makes it a bit tricky, however, for an example I could work with a simple console.log of the page value followed by the console.log of every word value stored within that page. I need the order preserved. so probably indexed. I will also need to be able to sort each page by the votes value, but I think I can figure the rest out for that.
I have found tutorials on how to search through key-value pairs, but I cannot find how to do all of the following with one solution:
A: access the value of word
B: maintain the order of the data-set, allowing me to append them to the matching html element
C: allows me the opportunity to change which set of elements I am appending to when i have finished looping through a single member of the parent index (the one recording the page)
I imagine it is some combination of for/of and for/in, but I'm getting a headache. Please help?
addl info:
function would run at app startup or when the dataset being examined is changed.
function would take a large dataset filled with around 200 page number values, each with 60+ sets of data like the one listed above, the contents of a single page index for example:
{"word":"a","votes":9326,"userMade":"FALSE","limiter":"FALSE"},
{"word":"aaron","votes":4129,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abandoned","votes":1289,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abc","votes":5449,"userMade":"FALSE","limiter":"FALSE"},
{"word":"aberdeen","votes":641,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abilities","votes":2210,"userMade":"FALSE","limiter":"FALSE"},
{"word":"ability","votes":7838,"userMade":"FALSE","limiter":"FALSE"},
{"word":"able","votes":8649,"userMade":"FALSE","limiter":"FALSE"},
{"word":"aboriginal","votes":1837,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abortion","votes":3232,"userMade":"FALSE","limiter":"FALSE"},
{"word":"about","votes":9295,"userMade":"FALSE","limiter":"FALSE"},
{"word":"above","votes":8818,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abraham","votes":867,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abroad","votes":4969,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abs","votes":2415,"userMade":"FALSE","limiter":"FALSE"},
{"word":"absence","votes":4934,"userMade":"FALSE","limiter":"FALSE"},
{"word":"absent","votes":2937,"userMade":"FALSE","limiter":"FALSE"},
{"word":"absolute","votes":5251,"userMade":"FALSE","limiter":"FALSE"},
{"word":"absolutely","votes":5936,"userMade":"FALSE","limiter":"FALSE"},
{"word":"absorption","votes":285,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abstract","votes":7946,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abstracts","votes":1907,"userMade":"FALSE","limiter":"FALSE"},
{"word":"abuse","votes":7238,"userMade":"FALSE","limiter":"FALSE"},
{"word":"academic","votes":7917,"userMade":"FALSE","limiter":"FALSE"},
{"word":"academics","votes":1706,"userMade":"FALSE","limiter":"FALSE"},
{"word":"academy","votes":6755,"userMade":"FALSE","limiter":"FALSE"},
{"word":"acc","votes":6469,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accent","votes":1020,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accept","votes":7547,"userMade":"FALSE","limiter":"FALSE"},
{"word":"acceptable","votes":4907,"userMade":"FALSE","limiter":"FALSE"},
{"word":"acceptance","votes":7273,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accepted","votes":7684,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accepting","votes":1789,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accepts","votes":1535,"userMade":"FALSE","limiter":"FALSE"},
{"word":"access","votes":9031,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accessed","votes":2932,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accessibility","votes":5702,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accessible","votes":5662,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accessing","votes":2096,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accessories","votes":8875,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accessory","votes":5661,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accident","votes":5664,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accidents","votes":2991,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accommodate","votes":1807,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accommodation","votes":8059,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accommodations","votes":3885,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accompanied","votes":2532,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accompanying","votes":664,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accomplish","votes":1070,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accomplished","votes":2419,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accordance","votes":6434,"userMade":"FALSE","limiter":"FALSE"},
{"word":"according","votes":8282,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accordingly","votes":3003,"userMade":"FALSE","limiter":"FALSE"},
{"word":"account","votes":8996,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accountability","votes":3029,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accounting","votes":7459,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accounts","votes":7507,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accreditation","votes":1605,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accredited","votes":3027,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accuracy","votes":6779,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accurate","votes":6427,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accurately","votes":1493,"userMade":"FALSE","limiter":"FALSE"},
{"word":"accused","votes":2853,"userMade":"FALSE","limiter":"FALSE"},
{"word":"acdbentity","votes":1389,"userMade":"FALSE","limiter":"FALSE"},
and the output would ultimately append the value paired with each word to a specific button through iteration, but also sorted by the page value.
each page is a set of buttons in a 3d object that looks like this:
the text is appended to each button which in turn is a 3d object embeded in an html object using aframe. I can make the appending code.
You can use Object.entries() to get the key value pairs of an object.
var words = [
{"word":"a","votes":9326,"userMade":"FALSE","limiter":"FALSE"},
{"word":"aaron","votes":4129,"userMade":"FALSE","limiter":"FALSE"}
];
words.forEach((wordEntry) => {
var keyValuePairs = Object.entries(wordEntry);
keyValuePairs.forEach((kv) => {
console.log(`key: ${kv[0]} value: ${kv[1]}`);
});
});
my latest attempt looks like this:
for (let p=1; p<129; p++){
for (let b=1; b<68; b++){
let pTpl = (p).toLocaleString(undefined, {minimumIntegerDigits: 3});
let bDbl = (b).toLocaleString(undefined, {minimumIntegerDigits: 2});
var `#fCont${pTpl}${bDbl}` = document.createElement('a-text');
`fCont${pTpl}${bDbl}`.setAttribute('value', 'engWordLib[p,b,0]');
`fCont${pTpl}${bDbl}`.setAttribute('votes', 'engWordLib[p,b,1]');
`fCont${pTpl}${bDbl}`.setAttribute('userMade', 'engWordLib[p,b,2]');
`fCont${pTpl}${bDbl}`.setAttribute('limiter', 'engWordLib[p,b,3]');
`fCont${pTpl}${bDbl}`.setAttribute('visible', 'false');
`fBtn${bDbl}`.appendChild(`#fCont${pTpl}${bDbl}`)
}
}
please note that I havent checked this for errors. I still think this code is to WET and I would prefer the key names for the properties be preserved in the datastructure rather than tacked on when it's appended to the page. I guess I could add a dimension to the array.... seems kind of messy when an object property value has the key value pairs right in it. cant get the iteration of objects in an array down though.... Will continue to persue a cleaner method.
I've been reading all around the MDN, but I get stuff like:
keyPath
The key path for the index to use. Note that it is possible to create an index with an empty keyPath, and also to pass in a sequence (array) as a keyPath.
Well, no s#!t, keyPath is a key path. But what is that?
In all examples, it's the same thing as the name of the column (or index, as they call it):
objectStore.createIndex("hours", "hours", { unique: false });
objectStore.createIndex("minutes", "minutes", { unique: false });
objectStore.createIndex("day", "day", { unique: false });
objectStore.createIndex("month", "month", { unique: false });
objectStore.createIndex("year", "year", { unique: false });
They say that you can pass:
An empty string - ""
A valid JavaScript identifier (I assume this means valid JS variable name)
Multiple javascript identifiers separated by periods, eg: "name.name2.foo.bar"
An array containing any of the above, eg.: ["foo.bar","","name"]
I can't imagine what purpose does this serve. I absolutely do not understand what keyPath is and what can I use it for. Can someone please provide example usage where keyPath is something else than the name of the column? An explanation what effect do values of keyPath have on the database?
Examples might help. If you use a key path with an object store, you can have keys plucked out of the objects being stored rather than having to specify them on each put() call. For example, with records that just have an id and name, you could use the record's id as a primary key for the object store:
store = db.createObjectStore('my_store', {keyPath: 'id'});
store.put({id: 987, name: 'Alice'});
store.put({id: 123, name: 'Bob'});
Which gives you this store:
key | value
------+-------------------
123 | {id: 123, name: 'Bob'}
987 | {id: 987, name: 'Alice'}
But if you want to look record up by name, you create an index:
name_index = store.createIndex('index_by_name', 'name');
Which gives you this index:
key | primary key | value
--------+-------------+--------------------------
'Alice' | 987 | {id: 987, name: 'Alice'}
'Bob' | 123 | {id: 123, name: 'Bob'}
(The index doesn't really store a copy of the value, just the primary key. But it's easier to visualize this way. This also explains the properties you'll see on a cursor if you iterate over the index.)
So now you can look up a record by name with:
req = name_index.get('Alice')
When records are added to the store, the key path is used to generate the key for the index.
Key paths with . separators can be used for lookups in more complex records. Key paths that are arrays can either produce compound keys (where the key is itself an array), or multiple index entries (if multiEntry: true is specified)
A great way to understand things is to think how you would design it yourself.
Let's take a step back, and look at how a very simple NoSQL database would store data, then add indices.
Each Store would look like a JavaScript object (Or dictionary in python, a hash in C# ) etc...
{
"1": {"name": "Lara", "age": "20"},
"2": {"name": "Sara", "age": "22"},
"3": {"name": "Joe", "age": "22"},
}
This structure is basically a list of values, plus an index to retrieve the values, like so:
//Raw JS:
var sara = data["2"]
// IndexedDB equivalent:
store.get(2)
This is super fast for direct object retrieval, but sucks for filtering. If you want to get people of a specific age, you need to loop over every value.
If you knew in advance you would be doing your queries by age, you could really speed up such queries by creating a new object which indexes the value by age:
{
"20": [<lara>],
"22": [<joe>, <sara>],
...
}
But how would you query that? You can't use the default indexing syntax (e.g. data[22] or store.get(22)) because those expect the key.
One way would be to name that second index, e.g. by_age_index and give our store object a way to access that index by name, so you could to this:
store.index('by_age_index').get(22) // Returns Joe and Sara
The last bit of the puzzle would be telling that index how to determine which records go against which key (because it has to keep itself updated when records are added/changed/removed)
In other words, how does it know Joe and Sara go against key 22, but Lara goes against key 20?
In our case, we want to use the field age from each record. This is what we mean by a keyPath.
So when defining an index, it makes sense that we would specify that as well as the name, e.g.
store.createIndex('by_age_index', 'age');
Of course, if you want to access your index like this:
store.index('age').get(22) // Returns Joe and Sara
Then you need to create your index like this:
store.createIndex('age', 'age');
Which is what most people do, which is why we see that in examples, which gives the impression that the first argument is the keyPath (whereas it's actually just the arbitrary name we give that index) leaving us unsure about what the second argument might be for.
I could have explained all this by saying:
The first parameter is the handle by which you access the index on the store, the second parameter is the name of the field on the record by which that index should group its records.
But maybe this rundown will help other people too :-)
A keypath is how you indicate to indexedDB which properties of your object play a special role. Similar to how you would indicate to an SQL database that a certain column in a table is the primary key of the table, or how you could tell a database to create an index on one or more particular columns in a table.
In other words, it is the path that the indexedDB implementation should follow when determining which property should be used for some calculation. For example, when searching for a value with a given key.
It is a path, and not a simple key, because it considers that object property values can also be objects. In other words, there is a hierarchy. For example, {a:{b:1}}. The "path" to the value 1 is "a.b". The path is the sequence of properties to visit to get to the value.
The key part of the name signifies that the columns play an important role. For example, in identifying the primary key property, or a particular indexed property.
Properties that are not part of the keypath are ignored in the sense that the indexedDB implementation just treats the whole object as a bag of properties, and only pays attention to those, or gains awareness of those, that are a part of a keypath.
I'm making a web app where a user gets data from PHP, and the data consists of MySQL rows, so I want to save the used ones in a global variable, something like a buffer, to prevent extra AJAX requests.
I'm doing this right now :
window.ray = []; // global variable
$(function(){
data = getDataWithAjax(idToSearch);
window.ray[data.id] = data.text;
});
but when the id is big, say 10 for now, window.ray becomes this :
,,,,,,,,42
so it contains 9 unnecessary spots. Or does it? Is it only visible when I'm doing console.log(window.ray);
If this is inefficient, I want to find a way like PHP, where I can assign only indices that I want, like :
$array['420'] = "abc";
$array['999'] = "xyz";
Is my current way as efficient as PHP, or does it actually contain unnecessary memory spots?
Thanks for any help !
Use an object instead of an array. The object will let you use the id as the key and be more efficient for non-sequential id values.
window.ray = {}; // global variable
$(function(){
data = getDataWithAjax(idToSearch);
window.ray[data.id] = data.text;
});
You can then access any element by the id:
var text = window.ray[myId];
If you are assigning values directly by property name, then it doesn't make any difference in terms of performance whether you use an Array or an Object. The property names of Arrays are strings, just like Objects.
In the following:
var a = [];
a[1000] = 'foo';
then a is (a reference to) an array with length 1,001 (always at least one greater than the highest index) but it only has one numeric member, the one called '1000', there aren't 1,000 other empty members, e.g.:
a.hasOwnProperty['999']; // false
Arrays are just Objects with a special, self–adjusting length property and some mostly generic methods that can be applied to any suitable object.
One feature of sparse arrays (i.e. where the numeric properties from 0 to length aren't contiguous) is that a for loop will loop over every value, including the missing ones. That can be avoided and significant performance gains realised by using a for..in loop and using a hasOwnProperty test, just like an Object.
But if you aren't going to use any of the special features of an Array, you might as well just use an Object as suggested by jfriend00.
So I am using mongoose and node.js to access a mongodb database. I want to bump up each result based on a number (they are ordered by date created if none are bumped up). For example:
{ name: 'A',
bump: 0 },
{ name: 'B',
bump: 0 },
{ name: 'C',
bump: 2 },
{ name: 'D',
bump: 1 }
would be retreived in the order: C, A, D, B. How can this be accomplished (without iterating through every entry in the database)?
Try something like this. Store a counter tracking the total # of threads, let's call it thread_count, initially set to 0, so have a document somewhere that looks like {thread_count:0}.
Every time a new thread is created, first call findAndModify() using {$inc : {thread_count:1}} as the modifier - i.e., increment the counter by 1 and return its new value.
Then when you insert the new thread, use the new value for the counter as the value for a field in its document, let's call it post_order.
So each document you insert has a value 1 greater each time. For example, the first 3 documents you insert would look like this:
{name:'foo', post_order:1, created_at:... } // value of thread_count is at 1
{name:'bar', post_order:2, created_at:... } // value of thread_count is at 2
{name:'baz', post_order:3, created_at:... } // value of thread_count is at 3
etc.
So effectively, you can query and order by post_order as ASCENDING, and it will return them in the order of oldest to newest (or DESCENDING for newest to oldest).
Then to "bump" a thread in its sorting order when it gets upvoted, you can call update() on the document with {$inc:{post_order:1}}. This will advance it by 1 in the order of result sorting. If two threads have the same value for post_order, created_at will differentiate which one comes first. So you will sort by post_order, created_at.
You will want to have an index on post_order and created_at.
Let's guess your code is the variable response (which is an array), then I would do:
response.sort(function(obj1, obj2){
return obj2.bump - obj1.bump;
});
or if you want to also take in mind name order:
response.sort(function(obj1, obj2){
var diff = obj2.bump - obj1.bump;
var nameDiff = (obj2.name > obj1.name)?-1:((obj2.name < obj1.name)?1:0);
return (diff == 0) ? nameDiff : diff;
});
Not a pleasant answer, but the solution you request is unrealistic. Here's my suggestion:
Add an OrderPosition property to your object instead of Bump.
Think of "bumping" as an event. It is best represented as an event-handler function. When an item gets "bumped" by whatever trigger in your business logic, the collection of items needs to be adjusted.
var currentOrder = this.OrderPosition
this.OrderPosition = currentOrder - bump; // moves your object up the list
// write a foreach loop here, iterating every item AFTER the items unadjusted
// order, +1 to move them all down the list one notch.
This does require iterating through many items, and I know you are trying to prevent that, but I do not think there is any other way to safely ensure the integrity of your item ordering - especially when relative to other pulled collections that occur later down the road.
I don't think a purely query-based solution is possible with your document schema (I assume you have createdDate and bump fields). Instead, I suggest a single field called sortorder to keep track of your desired retrieval order:
sortorder is initially the creation timestamp. If there are no "bumps", sorting by this field gives the correct order.
If there is a "bump," the sortorder is invalidated. So simply correct the sortorder values: each time a "bump" occurs swap the sortorder fields of the bumped document and the document directly ahead of it. This literally "bumps" the document up in the sort order.
When querying, sort by sortorder.
You can remove fields bump and createdDate if they are not used elsewhere.
As an aside, most social sites don't directly manipulate a post's display position based on its number of votes (or "bumps"). Instead, the number of votes is used to calculate a score. Then the posts are sorted and displayed by this score. In your case, you should combine createdDate and bumps into a single score that can be sorted in a query.
This site (StackOverflow.com) had a related meta discussion about how to determine "hot" questions. I think there was even a competition to come up with a new formula. The meta question also shared the formulas used by two other popular social news sites: Y Combinator Hacker News and Reddit.