Is there a way to retrieve the next unique index in a store based on a predicate on the record. For example if I have a book store full of objects like so:
{name: 'Hello Kitty', author: 'Me', pages: 5}
Would it be possible to return the next unique index on author, but base the uniqueness on the highest number of pages?
index.openKeyCursor('author', IDBCursor.nextunique).onsuccess = function(event) {
var cursor = event.target.result;
if (cursor) {
// How to filter the record by highest number of pages?
cursor.continue();
}
};
This is a bit tricky, but you can do. I will illustrate with my library https://bitbucket.org/ytkyaw/ydn-db but you can use IndexedDB API.
First you have to use compound index (only Firefox and Chrome supported) using array keyPath. Database schema for ydn-db is
var schema = {
stores: [{
name: 'book',
indexes: [{
name: 'author, pages',
keyPath: ['author', 'pages']
}]
}
};
var db = new ydn.db.Storage('db name', schema);
The index, 'author, pages' is sorted by author and then by pages. Then we prepare cursor or create iterator in ydn-db.
var iter = new ydn.db.IndexValueIterator('book', 'author, pages');
By default, order is in ascending. Here we want descending order to get highest pages value. This inadvertently make author to sort in descending order, but there is no way to avoid it.
iter = iter.reverse().unique(); // essentially 'PREV_UNIQUE'
Then, we open the iterator giving rise to cursor with descending ordering. The first cursor is what we want. On next iteration, we skip duplicate author name. This is done by using cursor.continue(next_key) method. next_key is given, such that it won't repeat what already got by giving lowest possible value with known author key.
db.open(function(cursor) {
var book = cursor.getValue();
console.log(book);
var effective_key = cursor.getKey();
var author_key = effective_key[0];
var next_key = [author_key];
return next_key; // continue to this or lower than this key.
}, iter);
Note that, we just need to iterate only unique author and no buffer memory require, and hence scalable.
Related
I am trying to loop through a dictionary of customers and save energy usage data, but for some customers when I try to change the values in their usage dictionary it will also change a completely different customer's value. I have a nested dictionary with customer utility information, the top-level key being a unique internal ID.
I stripped my code down to a single loop, looping through the top-level keys and setting the same month's usage for all customers in the dictionary to be the value of the iterator. After that, as shown in the code sample below, I log the values for three customers. After that, I increment only one of those customer's usage, and log the values again. The console shows that two over the customer's have dictionaries that are tied together somehow, but I can't figure out why or how to solve this. I can't discern any pattern in the keys of the linked customers, either.
Structure of the nested dictionary:
CustDict =
{"N0100000XXXXXX" =
{"name" = "XXXX"},
{"address" = "XXXX"},
{"meter_read_dates" =
{"2021-05-13" =
{"usage" = "XXXX"}
}
}
}
Stripped down code I used to demonstrate what is happening as simply as possible (real ID values):
Logger.log(custDict["N01000009700816"]["meter_read_dates"]["2021-05-13"]["usage"])
Logger.log(custDict["N01000000419887"]["meter_read_dates"]["2021-05-13"]["usage"])
Logger.log(custDict["N01000012580668"]["meter_read_dates"]["2021-05-13"]["usage"])
custDict["N01000009700816"]["meter_read_dates"]["2021-05-13"]["usage"] =
custDict["N01000009700816"]["meter_read_dates"]["2021-05-13"]["usage"] + 1
Logger.log(custDict["N01000009700816"]["meter_read_dates"]["2021-05-13"]["usage"])
Logger.log(custDict["N01000000419887"]["meter_read_dates"]["2021-05-13"]["usage"])
Logger.log(custDict["N01000012580668"]["meter_read_dates"]["2021-05-13"]["usage"])
Console Output:
11:54:56 AM Info 346.0
11:54:56 AM Info 346.0
11:54:56 AM Info 322.0
11:54:56 AM Info 347.0
11:54:56 AM Info 347.0
11:54:56 AM Info 322.0
Code used to create the CustDict dictionary:
stmtCR = conn.prepareStatement('SELECT cust_id, utility_account, cycle_id, read_cycle FROM customers')
results = stmtCR.executeQuery()
resultsMetaData = results.getMetaData()
numCols = resultsMetaData.getColumnCount();
results.last();
numRows = results.getRow();
i = 0
results.first()
var custDict = {}
while (i < numRows)
{
custDict[results.getString(1)] = {}
custDict[results.getString(1)]["id"] = results.getString(1)
custDict[results.getString(1)]["utility_account"] = results.getString(2)
custDict[results.getString(1)]["cycle_id"] = results.getString(3)
custDict[results.getString(1)]["read_cycle"] = results.getString(4)
results.next()
i++;
}
for (i = 0; i < Object.keys(custDict).length; i++)
{
tempCust = custDict[Object.keys(custDict)[i]]
tempCycleId = tempCust["cycle_id"]
tempReadCycle = tempCust["read_cycle"]
tempCust["meter_read_dates"] = cycleIdShdDict[tempCycleId][tempReadCycle]
custDict[Object.keys(custDict)[i]] = tempCust
}
cycleIdShdDict is a seperate dictionary that contains a set of dates associated with each cycle_id and read_cycle
I suspect the problem is that Object.keys(custDict) is returning the keys in a different order at different places in the for loop. So you're getting the object from one key, and then assigning it to a different key.
There's no need to assign back to custDict[Object.keys(custDict)[i]] since you're modifying the object in place, not a copy.
But instead of looping through the keys, loop through the values and modify them.
Object.values(custDict).forEach(tempCust => {
let tempCycleId = tempCust["cycle_id"];
let tempReadCycle = tempCust["read_cycle"];
tempCust["meter_read_dates"] = cycleIdShdDict[tempCycleId][tempReadCycle];
});
I am looping through a collection of blog posts to firstly push the username and ID of the blog author to a new array of arrays, and then secondly, count the number of blogs from each author. The code below achieves this; however, in the new array, the username and author ID are no longer separate items in the array, but seem to be concatenated into a single string. I need to retain them as separate items as I need to use both separately; how can I amend the result to achieve this?
var countAuthors = [];
blogAuthors = await Blog.find().populate('authors');
blogAuthors.forEach(function(blogAuthor){
countAuthors.push([blogAuthor.author.username, blogAuthor.author.id]);
})
console.log(countAuthors);
// Outputs as separate array items, as expected:
// [ 'author1', 5d7eed028c298b424b3fb5f1 ],
// [ 'author2', 5dd8aa254d74b30017dbfdd3 ],
var result = {};
countAuthors.forEach(function(x) {
result[x] = (result[x] || 0) + 1;
});
console.log(result);
// Username and author ID become a single string and cannot be accessed as separate array items
// 'author1,5d7eed028c298b424b3fb5f1': 15,
// 'author2,5dd8aa254d74b30017dbfdd3': 2,
Update:
Maybe I can explain a bit further WHY on what to do this. What I am aiming for is a table which displays the blog author's name alongside the number of blogs they have written. However, I also want the author name to link to their profile page, which requires the blogAuthor.author.id to do so. Hence, I need to still be able to access the author username and ID separately after executing the count. Thanks
You could use String.split().
For example:
let result = 'author1,5d7eed028c298b424b3fb5f1'.split(',')
would set result to:
['author1' , '5d7eed028c298b424b3fb5f1']
You can then access them individually like:
result[1] //'5d7eed028c298b424b3fb5f1'
Your issue is that you weren't splitting the x up in the foreach callback, and so the whole array was being converted to a string and being used as the key when inserting into the results object.
You can use array destructuring to split the author name and blog id, and use them to optionally adding a new entry to the result object, and then update that result.
countAuthors = [
['author1', 'bookId1'],
['author2', 'bookId2'],
['author1', 'bookId3'],
['author1', 'bookId4'],
['author2', 'bookId5']
]
var result = {};
countAuthors.forEach(([author, id]) => {
if (result[author] === undefined) {
result[author] = {count: 0, blogIds: []};
}
result[author].count += 1;
result[author].blogIds.push(id);
});
console.log(result);
We're building a site with ExpressionEngine. We are running a SQL query to gather up all member IDs for a specific member group. After that, we are using EE tags to get data from a custom member field for each member ID.
The ID and field data need to stay paired, as we will be populating a drop-down so that the ID is the value and the field data is the text, so we are currently putting them into a JS array as key/value pairs. The call is as follows:
var array= [
{exp:query sql="SELECT * FROM exp_members WHERE group_id = 5"}
{exp:member:custom_profile_data
member_id="{member_id}"}
{if company != ''}
{{member_id}:"{company}"},
{/if}
{/exp:member:custom_profile_data}
{/exp:query}
};
This gives us the output:
var array = [
{1:"name01"},
{2:"name02"},
{3:"name01"},
{4:"name03"}
];
Now, our problem. We need to remove objects based on duplicate field data (values) only, so the above array would look like this:
var array = [
{1:"name01"},
{2:"name02"},
{4:"name03"}
];
None of these IDs (keys) will ever be the same, but the field data (values) can be. So we want to keep the first KV pair that comes through with a unique value, but remove any subsequent dupes of that value - despite the fact that they will not be true "duplicate values" due to a different ID (key).
Keeping in mind that the KV pairs are all dynamic, is there any possible way to do this via JS so we can create a new array for the cleaned data to pass to the drop-down?
You could handle the duplications by modifying your MySQL query. (In my example, my custom field ID was 1.)
var myArray = [];
{exp:query sql="SELECT MIN(m.member_id) AS co_member_id, d.m_field_id_1 AS company FROM exp_members m INNER JOIN exp_member_data d ON m.member_id = d.member_id WHERE d.m_field_id_1 != '' AND m.group_id > 0 GROUP BY d.m_field_id_1;"}
myArray.push({{co_member_id}: "{company}"});
{/exp:query}
This query would use the first (in the ordinal sense) member_id found; you could also change the MIN to MAX and get the last.
This will give you a clean output in your source, without the need for any additional JS processing. I'd also recommend changing the names of the variables you're outputting as to not conflict in EE's parsing.
I would do it like...
function removeDups(arry){
var tmp = {}, retainIdx=[], newArry=[];
arry.forEach(function(obj, idx){
var val = obj[Object.keys(obj)[0]];
if(val && !tmp[val]){
retainIdx.push(idx);
tmp[val] = true;
}
});
retainIdx.forEach(function(i){
newArry.push(arry[i]);
});
return newArry;
};
Consider the following object store, with the domain key set as the keyPath
var tags = [
//codes: 0 - markdown wrap tag
// 1 - HTML wrap tag
// 2 - single tag
{ domain: "youtube",
bold:["*",0],
strikethrough:["-",0],
italic:["_",0]
},
{ domain: "stackoverflow",
bold:["<strong>",1],
italic:["<em>",1],
strikethrough:["<del>",1],
superscript:["<sup>",1],
subscript:["<sub>",1],
heading1:["<h1>",1],
heading2:["<h2>",1],
heading3:["<h3>",1],
blockquote:["<blockquote>",1],
code:["<code>",1],
newline:["<br>",2],
horizontal:["<hr>",2]
}
];
The above code works fine and lets me do look-ups easily and efficiently. However, there are many cases where two objects in the store are completely identical except for their domain attribute.
For example, I want to add objects for all of the Stack Exchange sites to the store, and all of those objects would be equal to the one for StackOverflow.
So, rather than create many separate objects, I want to do something like this:
var tags = [
//codes: 0 - markdown wrap tag
// 1 - HTML wrap tag
// 2 - single tag
{ domain: ["youtube"],
bold:["*",0],
strikethrough:["-",0],
italic:["_",0]
},
{ domain: ["stackoverflow","stackexchange",...],
bold:["<strong>",1],
italic:["<em>",1],
strikethrough:["<del>",1],
superscript:["<sup>",1],
subscript:["<sub>",1],
heading1:["<h1>",1],
heading2:["<h2>",1],
heading3:["<h3>",1],
blockquote:["<blockquote>",1],
code:["<code>",1],
newline:["<br>",2],
horizontal:["<hr>",2]
}
];
Would it be possible to use a KeyGen rather than a keyPath and set up some kind of index that took a value and searched for it in the arrays pointed to by the domain key?
Or would I have to use a cursor each time I want to do a look up?
Some potentially helpful references are:
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Basic_Concepts_Behind_IndexedDB
http://www.w3.org/TR/IndexedDB/#key-path-construct
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB
The solution is to use an index with the multiEntry key property set to true
see this link (thanks #kyaw Tun)
Each index also has a multiEntry flag. This flag affects how the index behaves when the result of evaluating the index's key path yields an Array. If the multiEntry flag is false, then a single record whose key is an Array is added to the index. If the multiEntry flag is true, then the one record is added to the index for each item in the Array. The key for each record is the value of respective item in the Array.
Armed with this index, a specific keyPath is no longer necessary, so you can just use a keyGen for simplicity.
So, to create the database:
request.onupgradeneeded = function(event)
{
var db = event.target.result;
var objectStore = db.createObjectStore("domains", {autoIncrement: true });
objectStore.createIndex("domain", "domain", { unique: true, multiEntry: true });
for(var i in tags)
{
objectStore.add(tags[i]);
console.log("added " + tags[i]["domain"] + " to the DB");
}
};
and an example of using a domain to query for an object:
var objectStore = db.transaction("domains").objectStore("domains");
var query = objectStore.index("domain").get(queryURL);
query.onsuccess = function(event){...};
I have store with multiEntry index on tags.
{ tags: [ 'tag1', 'tag2', 'tag3' ] }
And i have query that also list of tags.
[ 'tag2', 'tag1', 'tag4' ]
I need to get all records which contain one of tag in query (Similar to SQL OR statement).
Currently I cannot find any other solution except iterate over tags in query and search by the each tag in the store.
Is there any better solution?
Thank you.
You cannot retrieve all results with one query except with iteration. You can optimize the search result by opening a index from the lowest value to the highest:
IDBKeyRange.bound ('tag1', 'tag4');
Other Indexed-Db feature you can use is to open multiple queries and combine the result when the queries complete. This way would be much faster than the iteration.
IndexedDB has only range query as Deni Mf answered.
OR query is simply union of multiple queries. That may be OK.
If you want efficient query, you have to iterate the cursor and seek the cursor position as necessary. Using my library, it will be
tags = ['tag2', 'tag1', 'tag4'];
tags.sort();
iter = new ydn.db.KeyIterator('store name', 'tags', IDBKeyRange.bound(tags[0], tags[tags.length-1]);
keys = [];
i = 0;
var req = db.open(iter, function(cursor) {
if (tags.indexOf(cursor.indexKey()) >= 0) {
// we got the result
if (keys.indexOf(cursor.key()) == -1) { // remove duplicate
keys.push(cursor.key());
}
} else {
return tags[++i]; // jump to next index position.
}
);
req.done(function() {
db.list('store name', keys).done(function(results) {
console.log(results);
}
});
Notice that the algorithm has no false positive retrieval. Key only query is performed first, so that we don't waste on de-serilization. The results is retrieved only we get all the primary keys after removing deplicates.