Develop Reference

get a range of numbers from sql column of type varchar

I have a postgres server running with one column (say marks) of type VARCHAR(255), but is supposed to have numbers, like if i do a select *.. query , i will get ['100','50','21','14'...] etc.
i would like to run a range query on it, like user passes [10,30] and gets ['21','14'] as result. I think this would require casting at the time of running the BETWEEN query, but i cannot get it to work properly.
I am using sequalize.js which is generating the following query:
SELECT "id"
FROM "token_attributes" AS "token_attributes"
WHERE "token_attributes"."attributesDirectoryId" = 3
AND CAST('token_attributes.attributeValue' AS INTEGER) BETWEEN 10 AND 30;
on server also this query seems to fail. the sequalize query that is being created is :
{
where: {
attributesDirectoryId: 3,
attributeValue: Where { attribute: [Cast], comparator: '=', logic: [Object] }
},
attributes: [ 'id' ]
}
i have used the following code to create the where condition (cast and where were imported from sequelize):
let whereFilter ={}
let value = where(cast(`${tableName}.attributeValue`, 'integer'), {[Op.between]: rangeAsInt})
whereFilter['attributeValue'] = value
so this is basically calling table.findAll({where:whereFilter}) I am not sure how to either make sequelize create a correct sql api or what the actual correct SQL api would be. can anyone help?

found the issue, i missed the sequilize.col function :
let whereFilter ={}
let value = where(cast(col(`${tableName}.attributeValue`), 'integer'), {[Op.between]: rangeAsInt})
whereFilter['attributeValue'] = value
and the query would be :
SELECT "id"
FROM "token_attributes" AS "token_attributes"
WHERE "token_attributes"."attributesDirectoryId" = 3
AND CAST("token_attributes"."attributeValue" AS INTEGER) BETWEEN 10 AND 30;

Javascript ForEach on Array of Arrays

I am looping through a collection of blog posts to firstly push the username and ID of the blog author to a new array of arrays, and then secondly, count the number of blogs from each author. The code below achieves this; however, in the new array, the username and author ID are no longer separate items in the array, but seem to be concatenated into a single string. I need to retain them as separate items as I need to use both separately; how can I amend the result to achieve this?
var countAuthors = [];
blogAuthors = await Blog.find().populate('authors');
blogAuthors.forEach(function(blogAuthor){
countAuthors.push([blogAuthor.author.username, blogAuthor.author.id]);
})
console.log(countAuthors);
// Outputs as separate array items, as expected:
// [ 'author1', 5d7eed028c298b424b3fb5f1 ],
// [ 'author2', 5dd8aa254d74b30017dbfdd3 ],
var result = {};
countAuthors.forEach(function(x) {
result[x] = (result[x] || 0) + 1;
});
console.log(result);
// Username and author ID become a single string and cannot be accessed as separate array items
// 'author1,5d7eed028c298b424b3fb5f1': 15,
// 'author2,5dd8aa254d74b30017dbfdd3': 2,
Update:
Maybe I can explain a bit further WHY on what to do this. What I am aiming for is a table which displays the blog author's name alongside the number of blogs they have written. However, I also want the author name to link to their profile page, which requires the blogAuthor.author.id to do so. Hence, I need to still be able to access the author username and ID separately after executing the count. Thanks

You could use String.split().
For example:
let result = 'author1,5d7eed028c298b424b3fb5f1'.split(',')
would set result to:
['author1' , '5d7eed028c298b424b3fb5f1']
You can then access them individually like:
result[1] //'5d7eed028c298b424b3fb5f1'

Your issue is that you weren't splitting the x up in the foreach callback, and so the whole array was being converted to a string and being used as the key when inserting into the results object.
You can use array destructuring to split the author name and blog id, and use them to optionally adding a new entry to the result object, and then update that result.
countAuthors = [
['author1', 'bookId1'],
['author2', 'bookId2'],
['author1', 'bookId3'],
['author1', 'bookId4'],
['author2', 'bookId5']
]
var result = {};
countAuthors.forEach(([author, id]) => {
if (result[author] === undefined) {
result[author] = {count: 0, blogIds: []};
}
result[author].count += 1;
result[author].blogIds.push(id);
});
console.log(result);

JS and ExpressionEngine - Remove KV pairs by duplicate values only?

We're building a site with ExpressionEngine. We are running a SQL query to gather up all member IDs for a specific member group. After that, we are using EE tags to get data from a custom member field for each member ID.
The ID and field data need to stay paired, as we will be populating a drop-down so that the ID is the value and the field data is the text, so we are currently putting them into a JS array as key/value pairs. The call is as follows:
var array= [
{exp:query sql="SELECT * FROM exp_members WHERE group_id = 5"}
{exp:member:custom_profile_data
member_id="{member_id}"}
{if company != ''}
{{member_id}:"{company}"},
{/if}
{/exp:member:custom_profile_data}
{/exp:query}
};
This gives us the output:
var array = [
{1:"name01"},
{2:"name02"},
{3:"name01"},
{4:"name03"}
];
Now, our problem. We need to remove objects based on duplicate field data (values) only, so the above array would look like this:
var array = [
{1:"name01"},
{2:"name02"},
{4:"name03"}
];
None of these IDs (keys) will ever be the same, but the field data (values) can be. So we want to keep the first KV pair that comes through with a unique value, but remove any subsequent dupes of that value - despite the fact that they will not be true "duplicate values" due to a different ID (key).
Keeping in mind that the KV pairs are all dynamic, is there any possible way to do this via JS so we can create a new array for the cleaned data to pass to the drop-down?

You could handle the duplications by modifying your MySQL query. (In my example, my custom field ID was 1.)
var myArray = [];
{exp:query sql="SELECT MIN(m.member_id) AS co_member_id, d.m_field_id_1 AS company FROM exp_members m INNER JOIN exp_member_data d ON m.member_id = d.member_id WHERE d.m_field_id_1 != '' AND m.group_id > 0 GROUP BY d.m_field_id_1;"}
myArray.push({{co_member_id}: "{company}"});
{/exp:query}
This query would use the first (in the ordinal sense) member_id found; you could also change the MIN to MAX and get the last.
This will give you a clean output in your source, without the need for any additional JS processing. I'd also recommend changing the names of the variables you're outputting as to not conflict in EE's parsing.

I would do it like...
function removeDups(arry){
var tmp = {}, retainIdx=[], newArry=[];
arry.forEach(function(obj, idx){
var val = obj[Object.keys(obj)[0]];
if(val && !tmp[val]){
retainIdx.push(idx);
tmp[val] = true;
}
});
retainIdx.forEach(function(i){
newArry.push(arry[i]);
});
return newArry;
};

IndexedDB: Can you use an array element as a key or an index?

Consider the following object store, with the domain key set as the keyPath
var tags = [
//codes: 0 - markdown wrap tag
// 1 - HTML wrap tag
// 2 - single tag
{ domain: "youtube",
bold:["*",0],
strikethrough:["-",0],
italic:["_",0]
},
{ domain: "stackoverflow",
bold:["<strong>",1],
italic:["<em>",1],
strikethrough:["<del>",1],
superscript:["<sup>",1],
subscript:["<sub>",1],
heading1:["<h1>",1],
heading2:["<h2>",1],
heading3:["<h3>",1],
blockquote:["<blockquote>",1],
code:["<code>",1],
newline:["<br>",2],
horizontal:["<hr>",2]
}
];
The above code works fine and lets me do look-ups easily and efficiently. However, there are many cases where two objects in the store are completely identical except for their domain attribute.
For example, I want to add objects for all of the Stack Exchange sites to the store, and all of those objects would be equal to the one for StackOverflow.
So, rather than create many separate objects, I want to do something like this:
var tags = [
//codes: 0 - markdown wrap tag
// 1 - HTML wrap tag
// 2 - single tag
{ domain: ["youtube"],
bold:["*",0],
strikethrough:["-",0],
italic:["_",0]
},
{ domain: ["stackoverflow","stackexchange",...],
bold:["<strong>",1],
italic:["<em>",1],
strikethrough:["<del>",1],
superscript:["<sup>",1],
subscript:["<sub>",1],
heading1:["<h1>",1],
heading2:["<h2>",1],
heading3:["<h3>",1],
blockquote:["<blockquote>",1],
code:["<code>",1],
newline:["<br>",2],
horizontal:["<hr>",2]
}
];
Would it be possible to use a KeyGen rather than a keyPath and set up some kind of index that took a value and searched for it in the arrays pointed to by the domain key?
Or would I have to use a cursor each time I want to do a look up?
Some potentially helpful references are:
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Basic_Concepts_Behind_IndexedDB
http://www.w3.org/TR/IndexedDB/#key-path-construct
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB

The solution is to use an index with the multiEntry key property set to true
see this link (thanks #kyaw Tun)
Each index also has a multiEntry flag. This flag affects how the index behaves when the result of evaluating the index's key path yields an Array. If the multiEntry flag is false, then a single record whose key is an Array is added to the index. If the multiEntry flag is true, then the one record is added to the index for each item in the Array. The key for each record is the value of respective item in the Array.
Armed with this index, a specific keyPath is no longer necessary, so you can just use a keyGen for simplicity.
So, to create the database:
request.onupgradeneeded = function(event)
{
var db = event.target.result;
var objectStore = db.createObjectStore("domains", {autoIncrement: true });
objectStore.createIndex("domain", "domain", { unique: true, multiEntry: true });
for(var i in tags)
{
objectStore.add(tags[i]);
console.log("added " + tags[i]["domain"] + " to the DB");
}
};
and an example of using a domain to query for an object:
var objectStore = db.transaction("domains").objectStore("domains");
var query = objectStore.index("domain").get(queryURL);
query.onsuccess = function(event){...};

retrieve unique values for key from parse.com when class has > 1000 objects

In parse I have a class named "TestScore". Each object has a key named "quizName".
I need to get an array of unique "quizName" values. I wrote the below which queries the "TestClass" and loops through the results looking for unique "quizName" values.
At first seemed to do the job. But then I realized that the maximum number of returned objects is 1000. Soon there will be more than 1000 objects stored which means that this method will not guarantee that I end up will all values.
function loadTests(){
//create an array to hold each unique test name as we find them
var uniqueEntries = [];
//query parse to return TestScore objects
var TestScore = Parse.Object.extend("TestScore");
var query = new Parse.Query(TestScore);
query.limit(1000) //added this after realizing that the default query limit is only 100
query.find({
success: function(testScore) {
$(testScore).each(function(index, score) {
//here I loop though all of the returned objects looking at the "quizName" for each
if($.inArray(score.get("quizName"), uniqueEntries) === -1) {
//if the quiz name is not already in the "uniqueEntries" array, I add it to the array
uniqueEntries.push(score.get("quizName"));
}
});
//do stuff with quiznames here...., add them as options in select boxes mostly
}
});
}
I looked at {Parse.Query} notContainedIn(key, values) which looks promising but cant figure out if I can add values to the array as I find them. It seems like I would have to have an array to start with (defeating the whole point.)
This part of the docs "{Parse.Query} Returns the query, so you can chain this call." makes me think I might be able to chain queries together to get what I need, but that doesn't seem very efficient.
How can I retrieve unique values for key "quizName" when my class has > 1000 objects?

I'm sure you're long past this by now, but only way I know of to do it is to use one query after another by using a .skip(#) value for each query. So get 1000, then query again with .skip(1000), concatenate the items from the first list and second, then query again with .skip(2000), etc...
Be aware that I think there's a limit on skip values of 10,000. Don't take my word on that, just pointing you to something that I think is right that you should confirm if you think it applies to your situation.

I eventually found a tutorial online that I was able to modify and came up with the below. This effectively sets the return limit to 10,000 instead of 1,000 and allows setting several different parameters for the query.
My changes could surely be written better, maybe as an options object or similar but it works for my needs.
You can see a working demo here
function getStuff(){
// here we will setup and call our helper functions with callbacks to handle the results
var scheme =['SOTest',true]; // return all objects with value `true` in the `SOTest` column
// var scheme =['descending','createdAt']; // return all objects with sort order applied
// var scheme =''; // or just return all objects
// see `findChunk()` below for more info
var Remark = Parse.Object.extend("Remark");
schemePromise(Remark, scheme).done(function (all) {
console.log('Found ' + all.length+' Remarks');
$.each( all, function(i, obj){
$('#test').append(obj.get('Remark') +'<br>');
});
})
.fail(function (error) {
console.log("error: " + JSON.stringify(error));
});
}
getStuff(); // call our function
// helper functions used to get around parse's 1000 query limit
// raises the limit to 10,000 by using promises
function findChunk(model, scheme, allData) {
// if `scheme` was an empty string, convert to an array
// this is the default and returns all objects in the called class
if(scheme==''){ ['scheme',''] };
// will return a promise
var limit = 1000;
var skip = allData.length;
var findPromise = $.Deferred();
var query = new Parse.Query(model);
// to get all objects from the queried Class then sort them by some column
// pass `scheme` as an array like [ sort method, column to sort ]
if (scheme[0]=='descending') query.descending(scheme[1]);
else if (scheme[0]=='ascending') query.ascending(scheme[1]);
// to limt results to objects that have a certain value in a specific column
// pass `scheme` as an array like [ column name, value ]
else query.equalTo(scheme[0], scheme[1]);
// more options can easily be built in here using `scheme`
query
.limit(limit)
.skip(skip)
.find()
.then(function (results) {
findPromise.resolve(allData.concat(results), !results.length);
}, function (results) {
findPromise.reject(error);
});
return findPromise.promise();
}
function schemePromise(model, scheme, allResults, allPromise) {
// find a scheme at a time
var promise = allPromise || $.Deferred();
findChunk(model, scheme, allResults || [])
.done(function (results, allOver) {
if (allOver) {
// we are done
promise.resolve(results);
} else {
// may be more
schemePromise(model, scheme, results, promise);
}
})
.fail(function (error) {
promise.reject(error);
});
return promise.promise();
}