Develop Reference

Substring in Key parameter

I am working on my CouchDB project, where I want to create a specific view for my database.
It has proven that one of my key parameters has an ID as part of its name, but the other part is unique. ex "unique_ID_unique":"Value".
So brute force solutions like changing the name and/or how it is saved is not preferred.
To make it clear the ID is different for every entry (date).
I tried to modify it by using regex rules but it returns NULL for the key part.
emit(doc[/_unique$/], doc['something.else']);
Does someone have any idea why it is like that?
P.S: I already had a question like this yesterday, but due to the insufficient information that I gave, it led to wrong answers and I had to delete it.

Let's say you have a function that extract the unique key from a particular one:
var key = "blabla_120391029301923_blabla";
var keySplitted = key.split("_"); //Value: ["blabla","120391029301923","blabla"]
var uniqueKey = keySplitted[2]; //Value: blabla
From this, you can create a view what will map each documents and index them with your key.
function(doc){
var keySplitted = doc._id.split("_");
if(keySplitted.length == 3){
emit(keySplitted[2]);
}
}
The previous map ids from this:
evening_01012018_food
morning_02012018_musc
evening_01022018_food
To this:
food
musc
food
UPDATE
From offline discussion, I was able to understand that the objects to index had this content:
{
"_id": "doc_1",
"_rev": "1-89d017d9e5c82ee56d9beec2756fec99",
"type": "googrtt",
"ssrc_3685071425_send.bytesSent": "33621"
}
So with this document, the properties had to be split.
The final view content looks like this:
function (doc) {
for(key in doc){
var splitKey= key.split('_');
if(splitKey.length === 3){
emit(splitKey[2],doc[key]);
}
}
}

Why am I unable to sort list by difference of 2 fields?

So I have a list of items.
Each item has a like or dislike button. I wish to sort it by total score, which is = # likes less # dislikes.
I am trying to understand why:
The below handlebar + sort is not working on client side and how to make it work?
If a server side solution would be better and why? (server side takes up unnecessary disk space from what I have learnt in other posts and premature optimization - Yes, Im still in the early stages)
This is how I store it in the list in items.js within collections (ground 0 form).
var item = _.extend(itemAttributes, {
lovers: [],
likes: 0,
haters: [],
dislikes: 0
//popularity: likes - dislikes I tried inserting in collection but doesnt work too
});
So Im sorting it via a handlebar helper + extension of the main list controller
This is the handlebar segment
Template.registerHelper('popularity', function(likes, dislikes) {
var popularity = likes - dislikes;
return popularity;
This is the sorter in router.js
BestItemController = ItemListController.extend({
sort: {popularity: -1},
nextPath: function() {
return Router.routes.BestItem.path({itemsLimit: this.itemsLimit() + this.increment})
}
});
So the handlebar popularity calculations actually does work, the popularity score appears on addition of {{ popularity 123numbersxx }}
However the sorting doesnt work, probably because the sorting does not sort "on the surface" calculations, but rather on the actual item and its fields?
I tried to insert an additional schema field (see above commented line). However that causes errors which states likes are not defined.
Would anyone help guide me a little on this?
Also if you think my method of doing things is bad, appreciate any other ways? For example if sorting on the individual template helper.js files rather than on the main router.js files.
Many thanks!

You can achieve that client-side. Here is how I would proceed:
You probably display your item to be sorted using an {{#each item}} iteration. I would replace item in the #each by a custom helper and create a function, using a cursor or an array as an argument, that will sort your items using the current sorting settings.
Your helper could look like that:
Template.items.helpers({
sortedItems: function(){
return sortMyItems(items.find()) //add .fetch() if you need an array,
//or directly your array if you already have it in a variable.
}
});
And at the beginning of your file, you add the sortMyItems function where you return the sorted list of items.
sortMyItems = function(cursor) {
if(!cursor) {
return [];
}
var sortBy = Session.get("sortBy");// in your case, it would be set to "popularity"
var sortAscending = Session.get("sortAscending ");
if(typeof(sortAscending) == "undefined") sortAscending = true;
var sorted = [];
var raw = cursor.fetch();
// sort
if(sortBy) {
sorted= _.sortBy(raw, sortBy);
// descending?
if(!sortAscending) {
sorted= sorted.reverse();
}
}
return sorted;
}
here I use Session vars, but I advise you to rather use reactive variables or reactive dictionary, since this is a feature related to the current view only.

parse.com search for partial string in array

I'm trying to search a Parse.com field which is an array for a partial string.
When the field is in String format I can do the following:
// Update the filtered array based on the search text and scope.
// Remove all objects from the filtered search array
[self.searchResults removeAllObjects];
// Filter the array using NSPredicate
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"SELF.busnumber contains[c] %#", searchText];
self.searchResults = [NSMutableArray arrayWithArray:[self.objects filteredArrayUsingPredicate:predicate]];
This works, however the new field I want to search in is an Array.
It works when I change the it to the following:
PFQuery * query = [PFQuery queryWithClassName:#"Bus"];
[query whereKey:#"route" equalTo:[NSString stringWithFormat:#"%#", searchText]];
[query findObjectsInBackgroundWithBlock:^(NSArray *objects, NSError *error) {
NSLog(#"Objects: %#", objects);
if (error)
{
NSLog(#"ERROR: %#", error.localizedDescription);
}
else
{
[self.searchResults removeAllObjects];
[self.searchResults addObjectsFromArray:objects];
[self.searchDisplayController.searchResultsTableView reloadData];
}}];
However I need the exact String for this.
I want to be able to search for parts of a string though, but when I change it to:
[query whereKey:#"route" containsString:[NSString stringWithFormat:#"%#", searchText]];
I get:
[Error]: $regex only works on string fields (Code: 102, Version: 1.7.4)
Any ideas? Thanks :)

What you've attempted is rational, but the string qualifiers on PFQuery work only on strings.
I've seen this theme frequently on SO: PFQuery provides only basic comparisons for simple attributes. To do anything more, one must query for a superset and do app level computation to reduce the superset to the desired set. Doing so is expensive for two reasons: app-level compute speed/space, and network transmission of the superset.
The first expense is mitigated and the second expense is eliminated by using a cloud function to do the app level reduction of the superset. Unless you need the superset records on the client anyway, consider moving this query to the cloud.
Specific to this question, here's what I think the cloud function would resemble:
// very handy to have underscore available
var _ = require('underscore');
// return Bus objects whose route array contains strings which contain
// the passed routeSubstring (request.params.routeSubstring)
Parse.Cloud.define("busWithRouteContaining", function(request, response) {
// for now, don't deal with counts > 1k
// there's a simple adjustment (using step recursively) to get > 1k results
var query = new Parse.Query("Bus");
query.find().then(function(buses) {
var matching = _.select(buses, function(bus) {
var route = bus.get("route");
var routeSubstring = request.params.routeSubstring;
return _.contains(route, function(routeString) {
return routeString.includes(routeSubstring);
});
});
response.success(matching);
}, function(error) {
response.error(error);
});
});
If you do decide to perform the reduction on the client and need help with the code, I can edit that in. It will be a pretty simple switch to predicateWithBlock: with a block that iterates the array attribute and checks rangeOfString: on each.

How do I match this text faster?

I'm building an autosuggest for names. When the user types in the textbox, it hits the server and runs this:
var names = [ list of 1000 names ]; //I have a list of 1000 names, this is static.
var query = 'alex';
var matched_names = [];
//This is when it gets slow....
names.forEach(function(name){
if(name.indexOf(query) >= 0){
matched_names.push(name);
}
});
return matched_names;
How can I make this faster? I'm using Node.js

If the names are static then move this code to the client and run it there. The only reason to run code like this on the server is if the data source is dynamic in some way.
Doing this logic client-side will dramatically improve performance.

You should probably use filter instead, for one thing, because it's native:
var names = [ /* list of 1000 names */ ];
var query = 'alex';
var matched_names = names.filter(function(name) {
return name.indexOf(query) > -1;
});
return matched_names;

If you store the names in sorted order, then you can use binary search to find the region of names within the sorted order that start with the fragment of name that the user has typed so far, instead of checking all the names one by one.
On a system with a rather odd programming language, where I wanted to find all matches containing what the user had typed so far in any position, I got a satisfactory result for not much implementation effort by reviving http://en.wikipedia.org/wiki/Key_Word_in_Context. (Once at university I searched through a physical KWIC index, printed out from an IBM lineprinter, and then bound as a document for just this purpose.

I would suggest you to do this stuff on the client-side and prefer (for now) a while loop instead of a filter/forEach approach:
var names = [ /* list of 1000 names */ ]
, query = 'alex'
, i = names.length
, matched_names = [];
while(i--){
if(names[i].indexOf(query) > -1){
matched_names.push(names[i]);
}
}
return matched_names;
This will be much faster (even if filter/forEach are natively supported). See this benchmark: http://jsperf.com/function-loops/4

optimize search through large js string array?

if I have a large javascript string array that has over 10,000 elements,
how do I quickly search through it?
Right now I have a javascript string array that stores the description of a job,
and I"m allowing the user to dynamic filter the returned list as they type into an input box.
So say I have an string array like so:
var descArr = {"flipping burgers", "pumping gas", "delivering mail"};
and the user wants to search for: "p"
How would I be able to search a string array that has 10000+ descriptions in it quickly?
Obviously I can't sort the description array since they're descriptions, so binary search is out. And since the user can search by "p" or "pi" or any combination of letters, this partial search means that I can't use associative arrays (i.e. searchDescArray["pumping gas"] )
to speed up the search.
Any ideas anyone?

As regular expression engines in actual browsers are going nuts in terms of speed, how about doing it that way? Instead of an array pass a gigantic string and separate the words with an identifer.
Example:
String "flipping burgers""pumping gas""delivering mail"
Regex: "([^"]*ping[^"]*)"
With the switch /g for global you get all the matches. Make sure the user does not search for your string separator.
You can even add an id into the string with something like:
String "11 flipping burgers""12 pumping gas""13 delivering mail"
Regex: "(\d+) ([^"]*ping[^"]*)"
Example: http://jsfiddle.net/RnabN/4/ (30000 strings, limit results to 100)

There's no way to speed up an initial array lookup without making some changes. You can speed up consequtive lookups by caching results and mapping them to patterns dynamically.
1.) Adjust your data format. This makes initial lookups somewhat speedier. Basically, you precache.
var data = {
a : ['Ant farm', 'Ant massage parlor'],
b : ['Bat farm', 'Bat massage parlor']
// etc
}
2.) Setup cache mechanics.
var searchFor = function(str, list, caseSensitive, reduce){
str = str.replace(/(?:^\s*|\s*$)/g, ''); // trim whitespace
var found = [];
var reg = new RegExp('^\\s?'+str, 'g' + caseSensitive ? '':'i');
var i = list.length;
while(i--){
if(reg.test(list[i])) found.push(list[i]);
reduce && list.splice(i, 1);
}
}
var lookUp = function(str, caseSensitive){
str = str.replace(/(?:^\s*|\s*$)/g, ''); // trim whitespace
if(data[str]) return cache[str];
var firstChar = caseSensitive ? str[0] : str[0].toLowerCase();
var list = data[firstChar];
if(!list) return (data[str] = []);
// we cache on data since it's already a caching object.
return (data[str] = searchFor(str, list, caseSensitive));
}
3.) Use the following script to create a precache object. I suggest you run this once and use JSON.stringify to create a static cache object. (or do this on the backend)
// we need lookUp function from above, this might take a while
var preCache = function(arr){
var chars = "abcdefghijklmnopqrstuvwxyz".split('');
var cache = {};
var i = chars.length;
while(i--){
// reduce is true, so we're destroying the original list here.
cache[chars[i]] = searchFor(chars[i], arr, false, true);
}
return cache;
}
Probably a bit more code then you expected, but optimalisation and performance doesn't come for free.

This may not be an answer for you, as I'm making some assumptions about your setup, but if you have server side code and a database, you'd be far better off making an AJAX call back to get the cut down list of results, and using a database to do the filtering (as they're very good at this sort of thing).
As well as the database benefit, you'd also benefit from not outputting this much data (10000 variables) to a web based front end - if you only return those you require, then you'll save a fair bit of bandwidth.

I can't reproduce the problem, I created a naive implementation, and most browsers do the search across 10000 15 char strings in a single digit number of milliseconds. I can't test in IE6, but I wouldn't believe it to more than 100 times slower than the fastest browsers, which would still be virtually instant.
Try it yourself: http://ebusiness.hopto.org/test/stacktest8.htm (Note that the creation time is not relevant to the issue, that is just there to get some data to work on.)
One thing you could do wrong is trying to render all results, that would be quite a huge job when the user has only entered a single letter, or a common letter combination.

I suggest trying a ready made JS function, for example the autocomplete from jQuery. It's fast and it has many options to configure.
Check out the jQuery autocomplete demo

Using a Set for large datasets (1M+) is around 3500 times faster than Array .includes()
You must use a Set if you want speed.
I just wrote a node script that needs to look up a string in a 1.3M array.
Using Array's .includes for 10K lookups:
39.27 seconds
Using Set .has for 10K lookups:
0.01084 seconds
Use a Set.