Develop Reference

Discord.js searching on key terms

I'm trying to create a Discord bot for trading within certain games. So far I have most of the basic commands working--!create creates a trade listing in an SQL database, !find finds one--but it only finds it on the exact same word. What I'm trying to do is make the search less specific so the terms don't have to be exactly equal to show results.
My current code is pretty convoluted and, needless to say, very broken:
var searchTerms = args[1].split(" ");
var output = {};
for (var id in userData) {
for (var offer in userData[id].offers) {
var score = 0;
for (var key in searchTerms) {
if (offer.includes(key)) {
score ++;
}
}
if (score >= searchTerms.length / 2) {
output[id] = userData[id].offers[offer] + " - " + ((score / searchTerms.length) * 100) + "%";
}
}
}
if (output == {}) {
bot.sendMessage({
to: channelID,
message: 'No matching offers found.'
});
} else {
msg = ""
for (id in output) {
msg += '<#' + id + '> - ' + output[id] + " "
}
bot.sendMessage({
to: channelID,
message: Object.keys(output).length + ' offers found: ' + msg
});
}
I'm new to Javascript so I'm not really sure how to get this working. Any tips are appreciated, thanks!

It looks like what you're trying to implement is an mechanism called Fuzzy Search, which user can find similar results using typo or approximate strings.
( Reference: https://en.wikipedia.org/wiki/Approximate_string_matching )
It not really an easy feature for a programming beginner to implement on your own, either the database have to support some kind of fuzzy query, or you'll have to get all the data from database first, and use a JavaScript fuzzy search library to accomplish that.
If you still want to do it, I recommend using Fuse.js, which is able to accomplish fuzzy search in a few lines
//list to be searched
var books = [{
'ISBN': 'A',
'title': "Old Man's War",
'author': 'John Scalzi'
}, {
'ISBN': 'B',
'title': 'The Lock Artist',
'author': 'Steve Hamilton'
}]
// init the search
var options = {
keys: ['title', 'author'],
id: 'ISBN'
}
var fuse = new Fuse(books, options)
fuse.search('old')
// result
[
"A"
]
Fuzzy search is a complex computer science problem, if you want to know more about it and how Fuse.js is implemented, here are a few useful links
An intro to fuzzy string matching
source code of Fuse.js
bitap algorithm (used by fuse.js)

Passing in data from Mongodb to HTML table using javascript using Node.js framework

I'm quite new at using node.js. Right now I'm trying to pull data from MongoDB and display it in a table using Javascript + HTML. However, my table is populating with undefined in all the fields. I think something is definitely wrong with how I'm reading data through to the Javascript function b/c I am able to render the full results from the people.js file straight to the webpage. Thank you in advance!! Below is my code:
Code for my people.js file:
exports.getPeople = (req, res) => {
People.find((err, docs) => {
if (err) { return next(err); }
if (docs != null){
console.log(docs.length)
docs.forEach(function(docs, index) {
console.log(index + " key: " + docs.name)
});
res.render('people', { people: docs });
}
else{
res.render('people', { people: docs() });
}
});
};
My Javascript + HTML that's populating my webpage.
script(type='text/javascript', src='http://code.jquery.com/jquery-1.9.1.js', charset='UTF-8')
script.
$(document).ready(function(){
var obj= '$(people)'
var tbl = "<table>"
var content="";
for(i=0; i<obj.length;i++){
content +=
'<tr>+<td>' +obj[i]["name"]+
'</td><td>'+obj[i]["type"]+
'</td><td>'+obj[i]["min_hours"]+
'</td><td>'+obj[i]["max_hours"]+
'</td><td>'+obj[i]["email"]+
'</td><td>'+obj[i]["phone_number"]+
'</td><td>'+ '<input type="button" value = "Update" onClick="Javacsript:deleteRow(this)">' +
'</td><td>'+'<input type="button" value = "Delete" onClick="Javacsript:deleteRow(this)">';
'</td></tr>';
}
content += "</table>"
$('#myTableData').append(content);
});

As you mentioned, you can render the array results from the people.js file directly into the webpage. So, you don't have to read the data through a JavaScript function using jQuery. The template engine language is built on top of JavaScript and it supports plenty of methods and features to do what you're trying to achieve here. So, for example, you may use an iteration method like each..in to build your table (see docs - Iteration):
// ...
body
table(id="myTableData")
// for each person in the people array (from people.js) ...
each person in people
// build a new table row
tr
// insert table data
td #{person.name}
td #{person.type}
td #{person.min_hours}
td #{person.max_hours}
td #{person.email}
td #{person.phone_number}
// add the update and delete buttons
td
input(type="button" value = "Update" onclick=" ... ")
input(type="button" value = "Delete" onclick=" ... ")
// move to next person in the people array ...
The Problem
var obj = '$(people)' does not work as you may expect. You want obj to hold the people array from the people.js file so that you can loop over each object in the array, but this is not what's happening. obj is actually a string value of length 9, so the for loop evaluates 9 string values (not objects). This is why all of your fields are undefined.
To see what I mean, run this code snippet:
var obj = '$(people)';
for (var i = 0; i < obj.length; i++){
console.log(obj[i]);
console.log(obj[i]["name"]);
}
The reason $(people) does not evaluate to an object is mainly because the parent element, script. causes everything below it to evaluate to plain text. The . after the tag causes the template engine to render plain text (see docs: Block in a Tag).
If you wanted to assign people to obj in your inline script you may try it this way:
script
| var obj = #{people};
But this will cause an Unexpected identifier JavaScript error because of the _id field on each item in people. By default _id is an ObjectID hex value from MongoDb so you would have to either remove the _id field from the docs or add quotes to each doc._id so it evaluates to a string. This would all have to be done in person.js before you return the data.
To see what I mean about the Unexpected identifier error, run this code snippet:
// works
var obj = { _id: '583ab33cdaf857b543c76afe',
name: 'john'};
// Error: Unexpected identifier
var obj = { _id: 583ab33cdaf857b543c76afe,
name: 'john'};

Loop Through JSON, Insert Key/Value Between Objects?

UPDATE - Thanks for all the great answers and incredibly fast response. I've learned a great deal from the suggested solutions. I ultimately chose the answer I did because the outcome was exactly as I asked, and I was able to get it working in my application with minimal effort - including the search function. This site is an invaluable resource for developers.
Probably a simple task, but I can't seem to get this working nor find anything on Google. I am a Javascript novice and complex JSON confuses the hell out of me. What I am trying to do is make a PhoneGap Application (Phone Directory) for our company. I'll try to explain my reasoning and illustrate my attempts below.
I have JSON data of all of our employees in the following format:
[
{
"id":"1",
"firstname":"John",
"lastname":"Apple",
"jobtitle":"Engineer"
},
{
"id":"2",
"firstname":"Mark",
"lastname":"Banana",
"jobtitle":"Artist"
},
... and so on
]
The mobile framework (Framework 7) that I am using offers a "Virtual List" solution which I need to take advantage of as our directory is fairly large. The virtual list requires you to know the exact height of each list item, however, you can use a function to set a dynamic height.
What I am trying to do is create "headers" for the alphabetical listing based on their last name. The JSON data would have to be restructured as such:
[
{
"title":"A"
},
{
"id":"1",
"firstname":"John",
"lastname":"Apple",
"jobtitle":"Engineer"
},
{
"title":"B"
},
{
"id":"2",
"firstname":"Mark",
"lastname":"Banana",
"jobtitle":"Artist"
},
... and so on
]
I've been able to add key/value pairs to existing objects in the data using a for loop:
var letter, newLetter;
for(var i = 0; i < data.length; i++) {
newLetter = data[i].lastname.charAt(0);
if(letter != newLetter) {
letter = newLetter
data[i].title = letter;
}
}
This solution changes the JSON, thus outputting a title bar that is connected to the list item (the virtual list only accepts ONE <li></li> so the header bar is a div inside that bar):
{
"id":"1",
"firstname":"John",
"lastname":"Apple",
"jobtitle":"Engineer",
"title":"A"
},
{
"id":"1",
"firstname":"Mike",
"lastname":"Apricot",
"jobtitle":"Engineer",
"title":""
}
This solution worked until I tried implementing a search function to the listing. When I search, it works as expected but looks broken as the header titles ("A", "B", etc...) are connected to the list items that start the particular alphabetical section. For this reason, I need to be able to separate the titles from the existing elements and use them for the dynamic height / exclude from search results.
The question: How can I do a for loop that inserts [prepends] a NEW object (title:letter) at the start of a new letter grouping? If there is a better way, please enlighten me. As I mentioned, I am a JS novice and I'd love to become more efficient programming web applications.

var items = [
{ "lastname":"Apple" },
{ "lastname":"Banana" },
{ "lastname":"Box" },
{ "lastname":"Bump" },
{ "lastname":"Can" },
{ "lastname":"Switch" }
];
var lastC = null; //holds current title
var updated = []; //where the updated array will live
for( var i=0;i<items.length;i++) {
var val = items[i]; //get current item
var firstLetter = val.lastname.substr(0,1); //grab first letter
if (firstLetter!==lastC) { //if current title does not match first letter than add new title
updated.push({title:firstLetter}); //push title
lastC = firstLetter; //update heading
}
updated.push(val); //push current index
}
console.log(updated);

Well right now you have an array of objects - prefixing the title as its own object may be a bit confusing - a better structure may be:
[
{
title: "A",
contacts: [
{
"id":"1",
"firstname":"John",
"lastname":"Apple",
"jobtitle":"Engineer",
"title":"A"
}
]
Given your current structure, you could loop and push:
var nameIndexMap = {};
var newContactStructure = [];
for (var i = 0; i < data.length; i++) {
var letter = data[i].lastname.charAt(0);
if (nameIndexMap.hasOwnProperty(letter)) {
//push to existing
newContactStructure[nameIndexMap[letter]].contacts.push(data[i])
} else {
//Create new
nameIndexMap[letter] = newContactStructure.length;
newContactStructure.push({
title: letter,
contacts: [
data[i]
]
});
}
}
newContactStructure will now contain your sorted data.
Demo: http://jsfiddle.net/7s50k104/

Simple for loop with Array.prototype.splice will do the trick:
for (var i = 0; i < data.length; i++) {
if (i == 0 || data[i-1].lastname[0] !== data[i].lastname[0]) {
data.splice(i, 0, {title: data[i].lastname[0]});
i++;
}
}
Demo. Check the demo below.
var data = [
{"lastname":"Apple"},
{"lastname":"Banana"},
{"lastname":"Bob"},
{"lastname":"Car"},
{"lastname":"Christ"},
{"lastname":"Dart"},
{"lastname":"Dog"}
];
for (var i = 0; i < data.length; i++) {
if (i == 0 || data[i-1].lastname[0] !== data[i].lastname[0]) {
data.splice(i, 0, {title: data[i].lastname[0]});
i++;
}
}
alert(JSON.stringify( data, null, 4 ));

Implementing MongoDB 2.4's full text search in a Meteor app

I'm looking into adding full text search to a Meteor app. I know MongoDB now supports this feature, but I have a few questions about the implementation:
What's the best way to enable the text search feature (textSearchEnabled=true) in a Meteor app?
Is there a way to add an index (db.collection.ensureIndex()) from within your app?
How can you run a Mongo command (i.e. db.quotes.runCommand( "text", { search: "TOMORROW" } )) from within a Meteor app?
Since my goal is to add search to Telescope, I'm searching for a "plug-and-play" implementation that requires minimal command line magic and could even work on Heroku or *.meteor.com.

The simplest way without editing any Meteor code is to use your own mongodb. Your mongodb.conf should look something like this (on Arch Linux it is found at /etc/mongodb.conf)
bind_ip = 127.0.0.1
quiet = true
dbpath = /var/lib/mongodb
logpath = /var/log/mongodb/mongod.log
logappend = true
setParameter = textSearchEnabled=true
The key line is setParameter = textSearchEnabled=true, which, as it states, enables text search.
Start mongod up
Tell meteor to use your mongod not its own by specifying the MONGO_URL environmental variable.
MONGO_URL="mongodb://localhost:27017/meteor" meteor
Now say you have collection called Dinosaurs declared say in collections/dinosaurs.js
Dinosaurs = new Meteor.Collection('dinosaurs');
To create an text index for the collection create a file server/indexes.js
Meteor.startUp(function () {
search_index_name = 'whatever_you_want_to_call_it_less_than_128_characters'
// Remove old indexes as you can only have one text index and if you add
// more fields to your index then you will need to recreate it.
Dinosaurs._dropIndex(search_index_name);
Dinosaurs._ensureIndex({
species: 'text',
favouriteFood: 'text'
}, {
name: search_index_name
});
});
Then you can expose the search through a Meteor.method, for example in the file server/lib/search_dinosaurs.js.
// Actual text search function
_searchDinosaurs = function (searchText) {
var Future = Npm.require('fibers/future');
var future = new Future();
Meteor._RemoteCollectionDriver.mongo.db.executeDbCommand({
text: 'dinosaurs',
search: searchText,
project: {
id: 1 // Only take the ids
}
}
, function(error, results) {
if (results && results.documents[0].ok === 1) {
future.ret(results.documents[0].results);
}
else {
future.ret('');
}
});
return future.wait();
};
// Helper that extracts the ids from the search results
searchDinosaurs = function (searchText) {
if (searchText && searchText !== '') {
var searchResults = _searchEnquiries(searchText);
var ids = [];
for (var i = 0; i < searchResults.length; i++) {
ids.push(searchResults[i].obj._id);
}
return ids;
}
};
Then you can publish only documents that have been searched for in 'server/publications.js'
Meteor.publish('dinosaurs', function(searchText) {
var doc = {};
var dinosaurIds = searchDinosaurs(searchText);
if (dinosaurIds) {
doc._id = {
$in: dinosaurIds
};
}
return Dinosaurs.find(doc);
});
And the client side subscription would look something like this in client/main.js
Meteor.subscribe('dinosaurs', Session.get('searchQuery'));
Props to Timo Brinkmann whose musiccrawler project was the source of most this knowledge.

To create a text index and try to add like this I hope so it will be useful if there is still problem comment
From docs.mongodb.org:
Append scalar index fields to a text index, as in the following
example which specifies an ascending index key on username:
db.collection.ensureIndex( { comments: "text",
username: 1 } )
Warning You cannot include multi-key index field or geospatial index
field.
Use the project option in the text to return only the fields in the
index, as in the following:
db.quotes.runCommand( "text", { search: "tomorrow",
project: { username: 1,
_id: 0
}
}
)
Note: By default, the _id field is included in the result set. Since the example index did not include the _id field, you must
explicitly exclude the field in the project document.

client-side caching of jquery-ui-autocomplete: javascript/jquery regex to match a string "index" of an object

In my web app , i am using jquery ui autocomplete which data are stored in the database (Mysql).
In this application, i have the administration official data And i enable the users to enter their own data. Data will be displayed in the autocomplete drop down list .
The administration data are limited and never change thus are preloaded and cached in memory, however the user data aren't (since user data grows exponentially and there is no point of using a database if all is to be cached in memory).
So i am doing multi levels caching in my web app, of which, i am caching autocomplete's user data on the client side (instead of getting them from db on each search).
The scenario is as follow:
When the app loads, the admin data (that are preloaded in memory) are loaded and cached in a JS object to be used in the autocomplete mechanism.
Thus when a user searches for a word, it will check the admin data JS object and if it can't find it, it will go to db and find it in user data.
So for the case of searching user's data in db, What i am doing right now is the following:
$( "#search" ).autocomplete({
source:function(request, response) {
var results = $.ui.autocomplete.filter(data, request.term);
if (request.term.toUpperCase() in autocomplete_cache) {
response($.map(autocomplete_cache[request.term.toUpperCase()], function(item) {
return {value: item.members.name, md5: item.members.ID}
}))
return;
}
where autocomplete_cache is a javascript object:
var autocomplete_cache = {};
// ...it gets filled everytime a new $.post gets new data
...autocomplete_cache[request.term.toUpperCase()] = data;
For the moment, it is caching the exact term searched by a user.
For example, if a user writes "ABCD" and which is not already cached :
--> go to the database (MySQL) do :"WHERE term Like %ABCD%"
--> autocomplete_cache["ABCD"] = "data parsed from mysql"
So if the same USER now retypes "ABCD" it will get it from the autocomplete_cache Object. However if the USER type "ABC" it won't get it from the cache object because of the line:
if (request.term.toUpperCase() in autocomplete_cache)
So what i would like to do is to match what the User types in the cache, so in this scenario if the User types "ABC" (which is contained in cache's "ABCD" String) , the autocomplete will build a drop down list containing "ABCD".
Is there a way of doing that? the reason for this requirement instead of finding the exact String index in the cache object , it will find a wider range and thus limit probabilities of needing to go to db to find the new matches.

Instead of looking for a specific key, test each key whether the term is a substring:
var term = request.term.toUpperCase();
for (var key in autocomplete_cache) {
if (key.indexOf(term) > -1) {
// term is contained in key
}
}
And instead of calling response on the first match, collect all matches’ items in an additional array and call response at the end:
var term = request.term.toUpperCase(),
items = [];
for (var key in autocomplete_cache) {
if (key.indexOf(term) > -1) {
items.merge($.map(autocomplete_cache[key], function(item) {
return {value: item.members.name, md5: item.members.ID}
});
}
}
if (items.length > 0) response(items);
You might also want to remove any duplicates in items and sort the resulting items before calling response:
var term = request.term.toUpperCase(),
items = [],
index = {};
for (var key in autocomplete_cache) {
if (key.indexOf(term) > -1) {
$.each(autocomplete_cache[key], function(item) {
var id = item.members.ID;
if (!index[id]) {
items.push({value: item.members.name, md5: item.members.ID});
index[id] = true;
}
});
}
}
items.sort(function(a, b) { return a.name.localeCompare(b.name); });
if (items.length > 0) response(items);