Develop Reference

Quotationmark problem with Knex and SQL-database

I'm trying to save a query to a database using Knex. But if this query I'm trying to save includes quotation marks I'm getting an error.
Here's an example of how the code might look like:
db.query(`INSERT INTO test.searches VALUES ('TestUser', 'testqueryname', ''SELECT * FROM table WHERE team='rocket'')`, info, () =>{}
Of course in the real case I'm not sending in hardcoded values but variables as strings.
Trying to save this gets me the error
error: syntax error at or near "rocket"

To start with looks like your code has javascript syntax error at least closing parenthesis is missing.
Secondly knex does not have .query() method (maybe you are using database driver directly?). You should probably be using knex.raw() even that query above could be easily written with normal knex methods.
If you are using knex.raw you can do quoting with ?? replacements and value bindings wit ?.
If you are writing your code like in the question you have no advantage at all from using knex and you should be using database driver directly instead.

If you are running MySQL, you need to slash escape quotation marks:
db.query(`INSERT INTO test.searches VALUES ('TestUser', 'testqueryname', 'SELECT * FROM table WHERE team=\'rocket\'')`, info, () =>{}
If you are running PostgreSQL, you need to double escape quotation marks:
db.query(`INSERT INTO test.searches VALUES ('TestUser', 'testqueryname', 'SELECT * FROM table WHERE team=''rocket''')`, info, () =>{}
Please consider using parameterised SQL queries or at least appropriately escaping input to avoid SQL injection attacks.

Mongodb Nodejs Regex Query

I am using the module "mongodb" to query mongodb from node js APIs. I am trying to search contacts document by a search query. The selector for the same is as below:
selector['users.name'] = {$regex: new RegExp(params.query), $options:"i"}
Consider the name of the user is "Sagar Gopale" and the search query is "sagar" or even "gopale" then the above works fine and returns results properly. But if I input search query as "sagar gopale" it returns empty result. Can someone provide me a solution so that I can input the search query including the space?
Thanks in advance.

Actualy you will have many difficulties here if you stick with the regexp aproach, I suggest you to create a text index:
collection.createIndex({"users.name" : "text"});
And then when you will search:
db.collection('users').find({
$text: {
$search: params.query,
$caseSensitive: false,
}
});
In this way you will get the result when you use "Sagar", "sagar", "sagar gopa", "gopale sagar", and many combinations of those 2.
This aproach works very nicely if you have an autocomplete, or the client doesn't know exactly the name. Mongo dispose of strong text search algorithms that will match letters, even if they are not in proper order.

Best way to perform a full text search in MongoDB and Mongoose

I'm searching on Google since days and I tried many things but I still can not perform a good full text search on my user collection.
I tried ElasticSearch but was pretty impossible to query and paginate...
I tried many plugins for Mongoose like ElMongo, mongoose-full-text, Mongoosastic, etc... everyone are really bad documented and I don't know how to perform a good full text search.
So, my collection is a normal collection:
user = {
name: String,
email: String,
profile: {
something: String,
somethingElse: String
}
}
I have a search input in a page with a simple POST, if I type hello world what I need is to search on the entire collection fields the matching words of my search query and get the results.
It will be really nice also to have options to handle a pagination like 10 items per page or something...
What is the best solution to achieve this? I'm using MongoDB 2.6.* with Mongoose, NodeJS and ExpressJS.
Thanks.

You can add a text index to your Mongoose schema definition that lets you use the $text operator in your find queries to search all fields included in the text index.
To create an index to support text search on, say, name and profile.something:
var schema = new Schema({
name: String,
email: String,
profile: {
something: String,
somethingElse: String
}
});
schema.index({name: 'text', 'profile.something': 'text'});
Or if you want to include all string fields in the index, use the '$**' wildcard:
schema.index({'$**': 'text'});
This would enable you to perform a paged text search query like:
MyModel.find({$text: {$search: searchString}})
.skip(20)
.limit(10)
.exec(function(err, docs) { ... });
For more details, read the full MongoDB Text Indexes documentation.

This is not an extra answer but an addUp, but if the above answer by JohnnyHK is giving you an empty array [] .
Try something basic first without limit and skip
const results = await MyModel.find({ $text: { $search: "text" } });
Please check if the index was created properly using mongosh using
db.stories.getIndexes() or compass GUI both attached. Else create one via COMPASS GUI or via mongosh using docs
Then try changing searchText multiple times and try.
Even after all this it took me 7- 8 runs of same code to get proper results,

How do I create a "like" filter view in CouchDB

Here's an example of what I need in sql:
SELECT name FROM employ WHERE name LIKE %bro%
How do I create view like that in CouchDB?

The simple answer is that CouchDB views aren't ideal for this.
The more complicated answer is that this type of query tends to be very inefficient in typical SQL engines too, and so if you grant that there will be tradeoffs with any solution then CouchDB actually has the benefit of letting you choose your tradeoff.
1. The SQL Ways
When you do SELECT ... WHERE name LIKE %bro%, all the SQL engines I'm familiar with must do what's called a "full table scan". This means the server reads every row in the relevant table, and brute force scans the field to see if it matches.
You can do this in CouchDB 2.x with a Mango query using the $regex operator. The query would look something like this for the basic case:
{"selector":{
"name": {
"$regex": "bro"
}
}}
There do not appear to be any options exposed for case-sensitivity, etc. but you could extend it to match only at the beginning/end or more complicated patterns. If you can also restrict your query via some other (indexable) field operator, that would likely help performance. As the documentation warns:
Regular expressions do not work with indexes, so they should not be used to filter large data sets. […]
You can do a full scan in CouchDB 1.x too, using a temporary view:
POST /some_database/_temp_view
{"map": "function (doc) { if (doc.name && doc.name.indexOf('bro') !== -1) emit(null); }"}
This will look through every single document in the database and give you a list of matching documents. You can tweak the map function to also match on a document type, or to emit with a certain key for ordering — emit(doc.timestamp) — or some data value useful to your purpose — emit(null, doc.name).
2. The "tons of disk space available" way
Depending on your source data size you could create an index that emits every possible "interior string" as its permanent (on-disk) view key. That is to say for a name like "Dobros" you would emit("dobros"); emit("obros"); emit("bros"); emit("ros"); emit("os"); emit("s");. Then for a term like '%bro%' you could query your view with startkey="bro"&endkey="bro\uFFFF" to get all occurrences of the lookup term. Your index will be approximately the size of your text content squared, but if you need to do an arbitrary "find in string" faster than the full DB scan above and have the space this might work. You'd be better served by a data structure designed for substring searching though.
Which brings us too...
3. The Full Text Search way
You could use a CouchDB plugin (couchdb-lucene now via Dreyfus/Clouseau for 2.x, ElasticSearch, SQLite's FTS) to generate an auxiliary text-oriented index into your documents.
Note that most full text search indexes don't naturally support arbitrary wildcard prefixes either, likely for similar reasons of space efficiency as we saw above. Usually full text search doesn't imply "brute force binary search", but "word search". YMMV though, take a look around at the options available in your full text engine.
If you don't really need to find "bro" anywhere in a field, you can implement basic "find a word starting with X" search with regular CouchDB views by just splitting on various locale-specific word separators and omitting these "words" as your view keys. This will be more efficient than above, scaling proportionally to the amount of data indexed.

Unfortunately, doing searches using LIKE %...% aren't really how CouchDB Views work, but you can accomplish a great deal of search capability by installing couchdb-lucene, it's a fulltext search engine that creates indexes on your database that you can do more sophisticated searches with.
The typical way to "search" a database for a given key, without any 3rd party tools, is to create a view that emits the value you are looking for as the key. In your example:
function (doc) {
emit(doc.name, doc);
}
This outputs a list of all the names in your database.
Now, you would "search" based on the first letters of your key. For example, if you are searching for names that start with "bro".
/db/_design/test/_view/names?startkey="bro"&endkey="brp"
Notice I took the last letter of the search parameter, and "incremented" the last letter in it. Again, if you want to perform searches, rather than aggregating statistics, you should use a fulltext search engine like lucene. (see above)

You can use regular expressions. As per this table you can write something like this to return any id that contains "SMS".
{
"selector": {
"_id": {
"$regex": "sms"
}
}
}
Basic regex you can use on that includes
"sms$" roughly to LIKE "%sms"
"^sms" roughly to LIKE "sms%"
You can read more on regular expressions here

i found a simple view code for my problem...
{"getavailableproduct": {
"map": "function(doc) { var prefix = doc['productid'].match(/[A-Za-z0-9]+/g); if(prefix) for(var pre in prefix) { emit(prefix[pre],null); } }"
}
}
from this view code if i split a key sentence into a key word...
and i can call
?key="[search_keyword]"
but i need more complex code because if i run this code i can only find word wich i type (ex: eat, food, etc)...
but if i want to type not a complete word (ex: ea from eat, or foo from food) that code does not work..

I know it is an old question, but: What about using a "list" function? You can have all your normal views, andthen add a "list" function to the design document to process the view's results:
{
"_id": "_design/...",
"views": {
"employees": "..."
},
"lists": {
"by_name": "..."
}
}
And the function attached to "by_name" function, should be something like:
function (head, req) {
provides('json', function() {
var filtered = [];
while (row = getRow()) {
// We can retrive all row information from the view
var key = row.key;
var value = row.value;
var doc = req.query.include_docs ? row.doc : {};
if (value.name.indexOf(req.query.name) == 0) {
if (req.query.include_docs) {
filtered.push({ key: key, value: value, doc: doc});
} else {
filtered.push({ key: key, value: value});
}
}
}
return toJSON({ total_rows: filtered.length, rows: filtered });
});
}
You can, of course, use regular expressions too. It's not a perfect solution, but it works to me.

You could emit your documents like normal. emit(doc.name, null); I would throw a toLowerCase() on that name to remove case sensitivity.
and then query the view with a slew of keys to see if something "like" the query shows up.
keys = differentVersions("bro"); // returns ["bro", "br", "bo", "ro", "cro", "dro", ..., "zro"]
$.couch("db").view("employeesByName", { keys: keys, success: dealWithIt } )
Some considerations
Obviously that array can get really big really fast depending on what differentVersions returns. You might hit a post data limit at some point or conceivably get slow lookups.
The results are only as good as differentVersions is at giving you guesses for what the person meant to spell. Obviously this function can be as simple or complex as you like. In this example I tried two strategies, a) removed a letter and pushed that, and b) replaced the letter at position n with all other letters. So if someone had been looking for "bro" but typed in "gro" or "bri" or even "bgro", differentVersions would have permuted that to "bro" at some point.
While not ideal, it's still pretty fast since a look up in Couch's b-trees is fast.

why cann't we just use indexOf() in view?

CouchDB case in-sensitive and OR options like in mysql

I'm quite new with couchDB and I need a little support.
In MySQL I could run simply this query:
SELECT `name`, `id`, `desc` FROM `table`
WHERE `name`="jack" OR `cat` LIKE "%|52224|%";
And here are my two problems:
I started to create a view (still without Like option and everything):
function(doc) {
emit([doc.name, doc.cat], {
"name" : doc.name,
"desc" : doc.desc,
"id" : doc._id
});
}
1. When I use "emit([doc.name" the string must match 100% (also case sensitive).
-> How I make this option case un-sesnsitive?
That I can ask for ("Jack, jack, jAck, JAck,...) like in mysql?
2. How I create the OR option?
When I use [doc.name, doc.cat], I'm also forced to ask for both vars.
But when I have just one of them,
how I can query without creating for each option an own view?

To implement case insensitive search you just have to convert keys to lowercase:
emit([doc.name.toLowerCase(), doc.cat.toLowerCase()])
Now, if you convert your query to lowercase you'll have case insensitive matching.
The problem with this solution is that if you want both case sensitive and case insensitive search you can either emit both values:
[doc.name.toLowerCase(), doc.name, doc.cat]
and use startkey and endkey to filter results, or create a separate view.
The second question is a bit more tricky to implement.
First of all, if you need to filter by doc.name only you can send request with startkey=["jack",0] and endkey=["jack",'zzzzzz'] which will return all documents with doc.name="jack" and doc.cat between 0 and 'zzzzzz' (there probably is a better way to say "any cat", unfortunately I can't find it right now).
If you need a real OR, then you should emit two rows for each document:
emit(doc.name, doc); emit(doc.cat, doc);
This way you can POST needed keys with your request: {"keys": ["jack", "cat_name"]}
This will return every document with either "jack" OR "cat_name" key. However documents that have both will be returned twice, so you have to filter duplicates in your application code.
You can also use couchdb-lucene which will solve both of your problems and probably a lot more. It is a popular choice among couchdb users for implementing advanced queries.