I'm trying to come with the most practical search possible for an auto suggest search bar like the one on instagrams (it's able to match regex of strangers). So let's say I have a million user with their own unique handle. I want to be able to do the suggest search as the user is typing on the front-end. For ex if the current strings of the user type field are 'dav' its will match any unique handles in the entire Mongodb for that regex. I know mongo does support find field with regex but how do I limit the maximum of results for efficiency. I cannot find an answer on this for Mongoose. Say the user is search for 'dave' and there's like 3k results that match 'dave' but it will stop at search at the first 10 and return those result. So the user gonna have to be more specific with the unique handle. Thank you.
You can do that using limit() (and additionally skip() if you want to use pagination)
db.users.find(...).limit(10)
This will return the first 10 results.
Related
I am having troubles with firebase using the cloudstore .where query.
I want to query a big collection of documents (in my case posts) but I only want to query the posts in which the groupId matches any of the groups that the user is in. The reason for this is that I want to query a combined feed for the user with all the latest relevant data (using orderBy and limit).
I know that I can use array-contains, so I could for instance query all of the posts for user where the user is a member.
firebase.db.collection('posts').where('members','array-contains',firebase.uid)
This would work if I decided to keep track of the members in a group. Problem is if I would change members in a group, I would have to loop through all posts and change the array of members (not really good practice). Better would then be to have the post contain and id of which group it was posted in.
So let's say the user has an array containing all the groups he is in
user.groups = ['companyGroup', '{id}', '{id2}']
I would then like to query through the whole posts collection and get all the documents where the field groupId matches any of the values in user.groups something like this:
firebase.db.collection('posts').where('groupId','==',[any of user.groups])
or maybe the reverse:
firebase.db.collection('posts').where(user.groups,'array-contains','groupId')
^ I have not tried this one but I am certain it doesn't work according to the docs according to
The where() method takes three parameters: a field to filter on, a comparison operation, and a value. The comparison can be <, <=, ==, >, >=, or array_contains.
Is there a possible way to do something like this? I can't really query multiple locations at once and combine them because that defeats the purpose of being able to limit the data and orderBy fields. I understand that I could put a new collection called feed under every user and for every post use a firebase function that pushes to post to the relevant members feed (only the id and latestActivity), but then as soon as that post changes (I am going to use a field called latestActivity to order data according to relevancy, but also when deleting a post) I would need to loop through all docs under every user affected and change the value/delete doc.
Any ideas are greatly appreciated!
Currently, there is no way to pass an array of ids to the where() function and expect to get all the documents that corresponde to each particular id:
firebase.db.collection('posts').where('groupId','==',[any of user.groups])
The option that you have, is to store in the array either the ids as strings or references (path to group documents). There is no real advantage of storing references rather than strings, so it's up to you to decide which one you feel more comfortable with.
To get the all group documents, you should get the array that contains those ids/references and for each one separately create a new database request. So unfortunately there is no other way to get those group documents at once using a single query.
However, creating extra database calls it doesn't mean that fetching, let's say 3 documents, will be 3x slower as fetching one document. You can make some tests yourself.
I quote #Frank van Puffelen, "Firebase has a history of performing fine in such cases, since it pipelines the requests."
use array contain it works perfectly.
firebase.db.collection('posts').where(user.groups,'array-contains','groupId')
it works pretty good for me . you should try this.
When looking at products like DnD Insider and the Kindle app, users can quickly search for matching text strings in a large structure of text data. If I were to make a web application that allowed users to quickly search a "rulebook" (or similar text) for a matching entry and pull up the data to read, how should I organize the data?
I don't think it's a good idea to put all the data into memory. But if I stored it in some kind of database, what would be a good way to search the database and retrieve the appropriate matching entry?
So far, I believe I'm going to use the Boyer-Moore algorithm to actually do the searching. I can put the various sections of rule-text into different database entries. The user search will prioritize searching section titles over section body text. Since the text will be static and not user-editable, perhaps an array to store every word would work?
Typically some kind of inverted index is used for this purpose: https://en.wikipedia.org/wiki/Inverted_index
Basically this is a map from each word to a list of the places in which it appears. Each "place" could be a (document ID, occurrence count), or something more precise if you want to support phrase searching or if you want to give more weight to matches in titles, etc.
Search results are usually ranked with some variant of tf-idf: https://en.wikipedia.org/wiki/Tf%E2%80%93idf
Short Overview about the environment
I am writing an app on the CEAN Stack which should does the following:
The couchbase contains a big amount of large JSON documents
I fetch a bunch of JSON Keys from documents in the Couchbase
The user has a input field where he can enter filter criteria to find specific documents (e.g. "key1 = 10 & key2 > 20")
Current Approach
Currently I use the Angular Bootstrap Typeahead Plugin.
So when the user enters a letter the app should search in the fetched JSON keys to find keys which start with the entered search string. (e.g. user types "ke" and the app serves a list with ("key1", "key2", ...).
This is the common use-case for this plugin.
My Problem
I want the input box to behave like a text autocompletion. So when the user starts typing, the app guesses the key. (Let's say he types "ke" and selects key1). Now the user keeps on writing and the app should check for new words. (e.g. the content of the input is now "key1 = 10 & ke" and the app should now suggest the matching keys again( e.g. "key2"). Is this possible with this plugin or do you suggest an other solution for this problem?
Thanks in Advance!
example:
Plnkr
Plunker with current approach (I took the states because I can not provide the real keys but for the sake of clearifing the problem it should be enough)
So when I write "Alabama Te" it should suggest["Texas", Tennesse"] again.
On the website I'm developing, I have a search box with which I would like to incorporate a live autosuggest feature. (If you know a better way than the route I'm going please let me know)
Here's what I have in mind:
As the user types in the search bar, when the onchange event is triggered I want to send the query to the server (via ajax or websockets), then build a regex from the query (/^SOMELETTERS/gi), then search multiple fields (product names, brands, product numbers...) and match them to the regex. I want to gather the top 10 results and send them to the client side in json format to be used in an autosuggest script that's kind of like what google has.
These are my questions:
(1) Is there a better way?
(2) If no, how would I build a regex that matches items that begin with whatever the query is? ...I mean, what would the syntax be? I've tried a lot of things but they don't seem to work.
Call the constructor of the RegExp object. MSDN Docs.
If you have req.params.q you can:
var rx = new Regex(req.params.q);
I have a large DB having > 20,000 rows. I have two tables songs and albums:
table songs contains songid, albumid, songname and table albums contains albumid, albumname
Currently when user searches for a song I give results instantly as soon as he starts typing. Just like Google Instant.
What I am using is: Everytime user types I send that query string to my backend php file and there I execute that query in my DB like this:
SELECT * FROM songs, albums WHERE songs.albumid = albums.albumid AND songs.songname LIKE '%{$query_string}%';
But it's very inefficient to use DB queries everytime and also it's not scalable as my DB is growing everyday.
Therefore I want the same feature but faster and efficient and scalable.
Also, I dont want it to be exact pattern matching, for example:
Suppose, if user types "Rihana" instead of "Rihanna" then it should be able to give the results related to Rihanna.
Thanks.
You should index table Songs songname column on first n chars, say 6, to get better performance for the query.
Trigger the query search only after n chars have been typed, say 3 (jquery autocomplete has this option, for example)
You may also consider an in-memory DB if performance is truly crucial (sounds like it is) and the amount of data will not consume too much resident memory.
Google, btw, does not use a legacy RDBMS to perform its absurdly fast searches (continually amazed...)
First of all you should find MySQL's FULLTEXT search support far faster than your current approach.
I suspect with the kind of speed you'd like from this solution and the support for searching for mis-spelled words that you'd be better off investigating some kind of more featured full text search engine. These include:
Sphinx Search
Solr
ElasticSearch
Try full text search.
Indexing requires MyISAM tables though.
If you need ACID and full text search, use PostgreSQL.