I'm very new to elasticsearch and the documentation really just confuses me so please bear with me a little bit here.
I have an index called zproducts and under it a type called item which is mapped looks somewhat like this:
{
"item_name" : "Product A",
"category_ids" : [ "id1", "id2" ]
},
{
"item_name" : "Product B",
"category_ids" : [ "id1" ]
},
{
"item_name" : "Product C",
"category_ids" : [ "id2" ]
}
I want to be able to query for items that match at least one of the of the categories. ie. querying for id2 will return products A and C.
It would seem that I have the exact same problem that this guy had.
But the solution suggested there just doesn't work for me.
This is my current query:
/zproducts/items/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"category_ids": "id2"
}
}
]
}
}
}
Subsequent test query:
{
"query": {
"filtered": {
"filter": {
"term": {
"category_ids": "id2"
}
}
}
}
}
Both queries return ALL the items on store and not just the ones I'm trying to query for.
There was also the original query which, for some reason, used to work but stopped working altogether.
{
"query": {
"bool": {
"must": [
{
"match": {
"category_ids": {
"query": "id2",
"operator": "or"
}
}
}
]
}
}
}
So what am I doing wrong here? Can anyone please shed some light?
This query:
{
"query": {
"filtered": {
"filter": {
"term": {
"category_ids": "id2"
}
}
}
}
}
is correct. Make sure you are making your request as POST /zproducts/items/_search, rather than GET /zproducts/items/_search - getting all documents in an index back for a query is a good hint that the query body is being ignored, which is a good hint that you're using GET; many clients will not send a request body with a GET request, which ES interprets as a blank query.
Related
hey I am quite new to mongoose and can't get my head around search.
models
User->resumes[]->employments[]
UserSchema
{
resumes: [ResumeSchema],
...
}
ResumeSchema
{
employments: [EmploymentSchema],
...
}
EmploymentSchema
{
jobTitle: {
type: String,
required: [true, "Job title is required."]
},
...
}
Background
User has to enter job title and needs suggestions from the existing data of the already present resumes and their employment's job title
I have tried the following code.
let q = req.query.q; // Software
User.find({ "resumes.employments.jobTitle": new RegExp(req.query.q, 'ig') }, {
"resumes.employments.$": 1
}, (err, docs) => {
res.json(docs);
})
Output
[
{
_id: '...',
resumes:[
{
employments: [
{
jobTitle: 'Software Developer',
...
},
...
]
},
...
]
},
...
]
Expected OutPut
["Software Developer", "Software Engineer", "Software Manager"]
Problem
1:) The Data returned is too much as I only need jobTitle
2:) All employments are being returned whereas the query matched one of them
3:) Is there any better way to do it ? via index or via $search ? I did not find much of information in mongoose documentation to create search index (and I also don't really know how to create a compound index to make it work)
I know there might be a lot of answers but none of them helped or I was not able to make them work ... I am really new to mongodb I have been working with relational databases via SQL or through ORM so my mongodb concepts and knowledge is limited.
So please let me know if there is a better solution to do it. or something to make the current one working.
You can use one of the aggregation query below to get this result:
[
{
"jobTitle": [
"Software Engineer",
"Software Manager",
"Software Developer"
]
}
]
Query is:
First using $unwind twice to deconstructs the arrays and get the values.
Then $match to filter by values you want using $regex.
Then $group to get all values together (using _id: null and $addToSet to no add duplicates).
And finally $project to shown only the field you want.
User.aggregate({
"$unwind": "$resumes"
},
{
"$unwind": "$resumes.employments"
},
{
"$match": {
"resumes.employments.jobTitle": {
"$regex": "software",
"$options": "i"
}
}
},
{
"$group": {
"_id": null,
"jobTitle": {
"$addToSet": "$resumes.employments.jobTitle"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here
Also another option is using $filter into $project stage:
Is similar as before but using $filter instead of $unwind twice.
User.aggregate({
"$unwind": "$resumes"
},
{
"$project": {
"jobs": {
"$filter": {
"input": "$resumes.employments",
"as": "e",
"cond": {
"$regexMatch": {
"input": "$$e.jobTitle",
"regex": "Software",
"options": "i"
}
}
}
}
}
},
{
"$unwind": "$jobs"
},
{
"$group": {
"_id": null,
"jobTitle": {
"$addToSet": "$jobs.jobTitle"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here
I have database collection, that looks like this , how to remove this empty array .
Initially I have object in this (HUE) array
Which looks like this
"HUE": [{
"chartId": "timeseries",
"name": "TS",
}]
, but after deleting the objects, it does not delete the empty array
{
"userId": "adam",
"charts": {
"HUE": [],
"Color": [{
"chartId": "one",
"name": "TS",
}]
}
}
P.S I only want to delete the HUE array when its empty
delChartObj.updateOne(
{ 'userId': userId },
{ $pull: query } // this line actually find the chartId and delete it
// after the above line, I actually want to check, if after del the object , array became empty, then delete the array too
, function (err, obj) {
if (err) { res.send.err }
res.status(200).send("Deleted Successfully");
});
db.collection.update({userId, 'charts.HUE': {$exists:true, $size:0}}, { $unset: { 'charts.HUE': 1 } })
It works -> https://mongoplayground.net/p/uqU7d0Kp2eL
Update: The question changed and this solution is not completely correct. Users have to ask with more details.
If you want to filter the array and then if empty to delete the array, you can do the bellow
db.collection.update(
{
"$expr" : {
"$eq" : [ "$userId", "adam" ]
}
},
[ {
"$addFields" : {
"charts.HUE" : {
"$filter" : {
"input" : "$charts.HUE",
"as" : "hue",
"cond" : {
"$ne" : [ "$$hue.chartId", "timeseries" ]
}
}
}
}
}, {
"$addFields" : {
"charts.HUE" : {
"$cond" : [ {
"$eq" : [ {
"$size" : "$charts.HUE"
}, 0 ]
}, "$$REMOVE", "$charts.HUE" ]
}
}
} ]
)
Test code here
Its pipeline update requires MongoDB >= 4.2
Pipeline updates are very powerful, but for simple updates they can be more complicated.
$$REMOVE is a system variable, if a field gets this value, its removed.
Here the field gets this value only if empty array.
I am trying to do a simple wildcard query in a MongoDB database using a user-generated string variable. For example, if the user searches 'Bu', things like 'Burger' and 'Burger King' should be returned from the database. I have searched and tried several things but nothing seems to work. Any help would be greatly appreciated. Note: This is client-side JS.
var text = document.getElementById("Search_Box").value;
var regex = new RegExp(text + ".*");
client.login().then(() => db.collection('businesses')
.find({name:{$regex:regex}}).then(docs => {
console.log("[MongoDB Stitch] Connected to Stitch");
If you had the following documents:
{
"_id" : ObjectId("5a0f3a464d8a2fe38bec4e92"),
"name" : "Burger"
}
{
"_id" : ObjectId("5a0f3a464d8a2fe38bec4e94"),
"name" : "Burger King"
}
{
"_id" : ObjectId("5a0f3a464d8a2fe38bec4e96"),
"name" : "Booby's"
}
{
"_id" : ObjectId("5a0f3a464d8a2fe38bec4e98"),
"name" : "McDonald"
}
Starting with
To get everything starting with "Bu" you could do
db.collection('businesses').find({name: {$regex: '^Bu'}})
or
db.collection('businesses').find({name: {$regex: /^Bu/}})
In the middle of
If you needed anything that contained "Ki.*g" anywhere in the word you could do:
db.collection('businesses').find({name: {$regex: 'Ki.*g'}})
or
db.collection('businesses').find({name: {$regex: /Ki.*g/}})
Do the effort and go through the documentation. Everything is explained there, with a lot more details. https://docs.mongodb.com/manual/reference/operator/query/regex/
I'm having a similar problem. I'm sending:
async search (term) {
this.suggestions = await this.db.collection(this.collection).find({name: {$regex: term + '.*', $options: 'i'}}).sort({'_id': 1}).execute()
}
And I'm getting this back from Mongo
Stack Trace:
StitchError: $regex requires regular expression
at find (<native code>)
at apply (<native code>)
at executeServiceFunction (<anonymous>:10:10)
{
"service": "mongodb-atlas",
"name": "find",
"arguments": [
{
"database": "monsters",
"collection": "monsters",
"query": {
"name": {
"$regex": "Aart.*",
"$options": "i"
}
},
"project": null,
"sort": {
"_id": {
"$numberInt": "1"
}
}
}
]
}
I'm wondering how I can compare arrays of (nested) objects in Mongoose.
Considering the data below, I would like to get results when the name properties match. Could anyone help me with this?
Organisation.find( {
$or: [
{ "category_list": { $in: cat_list } },
{ "place_topics.data": { $in: place_tops } }
]
}
)
Let's say that this is the data stored in my MongoDB:
"category_list": [
{
"id": "197750126917541",
"name": "Pool & Billiard Hall"
},
{
"id": "197871390225897",
"name": "Cafe"
},
{
"id": "218693881483234",
"name": "Pub"
}
],
"place_topics": {
"data": [
{
"name": "Pool & Billiard Hall",
"id": "197750126917541"
},
{
"name": "Pub",
"id": "218693881483234"
}
]
}
And let's say that these are the arrays I want to compare against (almost the same data):
let cat_list = [
{
"id": "197750126917541",
"name": "Pool & Billiard Hall"
},
{
"id": "197871390225897",
"name": "Cafe"
},
{
"id": "218693881483234",
"name": "Pub"
}
]
let place_tops = [
{
"name": "Pool & Billiard Hall",
"id": "197750126917541"
},
{
"name": "Pub",
"id": "218693881483234"
}
]
When there are "multiple conditions" required for each array element is when you actually use $elemMatch, and in fact "need to" otherwise you don't match the correct element.
So to apply multiple conditions, you would rather make an array of conditions for $or instead of shortcuts with $in:
Organizations.find({
"$or": [].concat(
cat_list.map( c => ({ "category_list": { "$elemMatch": c } }) ),
place_tops.map( p => ({ "place_topics": { "$elemMatch": p } }) )
)
})
However, if you take a step back and think logically about it, you actually named one of the properties "id". This would generally imply in all good practice that the value is in fact ""unique".
Therefore, all you really should need to do is simply extract those values and stick with the original query form:
Organizations.find({
"$or": [
{ "category_list.id": { "$in": cat_list.map(c => c.id) } },
{ "place_topics.id": { "$in": place_tops.map(p => p.id) } }
]
})
So simply mapping both the values and the property to "match" onto the "id" value instead. This is a simple "dot notation" form that generally suffices when you have one condition per array element to test/match.
That is generally the most logical approach given the data, and you should apply which one of these actually suits the data conditions you need. For "multiple" use $elemMatch. But if you don't need multiple because there is a singular match, then simply do the singular match
I'm trying to get the latest records, grouped by the field groupId, which is a String like "group_a".
I followed the accepted answer of this question, but I've got the following error message:
Fielddata is disabled on text fields by default. Set fielddata=true on [your_field_name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.
In the Elasticsearch docs is written:
Before you enable fielddata, consider why you are using a text field for aggregations, sorting, or in a script. It usually doesn’t make sense to do so.
I'm using a text field, because groupId is a String. Does it make sense to set fielddata: true if I want to group by it?
Or are there alternatives?
Using "field": "groupId.keyword" (suggested here) didn't work for me.
Thanks in advance!
The suggest answer with .keyword is the correct one.
{
"aggs": {
"group": {
"terms": {
"field": "groupId.raw"
},
"aggs": {
"group_docs": {
"top_hits": {
"size": 1,
"sort": [
{
"timestamp (or wathever you want to sort)": {
"order": "desc"
}
}
]
}
}
}
}
}
}
with a mapping like that:
"groupId": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
}