How can I query elastic search based on the number key?
JSON field name
years_of_experience :
"{\"61\": \"10\", \"8240\": \"5\", \"8249\": \"2\", \"50\": \"0\", \"2079\": \"2\"}"
I want to filter years_of_experience like 50:0.
So, according to your sample, you have documents like the below:
POST myindex/_doc
{
"years_of_experience": {
"50": "0",
"61": "10",
"2079": "2",
"8240": "5",
"8249": "2"
}
}
So, you have an object for years_of_experience, and you want to do an exact match with the field name and values. You need to set all fields inside this field you want to set as a keyword type. First, you need to handle the mapping part of this problem. Here is a solution for this :
PUT myindex
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"dynamic_templates": [
{
"strings_as_keyword": {
"match_mapping_type": "string",
"path_match": "years_of_experience.*",
"mapping": {
"type": "keyword"
}
}
}
],
"properties": {
"years_of_experience": {
"type": "object"
}
}
}
}
While creating your index for this data, you need to use a dynamic template for the years_of_experience object. And all the fields inside this will be keyword type, and you can run term queries on these fields.
So now we can create the documents after creating an index with the above settings. And you can filter the data as below :
GET myindex/_search
{
"query": {
"term": {
"years_of_experience.50": "0"
}
}
}
Related
I have a collection named templates. It has a field 'sec_1' which is an array of objects with properties 'name' and 'des' both of type string. Now I'm using elastic search and when I try to execute a query that matches only title fields I get an error saying the sec_1 field is not nested. Upon checking the mappings here is what I get mapping type of sec_1 is text.
I checked online and saw the fix was to modify the mapping. I created a new index like:
`curl base_url/new_index_name
{
"mappings":
{"properties": {
"sec_1": {
"type":
"nested","properties": {
"name": { "type": "text" },
"des": { "type": "text" }
}
}
}
}
}
and then reindexed like this:
curl base_url/_reindex
{
"source": {
"index": "old_index"
},
"dest": {
"index": "new_index"
}
}`
First request is successful, the second one fails with this error:
{
"index": "new_index",
"type": "_doc",
"id": "be5123a4-d0e8-4d7b-a8f1-42f31d37fe55",
"cause": {
"type": "mapper_parsing_exception",
"reason": "object mapping for [sec_1] tried to parse field [null] as object, but found a concrete value"
},
"status": 400
},
I don't understand why this is happening and what I'm doing wrong. Please help. I do not know much about elastic search, I've looked for solutions online, tried chatgpt too but the same steps appear. I'm finding the reason for this error is that sec_1 is not nested type but I've checked in database it is an array of objects. What else could be wrong?
I tried creaing new index but cannot do that and without new index my search query cant function properly
I've got a lambda function, which acts as a trigger on a table with best scores of users to handle a leaderboard table.
In my leaderboard table, the sort key is the score, and the player's name is a separate entry with a list, because it's possible that there could be more than one player with the same score. Never mind.
So when adding a player I do:
var paramsNewEntry = {
"TableName": leaderboardTable,
"Key": {
"trackId": trackId,
"time": newValue
},
"UpdateExpression": "SET players = list_append(if_not_exists(players, :emptyList), :playersList),
"ExpressionAttributeValues": {
":playersList": [userId],
":emptyList":[]
},
"ReturnValues": "NONE"
};
And this works fine. I wanted to remove it this way:
var paramsOldEntry = {
"TableName": myTable,
"Key": {
"trackId": trackId,
"time": oldValue
},
"UpdateExpression": "DELETE players :playerToRemove",
"ExpressionAttributeValues": {
":playerToRemove": [userId]
},
"ReturnValues": "ALL_NEW"
}
But I get: Invalid UpdateExpression: Incorrect operand type for operator or function; operator: DELETE, operand type: LIST error.
The players attribute is a list, query response example:
{
"Items": [
{
"time": {
"N": "99994"
},
"players": {
"L": [
{
"S": "krystianPostman2"
}
]
},
"trackId": {
"S": "betaTrack001"
}
}
],
"Count": 1,
"ScannedCount": 1,
"LastEvaluatedKey": {
"time": {
"N": "99994"
},
"trackId": {
"S": "betaTrack001"
}
}
}
I've not seen any question on SO which would provide any details on this in javascript, when using the dynamodb Document API.
DynamoDB API doesn't have an option to delete the value from LIST datatype based on its value. However, if you know the index of the value to be deleted, you can use REMOVE to delete the entry from list.
The DELETE action only supports Set data types.
UpdateExpression: 'REMOVE players[0]'
If the LIST is going to have only name attribute, it is better to save it as SET rather than LIST DynamoDB datatype.
Creating Set:-
var docClient = new AWS.DynamoDB.DocumentClient();
docClient.createSet( ["v1", "v2"]);
Deleting the values from SET using DELETE
I'm trying to get the latest records, grouped by the field groupId, which is a String like "group_a".
I followed the accepted answer of this question, but I've got the following error message:
Fielddata is disabled on text fields by default. Set fielddata=true on [your_field_name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.
In the Elasticsearch docs is written:
Before you enable fielddata, consider why you are using a text field for aggregations, sorting, or in a script. It usually doesn’t make sense to do so.
I'm using a text field, because groupId is a String. Does it make sense to set fielddata: true if I want to group by it?
Or are there alternatives?
Using "field": "groupId.keyword" (suggested here) didn't work for me.
Thanks in advance!
The suggest answer with .keyword is the correct one.
{
"aggs": {
"group": {
"terms": {
"field": "groupId.raw"
},
"aggs": {
"group_docs": {
"top_hits": {
"size": 1,
"sort": [
{
"timestamp (or wathever you want to sort)": {
"order": "desc"
}
}
]
}
}
}
}
}
}
with a mapping like that:
"groupId": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
}
The server I'm working with changed the REST format from plain JSON:
{
"removedVertices": [
{
"id": "1",
"info": {
"host": "myhost",
"port": "1111"
},
"name": "Roy",
"type": "Worker"
}
],
"id": "2",
"time": 1481183401573
}
To Jackson format:
{
"removedVertices": [
"java.util.ArrayList",
[
{
"id": "1",
"info": [
"java.util.HashMap",
{
"host": "myhost",
"port": "1111"
}
]
"name": "Roy",
"type": "Worker",
}
]
"id": "2",
"time": 1482392323858
}
How can I parse it the way it was before in Angular/Javascript?
Assuming only arrays are affected, I would use underscore.js and write a recursive function to remove the Jackson type.
function jackson2json(input) {
return _.mapObject(input, function(val, key) {
if (_.isArray(val) && val.length > 1) {
// discard the Jackson type and keep the 2nd element of the array
return val[1];
}
else if (_.isObject(val)) {
// apply the transformation recursively
return jackson2json(val);
}
else {
// keep the value unchanged (i.e. primitive types)
return val;
}
});
}
If the api should be restful, then the server should not return none plain json results. I think the server site need to fix that.
I think it is because the server enabled the Polymorphic Type Handling feature.
Read Jackson Default Typing for object containing a field of Map and JacksonPolymorphicDeserialization.
Disable the feature and you will get result identical to plain json.
The main difference i see is that in arrays you have an additional string element at index 0.
If you always get the same structure you can do like this:
function jacksonToJson(jackson) {
jackson.removedVertices.splice(0, 1);
jackson.removedVertices.forEach((rmVert) => {
rmVert.info.splice(0, 1);
});
return jackson;
}
I try to find one object in document's array, and update its fields.
db.rescuemodels.findAndModify({
query: {
"features": {
$elemMatch: {
"properties.title": "W"
}
}
},
update: {
$set: {
"features": {
"properties": {
"title": "XXX"
}
}
}
}
})
Query is fine, result is one matching element, but how to make update method change just one field in this example title? Because now it create new array or object and clean old array.
MongoDB has "Dot Notation" for this purpose, as well as the positional $ operator for referencing matched elements of an array:
db.rescuemodels.findAndModify({
"query": { "features.properties.title":"W" },
"update": { "$set": { "features.$.properties.title":"XXX" } }
})
Note that this only works when there is a single array present as in:
{
"features": [
{ "properties": { "name": "A" } },
{ "properties": { "name": "W" } }
}
}
If you are nesting arrays then MongoDB cannot match in "positional operator beyond the "outer" array only:
{
"features": [
{ "properties": [{ "name": "A" }, { "name": "W" }] },
]
}
Postional matching will not work there because you cannot do features.$.properties.$.name and the matched element index would be 0 and not 1 as this refers to the outer array.
Also note that under nodejs the MongoDB driver syntax for .findAndModify() is quite different to the shell syntax. The "query" and "update" parts are separate arguments there rather than the document form as used by the shell,
To update an individual element in the array "features" you can use the positional operator, $. Your query would look something like this...
db.rescuemodels.findAndModify({
query: {
"features": {
$elemMatch: {
"properties.title": "W"
}
}
},
update: {
$set: {
"features.$.properties.title": "XXX"
}
}
})