while running script in elastic search, I got 504 Gateway timeout error.
{
"query": {
"bool": {
"filter": {
"script": {
"script": " doc['creted_date'].date.getMonthOfYear() == 12 "
}
}
}
},
"aggs": {
"test": {
"date_histogram": {
"field": "creted_date",
"interval": "month",
"format": "MMM"
},
"aggs": {
"cost": {
"sum": {
"field": "cost"
}
}
}
}
}
}
Error result :
{
"statusCode": 504,
"error": "Gateway Time-out",
"message": "Client request timeout"
}
whenever I am running this script over index having small number of documents , it gives perfect output. but on index having large number of documents , it gives above error.
Can we manually set the timeout for request in elastic search ? or Is there any other solution for this problem ?
Try this for Elasticsearch 6.x.
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"source": "doc['created_on'].date.getMonthOfYear() == params.month",
"params": {
"month": 5
}
}
}
}
}
}
}
Try this.
{
"query": {
"bool": {
"filter": {
"script": {
"lang": "expression",
"script": "doc['creted_date'].getMonth() == month-1",
"params": {
"month": 12
}
}
}
}
}
}
Related
I'm trying to make a data request from a URL based on the data provider's API specification.
My app is built using the Node.JS and NPM.
I'm subscribed to use the API and I've verified that all my identity credentials, policies and IAM keys that give me permission to access the API.
Could someone please help me define the correct value to place inside the 'Body' parameter of my API request based on the API specification provided by the URL.
Thanks!
My current request parameter code looks like below:
const request_parameter = {
DataSetId: "xxxxxac09b4xxxxxxxxxx",
RevisionId: "xxxxxxc8df94fxxxxxxx",
AssetId: "xxxxxxx2bde1fa3xxxxxxx",
Method: "POST",
Path: "/pxxx/graphql",
QueryStringParameters: {
param1: " ",
param2: " "
},
RequestHeaders: {
"Content-Type": "application/json"
},
Body: {\"body_param\":\"body_param_value\"} // FIX ME
};
The data provider's actual API specification looks like below:
{
"openapi": "3.0.1",
"info": {
"title": "image-services",
"description": "API Gateway for integration of Platform Services into AWS Data Exchange",
"version": "2022-03-03T20:04:18Z"
},
"servers": [
{
"url": ""
}
],
"paths": {
"/psdm/graphql": {
"post": {
"requestBody": {
"description": "GraphQL query for the Catalog in the form of:\n\n `{\n availablemetadatarecord(\n QUERYPARAMS\n ) { \n FIELDLIST\n } \n }`\n",
"content": {
"application/graphql": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchRequest"
},
"examples": {
"Limit returned data": {
"description": "Limit the number of records in a response.\n\nIn your query, simply specify `limit: <int>` \n",
"value": "{\n availablemetadatarecord(\n limit: 10\n content: { catalogProperties: { vendor: { eq: \"EYE\" } } }\n ) {\n assignedId\n }\n}\n"
}
}
}
},
"required": true
},
"responses": {
"200": {
"description": "200 response",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchResponse"
}
}
}
}
},
"security": [
{
"sigv4": []
}
]
}
}
},
"components": {
"schemas": {
"CatalogSearchRequest": {
"title": "CatalogSearchRequest",
"type": "object"
},
"CatalogSearchResponse": {
"title": "Catalog Search Response",
"required": [
"data",
"paginationInfo"
],
"type": "object",
"properties": {
"paginationInfo": {
"type": "object",
"properties": {
"recordsInPage": {
"type": "integer"
},
"nextOffset": {
"type": "integer"
}
}
},
"data": {
"type": "object",
"properties": {
"availablemetadatarecord": {
"type": "array",
"description": "Records returned by query",
"items": {
"type": "object"
}
}
}
}
}
}
},
"securitySchemes": {
"sigv4": {
"type": "apiKey",
"name": "Authorization",
"in": "header",
"x-amazon-apigateway-authtype": "awsSigv4"
}
}
}
}
I'm trying to write a GraphQL query to fetch data from an AWS DataExchange provider based on their company's API specification.
However, every query I've written so far isn't returning any data.
I've verified that all my AWS credentials and IAM keys are correct.
Could you please help me write the correct query based on the API specification given by the data provider.
My current request 'body' looks like below:
Body: JSON.stringify({
content: {
"application/graphql": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchRequest"
},
query: `query availablemetadatarecord(limit: 10 content: { catalogProperties: { vendor: { eq: "EYE" } } }) {
assignedId
}
}`
}
}
}),
The DataExchange provider's actual API specification looks like below.
{
"openapi": "3.0.1",
"info": {
"title": "image-services",
"description": "API Gateway for integration of Platform Services into AWS Data Exchange",
"version": "2022-03-03T20:04:18Z"
},
"servers": [
{
"url": ""
}
],
"paths": {
"/psdm/graphql": {
"post": {
"requestBody": {
"description": "GraphQL query for the Catalog in the form of:\n\n `{\n availablemetadatarecord(\n QUERYPARAMS\n ) { \n FIELDLIST\n } \n }`\n",
"content": {
"application/graphql": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchRequest"
},
"examples": {
"Limit returned data": {
"description": "Limit the number of records in a response.\n\nIn your query, simply specify `limit: <int>` \n",
"value": "{\n availablemetadatarecord(\n limit: 10\n content: { catalogProperties: { vendor: { eq: \"EYE\" } } }\n ) {\n assignedId\n }\n}\n"
}
}
}
},
"required": true
},
"responses": {
"200": {
"description": "200 response",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchResponse"
}
}
}
}
},
"security": [
{
"sigv4": []
}
]
}
}
},
"components": {
"schemas": {
"CatalogSearchRequest": {
"title": "CatalogSearchRequest",
"type": "object"
},
"CatalogSearchResponse": {
"title": "Catalog Search Response",
"required": [
"data",
"paginationInfo"
],
"type": "object",
"properties": {
"paginationInfo": {
"type": "object",
"properties": {
"recordsInPage": {
"type": "integer"
},
"nextOffset": {
"type": "integer"
}
}
},
"data": {
"type": "object",
"properties": {
"availablemetadatarecord": {
"type": "array",
"description": "Records returned by query",
"items": {
"type": "object"
}
}
}
}
}
}
},
"securitySchemes": {
"sigv4": {
"type": "apiKey",
"name": "Authorization",
"in": "header",
"x-amazon-apigateway-authtype": "awsSigv4"
}
}
}
}
I get the following error on a react component. I thought I had set a unique key when I look in graphQL playground explorer the id I am grabbing is unique for each element. So I am not sure why I am getting this error. I read that the elements inside my also need a key which I thought wasn't the case. Hopefully someone can help.
Error
Warning: Each child in a list should have a unique "key" prop.
GraphQL Query
query MyQuery {
allDatoCmsPricing {
edges {
node {
id //This was orignally there when I posted
details {
id //This was missing that was my error which was answered
task
}
}
}
}
}
GraphQL Result
{
"data": {
"allDatoCmsPricing": {
"edges": [
{
"node": {
"details": [
{
"id": "DatoCmsItem-1919839-en",
"task": "Client Consultation"
},
{
"id": "DatoCmsItem-1919840-en",
"task": "S.M.A.R.T Goal Setting"
},
{
"id": "DatoCmsItem-1919841-en",
"task": "Fitness Assessment"
},
{
"id": "DatoCmsItem-1919842-en",
"task": "Client Centered Exercises"
},
{
"id": "DatoCmsItem-1919843-en",
"task": "1-2 Sessions per week"
}
]
}
},
{
"node": {
"details": [
{
"id": "DatoCmsItem-1927942-en",
"task": "Client Consultation"
},
{
"id": "DatoCmsItem-1927943-en",
"task": "S.M.A.R.T Goal Setting testing breaking line"
},
{
"id": "DatoCmsItem-1927945-en",
"task": "Fitness Assessment"
},
{
"id": "DatoCmsItem-1927946-en",
"task": "Client Centered Exercises"
},
{
"id": "DatoCmsItem-1927947-en",
"task": "Injury Prevention"
},
{
"id": "DatoCmsItem-1927948-en",
"task": "Nutrition Advice"
},
{
"id": "DatoCmsItem-1927949-en",
"task": "Program Design"
},
{
"id": "DatoCmsItem-1927950-en",
"task": "Corrective Exercises"
},
{
"id": "DatoCmsItem-1927951-en",
"task": "3 or more Sessions per week"
}
]
}
},
{
"node": {
"details": [
{
"id": "DatoCmsItem-1927866-en",
"task": "Client Consultation"
},
{
"id": "DatoCmsItem-1927867-en",
"task": "S.M.A.R.T Goal Setting"
},
{
"id": "DatoCmsItem-1927868-en",
"task": "Fitness Assessment"
},
{
"id": "DatoCmsItem-1927869-en",
"task": "Client Centered Exercises"
},
{
"id": "DatoCmsItem-1927870-en",
"task": "Injury Prevention"
},
{
"id": "DatoCmsItem-1927872-en",
"task": "Nutrition Advice"
},
{
"id": "DatoCmsItem-1927873-en",
"task": "2-3 Sessions per week"
}
]
}
}
]
}
}
}
Component
{data.allDatoCmsPricing.edges.map(({ node: pricing }) => (
<div key={pricing.id} >
<ul className="details-list">{pricing.details.map(detailEntry => {
return <ListItem key={detailEntry.id}><span>{detailEntry.task}</span>
</ListItem>
})}</ul>
</div>
...
It looks like pricing.id is undefined. It's represented by node.id in your data, but you only have a details property present. Your graphql query reflects this. Adding id to the query should resolve the error:
query MyQuery {
allDatoCmsPricing {
edges {
node {
id # <--- Added this field
details {
id
task
}
}
}
}
}
I have elastic Search data in following format:
{
"is_cricketer": 1,
"name": "Abraham",
"cities": [
{ "name": "stellenbosch" },
{ "name": "Nelspruit" },
{ "name": "East London" }
]
},
{
"is_cricketer": 1,
"name": "Abraham",
"cities": [
{ "name": "Rustenburg" },
{ "name": "Nelspruit" },
{ "name": "East London" }
]
},
{
"is_cricketer": 0,
"name": "deVilliers",
"cities": [
{ "name": "Cape town" },
{ "name": "Nelspruit" },
{ "name": "East London" }
]
}
I need to query elastic search to get all the profiles with is_cricketer = 1 and an OR query over the field for cities.name and name field. ie
( profile.is_cricketer == 1 && (profile.name == 'Abraham' || profile.cities[i].name == 'Nelspruit' ))
To get the profiles with OR query on the fields cities.name and name field for matching query string is as follows and it works a expected:
"should": [
{
"nested": {
"path": "cities",
"query": {
"multi_match": {
"query": "Nelspruit",
"fields": [
"cities.name"
]
}
}
}
},
{
"multi_match": {
"query": "Abraham",
"fields": [
"name"
]
}
}
]
And the Must query to get all the profiles with field is_cricketer = 1 is follows:
{
"must": {
"match": {
"is_cricketer": "1"
}
}
}
Above both queries working fine and i am trying to combine both query as follows:
{
"query": {
"bool": {
"must": {
"match": {
"is_cricketer": "1"
}
},
"should": [
{
"nested": {
"path": "cities",
"query": {
"multi_match": {
"query": "Nelspruit",
"fields": [
"cities.name"
]
}
}
}
},
{
"multi_match": {
"query": "Abraham",
"fields": [
"name"
]
}
}
]
}
}
}
which is not returning expected results, its returning all the profiles with is_cricketer = 1 without filtering for name and cities.name.
I also tried to include the should query inside must query as follows:
{
"query": {
"bool": {
"must": [{
"match": {
"is_cricketer": "1"
}
}, {
"should": [
{
"nested": {
"path": "cities",
"query": {
"multi_match": {
"query": "Nelspruit",
"fields": [
"cities.name"
]
}
}
}
},
{
"multi_match": {
"query": "Abraham",
"fields": [
"name"
]
}
}
]
}]
}
}
}
But i got following error for the above query:
"Error: [parsing_exception] [should] query malformed, no start_object
after query name, with { line=1 & col=64 }
at respond (/GitRepo/project/node_modules/elasticsearch/src/lib/transport.js:307:15)
at checkRespForFailure (/GitRepo/project/node_modules/elasticsearch/src/lib/transport.js:266:7)
at HttpConnector. (/GitRepo/project/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)
at IncomingMessage.bound (/GitRepo/project/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1056:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9)"
How to combine both the queries to get desired result. Any help will be appreciated.
if you want a should query inside a must you can use it in the following way
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
... your query here
}
]
}
}
]
}
}
}
This worked for me on ES 6.0.
Setup
PUT test1
{
"mappings": {
"type1": {
"properties": {
"cities": {
"type": "nested"
}
}
}
}
}
POST test1/type1
{
"is_cricketer": 1,
"name": "Abraham",
"cities": [
{ "name": "stellenbosch" },
{ "name": "Nelspruit" },
{ "name": "East London" }
]
}
POST test1/type1
{
"is_cricketer": 1,
"name": "Abraham",
"cities": [
{ "name": "Rustenburg" },
{ "name": "Nelspruit" },
{ "name": "East London" }
]
}
POST test1/type1
{
"is_cricketer": 0,
"name": "deVilliers",
"cities": [
{ "name": "Cape town" },
{ "name": "Nelspruit" },
{ "name": "East London" }
]
}
Query
GET test1/type1/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"is_cricketer": {
"value": 1
}
}
}
],
"should": [
{
"term": {
"name.keyword": {
"value": "Abraham"
}
}
},
{
"nested": {
"path": "cities",
"query": {
"term": {
"cities.name.keyword": {
"value": "Nelspruit"
}
}
}
}
}
]
}
}
}
Results - 2 hits
"hits": {
"total": 2,
"max_score": 2.2685113,
"hits": [
{
"_index": "test1",
"_type": "type1",
"_id": "zgcesWIBVwCaLf8KSuDi",
"_score": 2.2685113,
"_source": {
"is_cricketer": 1,
"name": "Abraham",
"cities": [
{
"name": "stellenbosch"
},
{
"name": "Nelspruit"
},
{
"name": "East London"
}
]
}
},
{
"_index": "test1",
"_type": "type1",
"_id": "eAQesWIBbxh35CpKckEH",
"_score": 2.2685113,
"_source": {
"is_cricketer": 1,
"name": "Abraham",
"cities": [
{
"name": "Rustenburg"
},
{
"name": "Nelspruit"
},
{
"name": "East London"
}
]
}
}
]
}
I have a hard time translating aggregation query for elastic search into elastic.js. I am reading the documentation but I just can not figure it out. And the examples that you can find online are mostly about deprecated facets feature, that is not very useful.
The JSON for example aggregation is as follows:
{
"aggs": {
"foo": {
"filter": {
"bool": {
"must": [
{
"query": {
"query_string": {
"query": "*"
}
}
},
{
"terms": {
"shape": [
"wc"
]
}
}
]
}
},
"aggs": {
"field": {
"terms": {
"field": "shape",
"size": 10,
"exclude": {
"pattern": []
}
}
}
}
}
},
"size": 0
}
This is how you would nest terms aggregation into filter aggregation with elasticjs
ejs.Request()
.size(0)
.agg(ejs.FilterAggregation("foo").filter(ejs.BoolFilter()
.must(ejs.TermsFilter('shape', 'wc'))
.must(ejs.QueryFilter(ejs.QueryStringQuery().query("*"))))
.agg(ejs.TermsAggregation("field").field("shape").size(10).exclude("my_pattern"))
)
BTW you are filtering on shape and then doing aggregations on it. I am not sure what exactly you are trying.
I found their documentation pretty good, Also they have a great tool to check if your query is valid and right. This would help you a lot
Hope this helps!!
It seems like you have misplaced your query under the aggs element. Try this:
{
"size": 0,
"query": {
"bool": {
"must": [
{
"query": {
"query_string": {
"query": "*"
}
}
},
{
"terms": {
"shape": [
"wc"
]
}
}
]
}
},
"aggs": {
"foo": {
"terms": {
"field": "shape",
"size": 10,
"exclude": {
"pattern": []
}
}
}
}
}