Is it possible to analyze a field that has already been analyzed?
For example, suppose we broke down /Health & Beauty/Vitamins & Supplements/Supplements into the following using a custom analysis with a hierarchical token:
/Health & Beauty
/Health & Beauty/Vitamins & Supplements
/Health & Beauty/Vitamins & Supplements/Supplements
Would it be possible to then run a separate analysis on each new string and store the results with the corresponding string?
How would we do that with the following mapping:
PUT /my_index
{
"settings": {
"analysis": {
"analyzer": {
"path-analyzer": {
"type": "custom",
"tokenizer": "path-tokenizer"
},
"url-analyzer": {
"type": "custom",
"char_filter" : ["urlFormat"],
"filter": ["lowercase"],
"tokenizer": "path-tokenizer"
},
"cat-analyzer": {
"type": "custom",
"char_filter" : ["catName"],
"tokenizer": "keyword"
}
},
"char_filter" : {
"urlFormat":{
"type":"pattern_replace",
"pattern":"[^a-z|A-Z|/]+",
"replacement":"-"
},
"catName":{
"type":"pattern_replace",
"pattern":"[^/]+/",
"replacement":""
}
},
"tokenizer": {
"path-tokenizer": {
"type": "path_hierarchy"
}
}
}
},
"mappings": {
"my_type": {
"dynamic": "strict",
"properties": {
"group_path": {
"type": "string",
"index_analyzer": "url-analyzer",
"search_analyzer": "keyword",
"fields": {
"name": {
"type": "string",
"index_analyzer": "cat-analyzer",
"search_analyzer": "keyword"
}
}
}
}
}
}
}
Thank you for your consideration
Related
I am working on a Builder for a typescript sdk library and I am trying to extract all the sdk classes with their respective functions into a json that the Builder can use.
I am trying to accomplish this:
class Form {
propA: string
propB: string
funcA(value: boolean, text: string) {
...
}
}
to
{
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Form": {
"type": "object",
"properties": {
"propA": {
"type": "string"
},
"propB": {
"type": "string"
},
"NamedParameters<typeof funcA>": {
"type": "object",
"properties": {
"value": {
"type": "boolean"
},
"text": {
"type": "string"
}
},
"required": [
"value",
"text"
],
"additionalProperties": false
}
},
"additionalProperties": false
}
}
}
But there seems to be no way to accomplish this. Does anyone know an alternative or a solution?
I'm trying to make a data request from a URL based on the data provider's API specification.
My app is built using the Node.JS and NPM.
I'm subscribed to use the API and I've verified that all my identity credentials, policies and IAM keys that give me permission to access the API.
Could someone please help me define the correct value to place inside the 'Body' parameter of my API request based on the API specification provided by the URL.
Thanks!
My current request parameter code looks like below:
const request_parameter = {
DataSetId: "xxxxxac09b4xxxxxxxxxx",
RevisionId: "xxxxxxc8df94fxxxxxxx",
AssetId: "xxxxxxx2bde1fa3xxxxxxx",
Method: "POST",
Path: "/pxxx/graphql",
QueryStringParameters: {
param1: " ",
param2: " "
},
RequestHeaders: {
"Content-Type": "application/json"
},
Body: {\"body_param\":\"body_param_value\"} // FIX ME
};
The data provider's actual API specification looks like below:
{
"openapi": "3.0.1",
"info": {
"title": "image-services",
"description": "API Gateway for integration of Platform Services into AWS Data Exchange",
"version": "2022-03-03T20:04:18Z"
},
"servers": [
{
"url": ""
}
],
"paths": {
"/psdm/graphql": {
"post": {
"requestBody": {
"description": "GraphQL query for the Catalog in the form of:\n\n `{\n availablemetadatarecord(\n QUERYPARAMS\n ) { \n FIELDLIST\n } \n }`\n",
"content": {
"application/graphql": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchRequest"
},
"examples": {
"Limit returned data": {
"description": "Limit the number of records in a response.\n\nIn your query, simply specify `limit: <int>` \n",
"value": "{\n availablemetadatarecord(\n limit: 10\n content: { catalogProperties: { vendor: { eq: \"EYE\" } } }\n ) {\n assignedId\n }\n}\n"
}
}
}
},
"required": true
},
"responses": {
"200": {
"description": "200 response",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchResponse"
}
}
}
}
},
"security": [
{
"sigv4": []
}
]
}
}
},
"components": {
"schemas": {
"CatalogSearchRequest": {
"title": "CatalogSearchRequest",
"type": "object"
},
"CatalogSearchResponse": {
"title": "Catalog Search Response",
"required": [
"data",
"paginationInfo"
],
"type": "object",
"properties": {
"paginationInfo": {
"type": "object",
"properties": {
"recordsInPage": {
"type": "integer"
},
"nextOffset": {
"type": "integer"
}
}
},
"data": {
"type": "object",
"properties": {
"availablemetadatarecord": {
"type": "array",
"description": "Records returned by query",
"items": {
"type": "object"
}
}
}
}
}
}
},
"securitySchemes": {
"sigv4": {
"type": "apiKey",
"name": "Authorization",
"in": "header",
"x-amazon-apigateway-authtype": "awsSigv4"
}
}
}
}
I'm trying to write a GraphQL query to fetch data from an AWS DataExchange provider based on their company's API specification.
However, every query I've written so far isn't returning any data.
I've verified that all my AWS credentials and IAM keys are correct.
Could you please help me write the correct query based on the API specification given by the data provider.
My current request 'body' looks like below:
Body: JSON.stringify({
content: {
"application/graphql": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchRequest"
},
query: `query availablemetadatarecord(limit: 10 content: { catalogProperties: { vendor: { eq: "EYE" } } }) {
assignedId
}
}`
}
}
}),
The DataExchange provider's actual API specification looks like below.
{
"openapi": "3.0.1",
"info": {
"title": "image-services",
"description": "API Gateway for integration of Platform Services into AWS Data Exchange",
"version": "2022-03-03T20:04:18Z"
},
"servers": [
{
"url": ""
}
],
"paths": {
"/psdm/graphql": {
"post": {
"requestBody": {
"description": "GraphQL query for the Catalog in the form of:\n\n `{\n availablemetadatarecord(\n QUERYPARAMS\n ) { \n FIELDLIST\n } \n }`\n",
"content": {
"application/graphql": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchRequest"
},
"examples": {
"Limit returned data": {
"description": "Limit the number of records in a response.\n\nIn your query, simply specify `limit: <int>` \n",
"value": "{\n availablemetadatarecord(\n limit: 10\n content: { catalogProperties: { vendor: { eq: \"EYE\" } } }\n ) {\n assignedId\n }\n}\n"
}
}
}
},
"required": true
},
"responses": {
"200": {
"description": "200 response",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CatalogSearchResponse"
}
}
}
}
},
"security": [
{
"sigv4": []
}
]
}
}
},
"components": {
"schemas": {
"CatalogSearchRequest": {
"title": "CatalogSearchRequest",
"type": "object"
},
"CatalogSearchResponse": {
"title": "Catalog Search Response",
"required": [
"data",
"paginationInfo"
],
"type": "object",
"properties": {
"paginationInfo": {
"type": "object",
"properties": {
"recordsInPage": {
"type": "integer"
},
"nextOffset": {
"type": "integer"
}
}
},
"data": {
"type": "object",
"properties": {
"availablemetadatarecord": {
"type": "array",
"description": "Records returned by query",
"items": {
"type": "object"
}
}
}
}
}
}
},
"securitySchemes": {
"sigv4": {
"type": "apiKey",
"name": "Authorization",
"in": "header",
"x-amazon-apigateway-authtype": "awsSigv4"
}
}
}
}
I am trying to add a search function for users in my app. When I type the word "Josh" in the search bar people with the name "Joshua" do not show up. I have type the full name "Joshua". How can I add this look-ahead functionality?
Here is my current query:
"bool": {
"must": [
{
"multi_match": {
"fields": [
"first",
"first.edge",
"last",
"last.edge",
"title",
"title.edge"
],
"query": search,
"fuzziness": 1,
}
}
]
}
One way to solve it is using ngram.
PUT ngram_custom_example
{
"settings": {
"analysis": {
"analyzer": {
"ngram_analyzer": {
"tokenizer": "standard",
"filter": [ "3_5_grams" ]
}
},
"filter": {
"3_5_grams": {
"type": "edge_ngram",
"min_gram": 2,
"max_gram": 5
}
}
}
},
"mappings": {
"properties": {
"name":{
"type": "text",
"analyzer": "ngram_analyzer"
}
}
}
}
POST ngram_custom_example/_bulk
{"index":{}}
{"name":"josh"}
{"index":{}}
{"name":"joshua"}
GET ngram_custom_example/_search
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"fields": [
"name"
],
"query": "josh"
}
}
]
}
}
}
Given:
Say that I am defining a schema for Contacts. But, I can have "Primary Contact", "Student" or one who is both; and different properties that go with all three choices. The contact types are defined in an array of contact_type: [ "Primary Contact", "Student" ] which can be either one, or both.
Say that the fields are as such per contact type:
If Primary Contact, then I want phone_number
If Student, then I want first_name
If Student and Primary Contact then I want phone_number and first_name
Usage
I use Ajv library to validate in Node.js using a code like such:
function validator(json_schema){
const Ajv = require('ajv');
const ajv = new Ajv({allErrors: true});
return ajv.compile(json_schema)
}
const validate = validator(json_schema);
const valid = validate(input);
console.log(!!valid); //true or false
console.log(validate.errors)// object or null
Note: I've had trouble with allErrors: true while using anyOf for this, and I use the output of allErrors to return ALL the missing/invalid fields back to the user rather than returning problems one at a time. Reference: https://github.com/ajv-validator/ajv/issues/980
Schema
I have written the following schema and it works if I do either "Student" or "Primary Contact" but when I pass both, it still wants to validate against ["Student"] or ["Primary Contact"] rather than both.
{
"$schema": "http://json-schema.org/draft-07/schema",
"type": "object",
"required": [],
"properties": {},
"allOf": [
{
"if": {
"properties": {
"contact_type": {
"contains": {
"allOf": [
{
"type": "string",
"const": "Primary Contact"
},
{
"type": "string",
"const": "Student"
}
]
}
}
}
},
"then": {
"additionalProperties": false,
"properties": {
"contact_type": {
"type": "array",
"items": [
{
"type": "string",
"enum": [
"Student",
"Primary Contact"
]
}
]
},
"phone": {
"type": "string"
},
"first_name": {
"type": "string"
}
},
"required": [
"phone",
"first_name"
]
}
},
{
"if": {
"properties": {
"contact_type": {
"contains": {
"type": "string",
"const": "Student"
}
}
}
},
"then": {
"additionalProperties": false,
"properties": {
"contact_type": {
"type": "array",
"items": [
{
"type": "string",
"enum": [
"Student",
"Primary Contact"
]
}
]
},
"first_name": {
"type": "string"
}
},
"required": [
"first_name"
]
}
},
{
"if": {
"properties": {
"contact_type": {
"contains": {
"type": "string",
"const": "Primary Contact"
}
}
}
},
"then": {
"additionalProperties": false,
"properties": {
"contact_type": {
"type": "array",
"items": [
{
"type": "string",
"enum": [
"Student",
"Primary Contact"
]
}
]
},
"phone": {
"type": "string"
}
},
"required": [
"phone"
]
}
}
]
}
Example Valid Inputs:
For just ["Primary Contact"]:
{
"contact_type":["Primary Contact"],
"phone":"something"
}
For just ["Student"]:
{
"contact_type":["Student"],
"first_name":"something"
}
For ["Primary Contact", "Student"]
{
"contact_type":["Primary Contact", "Student"],
"phone":"something",
"first_name":"something"
}
Question:
I would like this to validate even if allErrors: true, is this possible? If not, how should I change the schema?
Footnotes
I don't want to change the "contact_type" from being an array unless it is the last resort. (it is a requirement, but can be broken only if there's no other way)
I can't allow any additionalItems, therefore I'm fully defining each object in the if statements although contact_type is common. If I move contact_type out, then I get error messages about passing contact_type as an additionalItem (it looks at the if statement's properties and doesn't see contact_type when it's taken out to the common place). This is why my initial properties object is empty.
Here's how I might go about solving the validation issue: https://jsonschema.dev/s/XLSDB
Here's the Schema...
(It's easier if you try to break up concerns)
{
"$schema": "http://json-schema.org/draft-07/schema",
"type": "object",
First, we want to define our conditional checking subschemas...
"definitions": {
"is_student": {
"properties": {
"contact_type": {
"contains": {
"const": "Student"
}
}
}
},
"is_primay_contact": {
"properties": {
"contact_type": {
"contains": {
"const": "Primary Contact"
}
}
}
}
},
Next, I'm assuming you always want contact_type
"required": ["contact_type"],
"properties": {
"contact_type": {
"type": "array",
"items": {
"enum": ["Primary Contact", "Student"]
}
},
And we need to define all the allowed properties in order to prevent additional properties. (draft-07 cannot "see through" applicator keywords like allOf. You can with draft 2019-09 and beyond, but that's another story)
"phone": true,
"first_name": true
},
"additionalProperties": false,
Now, we need to define our structural constraints...
"allOf": [
{
If the contact is a student, first name is required.
"if": { "$ref": "#/definitions/is_student" },
"then": { "required": ["first_name"] }
},
{
If the contact is a primary contact, then phone is required.
"if": { "$ref": "#/definitions/is_primay_contact" },
"then": { "required": ["phone"] }
},
{
However, additionally, if the contact is both a student and a primary contact...
"if": {
"allOf": [
{ "$ref": "#/definitions/is_student" },
{ "$ref": "#/definitions/is_primay_contact" }
]
},
Then we require both phone and first name...
"then": {
"required": ["phone", "first_name"]
},
Otherwise, one of phone or first name is fine (which one is covered by the previous section)
"else": {
"oneOf": [
{
"required": ["phone"]
},
{
"required": ["first_name"]
}
]
}
}
]
}
I'm not convinced this is the cleanest approach, but it does work for the requirements you've provided.
As for getting validation errors you can pass back to your END user... given the conditional requirements you lay out, it's not something you can expect with pure JSON Schema...
Having said that, ajv does provide an extension to add custom error messages, which given the way I've broken the validation down into concerns, might be useable to add custom errors as you're looking to do (https://github.com/ajv-validator/ajv-errors).