I'm using searchkit to try to build a basic text search. I think the query I want to build is fairly simple. It needs to be structured like this:
{
"query":{
"bool":{
"must":[
{
"multi_match":{
"query":"test search",
"type":"phrase_prefix",
"fields":[
"field_1^5",
"field_2^4",
"field_3"
]
}
},
{
"term":
{
"field_id": "3"
}
}
],
"must_not":[
{
"term":
{
"status": "archived"
}
}
]
}
},
"size":6,
"highlight":{
"fields":{
"field_1":{},
"field_2":{},
"field_3":{}
}
}
}
I've tried using the prefixQueryFields attribute, which gave me something fairly close to what I wanted except it was using a BoolShould rather than a BoolMust, plus it was always including the default SimpleQueryString query. It looked something like this:
const prefixQueryFields = [
'field_1^5',
'field_2^4',
'field_3',
];
...
<SearchBox
searchOnChange={true}
prefixQueryFields={prefixQueryFields}
/>
I couldn't figure out the issues there easily and decided to go with the queryBuilder attribute in SearchBox. This is what I came up with:
_queryBuilder(queryString) {
const prefixQueryFields = [
'field_1^5',
'field_2^4',
'field_3',
];
return new ImmutableQuery()
.addQuery(BoolMust([
MultiMatchQuery(queryString, {
type: 'phase_prefix',
fields: prefixQueryFields,
})
]))
.addQuery(BoolMustNot([
TermQuery('status', 'archived'),
]));
}
...
<SearchBox
searchOnChange={true}
queryBuilder={this.queryBuilder}
/>
This query came out even more messed up, and I have no idea what to try next after checking the documentation and a cursory look at the source code.
(For the sake of brevity, I will not bother including the incorrect queries these two attempts created unless someone thinks that info will be useful.)
Figured it out. Using the QueryDSL structures wasn't working out very well, but apparently you can create the query with pure JSON, which worked great. Basically updated my query builder to return as so:
return {
bool: {
must: [
{
multi_match:{
query: queryString,
type: 'phrase_prefix',
fields: prefixQueryFields,
}
}
],
must_not: [
{
term: {
status: 'archived',
}
}
]
}
};
Related
I have a problem with parsing an XML file.
I want to remove strings with characters like \t\n.
XML File: http://ftp.thinkimmo.com/home/immoanzeigen24/immo.xml
{
trim: true,
normalize: true,
attrValueProcessors: [cleanValue, name => name],
valueProcessors: [cleanValue, name => name]
}
cleanValue:
const cleanValue = value => {
return value.toString().trim().replace("\t","atest");
};
I tried cleaning it with a lot of regex I've found online - but value always stays like following:
"verwaltung_objekt": {
"objektadresse_freigeben": "0",
"verfuegbar_ab": "nachaasjkdhkjshadjkashdAbsprache",
"bisdatum": "2016-01-15",
"min_mietdauer": "\n\t\t\t\t",
"max_mietdauer": "\n\t\t\t\t",
}
This is a difficult one!
I'd suggest following a simple strategy and pre-processing the xml data before you parse it.
This should resolve your issue at least.
If you just do something like:
function trimXml(xml) {
return xml.replace(/>\s+</g, "><");
}
xml = trimXml(xml);
Then parse the trimmed xml data. You should see the output now looks like so:
"verwaltung_objekt": [
{
"objektadresse_freigeben": [
"1"
],
"abdatum": [
"2017-03-01"
],
"min_mietdauer": [
""
],
"max_mietdauer": [
""
]
}
],
Which is a bit more like what you want!
I'm using ES with my node server via the package "elasticsearch": "12.1.3".
I do bulk inserts of my documents. Excerpt:
var body = [];
_.each(rows, function(doc) {
body.push({
update: {
_index: 'mytest',
_type: 'mydoc',
_id: doc.id,
_retry_on_conflict: 3
}
});
body.push({
doc: doc,
doc_as_upsert: true
});
});
client.bulk({
body: body
}, ...
On demand, to individually update documents, I have this in place:
client.index({
index: 'mytest',
type: 'mydoc',
id: doc.id,
body: doc.body
}, ...);
Everything works as expected so far. Now I'm trying to add basic 'light_english' stemming.
Looking at the Docs here
and for the JS package here
I want certain fields in my document to be "fuzzy" matched, therefore I think stemming is the way to go?
It is not clear to me how I would set this up.
Assuming I use the example settings from the link above, would this be the right way to do it:
client.cluster.putSettings({
"settings": {
"analysis": {
"filter": {
"no_stem": {
"type": "keyword_marker",
"keywords": [ "skies" ]
}
},
"analyzer": {
"my_english": {
"tokenizer": "standard",
"filter": [
"lowercase",
"no_stem",
"porter_stem"
]
}
}
}
}
});
And would this then work permanently for my two code examples above, if applied once?
Bonus question: What would be a good default analyzer plugin (or settings) I can use? My main goal is that searches for example: "Günther" would also match "gunther" and vice versa.
Might it be better to do this manually before inserting/updating documents, so that strings are lower-cased, diacritics removed etc.?
I've been stuck on this for a while. Take the following code as an example:
models.Summoner.findOne({
include: [{ model: models.RankedStats, as: 'SummonerRankedStats', required: true }],
where: { summonerId: summonerId, server: server }
}).then(function(summoner) {
models.RankedStats.create({
totalWins: 0,
totalLosses: 0
}).then(function(rankedStats) {
summoner.setSummonerRankedStats(rankedStats).then(function() {
console.log(summoner.SummonerRankedStats)
//This outputs undefined
summoner.getSummonerRankedStats().then(function(srs) {
console.log(srs)
//This outputs the RankedStats that were just created
})
models.Summoner.findOne({
include: [{ model: models.RankedStats, as: 'SummonerRankedStats', required: true }],
where: { summonerId: summonerId, server: server }
}).then(function(summoner) {
console.log(summoner.SummonerRankedStats)
//This outputs the SummonerRankedStats object
})
})
})
})
So, to put it simply... If I have a Summoner (var summoner) and perform a .setAssociation() or .createAssociation() on it, and then log summoner, the data created isn't there. If I fetch it again from the database (with .getAssociation() or by searching for that Summoner again) I can access it, but I was hoping to avoid that extra DB call.
Is there a way to add this information to the original object when using .create() or .set()? It can be achieved by doing something like:
summoner.dataValues.SummonerRankedStats = rankedStats
But that seems somewhat hacky :)
Is there a correct way to do it, or does it even make any sense?
Thanks in advance!
I am trying to execute a search query with search_type of count with the elasticsearch.angular.js build from the npm module.
I can execute the query like this:
POST /index1/type1/_search?search_type=count
{
"aggs": {
"reviews": {
"nested": {
"path": "reviews"
}
}
}
}
but when I try to translate the query to the .js api, I get an error. My code looks like this:
var requestObject = {
index:'index1',
type:'type1',
searchType: 'count',
body: {
query:{
aggs: {
reviews: {
nested: {
path: "reviews"
}
}
}
}
};
esClient.search(requestObject)
The trace looks like this:
console.js:1 DEBUG: 2015-08-04T15:28:59Z
starting request { method: 'POST',
path: '/index1/type1/_search',
body: { aggs: { reviews: [Object] } },
query: { search_type: 'count' } }
That looks OK to an elasticsearch newbie, but the request completes with an error: ReferenceError: count is not defined.
What am I missing here please?
It turned out that my problem was a stupid error on my part (thanks #robertklep for pointing it out). The code above actually works correctly. As I was not able to find an example of using using searchType from the api, I am leaving this here in the hope it will be useful to somebody else.
Currently I am using the javascript library Clusterfck to perform a data grouping. This library generates an array in the following format:
[
{"canonical":[20,120,102],
"size":1
},
{"canonical":[250,255,253],
"left":{
"canonical":[250,255,253],
"size":1
},
"right":{
"canonical":[255,255,240],
"size":1
},
"size":2
},
{"canonical":[100,54,300],
"size":1
}
]
But javascript libraries for data visualization how D3.js and Jit using JSON structures as shown below:
{
"canonical":"Pai",
"children":[
{"canonical":[20,120,102], "size":1},
{
"canonical":[250,255,253],
"children": [
{"canonical":[250,255,253], "size":1},
{"canonical":[255,255,240], "size":1}
],
"size":2
},
{
"canonical":[100,54,300],
"size":1
}
]
}
I would like to convert these structures, using JavaScript or PHP. Could anyone help me?
If you just want to convert between these structures, this is pretty simple. See working fiddle: http://jsfiddle.net/nrabinowitz/vuk94/
function convert(input, rootName) {
// top level
if (Array.isArray(input)) {
return {
"canonical": rootName,
"children": input.map(convert)
};
}
// node
else {
['left', 'right'].forEach(function(side) {
if (input[side]) {
input.children = input.children || [];
input.children.push(convert(input[side]));
delete input[side];
}
});
return input;
}
}
This uses a couple of ECMAScript 1.5 features (Array.isArray, forEach, and map), but if you're using d3 you're probably targeting browsers that support this anyway.