Description and usage of JsonQueryElasticsearch processor:
A processor that allows the user to run a query (with aggregations) written with the ElasticSearch JSON DSL. It does not automatically paginate queries for the user. If an incoming relationship is added to this processor, it will use the flowfile’s content for the query. Care should be taken on the size of the query because the entire response from ElasticSearch will be loaded into memory all at once and converted into the resulting flowfiles.
Tags:
elasticsearch, elasticsearch 5, query, read, get, json
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the Expression Language Guide.
Name |
Default Value |
Allowable Values |
Description |
Query |
A query in JSON syntax, not Lucene syntax. Ex: {"query":{"match":{"somefield":"somevalue"}}}. If this parameter is not set, the query will be read from the flowfile content. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Query Attribute |
If set, the executed query will be set on each result flowfile in the specified attribute. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Index |
The name of the index to use. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Type |
The type of this document (used by Elasticsearch for indexing and searching) Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Client Service |
Controller Service API: ElasticSearchClientService Implementation: ElasticSearchClientServiceImpl |
An ElasticSearch client service to use for running queries. | |
Split up search results |
splitUp-no |
* No ![]() * Yes ![]() |
Split up search results into one flowfile per result. |
Split up aggregation results |
splitUp-no |
* No ![]() * Yes ![]() |
Split up aggregation results into one flowfile per result. |
Relationships:
Name |
Description |
aggregations | Aggregations are routed to this relationship. |
failure | All FlowFiles that cannot be read from Elasticsearch are routed to this relationship |
original | All original flowfiles that don't cause an error to occur go to this relationship. This applies even if you select the "split up hits" option to send individual hits to the "hits" relationship. |
hits | Search hits are routed to this relationship. |
Reads Attributes:
None specified.
Writes Attributes:
Name |
Description |
mime.type | application/json |
aggregation.name | The name of the aggregation whose results are in the output flowfile |
State management:
This component does not store state.
Restricted:
This component is not restricted.
Input requirement:
This component requires an incoming relationship.
System Resource Considerations:
None specified.
Summary:
This processor is intended for use with the ElasticSearch JSON DSL and ElasticSearch 5.X and newer. It is designed to be able to take a query from Kibana and execute it as-is against an ElasticSearch cluster. Like all processors in the “rest api” bundle, it uses the official Elastic client APIs, so it supports leader detection.
The query to execute can be provided either in the Query configuration property or in an attribute on a flowfile. In the latter case, the name of the attribute (Expression Language is supported here) must be provided in the Query Attribute property.
Additionally, search results and aggregation results can be split up into multiple flowfiles. Aggregation results will only be split at the top level because nested aggregations lose their context (and thus lose their value) if separated from their parent aggregation. The following is an example query that would be accepted:
{
"query": {
"match": {
"restaurant.keyword": "Local Pizzas FTW Inc"
}
},
"aggs": {
"weekly_sales": {
"date_histogram": {
"field": "date",
"interval": "week"
},
"aggs": {
"items": {
"terms": {
"field": "product",
"size": 10
}
}
}
}
}
}