Description and usage of JsonQueryElasticsearch processor:

A processor that allows the user to run a query (with aggregations) written with the ElasticSearch JSON DSL. It does not automatically paginate queries for the user. If an incoming relationship is added to this processor, it will use the flowfile’s content for the query. Care should be taken on the size of the query because the entire response from ElasticSearch will be loaded into memory all at once and converted into the resulting flowfiles.

Tags:

elasticsearch, elasticsearch 5, query, read, get, json

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the Expression Language Guide.

Name	Default Value	Allowable Values	Description
Query			A query in JSON syntax, not Lucene syntax. Ex: {"query":{"match":{"somefield":"somevalue"}}}. If this parameter is not set, the query will be read from the flowfile content. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Query Attribute			If set, the executed query will be set on each result flowfile in the specified attribute. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Index			The name of the index to use. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Type			The type of this document (used by Elasticsearch for indexing and searching) Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Client Service		Controller Service API:  ElasticSearchClientService Implementation:  ElasticSearchClientServiceImpl	An ElasticSearch client service to use for running queries.
Split up search results	splitUp-no	* No  * Yes	Split up search results into one flowfile per result.
Split up aggregation results	splitUp-no	* No  * Yes	Split up aggregation results into one flowfile per result.

Relationships:

Name	Description
aggregations	Aggregations are routed to this relationship.
failure	All FlowFiles that cannot be read from Elasticsearch are routed to this relationship
original	All original flowfiles that don't cause an error to occur go to this relationship. This applies even if you select the "split up hits" option to send individual hits to the "hits" relationship.
hits	Search hits are routed to this relationship.

Reads Attributes:

None specified.

Writes Attributes:

Name	Description
mime.type	application/json
aggregation.name	The name of the aggregation whose results are in the output flowfile

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.

Summary:

This processor is intended for use with the ElasticSearch JSON DSL and ElasticSearch 5.X and newer. It is designed to be able to take a query from Kibana and execute it as-is against an ElasticSearch cluster. Like all processors in the “rest api” bundle, it uses the official Elastic client APIs, so it supports leader detection.

The query to execute can be provided either in the Query configuration property or in an attribute on a flowfile. In the latter case, the name of the attribute (Expression Language is supported here) must be provided in the Query Attribute property.

Additionally, search results and aggregation results can be split up into multiple flowfiles. Aggregation results will only be split at the top level because nested aggregations lose their context (and thus lose their value) if separated from their parent aggregation. The following is an example query that would be accepted:

{
        "query": {
            "match": {
                "restaurant.keyword": "Local Pizzas FTW Inc"
            }
        },
        "aggs": {
            "weekly_sales": {
              "date_histogram": {
                "field": "date",
                "interval": "week"
              },
              "aggs": {
                "items": {
                  "terms": {
                    "field": "product",
                    "size": 10
                  }
                }
              }
            }
        }
}