Description and usage of GetMongoRecord processor:

A record-based version of GetMongo that uses the Record writers to write the MongoDB result set.

Tags:

mongo, mongodb, get, fetch, record, json

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the Expression Language Guide.

Name

Default Value

Allowable Values

Description

Client Service

Controller Service API: 


MongoDBClientService

Implementation: 

MongoDBControllerService


If configured, this property will use the assigned client service for connection pooling.

Record Writer

Controller Service API: 


RecordSetWriterFactory

Implementations: 

JsonRecordSetWriter


CSVRecordSetWriter


ScriptedRecordSetWriter


FreeFormTextRecordSetWriter


AvroRecordSetWriter


XMLRecordSetWriter


The record writer to use to write the result sets.

Mongo Database Name

The name of the database to use

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)


Mongo Collection Name

The name of the collection to use

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)


Schema Name

${schema.name} The name of the schema in the configured schema registry to use for the query results.

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)


Query Output Attribute If set, the query will be written to a specified attribute on the output flowfiles.

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)


Query The selection criteria to do the lookup. If the field is left blank, it will look for input from an incoming connection from another processor to provide the query as a valid JSON document inside of the FlowFile's body. If this field is left blank and a timer is enabled instead of an incoming connection, that will result in a full collection fetch using a "{}" query.

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)


Projection The fields to be returned from the documents in the result set; must be a valid BSON document

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)


Sort The fields by which to sort; must be a valid BSON document

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)


Limit The maximum number of elements to return

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)


Batch Size The number of elements to be returned from the server in one batch

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)


Relationships:

Name

Description

success All FlowFiles that have the results of a successful query execution go here.
failure All input FlowFiles that are part of a failed query execution go here.
original All input FlowFiles that are part of a successful query execution go here.

Reads Attributes:

None specified.

Writes Attributes:

Name

Description

mongo.database.name The database where the results came from.
mongo.collection.name The collection where the results came from.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.

Summary:

This processor runs queries against a MongoDB instance or cluster and writes the results to a flowfile. It allows input, but can run standalone as well. It is a record-aware version of the GetMongo processor.

Specifying the Query

The query can be specified in one of three ways:

  • Query configuration property.
  • Query Attribute configuration property.
  • FlowFile content.

If a value is specified in either of the configuration properties, it will not look in the FlowFile content for a query.

Limiting/Shaping Results

The following options for limiting/shaping results are available:

  • Limit - limit the number of results. This should not be confused with the “batch size” option which is a setting for the underlying MongoDB driver to tell it how many items to retrieve in each poll of the server.
  • Sort - sort the result set. Requires a JSON document like { “someDate”: -1 }
  • Projection - control which fields to return. Example, which would remove _id{ “_id”: 0 }

Misc Options

Results Per FlowFile, if set, creates a JSON array out of a batch of results and writes the result to the output. Pretty Print, if enabled, will format the JSON data to be easy read by a human (ex. proper indentation of fields).