Description and usage of GetMongoRecord processor:
A record-based version of GetMongo that uses the Record writers to write the MongoDB result set.
Tags:
mongo, mongodb, get, fetch, record, json
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the Expression Language Guide.
Name |
Default Value |
Allowable Values |
Description |
Client Service |
Controller Service API: MongoDBClientService Implementation: MongoDBControllerService |
If configured, this property will use the assigned client service for connection pooling. | |
Record Writer |
Controller Service API: RecordSetWriterFactory Implementations: JsonRecordSetWriter CSVRecordSetWriter ScriptedRecordSetWriter FreeFormTextRecordSetWriter AvroRecordSetWriter XMLRecordSetWriter |
The record writer to use to write the result sets. | |
Mongo Database Name |
The name of the database to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Mongo Collection Name |
The name of the collection to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Schema Name |
${schema.name} |
The name of the schema in the configured schema registry to use for the query results. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
|
Query Output Attribute |
If set, the query will be written to a specified attribute on the output flowfiles. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Query |
The selection criteria to do the lookup. If the field is left blank, it will look for input from an incoming connection from another processor to provide the query as a valid JSON document inside of the FlowFile's body. If this field is left blank and a timer is enabled instead of an incoming connection, that will result in a full collection fetch using a "{}" query. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Projection |
The fields to be returned from the documents in the result set; must be a valid BSON document Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Sort |
The fields by which to sort; must be a valid BSON document Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Limit |
The maximum number of elements to return Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
||
Batch Size |
The number of elements to be returned from the server in one batch Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
Relationships:
Name |
Description |
success | All FlowFiles that have the results of a successful query execution go here. |
failure | All input FlowFiles that are part of a failed query execution go here. |
original | All input FlowFiles that are part of a successful query execution go here. |
Reads Attributes:
None specified.
Writes Attributes:
Name |
Description |
mongo.database.name | The database where the results came from. |
mongo.collection.name | The collection where the results came from. |
State management:
This component does not store state.
Restricted:
This component is not restricted.
Input requirement:
This component requires an incoming relationship.
System Resource Considerations:
None specified.
Summary:
This processor runs queries against a MongoDB instance or cluster and writes the results to a flowfile. It allows input, but can run standalone as well. It is a record-aware version of the GetMongo processor.
Specifying the Query
The query can be specified in one of three ways:
- Query configuration property.
- Query Attribute configuration property.
- FlowFile content.
If a value is specified in either of the configuration properties, it will not look in the FlowFile content for a query.
Limiting/Shaping Results
The following options for limiting/shaping results are available:
- Limit - limit the number of results. This should not be confused with the “batch size” option which is a setting for the underlying MongoDB driver to tell it how many items to retrieve in each poll of the server.
- Sort - sort the result set. Requires a JSON document like { “someDate”: -1 }
- Projection - control which fields to return. Example, which would remove _id: { “_id”: 0 }
Misc Options
Results Per FlowFile, if set, creates a JSON array out of a batch of results and writes the result to the output. Pretty Print, if enabled, will format the JSON data to be easy read by a human (ex. proper indentation of fields).