Sends the contents of a FlowFile as a message to Apache Kafka. The messages to send may be individual FlowFiles or may be delimited, using a user-specified delimiter, such as a new-line.


Apache, Kafka, Put, Send, Message, PubSub


In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the Expression Language Guide.

Name Default Value Allowable Values Description
Known Brokers A comma-separated list of known Kafka Brokers in the format <host>:<port>
Topic Name The Kafka Topic of interest
Supports Expression Language: true
Partition Strategy Round Robin *Round Robin
Specifies how messages should be partitioned when sent to Kafka
Partition Specifies which Kafka Partition to add the message to. If using a message delimiter, all messages in the same FlowFile will be sent to the same partition. If a partition is specified but is not valid, then all messages within the same FlowFile will use the same partition but it remains undefined which partition is used.
Supports Expression Language: true
Kafka Key The Key to use for the Message
Supports Expression Language: true
Delivery Guarantee 0 *Best Effort
*Guarantee Single Node Delivery
*Guarantee Replicated Delivery
Specifies the requirement for guaranteeing that a message is sent to Kafka
Message Delimiter Specifies the delimiter (interpreted in its UTF-8 byte representation) to use for splitting apart multiple messages within a single FlowFile. If not specified, the entire content of the FlowFile will be used as a single message. If specified, the contents of the FlowFile will be split on this delimiter and each section sent as a separate Kafka message. Note that if messages are delimited and some messages for a given FlowFile are transferred successfully while others are not, the messages will be split into individual FlowFiles, such that those messages that were successfully sent are routed to the 'success' relationship while other messages are sent to the 'failure' relationship.
Supports Expression Language: true
Max Buffer Size 5 MB The maximum amount of data to buffer in memory before sending to Kafka
Max Record Size 1 MB The maximum size that any individual record can be.
Communications Timeout 30 secs The amount of time to wait for a response from Kafka before determining that there is a communications error
Batch Size 200 The number of messages to send in one batch. The producer will wait until either this number of messages are ready to send or "Queue Buffering Max Time" is reached. NOTE: This property will be ignored unless the 'Message Delimiter' property is specified.
Queue Buffering Max Time 5 secs Maximum time to buffer data before sending to Kafka. For example a setting of 100 ms will try to batch together 100 milliseconds' worth of messages to send at once. This will improve throughput but adds message delivery latency due to the buffering.
Compression Codec none * None
* Snappy
This parameter allows you to specify the compression codec for all data generated by this producer.
Client Name Client Name to use when communicating with Kafka

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.

Name Value Description
The name of a Kafka configuration property. The value of a given Kafka configuration property. These properties will be added on the Kafka configuration after loading any provided configuration properties. In the event a dynamic property represents a property that was already set as part of the static properties, its value wil be overriden with warning message describing the override. For the list of available Kafka properties please refer to:


Name Description
failure Any FlowFile that cannot be sent to Kafka will be routed to this Relationship
success Any FlowFile that is successfully sent to Kafka will be routed to this Relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.


This Processors puts the contents of a FlowFile to a Topic in Apache Kafka. The full contents of a FlowFile becomes the contents of a single message in Kafka. This message is optionally assigned a key by using the <Kafka Key> Property.

The Processor allows the user to configure an optional Message Delimiter that can be used to send many messages per FlowFile. For example, a \n could be used to indicate that the contents of the FlowFile should be used to send one message per line of text. If the property is not set, the entire contents of the FlowFile will be sent as a single message. When using the delimiter, if some messages are successfully sent but other messages fail to send, the FlowFile will be FORKed into two child FlowFiles, with the successfully sent messages being routed to ‘success’ and the messages that could not be sent going to ‘failure’.