“Tails” a file, ingesting data from the file as it is written to the file. The file is expected to be textual. Data is ingested only when a new line is encountered (carriage return or new-line character or combination). If the file to tail is periodically “rolled over”, as is generally the case with log files, an optional Rolling Filename Pattern can be used to retrieve data from files that have rolled over, even if the rollover occurred while NiFi was not running (provided that the data still exists upon restart of NiFi). It is generally advisable to set the Run Schedule to a few seconds, rather than running with the default value of 0 secs, as this Processor will consume a lot of resources if scheduled very aggressively. At this time, this Processor does not support ingesting files that have been compressed when ‘rolled over’.
tail, file, log, text, source
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.
|Name||Default Value||Allowable Values||Description|
|File to Tail||Fully-qualified filename of the file that should be tailed|
|Rolling Filename Pattern||If the file to tail "rolls over" as would be the case with log files, this filename pattern will be used to identify files that have rolled over so that if NiFi is restarted, and the file has rolled over, it will be able to pick up where it left off. This pattern supports wildcard characters * and ? and will assume that the files that have rolled over live in the same directory as the file being tailed.|
|State File||Specifies the file that should be used for storing state about what data has been ingested so that upon restart NiFi can resume from where it left off|
|Initial Start Position||Beginning of File||
*Beginning of Time
*Beginning of File
|When the Processor first begins to tail data, this property specifies where the Processor should begin reading data. Once data has been ingested from the file, the Processor will continue from the last point from which it has received data.|
|Specifies where the file is located, so that state can be stored appropriately in order to ensure that all data is consumed without duplicating data upon restart of NiFi|
|success||All FlowFiles are routed to this Relationship.|