Description:

Fetches files from an SFTP Server and creates FlowFiles from them

Tags:

sftp, get, retrieve, files, fetch, remote, ingest, source, input

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the Expression Language Guide, and whether a property is considered “sensitive”, meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

Name Default Value Allowable Values Description
Hostname The fully qualified hostname or IP address of the remote system
Supports Expression Language: true
Port 22 The port that the remote system is listening on for file transfers
Username Username
Password Password for the user account
Sensitive Property: true
Private Key Path The fully qualified path to the Private Key file
Private Key Passphrase Password for the private key
Sensitive Property: true
Remote Path The path on the remote system from which to pull or push files
Supports Expression Language: true
File Filter Regex Provides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetched
Path Filter Regex When Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scanned
Polling Interval 60 sec Determines how long to wait between fetching the listing for new files
Search Recursively false * true
* false
If true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectories
Ignore Dotted Files true * true
* false
If true, files whose names begin with a dot (".") will be ignored
Delete Original true * true
* false
Determines whether or not the file is deleted from the remote system after it has been successfully transferred
Connection Timeout 30 sec Amount of time to wait before timing out while creating a connection
Data Timeout 30 sec When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems
Host Key File If supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used
Max Selects 100 The maximum number of files to pull in a single connection
Remote Poll Batch Size 5000 The value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.
Strict Host Key Checking false * true
* false
Indicates whether or not strict enforcement of hosts keys should be applied
Send Keep Alive On Timeout true * true
* false
Indicates whether or not to send a single Keep Alive message when SSH socket times out
Use Compression false * true
* false
Indicates whether or not ZLIB compression should be used when transferring files
Use Natural Ordering false * true
* false
If true, will pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not defined

Relationships:

Name Description
success All FlowFiles that are received are routed to success

Reads Attributes:

None specified.

Writes Attributes:

Name Description
filename The filename is set to the name of the file on the remote server
path The path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursively> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3
file.lastModifiedTime The date and time that the source file was last modified
file.owner The numeric owner id of the source file
file.group The numeric group id of the source file
file.permissions The read/write/execute permissions of the source file
absolute.path The full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative path

See Also:

PutSFTP