Description:
Write FlowFile data to Hadoop Distributed File System (HDFS)
Tags:
hadoop, HDFS, put, copy, filesystem
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the Expression Language Guide.
| Name | Default Value | Allowable Values | Description |
| Hadoop Configuration Resources | A file or comma separated list of files which contains the Hadoop file system configuration. Without this, Hadoop will search the classpath for a 'core-site.xml' and 'hdfs-site.xml' file or will revert to a default configuration. | ||
| Kerberos Principal | Kerberos principal to authenticate as. Requires nifi.kerberos.krb5.file to be set in your nifi.properties | ||
| Kerberos Keytab | Kerberos keytab associated with the principal. Requires nifi.kerberos.krb5.file to be set in your nifi.properties | ||
| Kerberos Relogin Period | 4 hours | Period of time which should pass before attempting a kerberos relogin | |
| Directory |
The parent HDFS directory to which files should be written Supports Expression Language: true |
||
| Conflict Resolution Strategy | fail |
* replace * ignore * fail |
Indicates what should happen when a file with the same name already exists in the output directory |
| Block Size | Size of each block as written to HDFS. This overrides the Hadoop Configuration | ||
| IO Buffer Size | Amount of memory to use to buffer file contents during IO. This overrides the Hadoop Configuration | ||
| Replication | Number of times that HDFS will replicate each file. This overrides the Hadoop Configuration | ||
| Permissions umask | A umask represented as an octal number which determines the permissions of files written to HDFS. This overrides the Hadoop Configuration dfs.umaskmode | ||
| Remote Owner | Changes the owner of the HDFS file to this value after it is written. This only works if NiFi is running as a user that has HDFS super user privilege to change owner | ||
| Remote Group | Changes the group of the HDFS file to this value after it is written. This only works if NiFi is running as a user that has HDFS super user privilege to change group | ||
| Compression codec | NONE |
*NONE *DEFAULT *BZIP *GZIP *LZ4 *SNAPPY *AUTOMATIC |
No Description Provided. |
Relationships:
| Name | Description |
| failure | Files that could not be written to HDFS for some reason are transferred to this relationship |
| success | Files that have been successfully written to HDFS are transferred to this relationship |
Reads Attributes:
None specified.
Writes Attributes:
| Name | Description |
| filename | The name of the file written to HDFS comes from the value of this attribute. |
See Also:
GetHDFS