Description:
Lists blobs in an Azure Storage container. Listing details are attached to an empty FlowFile for use with FetchAzureBlobStorage. This Processor is designed to run on Primary Node only in a cluster. If the primary node changes, the new Primary Node will pick up where the previous node left off without duplicating all of the data.
Tags:
azure, microsoft, cloud, storage, blob
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered “sensitive”, meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.
| Name | Default Value | Allowable Values | Description |
| Storage Account Name | The storage account name. There are certain risks in allowing the account name to be stored as a flow file attribute. While it does provide for a more flexible flow by allowing the account name to be fetched dynamically from a flow file attribute, care must be taken to restrict access to the event provenance data (e.g. by strictly controlling the policies governing provenance for this Processor). In addition, the provenance repositories may be put on encrypted disk partitions.</br> Sensitive Property: true</br> Supports Expression Language: true | ||
| Storage Account Key | The storage account key. There are certain risks in allowing the account key to be stored as a flow file attribute. While it does provide for a more flexible flow by allowing the account key to be fetched dynamically from a flow file attribute, care must be taken to restrict access to the event provenance data (e.g. by strictly controlling the policies governing provenance for this Processor). In addition, the provenance repositories may be put on encrypted disk partitions.</br> Sensitive Property: true</br> Supports Expression Language: true | ||
| Container name | Name of the azure storage container.</br> Supports Expression Language: true | ||
| Prefix | Search prefix for listing</br> Supports Expression Language: true |
Relationships:
| Name | Description |
| success | All FlowFiles that are received are routed to success |
Reads Attributes:
None specified.
Writes Attributes:
| Name | Description |
| azure.container | The name of the Azure container |
| azure.blobname | The name of the Azure blob |
| azure.primaryUri | Primary location for blob content |
| azure.secondaryUri | Secondary location for blob content |
| azure.etag | Etag for the Azure blob |
| azure.length | Length of the blob |
| azure.timestamp | The timestamp in Azure for the blob |
| mime.type | MimeType of the content |
| lang | Language code for the content |
| azure.blobtype | This is the type of blob and can be either page or block type |
State management:
| Scope | Description |
| CLUSTER | After performing a listing of blobs, the timestamp of the newest blob is stored. This allows the Processor to list only blobs that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node can pick up where the previous node left off, without duplicating the data. |
Restricted:
This component is not restricted.
Input requirement:
This component does not allow an incoming relationship.
See Also:
FetchAzureBlobStorage, PutAzureBlobStorage