Description and usage of InvokeHttp processor:
An HTTP client processor which can interact with a configurable HTTP Endpoint. The destination URL and HTTP Method are configurable. FlowFile attributes are converted to HTTP headers and the FlowFile contents are included as the body of the request (if the HTTP Method is PUT or POST).
Tags:
http, https, rest, client
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the Expression Language Guide, and whether a property is considered “sensitive”, meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.
| Name | Default Value | Allowable Values | Description |
| HTTP Method | GET |
HTTP request method (GET, POST, PUT, DELETE, HEAD, OPTIONS). Arbitrary methods are also supported. Methods other than POST and PUT will be sent without a message body. Supports Expression Language: true |
|
| Remote URL |
Remote URL which will be connected to, including scheme, host, port, path. Supports Expression Language: true |
||
| SSL Context Service |
Controller Service API: SSLContextService Implementation: StandardSSLContextService |
The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections. | |
| Connection Timeout | 5 secs | Max wait time for connection to remote service. | |
| Read Timeout | 15 secs | Max wait time for response from remote service. | |
| Include Date Header | True |
* True * False |
Include an RFC-2616 Date header in the request. |
| Follow Redirects | True |
* True * False |
Follow HTTP redirects issued by remote server. |
| Attributes to Send | Regular expression that defines which attributes to send as HTTP headers in the request. If not defined, no attributes are sent as headers. Also any dynamic properties set will be sent as headers. The dynamic property key will be the header key and the dynamic property value will be interpreted as expression language will be the header value. | ||
| Basic Authentication Username | The username to be used by the client to authenticate against the Remote URL. Cannot include control characters (0-31), ':', or DEL (127). | ||
| Basic Authentication Password |
The password to be used by the client to authenticate against the Remote URL. Sensitive Property: true |
||
| Proxy Host | The fully qualified hostname or IP address of the proxy server | ||
| Proxy Port | The port of the proxy server | ||
| Put Response Body In Attribute |
If set, the response body received back will be put into an attribute of the original FlowFile instead of a separate FlowFile. The attribute key to put to is determined by evaluating value of this property. Supports Expression Language: true |
||
| Max Length To Put In Attribute | 256 | If routing the response body to an attribute of the original (by setting the "Put response body in attribute" property or by receiving an error status code), the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will be truncated to fit. Consider making this smaller if able. | |
| Use Digest Authentication | false |
* true * false |
Whether to communicate with the website using Digest Authentication. 'Basic Authentication Username' and 'Basic Authentication Password' are used for authentication. |
| Always Output Response | false |
* true * false |
Will force a response FlowFile to be generated and routed to the 'Response' relationship regardless of what the server status code received is or if the processor is configured to put the server response body in the request attribute. In the later configuration a request FlowFile with the response body in the attribute and a typical response FlowFile will be emitted to their respective relationships. |
| Trusted Hostname | Bypass the normal truststore hostname verifier to allow the specified remote hostname as trusted. Enabling this property has MITM security implications, use wisely. Will still accept other connections based on the normal truststore hostname verifier. Only valid with SSL (HTTPS) connections. | ||
| Add Response Headers to Request | false |
* true * false |
Enabling this property saves all the response headers to the original request. This may be when the response headers are needed but a response is not generated due to the status code received. |
| Content-Type | ${mime.type} |
The Content-Type to specify for when content is being transmitted through a PUT or POST. In the case of an empty value after evaluating an expression language expression, Content-Type defaults to application/octet-stream Supports Expression Language: true |
|
| Use Chunked Encoding | false |
* true * false |
When POST'ing or PUT'ing content set this property to true in order to not pass the 'Content-length' header and instead send 'Transfer-Encoding' with a value of 'chunked'. This will enable the data transfer mechanism which was introduced in HTTP 1.1 to pass data of unknown lengths in chunks. |
| Penalize on "No Retry" | false |
* true * false |
Enabling this property will penalize FlowFiles that are routed to the "No Retry" relationship. |
Dynamic Properties:
Dynamic Properties allow the user to specify both the name and value of a property.
| Name | Value | Description |
| Header Name | Attribute Expression Language |
Send request header with a key matching the Dynamic Property Key and a value created by evaluating the Attribute Expression Language set in the value of the Dynamic Property. Supports Expression Language: true |
Relationships:
| Name | Description |
| Retry | The original FlowFile will be routed on any status code that can be retried (5xx status codes). It will have new attributes detailing the request. |
| No Retry | The original FlowFile will be routed on any status code that should NOT be retried (1xx, 3xx, 4xx status codes). It will have new attributes detailing the request. |
| Response | A Response FlowFile will be routed upon success (2xx status codes). If the 'Output Response Regardless' property is true then the response will be sent to this relationship regardless of the status code received. |
| Failure | The original FlowFile will be routed on any type of connection failure, timeout or general exception. It will have new attributes detailing the request. |
| Original | The original FlowFile will be routed upon success (2xx status codes). It will have new attributes detailing the success of the request. |
Reads Attributes:
None specified.
Writes Attributes:
| Name | Description |
| invokehttp.status.code | The status code that is returned |
| invokehttp.status.message | The status message that is returned |
| invokehttp.response.body | In the instance where the status code received is not a success (2xx) then the response body will be put to the 'invokehttp.response.body' attribute of the request FlowFile. |
| invokehttp.request.url | The request URL |
| invokehttp.tx.id | The transaction ID that is returned after reading the response |
| invokehttp.remote.dn | The DN of the remote server |
| user-defined | If the 'Put Response Body In Attribute' property is set then whatever it is set to will become the attribute key and the value would be the body of the HTTP response. |
Properties and usage
HTTP Method: Specifies the HTTP Method.
Remote URL: Specifies the remote URL to be connected.
SSL Context Service: It is used to provide client certificate information.
Connection Timeout: Specifies maximum wait time for the connection to remote service.
Read Timeout: Maximum wait time for reading response from remote service.
Include Date Header: Includes an RFC-2616 date header in the request.
Follow Redirects: Used to follow HTTP redirects specified by the remote service or not.
Attributes to Send: It is used to set HTTP headers in the request using regular expression. If it is not defined, no attributes are sent as headers. Also, any dynamic properties added will be sent as headers.
Basic Authentication Username: Mentions the username for authenticating against remote URL.
Basic Authentication Password: Mentions the password for authenticating against remote URL.
Proxy Host: Specifies the hostname of the proxy server.
Proxy Port: Specifies the port number of the proxy server.
Proxy Username: Used to set the username when authenticating against proxy.
Proxy Password: Used to set the password when authenticating against proxy.
Put Response Body In Attribute: Specifies the response added as attribute into the original flow file instead of sending as new flow file.
Max Length To Put In Attribute: It is used to specify the maximum number of characters allowed to put as attribute when using put response body in attribute property. If the response is longer than the specified value, it will be truncated.
Use Digest Authentication: Used to communicate with the website. ‘Basic authentication username’ and ‘Basic authentication password’ are used for authentication.
Always Output Response: This property is set to force the processor to send a response as flow file regardless of what the server status code is received or if the processor is configured to send the response as attribute.
Trusted Hostname: This property is to set the remote hostname for bypassing the normal trust store hostname verifier to allow it as trusted.
Add Response Headers to Request: This property is used to save all the response headers to the original request. This is used when the response headers are needed but a response is not generated due to the status code received.
Content-Type: It is used to specify the type of content when content is being transmitted through Put or Post.
Send Message Body: It is used to send the HTTP message body on Post/Put/Patch requests by default.
Use Chunked Encoding: Set this property to true to pass the ‘Transfer-Encoding’ with a value of ‘chunked’ instead of passing the ‘content-length’ header. Penalize on “no retry”: Enable this property to penalize flow files that are routed to the “no retry” relationship.
Sample for InvokeHttp processor
This sample explains how to call the Rest API from the Data Integration Platform and save its response in a local file system.
Overview
The following processors are used in this sample:
|
Processor |
Comments |
| InvokeHTTP | To get data from secured URL. |
| UpdateAttribute | Sets filename to store the response. |
| PutFile | Sets file location to store the response generated. |
Step 1: Configure InvokeHttp processor
Drag and drop the InvokeHttp processor to invoke any REST API. Double click the processor and specify the API to be invoked as follows.
NOTE:
1.To access the HTTPS URL with client certificate information, then configure the StandardSSLContextService controller service as follows.
Refer to this link to generate TrustStore and KeyStore.
Now, go to your StandardSSLContextService and enter the following properties and enable the controller service.
KeyStore Filename: Enter the KeyStore file path generated.
KeyStore Password: Enter the password that generated.
KeyStore Type: JKS (Java Key Store)
TrustStore Filename: Enter the TrustStore file path generated.
TrustStore Password: Enter the password that you used.
TrustStore Type: JKS
TLS Protocol: TLS (Transport Layer Security)
2.To perform the Basic Authentication for REST API, specify the username and password or token as follows.
Step 2: Rename the response file
Drag and drop the UpdateAttribute processor and add the ‘filename’ property to rename the incoming file. Double click the processor and configure as follows.
Step 3: Store the response file
Drag and drop the PutFile processor and specify the location to store the file. Double click the processor and configure as follows.
Note: The InvokeHTTP processor can also used to invoke REST APIs that are created using the spring @RestController service running on the local tomcat server.
For more examples on InvokeHttp processor, please refer to the following articles:
https://www.syncfusion.com/kb/8578/how-to-download-image-using-hyperlinks-in-json-file
https://www.syncfusion.com/kb/8031/how-to-use-post-method-url-in-invokehttp-processor
https://www.syncfusion.com/kb/9332/get-data-from-soap-api-using-syncfusion-data-integration-platform