RestLookupService in Data Intergration

Use a REST service to look up values.

Tags:

rest, lookup, json, xml, http.

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the Expression Language Guide.

Name

Default Value

Allowable Values

Description

URL

 The URL for the REST endpoint. Expression language is evaluated against the lookup key/value pairs, not flowfile attributes.

Supports Expression Language: true (will be evaluated using variable registry only)

Record Reader

Controller Service API: 


RecordReaderFactory

Implementations: 

AvroReader


SyslogReader


ScriptedReader


ParquetReader


XMLReader


 The record reader to use for loading the payload and handling it as a record set.
Record Path An optional record path that can be used to define where in a record to get the real data to merge into the record set to be enriched. See documentation for examples of when this might be useful.

Supports Expression Language: true (will be evaluated using variable registry only)
SSL Context Service

Controller Service API: 


SSLContextService

Implementations: 

StandardRestrictedSSLContextService


StandardSSLContextService


 The SSL Context Service used to provide client certificate information for TLS/SSL connections.
Proxy Configuration Service

Controller Service API: 


ProxyConfigurationService

Implementation: 

StandardProxyConfigurationService


 Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS
Basic Authentication Username The username to be used by the client to authenticate against the Remote URL. Cannot include control characters (0-31), ':', or DEL (127).

Supports Expression Language: true (will be evaluated using variable registry only)
Basic Authentication Password The password to be used by the client to authenticate against the Remote URL.

Sensitive Property: true


Supports Expression Language: true (will be evaluated using variable registry only)
Use Digest Authentication false
  • true
  • false
Whether to communicate with the website using Digest Authentication. 'Basic Authentication Username' and 'Basic Authentication Password' are used for authentication.

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.

Name

Value

Description
* * All dynamic properties are added as HTTP headers with the name as the header name and the value as the header value.

Supports Expression Language: false

State management:

This component does not store state.

Restricted:

This component is not restricted.

System Resource Considerations:

None specified.

Summary:

General

This lookup service has the following optional lookup coordinate keys:

  • request.method; defaults to ‘get’, valid values:

    • delete
    • get
    • post
    • put
  • body; contains a string representing JSON, XML, etc. to be sent with any of those methods except for “get”.
  • mime.type; specifies media type of the request body, required when ‘body’ is passed.
  • *; any other keys can be configured to pass variables to resolve target URLs. See ‘Dynamic URLs’ section below.
    The record reader is used to consume the response of the REST service call and turn it into one or more records. The record path property is provided to allow for a lookup path to either a nested record or a single point deep in the REST response. Note: a valid schema must be built that encapsulates the REST response accurately in order for this service to work.

Headers

Headers are supported using dynamic properties. Just add a dynamic property and the name will be the header name and the value will be the value for the header. Expression language powered by input from the variable registry is supported.

Dynamic URLs

The URL property supports expression language through the lookup key/value pairs configured on the component using this lookup service (e.g. LookupRecord processor). The configuration specified by the user will be passed through to the expression language engine for evaluation. Note: flowfile attributes will be disregarded here for this property.
Ex. URL: http://example.com/service/${user.name}/friend/${friend.id}, combined with example record paths at LookupRecord processor:

  • user.name => “/example/username”
  • friend.id => “/example/first_friend”

Would dynamically produce an endpoint of http://example.com/service/john.smith/friend/12345

Using Variable Registry with URLs

In addition to the lookup key/value pairs, Variable Registry can be referred from expression languages configured at the URL property.
Ex. URL: http://${apiServerHostname}:${apiServerPort}/service/${user.name}/friend/${friend.id}, combined with the previous example record paths, and variable registry:

  • apiServerHostname => “test.example.com”
  • apiServerPort => “8080”

Would dynamically produce an endpoint of http://test.example.com:8080/service/john.smith/friend/12345