Description:

Manages pool of Spark sessions over HTTP

Tags:

Livy, REST, Spark, http

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the Expression Language Guide.

Name	Default Value	Allowable Values	Description
Livy Host			The hostname (or IP address) of the Livy server. Supports Expression Language: true (will be evaluated using variable registry only)
Livy Port	8998		The port number for the Livy server. Supports Expression Language: true (will be evaluated using variable registry only)
Session Pool Size	2		Number of sessions to keep open Supports Expression Language: true (will be evaluated using variable registry only)
Session Type	spark	* spark * pyspark * pyspark3 * sparkr	The type of Spark session to start (spark, pyspark, pyspark3, sparkr, e.g.)
Session Manager Status Interval	2 sec		The amount of time to wait between requesting session information updates. Supports Expression Language: true (will be evaluated using variable registry only)
SSL Context Service		Controller Service API:  SSLContextService Implementations:  StandardRestrictedSSLContextService StandardSSLContextService	The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections.
Connection Timeout	5 secs		Max wait time for connection to remote service.
Session JARs			JARs to be used in the Spark session. Supports Expression Language: true (will be evaluated using variable registry only)
Session Files			Files to be used in the Spark session. Supports Expression Language: true (will be evaluated using variable registry only)
Kerberos Credentials Service		Controller Service API:  KerberosCredentialsService Implementation:  KeytabCredentialsService	Specifies the Kerberos Credentials Controller Service that should be used for authenticating with Kerberos

State management:

This component does not store state.

Restricted:

This component is not restricted.

System Resource Considerations:

None specified.