Description:
Manages pool of Spark sessions over HTTP
Tags:
Livy, REST, Spark, http
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the Expression Language Guide.
Name |
Default Value |
Allowable Values |
Description |
Livy Host |
The hostname (or IP address) of the Livy server. Supports Expression Language: true (will be evaluated using variable registry only) |
||
Livy Port |
8998 |
The port number for the Livy server. Supports Expression Language: true (will be evaluated using variable registry only) |
|
Session Pool Size |
2 |
Number of sessions to keep open Supports Expression Language: true (will be evaluated using variable registry only) |
|
Session Type |
spark |
* spark * pyspark * pyspark3 * sparkr |
The type of Spark session to start (spark, pyspark, pyspark3, sparkr, e.g.) |
Session Manager Status Interval |
2 sec |
The amount of time to wait between requesting session information updates. Supports Expression Language: true (will be evaluated using variable registry only) |
|
SSL Context Service |
Controller Service API: SSLContextService Implementations: StandardRestrictedSSLContextService StandardSSLContextService |
The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections. | |
Connection Timeout |
5 secs | Max wait time for connection to remote service. | |
Session JARs |
JARs to be used in the Spark session. Supports Expression Language: true (will be evaluated using variable registry only) |
||
Session Files |
Files to be used in the Spark session. Supports Expression Language: true (will be evaluated using variable registry only) |
||
Kerberos Credentials Service |
Controller Service API: KerberosCredentialsService Implementation: KeytabCredentialsService |
Specifies the Kerberos Credentials Controller Service that should be used for authenticating with Kerberos |
State management:
This component does not store state.
Restricted:
This component is not restricted.
System Resource Considerations:
None specified.