Script Jobs

Configuration Properties

Run a custom Scala script as a Fusion job.

Configuration Properties

Property Description, Type

envOptions

Env Options

Additional environment variables to set for Spark driver before running this job

type: array of object

object attributes: {
  key (required) : {
    display name: Parameter Name
    type: string
    }
  value : {
    display name: Parameter Value
    type: string
    }
  }

Spark Job ID

required

The ID for this Spark job. Used in the API to reference this job. Allowed characters: a-z, A-Z, dash (-) and underscore (_)

type: string

maxLength: 128

pattern: ^[A-Za-z0-9_\-]+$

script

Scala Script

required

Custom script written in Scala to be executed in Fusion as a Spark job.

type: string

shellOptions

Spark Shell Options

Additional options to pass to the Spark shell when running this job.

type: array of object

object attributes: {
  key (required) : {
    display name: Parameter Name
    type: string
    }
  value : {
    display name: Parameter Value
    type: string
    }
  }

sparkConfig

Spark Settings

Spark configuration settings.

type: array of object

object attributes: {
  key (required) : {
    display name: Parameter Name
    type: string
    }
  value : {
    display name: Parameter Value
    type: string
    }
  }

type

Spark Job Type

required

type: string

default value: 'script'

enum: { script }