Apache Spark can power a wide variety of data analysis jobs. In Fusion, Spark jobs are especially useful for generating recommendations.Documentation Index
Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt
Use this file to discover all available pages before exploring further.
Spark job subtypes
For the Spark job type, the available subtypes are listed below.- Aggregation
Define an aggregation job. - Custom Spark job
Run a custom Spark job. - Script
Run a custom Scala script as a Fusion job.
Additional Spark jobs are available with a Fusion AI license.
Spark job configuration
Spark jobs can be created and modified using the Fusion UI or the Spark Jobs API. They can be scheduled using the Fusion UI or the Jobs API. For the complete list of configuration parameters for all Spark job subtypes, see the Jobs Configuration Reference.Learn more
Troubleshoot Custom Spark Jobs
Troubleshoot Custom Spark Jobs
To troubleshoot problems with a custom job, start by looking for errors in the script-job driver log,
/opt/fusion/latest.x*/var/log/api/spark-driver-launcher.log (on Unix) or C:\lucidworks\var\fusion\latest.*x\var\log\api\spark-driver-launcher.log (on Windows).For general Spark job troubleshooting steps, see Spark troubleshooting.