Fusion

Version 5.1
How To
Documentation
    Learn More

      Jobs and Scheduling

      Table of Contents

      A job is a runnable Fusion object that performs a specific activity. Datasource jobs, Spark jobs, and tasks are all types of jobs. Examples of the activities include:

      • Aggregating signal data

      • Training a shallow neural model and projecting each document onto the resulting vector embedding space

      • Loading data into a data source

      Job types

      Each job type is a type of Fusion object that you can run or schedule to be run.

      datasource

      A job to ingest data according to the specified datasource configuration, such as datasource:movie-db. Datasources are created using the Connector Datasources API or the Fusion UI.

      spark

      A Spark job to process data, such as spark:dailyMetricsRollup-counters. Spark jobs are created using the Spark Jobs API or the Fusion UI.

      See Spark Jobs.

      task

      A job to perform an HTTP call or log cleanup, such as task:delete-old-system-logs. Tasks are created using the Tasks API or the Fusion UI.

      See Tasks.

      The Jobs manager

      The Jobs manager, available in the Fusion workspace at Collections > Jobs, provides a simple interface for viewing and scheduling jobs. You can also create tasks and Spark jobs here.

      Jobs

      You cannot create, run, or schedule datasource jobs in the Jobs manager. You must create them on the Datasources page, in the Index Workbench, or in the Quickstart. To run a datasource job, use the scheduler.