Jobs and Schedules

A job is a runnable Fusion object that performs a specific activity. Datasource jobs, Spark jobs, and tasks are all types of jobs. Examples of the activities include:

  • Aggregating signal data

  • Training a shallow neural model and projecting each document onto the resulting vector embedding space

  • Loading data into a data source

A schedule defines when Fusion will perform a job.

You can schedule a job using cron notation, ISO-8601 interval definitions, or triggers that depend on the results of other jobs.

You can also run jobs on demand.

The Jobs API lets you view jobs and their run histories, configure their schedules, and control them directly.

Job types

Each job type is a type of Fusion object that you can run or schedule to be run.

datasource

A job to ingest data according to the specified datasource configuration, such as datasource:movie-db. Datasources are created using the Connector Datasources API or the Fusion UI.

spark

A Spark job to process data, such as spark:dailyMetricsRollup-counters. Spark jobs are created using the Spark Jobs API or the Fusion UI.

See Spark Jobs.

task

A job to perform an HTTP call or log cleanup, such as task:delete-old-system-logs. Tasks are created using the Tasks API or the Fusion UI.

See Tasks.

The Jobs manager

The Jobs manager, available in the Fusion workspace at Collections > Jobs, provides a simple interface for viewing and scheduling jobs. You can also create tasks and Spark jobs here.

Jobs

Note
You can’t create, run, or schedule datasource jobs in the Jobs manager. You must create them on the Datasources page, in the Index Workbench, or in the Quickstart. To run a datasource job, use the scheduler.
To create a new job
  1. In the Fusion workspace, navigate to Collections > Jobs.

  2. Click Add and select the job type.

    New Job

    The New Job Configuration panel appears. This panel is different for each job type. See the job types above for details.

  3. Configure the new job as needed.

  4. Click Save.

To run a job once
  1. In the Fusion workspace, navigate to Collections > Jobs.

  2. Select the job from the job list.

  3. Click Run.

    Start Job

  4. Click Start.

To schedule a job
  1. In the Fusion workspace, navigate to Collections > Scheduler.

  2. Select the job from the job list.

  3. Click Run.

  4. Click New Schedule.

    Schedule Job

  5. Select a job trigger:

    • After Another Job Completes

      Enter the job ID and job result that trigger this one.

      After Another Jobs Completes

    • Cron String

      Enter a Quartz cron expression. See the Quartz documentation for details.

      Cron String

    • Start + Interval

      Enter a start date/time, an interval, and the interval units.

      Start + Interval

The Scheduler

The Scheduler, available in the Fusion workspace at System > Scheduler, provides an interface for scheduling jobs.

Scheduler

You can’t create jobs in the Scheduler. To create new jobs, use the Jobs manager or the REST API for each job type.

Scheduler