> ## Documentation Index
> Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Machine Learning Jobs

export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => {
  const [isLoaded, setIsLoaded] = useState(false);
  useEffect(() => {
    const timer = setTimeout(() => {
      setIsLoaded(true);
    }, 500);
    return () => clearTimeout(timer);
  }, []);
  return <div className="lw-template-container">
      <Card title={title} icon={icon}>
        {isLoaded && <span dangerouslySetInnerHTML={{
    __html: `<lw-template id="a029c1a9-28be-427e-b0e1-5d918920246a"></lw-template
            >`
  }} />}
        <Link href={linkHref} className="agent-studio-link text-left text-gray-600 gap-2 dark:text-gray-400 text-sm font-medium flex flex-row items-center hover:text-primary dark:hover:text-primary-light group-hover:text-primary group-hover:dark:text-primary-light">Powered by Lucidworks Agent Studio</Link>
      </Card>
    </div>;
};

[localhost link]: http://localhost:3000/docs/4/fusion-ai/concepts/machine-learning/ml-jobs

[mintlify link]: https://doc.lucidworks.com/docs/4/fusion-ai/concepts/machine-learning/ml-jobs

[old doc.lw link]: https://doc.lucidworks.com/fusion/5.9/494

Fusion AI provides these job types to perform machine learning tasks.

<LwTemplate />

## Signals analysis

These jobs analyze a collection of signals in order to perform [query rewriting](#query-rewriting), [signals aggregation](#signals-aggregation), or [experiment analysis](#experiment-analysis).

* [Ground Truth](/docs/4/fusion-ai/reference/jobs/ground-truth)\
  Estimate ground truth queries using click signals and query signals, with document relevance per query determined using a click/skip formula.

### Query rewriting

These jobs produce data that can be used for query rewriting or to inform updates to the [synonyms.txt file](/docs/5/fusion/getting-data-out/query-enhancement/synonyms/overview).

* [Head/Tail Analysis](/docs/4/fusion-ai/reference/jobs/head-tail-analysis)\
  Perform head/tail analysis of queries from collections of raw or aggregated signals, to identify underperforming queries and the reasons. This information is valuable for improving overall conversions, Solr configurations, auto-suggest, product catalogs, and SEO/SEM strategies, in order to improve conversion rates.
* [Phrase Extraction](/docs/4/fusion-ai/reference/jobs/phrase-extraction)\
  Identify multi-word phrases in signals.
* [Synonym and Similar Queries Detection Jobs](/docs/4/fusion-ai/reference/jobs/synonym-and-similar-queries-detection)\
  Use this job to generate pairs of synonyms and pairs of similar queries. Two words are considered potential synonyms when they are used in a similar context in similar queries.
* [Token and Phrase Spell Correction](/docs/4/fusion-ai/reference/jobs/token-and-phrase-spell-correction)\
  Detect misspellings in queries or documents using the numbers of occurrences of words and phrases.

### Signals aggregation

* [Parameterized SQL Aggregation](/docs/4/fusion-ai/reference/jobs/parameterized-sql-aggregation)\
  A Spark SQL aggregation job where user-defined parameters are injected into a built-in SQL template at runtime.

### Experiment analysis

* [Ranking Metrics](/docs/4/fusion-ai/reference/jobs/ranking-metrics)\
  Calculate relevance metrics (nDCG and so on) by replaying ground truth queries against catalog data using variants from an experiment.
* [SQL-Based Experiment Metric](/docs/4/fusion-ai/reference/jobs/sql-based-experiment-metric) (deprecated)\
  This job is created by an experiment in order to calculate an objective.

  <Note>
    SQL-Based Experiment Metric job is deprecated as of Fusion AI 4.0.2.
  </Note>

## Collaborative recommenders

These jobs analyze signals and generate matrices used to provide [collaborative recommendations](/docs/4/fusion-ai/concepts/boosting/recommendation-methods).

* [ALS Recommender](/docs/4/fusion-ai/reference/jobs/als-recommender)\
  Use this job when you want to compute user recommendations or item similarities using a collaborative filtering recommender. You can also implement a user-to-item recommender in the advanced section of this job’s configuration UI. This job uses [SparkML’s Alternating Least Squares (ALS)](https://spark.apache.org/docs/latest/ml-collaborative-filtering.html).
* [Query-to-Query Similarity](/docs/4/fusion-ai/reference/jobs/query-to-query-similarity-computation)\
  Train a collaborative filtering matrix decomposition recommender using [SparkML’s Alternating Least Squares (ALS)](https://spark.apache.org/docs/latest/ml-collaborative-filtering.html) to batch-compute query-query similarities. This can be used for [items-for-query recommendations](/docs/4/fusion-ai/concepts/boosting/items-for-query) as well as [queries-for-query recommendations](/docs/4/fusion-ai/concepts/boosting/queries-for-query).

## Content-based recommenders

[Content-based recommenders](/docs/4/fusion-ai/concepts/boosting/recommendation-methods) create matrices of similar items based on their content.

## Content analysis

* [Cluster Labeling](/docs/4/fusion-ai/reference/jobs/cluster-labeling)\
  Use this job when you already have clusters or well-defined document categories, and you want to discover and attach keywords to see representative words within those existing clusters. (If you want to create new clusters, use the [Document Clustering job](/docs/4/fusion-ai/reference/jobs/document-clustering).)
* [Collection Analysis](/docs/4/fusion-ai/reference/jobs/collection-analysis)\
  Use this job when you want to compute basic metrics about your collection, like average word length, phrase percentages, and outlier documents (with very many or very few documents).
* [Document Clustering](/docs/4/fusion-ai/reference/jobs/document-clustering)\
  Cluster a set of documents and attach cluster labels.
* [Logistic Regression Classifier Training](/docs/4/fusion-ai/reference/jobs/logistic-regression-classifier-training)\
  Train a regularized logistic regression model for text classification.
* [Outlier Detection](/docs/4/fusion-ai/reference/jobs/outlier-detection)\
  Use this job when you want to find outliers from a set of documents and attach labels for each outlier group.
* [Random Forest Classifier Training](/docs/4/fusion-ai/reference/jobs/random-forest-classifier-training) (deprecated)\
  Train a [random forest classifier](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.classification.RandomForestClassificationModel.html) for text classification.
* [Word2Vec Model Training](/docs/4/fusion-ai/reference/jobs/word2vec-model-training) (Deprecated)\
  Train a shallow neural model, and project each document onto this vector embedding space.

## Data ingest

* [Parallel Bulk Loader](/docs/4/fusion-ai/reference/jobs/parallel-bulk-loader)\
  The Parallel Bulk Loader (PBL) job enables bulk ingestion of structured and semi-structured data from big data systems, NoSQL databases, and common file formats like Parquet and Avro.

## Legacy machine learning jobs

* [Legacy Item Recommender](/docs/4/fusion-ai/reference/jobs/legacy-item-recommender)\
  Compute user recommendations based on a pre-computed item similarity model.
* [Legacy Item Similarity](/docs/4/fusion-ai/reference/jobs/legacy-item-similarity)\
  Use this job when you only want to compute item-to-item similarities. This method is more lightweight than the generic Recommendations job.

  <Note>
    Legacy Item Similarity job is deprecated as of Fusion AI 4.1.0. Use the [ALS recommender job](/docs/4/fusion-ai/reference/jobs/als-recommender) instead.
  </Note>