Jobs Configuration Reference

These reference topics provide complete information about configuration properties for the Spark jobs that are enabled with a Fusion AI license.

For conceptual information and instructions for configuring and scheduling jobs, see Jobs and Schedules.

Additional jobs are available as part of the basic Fusion Server feature set.

  • ALS Recommender

    Use this job when you want to compute user recommendations or item similarities using a collaborative filtering recommender. You can also implement a user-to-item recommender in the advanced section of this job’s configuration UI.

  • Cluster Labeling

    Use this job when you already have clusters or well-defined document categories, and you want to discover and attach keywords to see representative words within those existing clusters. (If you want to create new clusters, use the Document Clustering job.)

  • Collection Analysis

    Use this job when you want to compute basic metrics about your collection, like average word length, phrase percentages, and outlier documents (with very many or very few documents).

  • Document Clustering

    Cluster a set of documents and attach cluster labels.

  • Ground Truth

    Estimate ground truth queries using click signals and query signals, with document relevance per query determined using a click/skip formula.

  • Head/Tail Analysis

    Perform head/tail analysis of queries from collections of raw or aggregated signals, to identify underperforming queries and the reasons. This information is valuable for improving overall conversions, Solr configurations, auto-suggest, product catalogs, and SEO/SEM strategies, in order to improve conversion rates.

  • Legacy Item Recommender

    Compute user recommendations based on a pre-computed item similarity model.

  • Legacy Item Similarity

    Use this job when you only want to compute item-to-item similarities. This method is more lightweight than the generic Recommendations job.

  • Logistic Regression Classifier Training

    Train a regularized logistic regression model for text classification.

  • Outlier Detection

    Use this job when you want to find outliers from a set of documents and attach labels for each outlier group.

  • Parallel Bulk Loader

    Run a Parallel Bulk Loader job.

  • Parameterized SQL Aggregation

    A SQL aggregation job where user-defined parameters are injected into a built-in SQL template at runtime.

  • Phrase Extraction

    Use this job when you want to identify phrases in your content.

  • Query-to-Query Similarity Computation

    Train a collaborative filtering matrix decomposition recommender using SparkML’s Alternating Least Squares (ALS) to batch-compute query-query similarities.

  • Random Forest Classifier Training

    Train a random forest classifier for text classification.

  • Ranking Metrics

    Calculate relevance metrics (nDCG and so on) by replaying ground truth queries against catalog data using variants from an experiment.

  • SQL-Based Experiment Metric (deprecated)

    This job is created by an experiment in order to calculate an objective.

    This job is deprecated as of Fusion AI 4.0.2.
  • Token and Phrase Spell Correction

    Detect misspellings in queries or documents using the numbers of occurrences of words and phrases. This job extracts tail tokens (one word) and phrases (two words) and finds similarly-spelled head tokens and phrases. For example, if two queries are spelled similarly, but one leads to a lot of traffic (head) and the other leads to a little or zero traffic (tail), then it’s likely that the tail query is misspelled and the head query is its correction.

  • Word2Vec Model Training

    Train a shallow neural model, and project each document onto this vector embedding space.