Machine Learning Jobs

Estimate ground truth queries using click signals and query signals, with document relevance per query determined using a click/skip formula.

Use this job when you want to compute user recommendations or item similarities using a collaborative filtering recommender. You can also implement a user-to-item recommender in the advanced section of this job’s configuration UI. This job uses SparkML’s Alternating Least Squares (ALS).

Train a collaborative filtering matrix decomposition recommender using SparkML’s Alternating Least Squares (ALS) to batch-compute query-query similarities. This can be used for items-for-query recommendations as well as queries-for-query recommendations.

Use this job when you already have clusters or well-defined document categories, and you want to discover and attach keywords to see representative words within those existing clusters. (If you want to create new clusters, use the Document Clustering job.)

Use this job when you want to compute basic metrics about your collection, like average word length, phrase percentages, and outlier documents (with very many or very few documents).

Cluster a set of documents and attach cluster labels.

Train a regularized logistic regression model for text classification.

Use this job when you want to find outliers from a set of documents and attach labels for each outlier group.

Train a random forest classifier for text classification.

Train a shallow neural model, and project each document onto this vector embedding space.

The Parallel Bulk Loader (PBL) job enables bulk ingestion of structured and semi-structured data from big data systems, NoSQL databases, and common file formats like Parquet and Avro.

Compute user recommendations based on a pre-computed item similarity model.

Use this job when you only want to compute item-to-item similarities. This method is more lightweight than the generic Recommendations job.