Recommendations and Boosting

Apps can call Fusion to return documents for a user to interact with in various contexts. These include:

  • Search contexts: A user provides a query and performs a search.

  • Non-search contexts: A user doesn’t perform a search, but rather lands on app page, for example, one that lists products or displays details for a single product. There is a query, but it comes from the app, not from the user.

The primary goal for the user and the business is relevance. Display the items that are most useful for the user, or for which the user is most likely to take some action, for example, to make a purchase.

In a basic search, a user enters search terms and Fusion uses an algorithm to rank the documents that it returns to the user. The matching and ranking depend solely on a match between the search terms and the contents of the documents (and possibly document metadata).

Recommendations are documents that a recommender stage in a query pipeline chooses to recommend to a user based on some criteria. But they aren’t recommendations for a user yet. Whether the documents are recommended to the user depends on the remainder of the query pipeline.

Boosts are changes in numeric scores associated with documents that match a query. Changing the scores can move documents up or down in the query response. Positive boosts can bring documents into the response that wouldn’t have been included without boosting. Negative boosts (which are actually fractional) can move documents out of the response.

Item similarities also play a role in some approaches for recommendations and boosting. When a user searches for an item, a search application can recommend and/or boost similar items.

Recommendation methods

There are two ways to get recommendations from Fusion: the Recommendations API or recommender stages for query pipelines.

Below is a more detailed comparison of the two methods:

Recommendations API

Recommender query stages

Customization

Limited customization

Highly configurable

"Cold start"

When no pertinent signals exist, no recommendations are returned.

When no pertinent signals exist, the query’s normal search results are returned (if recommendations are applied as boosts). If recommendations are returned as the result, then no recommendations are returned.

For the Recommend Items for User and Recommend Items for Items stages, the optional configuration setting Estimate Recent Results (enabled by default) estimates results since the last run of the recommender jobs.

Recommendation types

The Recommendations API provides the following kinds of recommendations:

  • A list of items recommended for the given query (the itemsForQuery endpoint).

  • A list of queries associated with the given item, using aggregated clicks (the queriesForItem endpoint).

The query stages provide different kinds of recommendations and/or boosting:

Content-based filtering:

Collaborative filtering:

Using the Recommendations API

In order to use the REST API Recommendations service to get recommendations for items in some collection, that collection must have associated signals and aggregated-signals system collections. How good recommendations are depends on how well the information in the signals and aggregated signals collections, which is derived from observed user behavior, matches user behavior going forward.

See the the Recommendations API for details.

Note: Recommendations based on collaborative filtering aren’t available through the Recommendations API. Instead, use the query-pipeline stages Recommend Items for User and Recommend Items for Item.

Enable Spark-based Collaborative Recommendations

Enable Recommendations in the Fusion UI enables Spark-based collaborative recommendations, that is, items-for-user and items-for-item recommendations (and users-for-item recommendations if you enable those). You don’t need to give this command to set up the recommendations infrastructure for items-for-query collaborative recommendations (Boost with Signals) or for content-based filtering.

To enable collaborative recommendations:

  1. Log in to Fusion as the admin user.

  2. From the Collections drop-down list, choose the collection for which you want to enable collaborative recommendations.

  3. Click Search.

  4. Click Settings settings > Enable Recommendations.

Enabling recommendations creates these objects:

  • Collections:

    Object Description

    <collection>_signals

    Collection to hold user interactions. If the signals feature is enabled for this collection (which it is by default), then this collection will already exist.

    <collection>_signals_aggr

    Collection to hold aggregated user interactions. If the signals feature is enabled for this collection (which it is by default), then this will already exist.

    The default aggregation groups together all signals of type click by the joint key (user_id_s, doc_id_s) from the <collection>_signals collection, and performs a time-decayed count of these results to generate an estimate of the "implicit preference" that user_id_s has for doc_id_s.

    <collection>_items_for_item_recommendations

    Collection to hold generated item-item similarities (by default 10 per item). No user_id_s data is present. A Recommend Items for Item query-pipeline stage can use the similarities to return item recommendations. For example, a query in which doc_id_s = docA would return an ordered list of other doc_id_s values for documents that are similar to document docA, along with the similarities. For example: [("docB", 0.83), ("docC", 0.55), ("docD", 0.43), …​, ("docK", 0.22)].

    <collection>_items_for_user_recommendations

    Collection to hold recommended items for a user. By default the job creates 10 recommendations per user.

  • Jobs:

    Object Description

    <collection>_item_aggregation

    Aggregates user-item interactions to generate weights. This aggregates user-item interactions and calculates a weight based on the recency of the interaction (so that more recent interactions have more impact on recommendations).

    <collection>_item_recommendations

    Runs a Spark job to train an ALS-based (Alternating Least Squares) recommendation model, and then uses that model to generate recommendations. By default, this job generates both items-for-user recommendations and items-for-item recommendations. The job stores the results in the <collection>_items_for_user_recommendations and <collection>_items_for_item_recommendations collections.

  • Job schedules:

    Object Description

    Schedule for the job <collection>_item_aggregation

    Runs the aggregation job once a day.

    Schedule for the job <collection>_item_recommendations

    Runs the recommender job after successful completion of the aggregation job.

  • Query pipelines:

    Object Description

    <collection>_items_for_user_recommendations

    Query pipeline to generate recommendations of items for a user

    <collection>_items_for_item_recommendations

    Query pipeline to generate recommendations of items similar to an item

Disable Spark-based Collaborative Recommendations

Disable Recommendations in the Fusion UI disables Spark-based collaborative recommendations (items-for-user, items-for-item, and users-for-item).

When you disable recommendations, Fusion disables these schedules:

  • Schedule for the job <collection>_item_aggregation

  • Schedule for the job <collection>_item_recommendations

This stops the production of new collaborative recommendations. Query pipeline stages continue to return recommendations based on existing data. No data is deleted when you disable recommendations.

Disabling recommendations has no effect on items-for-query recommendations (Boost with Signals) or on the recommendations infrastructure for content-based filtering.

To disable collaborative recommendations:

  1. Log in to Fusion as the admin user.

  2. From the Collections drop-down list, choose the collection for which you want to disable collaborative recommendations.

  3. Click Search.

  4. Click Settings settings > Disable Recommendations.