Improve RelevancyGetting Started with Managed Fusion

Table of Contents

Before you begin
Format display fields
Enable synthetic signals
Generate signals
Explore the raw signals
- Optional: Format signals fields
Explore the aggregated signal data
View the search results with and without default boosting

Signals are events that can be aggregated and used for automatic boosting or recommendations, which are methods to make search results more relevant.

As an example of boosting, the most popular search results for certain queries can be boosted so they display first (or at least nearer the top) when other users make similar queries.

Similarly for recommendations, click events or purchase events can be collected as signals and used to display "Customers who viewed this also viewed" or "Best-selling holiday items".

In a production environment, users' actions generate signals. For the purposes of this tutorial, you will use Query Workbench to generate click signals.

Lucidworks offers free training to help you get started.

The Learning Path for Refining Search Results focuses on the Fusion features that help you maximize the relevancy of your search results:

Visit the LucidAcademy to see the full training catalog.

Before you begin

To perform the steps in this part of the tutorial, you must first complete Part 1 - Create a Managed Fusion application, Part 2 - Index Data, and Part 3 - Query Data, which give you an indexed dataset that is configured for faceted search.

Format display fields

Sign in to Managed Fusion if it is not currently open.
In the Managed Fusion launcher, click the Movie Search app.
Click Querying > Query Workbench.
At the top right, select Display Fields.
In the Name field, click the row to display a dropdown of possible values.
Select title_txt. You can filter the list of possible values.
In the Description field, select id.
Click Display Fields to close the Display Fields panel.

Enable synthetic signals

To complete this part of the tutorial, you must generate more signals because not enough signals were created in Part 3 - Query Data. Since this example is a prototype app, users are not generating signals. To create more signals, you must enable synthetic signals in Query Workbench.

At the bottom of the Query Workbench page, click Format Results.
Select Show signal generators and Send click signals.
Click Save.
Hover over one of the search results. Query Workbench displays controls that include a Simulate button next to a field that lets you specify the number of signals to simulate:

Generate signals

With synthetic signals enabled, you will generate a simple set of signal data that you can use to generate meaningful recommendations.

For this tutorial, you will generate signals you can use to boost your favorite sci-fi titles so that they display first.

Search for star wars.

The top results are not your favorite titles:
In Format Results, Set results per page to 20. This should allow you to scroll for this next part instead of explicitly searching for the titles.

Next you will generate signals you can use to boost certain titles. Signals are tied to the search query, so your boosted titles will display first in the search results only when users search for star wars.
Hover over "Star Wars: Episode IV - A New Hope".
Set the number of signals to 4000 and click Simulate.
Hover over "Star Wars: Episode V - The Empire Strikes Back".
Set the number of signals to 3000 and click Simulate.
Hover over "Star Wars: Episode VI - Return of the Jedi".
Set the number of signals to 2000 and click Simulate.

With synthetic signals enabled, you can also send a single signal by clicking the underlined movie title.
In the upper right, click Save. The Save Pipeline window displays. By default, you will overwrite the existing query pipeline for the selected collection (in this case, Movie_Search_signals).
Click Save pipeline.

Explore the raw signals

When you create a collection, two corresponding collections are also created automatically:

COLLECTION_NAME_signals for raw signals. For example, Movie_Search_signals.
COLLECTION_NAME_signals_aggr for aggregated signals. For example, Movie_Search_signals_aggr.

You can use Query Workbench to explore the data in the _signals collection.

In the collection picker in the upper left, select Movie_Search_signals.
Click Querying > Query Workbench. The signals data displays.

Your signals collection contains several types of signals, such as click signals and search result signals.
To display click signals, enter type:click and click Search .
For any of the results, click show fields.

The count_i field displays the number of click signals you generated for this event. For example, given the corresponding doc_id for Star Wars: Episode IV - A New Hope, the count_i equals 4000.
Click hide fields.

Optional: Format signals fields

You can configure your search view to display fields that are meaningful for your investigation. For example, you can display the document ID and the number of click signals.

Click Display Fields.
For the Name field, select doc_id.
For the Description field, select count_i.

Explore the aggregated signal data

Aggregation jobs are created automatically when you create an app. However, you need to run the aggregation job manually:

Navigate to Collections > Jobs.
In the job list, select Movie_Search_click_signals_aggregation.
Click Run.
Click Start. The job may take a few minutes to complete.
When the success message displays, in the collections picker in the upper left, select Movie_Search_signals_aggr.
Click Querying > Query Workbench.

The aggregated signals data displays. If the aggregated signals data does not display, wait a minute and then reload your browser, or click Search in Query Workbench.
Click Display Fields.
In the Name field, select doc_id.
In the Description field, select aggr_count_i.
For the result named 1210, click show fields.

The fields for aggregated signals are very similar to the fields for raw signals, with additional fields to describe the aggregation:
- aggr_count_i. Number of signals that have been aggregated. For example, 3000.
- aggr_id_s. Name of the aggregation job.
- aggr_job_id_s. Job ID.
- aggr_type_s. Aggregation type.
In the upper right, click Save.

The Save Pipeline screen displays. By default, you will overwrite the existing query pipeline for the selected collection. For example, Movie_Search.
Click Save pipeline.

View the search results with and without default boosting

In the collections picker in the upper left, select Movie_Search.
Open Query Workbench. Navigate to Querying > Query Workbench.
Search for star wars.

"Star Wars: Episode IV - A New Hope" is the first search result, followed by Episode V and then VI. These search results are automatically boosted by the default configuration of the Boost with Signals query pipeline stage, which boosts on the id field.
Click Compare. Another preview panel opens. Now the working pipeline is on the right and a static snapshot of that same pipeline is on the left.

In this view, you can compare results from one query pipeline side by side with another query pipeline. In this case, you will compare results for the same pipeline (Movie_Search) with and without the Boost with Signals stage enabled.
Turn off the Boost with Signals stage.

The search results on the right display as they did before you generated synthetic click signals. Rank-change indicators indicate which results moved up or down as a result of turning off boosting.
Turn on the Boost with Signals stage again to restore the boosted results.
To close the comparison preview panel, click Close .