Product Selector

Fusion 5.12
    Fusion 5.12

    Signals Data Flow

    This diagram shows the flow of signals data from the search app through Fusion AI. The numbered steps are explained below.

    Signals data flow

    1. The search app sends a query to a Fusion query pipeline.

      The query request should include a user ID and session query parameter to identify the user.

    2. Optionally, the Fusion query pipeline queries the _signals_aggr collection to get boosts for the main query based on aggregated click data.

    3. The search app also sends a request signal to the Fusion /signals endpoint.

      The primary intent of a request signal is to capture the raw user query and contextual information about the user’s current activity in the app, such as the user agent and the page where they generated the query. The request signal does not contain any information about the results sent to Solr; it is created before a query is processed.

    4. Once Solr returns the response to Fusion, the SearchLogger component indexes the complete request/response data into the _signals collection as a response signal using the _signals_ingest pipeline. Therefore, the response signal captures all results from Fusion as it related to the original query.

      This is a departure from pre-4.0 versions of Fusion where query impressions were logged in a separate _logs collection. Query activity is no longer indexed into the _logs collection. All response signals use the fusion_query_id (see below) as the unique document ID in Solr.

    5. When the user clicks a link in the search results, the search app sends a click event to the Fusion signals endpoint (which invokes the _signals_ingest pipeline behind the scenes).

      The click signal must include a field named fusion_query_id in the params object of the raw click signal. The fusion_query_id field is returned in the query response (from step 1) in a response header named x-fusion-query-id. This allows Fusion to associate a click signal with the response signal generated in step 4. The fusion_query_id is also used by Fusion to associate click signals with experiments. For experiments to work, each click signal must contain the corresponding fusion_query_id that produced the document/item that was clicked.

    6. The _signals_ingest pipeline enriches signals before indexing into the _signals collection.

      This enrichment includes field mapping, geolocation resolution, and updating the has_clicks flag to "true" on request signals when the first click signal is encountered for a given request using the Update Related Document index stage.

    7. Fusion’s App Insights queries the _signals collection through a Fusion query pipeline to generate query analytics reports from raw signals.

      Note that App Insights app uses Fusion security for authentication.

    8. Behind the scenes, the SQL aggregation framework aggregates click signals to compute a weight for each query + doc_id + filters group.

      The resulting metrics are saved to the _signals_aggr collection to generate boosts on queries to the main collection (step 2 above).

    9. Recommendations also use aggregated documents in the _signals_aggr collection to build a collaborative filtering-based recommender model.