> ## Documentation Index
> Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Signals Data Flow
export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => {
const [isLoaded, setIsLoaded] = useState(false);
useEffect(() => {
const timer = setTimeout(() => {
setIsLoaded(true);
}, 500);
return () => clearTimeout(timer);
}, []);
return
{isLoaded && `
}} />}
Powered by Lucidworks Agent Studio
;
};
[localhost link]: http://localhost:3000/docs/4/fusion-ai/concepts/signals-and-aggregations/signals/data-flow
[mintlify link]: https://doc.lucidworks.com/docs/4/fusion-ai/concepts/signals-and-aggregations/signals/data-flow
[old doc.lw link]: https://doc.lucidworks.com/fusion/5.9/467
This diagram shows the flow of signals data from the search app through Fusion AI. The numbered steps are explained below.
1. The search app sends a query to a Fusion query pipeline.\
The query request should include a user ID and session query parameter to identify the user.
2. Optionally, the Fusion query pipeline queries the `_signals_aggr` collection to get boosts for the main query based on aggregated click data.
3. The search app also sends a request signal to the Fusion [`/signals` endpoint](/docs/4/fusion-ai/reference/api/signals-api).\
The primary intent of a request signal is to capture the raw user query and contextual information about the user’s current activity in the app, such as the user agent and the page where they generated the query. The request signal does not contain any information about the results sent to Solr; it is created before a query is processed.
4. Once Solr returns the response to Fusion, the SearchLogger component indexes the complete request/response data into the `_signals` collection as a response signal using the `_signals_ingest` pipeline. Therefore, the response signal captures all results from Fusion as it related to the original query.
This is a departure from pre-4.0 versions of Fusion where query impressions were logged in a separate `_logs` collection. Query activity is no longer indexed into the `_logs` collection. All response signals use the `fusion_query_id` (see below) as the unique document ID in Solr.
5. When the user clicks a link in the search results, the search app sends a click event to the Fusion signals endpoint (which invokes the `_signals_ingest` pipeline behind the scenes).\
The click signal *must* include a field named `fusion_query_id` in the `params` object of the raw click signal. The `fusion_query_id` field is returned in the query response (from step 1) in a response header named `x-fusion-query-id`. This allows Fusion to associate a click signal with the response signal generated in step 4. The `fusion_query_id` is also used by Fusion to associate click signals with [experiments](/docs/4/fusion-ai/concepts/experiments/overview). For experiments to work, each click signal must contain the corresponding `fusion_query_id` that produced the document/item that was clicked.
6. The `_signals_ingest` pipeline enriches signals before indexing into the `_signals` collection.\
This enrichment includes field mapping, geolocation resolution, and updating the `has_clicks` flag to "true" on request signals when the first click signal is encountered for a given request using the [Update Related Document index stage](/docs/4/fusion-server/reference/pipeline-stages/indexing/update-related-document-index-stage).
7. Fusion’s [App Insights](/docs/4/fusion-ai/concepts/insights/overview) queries the `_signals` collection through a Fusion query pipeline to generate query analytics reports from raw signals.\
Note that App Insights app uses Fusion security for authentication.
8. Behind the scenes, the [SQL aggregation](/docs/4/fusion-ai/concepts/signals-and-aggregations/aggregations/sql-aggregations) framework aggregates click signals to compute a weight for each query + `doc_id` + filters group.\
The resulting metrics are saved to the `_signals_aggr` collection to generate boosts on queries to the main collection (step 2 above).
9. Recommendations also use aggregated documents in the `_signals_aggr` collection to build a [collaborative filtering-based recommender model](/docs/4/fusion-ai/concepts/boosting/recommendation-methods).
1. The search app sends a query to a Fusion query pipeline.\
The query request should include a user ID and session query parameter to identify the user.
2. Optionally, the Fusion query pipeline queries the `_signals_aggr` collection to get boosts for the main query based on aggregated click data.
3. The search app also sends a request signal to the Fusion [`/signals` endpoint](/docs/4/fusion-ai/reference/api/signals-api).\
The primary intent of a request signal is to capture the raw user query and contextual information about the user’s current activity in the app, such as the user agent and the page where they generated the query. The request signal does not contain any information about the results sent to Solr; it is created before a query is processed.
4. Once Solr returns the response to Fusion, the SearchLogger component indexes the complete request/response data into the `_signals` collection as a response signal using the `_signals_ingest` pipeline. Therefore, the response signal captures all results from Fusion as it related to the original query.