Default Signals Index Pipeline
When indexing signals, Fusion uses a default index pipeline named _signals_ingest
unless you specify a different index pipeline.
The _signals_ingest
index pipeline has five stages:
-
Update
has_clicks
flag stageThe Update
has_clicks
flag stage is an instance of the Update Related Document stage that updates thehas_clicks
flag to "true" on an existing request signal after the first click signal is processed for the request.The update stage works as follows:
-
When a click signal is encountered (type == click)
-
Look at the incoming click signal for a field named
request_id_s
, which gets set by the Format Signals stage using a distributed cache of recently processed request signals.If the
request_id_s
field is set, then send a real-timeGET
query to Solr to find a request signal with ID equal to the value of therequest_id_s
field on the click signal. To avoid re-updating request signals, the RTG query also filters onhas_clicks==false
, which avoids duplicate atomic updates on the same document in Solr. Real-timeGET
is used to avoid timing issues between a request signal being sent to Solr and when it gets committed. This prevents missing updates when clicks occur soon after the initial request signal is sent by the search app. -
If the click signal does not have the
request_id_s
field set, then do a normal Solr lookup for the request signal using:+query_id:"${query_id}" +type:request +has_clicks:false
. A click signal may not have arequest_id_s
if there is a cache miss in the distributed cache used by the Format Signals stage. -
If the stage performs a normal query, there may be multiple request signals that have the same
query_id
. This is because thequery_id
is based onsession
+query
+filter
, so if a user sends the samequery
+filter
during the same session, there will be multiple request signals with the samequery_id
value. Thus, the stage sorts to get the latest request signal to update. -
If a related document is found (in this case a request signal), then the stage updates the
has_clicks
field to true and performs an atomic update in Solr.
This stage performs its work in a background thread, so it does not impact the indexing performance of the click signal.
-