Boost with SignalsQuery pipeline stage configuration specifications
The Boost with Signals query pipeline stage uses aggregated signals to selectively boost items in the set of search results.
Using the main query and the stage configuration parameters, this stage performs a secondary query to the COLLECTION_NAME_signals_aggr collection and returns updated boost weights for the items in the main query’s search results. Items that have received more user interaction also receive higher boost weights.
|
|
This stage accesses the signals_aggr collection. Before using it, verify that the following permission is set:
GET:/solr/COLLECTION_NAME_signals_aggr/select
|
The fields below are especially useful to understand when configuring this stage.
Number of Recommendations
|
Sets the rows query param in the main query as the maximum number of query results which will be boosted by this pipeline stage.
|
|
Sets the rows query param in the query that searches the COLLECTION_NAME_signals_aggr collection, so only the specified number of aggregated signals are retrieved and used for boosting. When signals boosting is applied to a query, aggregated signals records are queried from the appropriate _signals_aggr collection to find out the popularity or boost weight for documents which have signals. numSignals limits the number of records to be queried from a }}{{_signals_aggr collection and used to calculate this boost.
|
|
A filter to retrieve aggregated signals in the COLLECTION_NAME_signals_aggr collection per each aggregated signal’s aggr_type_s field value.
|
|
The document field in the main collection on which to perform boosting. Typically it should use default field, which is id.
This field corresponds to the Rollup Field/rollupField field. Together, these two fields act like a FIELD:VALUE pair in the query modification for boosting.
|
|
This adds a query parameter to the original query, either “query-param” or “query-parser”. The result is (“query-param” or “query-parser”) + Boost Param(“boost” or `bq)`, as in the examples below:
“query-param”+"boost", result boost query param
boost="map(query({!field f='id' v=‘6239046,13026192}), 0, 0, 1, 27.1705)"
“query-parser”+"boost", result boost query param
bp_xxx_bbqx="map(query({!field f='id' v=‘6239046,13026192}), 0, 0, 1, 27.1705)"
|
|
When Boost Param uses bq, similar logic applies. When Boost Param/boostingParam uses “boost”, it works with both “query-param” and “query-parser”.
|
|
|
Indicates which aggregated signal document field the boost parameter will use for the final boosting. It works in combination with the Solr Field to Boost On/boostId field.
This should be set to the field in the aggregated signal collection that stores the doc list that is aggregated as one record. By default it’s set to doc_id_s, because by default this is used in the click_signal_aggr SQL job.
|
|
Indicates the final boost weight used to calculate the new score for docs retrieved by the main query.
Similar to Rollup Field/rollupField above, this should be set to the field in the aggregated signal collection that stores the final weight that was calculated. By default it’s weight_d, because by default this is used in the click_signal_aggr SQL job.
|
Final Boost Weight Expression
|
Calculates the final weight using the weight and score retrieved from the COLLECTION_NAME_signals_aggr collection.
The default value is math:log(weight_d + 1) + 10 * math:log(score+1).
|
These parameters are used in the Solr Query parameters/queryParams field for retrieving signal aggregation docs from the COLLECTION_NAME_signals_aggr collection. These Solr query params will affect which aggregated signals are used for producing the boosting parameter on the main query.
|
Defines which field to query. In the default case, the query searches on the query_t field of aggregated signal docs.
|
|
Boosts docs within the set of retrieved docs using phrase matching.
|
|
pf2 is similar to pf; the difference is that pf2 works on bigram phrases.
|
|
pf3 is similar to pf; the difference is that pf3 works on trigram phrases.
|
If there is fq in the main query, how is it matched with the correct aggregated signal?
In this case, you need to use the lw.rec.fq query parameter in the main query. lw.rec.fq can be parsed by the Boost with Signals stage, and therefore the filters specified in it can be added to the Solr query that is retrieving the aggregated signals.
For example, if we have filter query param fq=format:CD&fq=name:Latin, this needs to be translated into lw.rec.fq=filters_s:"format:cd $ name:latin". Values must be lowercase. The final main query should be:
http://EXAMPLE_COMPANY.lucidworks.cloud/api/apps/demo_app/query-pipelines/demo_app/collections/demo_app/select?echoParams=all&wt=json&json.nl=arrarr&sort&start=0&q=apple&debug=true&rows=10&lw.rec.fq=filters_s:"format:cd $ name:latin"
|
|
Replace EXAMPLE_COMPANY with the name provided by your Lucidworks representative.
|
Now the Boost with Signals stage will only retrieve aggregated signals that have the same filter query.
|
|
If there are multiple fq values (for example, format:cd and name:latin), they are ordered alphabetically as strings and joined with " $ " (a $ with a space on each side). In the example, "format:cd $ name:latin".
|
What if my aggregated signals are in a different collection?
You can point the Boost with Signals stage to a different signal collection by adding a collection parameter in the Solr Query Parameters section.
|
|
When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.
|
Uses recommended items for search time boosting
skip - boolean
Set to true to skip this stage.
Default: false
label - string
A unique label for this stage.
<= 255 characters
condition - string
Define a conditional script that must result in true or false. This can be used to determine if the stage should process or not.
asyncConfig - Asynchronous Execution Config
enabled - boolean
Run the expensive data loading or processing part of this stage in a separate thread allowing the pipeline to continue executing. The results of this asynchronous execution can be merged into the pipeline request using a downstream "Merge Async Results" stage.
Default: false
asyncId - string
A unique value to use as reference in downstream "Merge Async Results" stages.
numRecommendations - integer
Number of documents to set for the main query to return.
Default: 10
numSignals - integer
Number of signals to process when getting recommended items.
Default: 100
aggrType - string
Default: click@doc_id,filters,query
boostId - string
Which Solr field to use when applying recommendation boosts.
Default: id
boostingMethod - stringrequired
The boost method to use. query-parser should be chosen if defType!=edismax for main query.
Default: query-param
Allowed values: query-paramquery-parser
boostingParam - stringrequired
’Boost' multiplies scores by the boost values whereas 'bq' adds optional clauses to main query.
Default: boost
Allowed values: boostbq
scaleRange - Scale Boosts
Scale the boost values to a [min,max] range
scaleMin - number
scaleMax - number
queryParams - array[object]
Parameters for querying Signal aggregation collection
Default: {"key":"qf","value":"query_t"}{"key":"pf","value":"query_t^50"}{"key":"pf","value":"query_t~3^20"}{"key":"pf2","value":"query_t^20"}{"key":"pf2","value":"query_t~3^10"}{"key":"pf3","value":"query_t^10"}{"key":"pf3","value":"query_t~3^5"}{"key":"mm","value":"50%"}{"key":"boost","value":"map(query({!field f=query_s v=$q}),0,0,1,20)"}{"key":"defType","value":"edismax"}{"key":"sort","value":"score desc, weight_d desc"}{"key":"fq","value":"weight_d:[* TO *]"}
object attributes:{key required : {
display name: Parameter Name
type: string
}value : {
display name: Parameter Value
type: string
}}
rollupField - string
Field to use for rolling up documents that have same doc id's
rollupWeightField - string
Field to use for signal weights
rollupWeightStrategy - string
Strategy to use for rolling up the weight: max or sum
Allowed values: summax
weightExpression - string
Optional expression to compute the final boost weight using a combination of fields returned by Solr, such as score and weight_d. Set to weight_d for similar behavior as older versions
Default: math:log(weight_d + 1) + 10 * math:log(score+1)
contextKey - string
Context key to save docId:weight_d boosts to
queryParamToBoost - string
Name of the parameter in the request containing query to boost, defaults to 'q'
Default: q
includeEnrichedQuery - boolean
If enabled, the stage combines the user's original query with the output of any stages that enrich the query, such as the tagger stage. Enabling this option expands the recall of the boost lookup but may impact precision. If enabled, the stage may change the configured 'mm' parameter to accommodate additional terms added to the boost lookup query.
