0..inf
to 0..1
.
This setting helps prevent the lexical query from dominating the final score, and normalizes the score into a range that works well with vector similarity scores.
Additionally, it helps prevent the vanishing gradient problem, which occurs when very high lexical scores are mapped to values extremely close to 1
, such as 0.99999999
.
During the hybrid search calculation, these near-1 values can cause the system to lose sensitivity to subtle differences in lexical relevance, effectively ‘squashing’ the gradient and reducing the impact of lexical scoring.
Lucidworks recommends setting the Lexical Query Squash Factor to the inverse of the maximum lexical score observed across your queries.
This helps balance the impact of lexical and vector scores, leading to more accurate and nuanced search results.
preFilterKey
object to the Javascript ctx
object.preFilterKey
object, as in this example:
\t
for the tab character. When entering configuration values in the API, use escaped characters, such as \\t
for the tab character.