> ## Documentation Index > Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt > Use this file to discover all available pages before exploring further. # Fine-tune Neural Hybrid Search pipelines > [card] Key questions to get you started > > Powered by Lucidworks Agent Studio Neural Hybrid Search (NHS) combines three interacting scoring systems: vector similarity, powered by an embedding model and Solr's Hierarchical Navigable Small World (HNSW) implementation; lexical scoring, which is Best Matching 25 (BM25) by default; and post-retrieval boosting, such as business rules and signals metadata. Each system produces scores on a different numeric scale and exposes its own controls, and the Neural Hybrid Query (NHQ) stage aligns them into a comparable score space. With three hyperparameters to tune, the combination space is vast, but many combinations produce similar results. Start simple and keep each hyperparameter between 0 and 1. Higher values are possible, but the wider the range, the harder tuning becomes. Follow the sequence below instead of testing every combination. ## Prepare your pipeline Before tuning, confirm your pipeline follows the stage ordering requirements in [Choosing, implementing, and troubleshooting NHS stages](https://doc.lucidworks.com/docs/5/fusion/hybrid-search/hybrid-stage-differences.md). Then apply the following guidance. 1. **Encode the raw user query.** Vectorize the original query the user typed, not an augmented version of it. Place the Vectorize Query stage as early as possible in the pipeline, before any stage that rewrites the query. If an earlier stage, such as spell correction, changes the query, save the raw query to a separate parameter first and pass that parameter into the Vectorize Query stage. > [!WARNING] > > Avoid passing a Solr-respelled or rule-rewritten query into the Vectorize Query stage, especially for custom-trained embedding models. > Spelling or rewrite changes can introduce incorrect associations that a custom model wasn't trained to handle. 2. **Fix highlighting before the NHQ stage.** Highlighting can conflict with hybrid stages, causing query failures such as `Index _ out of bounds for length _`. To resolve this, set `hl.q` to the lexical-only query parameter (``) before the hybrid stage, or to `` after it. 3. **Preserve the pre-NHQ query for boosting stages.** The NHQ stage rewrites `q` for the Solr query, so any boosting stage that runs after NHQ can't reliably use `q` to reference the original query text. Before the NHQ stage fires, copy `q` to a separate parameter for boosting stages to reference, or use ``, which the NHQ stage populates with the original query. > [!NOTE] > > Add a "Copy Original Query" stage at the very beginning of the pipeline to make this copy operation explicit, and use that parameter as the input for your vectorize, boost, and highlighting stages. 4. **Move security and ACL filters into vector prefiltering.** If your pipeline applies security or access-control filters, such as graph joins, site restrictions, or user-group exclusions, review where those filters run relative to vector search. Filters applied after vector retrieval let semantically similar but disallowed documents occupy the candidate pool. They're removed from the response, but they've already displaced valid results. Uncheck **Block pre-filtering** in the NHQ stage, then follow the linked procedure to configure vector prefilters through a Graph Security Trimming or JavaScript stage after NHQ. Retain top-level security filters so access restrictions also apply to lexical matches. For implementation details, see [Neural Hybrid Query stage](https://doc.lucidworks.com/docs/5/fusion/reference/config-ref/pipeline-stages/query-stages/neural-hybrid-query.md). 5. **Plan your index encoding string.** Decide what content makes up the string sent to the vectorize stage at index time, for example: ``` encoding_string = title + " " + description + " " + body ``` Cap the string length, for example at 10,000 characters, to keep indexing time predictable and truncation behavior consistent. Consider tracking the word count of the encoding string in a dedicated index field for monitoring and analysis. > [!NOTE] > > Embedding models have a fixed token limit, and content beyond that limit is silently truncated regardless of how long your encoding string is. > For knowledge management use cases with long documents, passing the full document as-is means only the opening content gets semantically represented. > Content deeper in the document isn't discoverable through vector search no matter how well you tune the hybrid parameters. > Chunk long content instead, or fall back to an AI-generated summary or extracted keywords as the encoding input. > See [Chunking](https://doc.lucidworks.com/docs/5/fusion/hybrid-search/chunking.md). ### Minimum recommended stage order 1. Vectorize Query 2. Query Fields 3. Neural Hybrid Query 4. Boost with Signals 5. Apply Rules 6. Solr Query ## Set a baseline for evaluation Evaluating against a golden or silver dataset with offline metrics is essential. Without it, you can't measure whether a change improves or hurts your results. See [Semantic vector search test guidelines](https://doc.lucidworks.com/docs/5/fusion/hybrid-search/testing-svs.md) for guidance on building a golden dataset and calculating metrics. > [!WARNING] > > If you use the Boost with Signals stage and build your golden set from the same signals, you may end up evaluating a simple lookup rather than search quality, since boosting operates on the same signals used to create the evaluation set. > Your metrics can end up biased, tainted by noisy signals, or misaligned with business priorities. > [!WARNING] > > If a site previously ran lexical-only search, its historical signals reflect what users clicked in lexical results, not what they would have clicked in semantic results. > An evaluation dataset built from those signals will structurally favor the lexical baseline. > For an unbiased evaluation, use a human-curated query-document relevance set built independently of historical click behavior, or plan to recollect signals after NHS is live and use them for a follow-up evaluation pass. Recommended primary metric by use case: | Use case | Metric | Why | | - | - | - | | E-commerce | NDCG\@5 | Incorporates ranking and recall of all relevant products | | Knowledge management | MRR\@3 | Focuses on early relevance | ## Tune parameters step by step Start broad, then narrow, changing one variable at a time so you can attribute changes in your offline metrics to a specific parameter. **1. Set initial NHQ parameters wide open** Before running any evaluations, set: * Number of Vector Results (`topK`): 1000 or higher * Min Return Vector Similarity: 0.0 to 0.5 Early tuning needs recall-first behavior so you can see scoring dynamics. The stage configuration accepts `topK` values from 1 to 1000. To test higher values, add an Additional Query Parameters stage after the NHQ stage that replaces the `vec_sim_q` parameter with `{!knn f=$vec_field v=$vec_q topK=$top_k preFilter=$vectorPreFilter}` and sets `top_k` to your value. Fusion 6 raises this limit. Tight similarity cutoffs or a low `topK` hide downstream tuning effects and starve boosting stages of candidates to act on. HNSW recall is imperfect, so Solr's implementation often needs a higher `topK` to ensure the best candidates are retrieved at all. An ensemble of scoring algorithms needs a good pool of candidates from each one. Limiting `topK` or raising the similarity cutoff too early starves the re-ranking step of candidates before boosting or weighting ever gets a chance to act. **2. Remove complexity** Turn off the Boost with Signals stage and any large filters. Boosting strongly influences result order and can mask the quality of the underlying hybrid search. Turn it off first so you can tune the best possible NHS configuration, then re-enable boosting to add further lift. Large or complex filters similarly make it hard to tell whether a change in results came from your hybrid tuning or the filters, so simplify until the base configuration is set. **3. Choose an embedding model** Lucidworks hosts pre-trained open-source models and supports custom trainable models, as well as Ray/Seldon deployments for client-managed models. * **Pre-trained models** are the right choice for a proof of value on an expedited timeline, or when you don't yet have enough signals to train a custom model. `snowflake-arctic-embed-m-v2.0` is a good default: a solid performance/speed trade-off with multilingual support. For this model, reducing dimensions to 256 (from 768) with int8 quantization optimizes Solr storage. * **Custom embedding models**, trained on your own data, generally produce the highest accuracy once you have enough signals (as a rule of thumb, more than 1,000 unique queries). E-commerce datasets tend to see the largest improvement from a custom model relative to a pre-trained one. * **Client-managed (Ray/Seldon) models** are supported for teams that want to deploy and manage their own model, but this path works best when you also have the data science resources to tune it. > [!NOTE] > > Embedding models have a maximum token context window, and content beyond that limit is truncated before vectorization. > Lucidworks pre-trained models are currently limited to 512 tokens, roughly 2,000 characters. > Separately, the vectorization API rejects a raw input payload larger than 1 MB. > If document body size is variable, cap your encoding string at a reasonable character limit, for example 20,000 characters, to avoid payload errors and keep truncation predictable. **4. Tune the squash factor** With a model selected and vectors indexed, run your first round of evaluations varying only the squash factor in the NHQ stage, with boosting still disabled. Run at least 5 iterations, then select the best-performing value before moving on. **Suggested starting values:** 0.001, 0.01, 0.05 **How squash works:** the NHQ stage compresses the raw lexical (BM25) score into a comparable range using: ``` squashed_score = tanh(α × lexical_score) final_score = squashed_score × lexical_weight + vector_score × vector_weight ``` The `tanh` function produces an S-shaped curve: it rises steeply near zero, passes through a roughly linear "sweet spot," then flattens asymptotically toward 1.0 as inputs grow large. The squash factor (α) controls how quickly the curve rises, and therefore which range of BM25 scores falls in the useful, linear portion of the curve. The question to ask when choosing α is where your relevant documents' BM25 scores typically fall. You want that range to land in the rising, non-saturated part of the curve, where score differences between documents survive squashing. If those scores fall in the flat region instead, they collapse to nearly the same value and the lexical ranking signal is lost. > [!TIP] > > To detect saturation, inspect the Solr explain output for your top results. > If squashed values cluster near `0.999`, decrease the squash factor (α), because it's too high and every high BM25 score is being collapsed to the same value. > If lexical scores seem to dominate despite a low lexical weight, increase the squash factor (α), because it may be too low. > > As a starting point, if you know the approximate minimum and maximum BM25 scores among your top relevant results, this formula maps the midpoint of that range to the center of the `tanh` sweet spot: > > ``` > α ≈ 2.0 / (x_min + x_max) > ``` > > Only α is configurable. > The shape of the `tanh` function itself is fixed. **5. Tune lexical and vector weights** Keep your squash factor fixed while you test weight combinations. Vector and lexical weights should add up to 1. **Starting values to test:** | Lexical weight | Vector weight | | - | - | | 0.9 | 0.1 | | 0.5 | 0.5 | | 0.1 | 0.9 | Don't let a preferred weight combination, whether from experience or a stakeholder request, stop you from testing the full range. The goal is the best-performing setup, not the most familiar one. Production e-commerce deployments with a strong vector model have landed on weight combinations as lopsided as 0.99 vector to 0.01 lexical, which can look counterintuitive until your evaluation metrics back it up. > [!TIP] > > Treat squash and weights as a coupled pair rather than independent settings. > In a controlled evaluation of a knowledge-management use case across several embedding models, a vector-heavy weighting (20% lexical / 80% vector) performed best paired with a low squash factor (≈0.02), while a lexical-heavy weighting (80% lexical / 20% vector) needed a much smaller squash factor (≈0.0002) to preserve lexical score differentiation. > The vector-heavy weighting paired with the smaller squash factor consistently underperformed across every model tested, so avoid that combination. > The best-performing combination improved MRR\@5 and MAP\@5 by roughly 14% over a lexical-only baseline. > If a weight combination isn't performing as expected, check whether your squash factor is still aligned with it. **6. Tune boosting scale** With squash and weights fixed, enable your boosting stage. Run a baseline with boost scaling disabled, then test the Boost with Signals stage's **Scale Boosts** setting with the following ranges. **Starting values to test:** | Scaling minimum | Scaling maximum | | - | - | | 0.01 | 0.9 | | 0.5 | 50 | | 1 | 100 | If scaling has little effect on your metrics, a minimum of 0.01 and a maximum of 0.9 is a reasonable default. This keeps scores within the 0 to 0.9 range, which aligns with Solr's hybrid scoring. > [!TIP] > > If your pipeline had signals-based boosting tuned before you enabled NHS, revisit those boost ranges. > They were likely calibrated against unsquashed BM25 scores and are now operating in the wrong scale. > One deployment needed its signals boost range widened substantially after enabling squash, from a single-digit multiplier range to a much wider one, along with an updated boost weight expression to compensate for the compressed score space. > If signals-based recommendations seem to have less influence than expected after enabling NHS, squash-induced score compression is a likely cause. **7. Tune topK** Keep boosting enabled and adjust Number of Vector Results (`topK`). Smaller values keep response time predictable, but Solr's HNSW implementation hardcodes `efSearch` to `topK × 2`, which has a large influence on recall quality. As a result, a higher `topK` sometimes improves recall enough to be worth the added latency. **Starting values to test:** 200, 500, 1000, and, if you use the `top_k` override described earlier, 2000 and 5000 > [!NOTE] > > The actual number of vector candidates retrieved is `topK × shard count`. > A single shard is common in e-commerce deployments, but enterprise document repositories often use multiple shards. > In a multi-shard environment, a `topK` of 200 across 5 shards yields up to 1,000 candidates before re-ranking. > Factor this into your latency expectations. **8. Tune the cosine similarity cutoff** Keep boosting enabled and set Min Return Vector Similarity, which filters out document similarities below a score threshold rather than limiting by `topK`. This parameter isn't strictly necessary, because leaving it low allows full recall of the `topK` candidates. However, it can reduce irrelevant results in the candidate pool, improve offline metrics, and make relevancy more consistent when filters are applied to results beyond the first page. Setting it too high can return null or zero results, so validate against your evaluation set as you raise it. **Starting values to test:** 0.5, 0.6, 0.7, 0.8 At this point your pipeline should have its best combination of hybrid parameters and boost scaling. Re-enable anything you turned off earlier, such as large filters, and move on to production readiness. ## Backfill embeddings on existing collections When you enable NHS on an existing Solr collection, documents indexed before NHS was implemented won't have vector fields and won't participate in vector search, which can produce inconsistent hybrid results. 1. Configure a datasource that queries for documents missing an embedding. Solr doesn't support existence queries (`field:*`) on vector field types, so use the boolean field the indexing vectorization stage writes whenever it successfully adds an embedding (`_b`) for existence filtering instead: ``` -_b:* AND :* ``` Match `` to whichever fields your pipeline uses to build the encoding string. 2. Feed the results through your index pipeline's embedding stages using a partial update pipeline, which writes only the embedding-related fields and doesn't overwrite other document data. 3. Set `rejectUpdatesIfDocNotPresent=true` on the partial update stage so the backfill job only modifies documents that already exist in the collection, which prevents accidentally creating incomplete records if a document was deleted between query time and update time. 4. Monitor backfill progress by querying for remaining documents where the boolean presence field is absent. > [!NOTE] > > Documents without embeddings continue to return lexical-only results until they're backfilled. > You don't need to pause production traffic during backfill, because the pipeline serves the best available result type per document. ## Verify production readiness Before promoting an NHS-enabled pipeline to production, review every stage for logging or diagnostic behavior intended for development use. Verbose logging under production query load can cause log noise and performance degradation, and stages that append diagnostic data to query responses can expose internal pipeline state to API consumers. Disable or remove these stages before go-live. Also confirm: * The query profile routes to the correct pipeline. * Solr configs are enriched with the vector field definitions and parsers your pipeline expects. * The Lucidworks AI gateway production account is configured and enabled for relevant users. * Every datasource feeding the collection uses the NHS-enabled index pipeline, if applicable. * Embedding coverage is verified. If needed, run a backfill first. * Hybrid search parameters are tested under production-scale query load. * Security/ACL filter migration to vector prefiltering is tested with production ACL data. * Signals boost ranges are recalibrated for the hybrid score scale, if signals boosting is in use. * A rollback plan is confirmed: know which query profile change reverts to the legacy lexical pipeline. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.