Product Selector

Fusion 5.12
    Fusion 5.12

    Configure Lucidworks AI vector searchLucidworks AI

    You can use Lucidworks AI models to vectorize text for Neural Hybrid Search.

    This feature is currently only available to clients who have contracted with Lucidworks for features related to Neural Hybrid Search and Lucidworks AI.

    This feature is only available in Managed Fusion 5.9.5.

    Configure LWAI Vectorize Field Index stage

    To vectorize the index pipeline fields:

    1. Sign into Managed Fusion, go to Indexing > Index Pipelines, then select an existing pipeline or create a new one. You can also use the default LWAI vectorize pipeline.

    2. Click Add a new pipeline stage, then select LWAI Vectorize Field.

    3. In the Label field, enter a unique identifier for this stage or leave blank to use the default value.

    4. In the Condition field, enter a script that results in true or false, which determines if the stage should process, or leave blank.

    5. Select the Account Name from the dropdown list. If Lucidworks AI is configured, your account name appears in this list.

    6. Select the Model from the dropdown list to use for encoding. For more information, see:

    7. Enter the Source Field. For example, body_t. Enter the name of the string field where the value should be submitted to the model for encoding. If the field is blank or does not exist, this stage is not processed. Template expressions are supported. For example, body_t.

    8. Enter the Destination Field. For example, body_512v. In the Destination field, enter the name of the field where the vector value from the model response is saved.

    9. In the Use Case Configuration section, click the + sign to enter the parameter name and value to send to Lucidworks AI. For example, a parameter name of dataType and a value of passage or query

      • This prefix can improves results when using the e5-small-v2 model. For more information refer to the Hugging Face FAQ.

    10. The Model Configuration section is not applicable to vector search.

    11. Select the Fail on Error checkbox to generate an exception if an error occurs while generating a prediction for a document.

    12. Click Save.

    Index data using the new pipeline. Verify the vector field is indexed by confirming the field is present in documents.

    For reference information, see Lucidworks AI Vectorize Field.

    Configure LWAI Vectorize Query stage

    To vectorize the query in the query pipeline:

    1. Sign into Managed Fusion, go to Querying > Query Pipelines, then select an existing pipeline or create a new one. You can also use the default LWAI Neural Hybrid Search pipeline.

    2. Click Add a new pipeline stage, then select LWAI Vectorize Query.

    3. In the Label field, enter a unique identifier for this stage or leave blank to use the default value.

    4. In the Condition field, enter a script that results in true or false, which determines if the stage should process, or leave blank.

    5. Select the Account Name from the dropdown list. If Lucidworks AI is configured, your account name appears in this list.

    6. Select the Model from the dropdown list to use for encoding. For more information, see:

    7. In the Query Input field, enter the location from which the query is retrieved.

    8. In the Output Context Variable field, enter the name of the variable where the vector value from the response is saved.

    9. In the Use Case Configuration section, click the + sign to enter the parameter name and value to send to Lucidworks AI. The useCaseConfig parameter that is common to embedding use cases is dataType, but each use case may have other parameters. The value for the query stage is query.

    10. The Model Configuration section is not applicable to vector search.

    11. Select the Fail on Error checkbox to generate an exception if an error occurs during this stage.

    12. Click Save.

    This query stage must be placed before the Solr Query stage. For more information, see Reorder Query Pipeline Stages.

    For reference information, see Lucidworks AI Vectorize Query.

    Perform vector searches

    After setting up the stages, you can perform vector searches via the knn query parser as you would with Solr. Specify the search vector and include it in the query. For example, change the q parameter to a knn query parser string.

    The LWAI Vectorize Query stage will encode user queries using the specified model and modify the q parameter to use the knn query parser, turning the query into a vector search.