Product Selector

Fusion 5.9
    Fusion 5.9

    Configure Lucidworks AI vector searchLucidworks AI

    You can use Lucidworks AI models to vectorize text for Neural Hybrid Search. This configuration uses the LWAI Vectorize Field and the LWAI Vectorize Query stages.

    Prerequisites

    In order to set up the Lucidworks AI index and query stages, you need to first set up your Lucidworks AI Gateway integration.

    Configure Lucidworks Vectorize Field index stage

    1. Sign in to Fusion and click Indexing > Index Pipelines.

    2. Click the pipeline you want to associate with Lucidworks AI.

    3. To vectorize text, click Add a new pipeline stage.

    4. In the AI section, click LWAI Vectorize Field.

    5. In the Label field, enter a unique identifier for this stage.

    6. In the Condition field, enter a script that results in true or false, which determines if the stage should process.

    7. In the Account Name field, select the Lucidworks AI API account name defined in Lucidworks AI Gateway.

      If your account name does not appear in the list or you are unsure which one to select, check your Lucidworks AI Gateway configuration.

    8. In the Model field, select the Lucidworks AI model to use for encoding.

      If your model does not appear in the list or you are unsure which one to select, check your Lucidworks AI Gateway configuration.

      For more information, see:

    9. In the Source field, enter the name of the string field where the value should be submitted to the model for encoding. If the field is blank or does not exist, this stage is not processed. Template expressions are supported.

    10. In the Destination field, enter the name of the field where the vector value from the model response is saved.

      • If a value is entered in this field, the following information is added to the document:

        • {Destination Field}_b is the boolean value if the vector has been indexed.

        • {Destination Field} is the vector field.

    11. In the Use Case Configuration section, click the + sign to enter the parameter name and value to send to Lucidworks AI. The useCaseConfig parameter that is common to embedding use cases is dataType, but each use case may have other parameters. The value for the query stage is query.

    12. The Model Configuration section is not currently available.

    13. The Call Asynchronously? check box is not currently available.

    14. Select the Fail on Error checkbox to generate an exception if an error occurs while generating a prediction for a document.

    15. Click Save.

    16. Index data using the new pipeline. Verify the vector field is indexed by confirming the field is present in documents. For reference information, see LWAI Vectorize Field.

    Configure Lucidworks Vectorize query stage

    1. Sign in to Fusion and click Querying > Query Pipelines.

    2. Click the pipeline you want to associate with Lucidworks AI.

    3. To vectorize text, click Add a new pipeline stage.

    4. Click LWAI Vectorize Query.

    5. In the Label field, enter a unique identifier for this stage.

    6. In the Condition field, enter a script that results in true or false, which determines if the stage should process.

    7. In the Account Name field, select the name of the Lucidworks AI account.

      If your account name does not appear in the list or you are unsure which one to select, check your Lucidworks AI Gateway configuration.

    8. In the Model field, select the Lucidworks AI model to use for encoding.

      If your model does not appear in the list or you are unsure which one to select, check your Lucidworks AI Gateway configuration.

      For more information, see:

    9. In the Query Input field, enter the location from which the query is retrieved.

    10. In the Output context variable field, enter the name of the variable where the vector value from the response is saved.

    11. In the Use Case Configuration section, click the + sign to enter the parameter name and value to send to Lucidworks AI. The useCaseConfig parameter that is common to embedding use cases is dataType, but each use case may have other parameters. The value for the query stage is query.

    12. The Model Configuration section is not currently available.

    13. Select the Fail on Error checkbox to generate an exception if an error occurs during this stage.

    14. Click Save.

    The Top K setting is set to 100 by default. We recommend leaving this as 100 or setting it to 200.

    This query stage must be placed before the Solr Query stage. For more information, see Reorder Query Pipeline Stages.

    Using additional pipeline stages

    Vector Search does not support all available pipeline stages. At minimum, use the Solr Query and LWAI Vectorize Query stages. Do not use the Query Fields stage when setting up vector search.

    Perform vector searches

    After setting up the stages, you can perform vector searches via the knn query parser as you would with Solr. Specify the search vector and include it in the query. For example, change the q parameter to a knn query parser string.

    The LWAI Vectorize Query stage will encode user queries using the specified model and modify the q parameter to use the knn query parser, turning the query into a vector search.