Product Selector

Fusion 5.9
    Fusion 5.9

    Configure the Smart Answers Pipelines (5.3 and later)

    Before beginning this procedure, train a machine learning model using either the FAQ method or the cold start method.

    Regardless of how you set up your model, the deployment procedure is the same:

    1. Create the Milvus collection.

    2. Configure the smart-answers index pipeline.

    3. Configure the smart-answers query pipeline.

    See also Best Practices, Advanced Model Training Configuration for Smart Answers, and Smart Answers Detailed Pipeline Setup.

    1. Create the Milvus collection

    For complete details about job configuration options, see the Create Collections in Milvus job.

    1. Navigate to Collections > Jobs > Add + and select Create Collections in Milvus.

    2. Configure the job:

      1. Enter an ID for this job.

      2. Under Collections, click Add.

      3. Enter a collection name.

      4. In the Dimension field, enter the dimension size of vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encoding model. For example, the Smart Answers Pre-trained Coldstart models outputs vectors of 512 dimension size. Dimensionality of encoders trained by Smart Answers Supervised Training job depends on the provided parameters and printed in the training job logs. See Smart Answers Supervised model training for more details.

    3. Click Save.

      The Create Collections in Milvus job can be used to create multiple collections at once. In this image, the first collection is used in the indexing and query steps. The other two collections are used in the example. Create Collections in Milvus job

    4. Click Run > Start to run the job.

    2. Configure the index pipeline

    1. Open the Index Workbench.

    2. Load or create your datasource using the default smart-answers index pipeline.

      smart-answers default index pipeline

    3. Configure the Encode into Milvus stage:

      1. change the value of Model ID to match the model deployment name you chose when you configured the model training job.

      2. Change Field to Encode to the document field name to be processed and encoded into dense vectors.

      3. Ensure the Encoder Output Vector matches the output vector from the chosen model.

      4. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.

        To test out your settings, turn on Fail on Error in the Encode into Milvus stage and Apply the changes. This will cause an error message to display if any settings need to be changed.

        Encode Into Milvus index stage

    4. Save the datasource.

    5. Index your data.

    3. Configure the query pipeline

    1. Open the Query Workbench.

    2. Load the default smart-answers query pipeline.

      smart-answers default query pipeline

    3. Configure the Milvus Query stage:

      1. Change the Model ID value to match the model deployment name you chose when you configured the model training job.

      2. Ensure the Encoder Output Vector matches the output vector from the chosen model.

      3. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.

      4. Milvus Results Context Key can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score.

        Milvus Query stage

    4. In the Milvus Ensemble Query stage, update the Ensemble math expression as needed based on your model and the name used in the prior stage for the storing the Milvus results.

      In versions 5.4 and later, you can also set the Threshold so that the Milvus Ensemble Query Stage will only return items with a score greater than or equal to the configured value.

      Milvus Ensemble Query stage

    5. Save the query pipeline.

    Pipeline Setup Example

    Index and retrieve the question and answer together

    To show question and answer together in one document (that is, treat the question as the title and the answer as the description), you can index them together in the same document. You can still use the default smart-answers index and query pipelines with a few additional changes.

    Prior to configuring the Smart Answers pipelines, use the Create Milvus Collection job to create two collections, question_collection and answer_collection, to store the encoded "questions" and the encoded "answers", respectively.

    Index Pipeline

    As shown in the pictures below, you will need two Encode into Milvus stages, named Encode Question and Encode Answer respectively.

    Encode Question (Encode Into Milvus) stage

    Pipeline setup example - Encode Question stage

    Encode Answer (Encode Into Milvus) stage

    Pipeline setup example - Encode Answer stage

    In the Encode Question stage, specify Field to Encode to be title_t and change the Milvus Collection Name to match the new Milvus collection, question_collection.

    In the Encode Answer stage, specify Field to Encode to be description_t and change the Milvus Collection Name to match the new Milvus collection, answer_collection.

    (For more detail, see Smart Answers Detailed Pipeline Setup.)

    Query Pipeline

    Since we have two dense vectors generated during indexing, at query time we need to compute both query to question distance and query to answer distance. This can be set up as the pictures shown below with two Milvus Query Stages, one for each of the two Milvus collections. To store those two distances separately, the Milvus Results Context Key needs to be different in each of these two stages.

    In the Query Questions stage, we set the Milvus Results Context Key to milvus_questions and the Milvus collection name to question_collection. .Query Questions (Milvus Query) stage Pipeline setup example - Query Questions stage

    In the Query Answers stage, we set the Milvus Results Context Key to milvus_answers and the Milvus collection name to answer_collection. .Query Answers (Milvus Query) stage Pipeline setup example - Query Answers stage

    Now we can ensemble them together with the Milvus Ensemble Query Stage with the Ensemble math expression combining the results from the two query stages. If we want the question scores and answer scores weighted equally, we would use: 0.5 * milvus_questions + 0.5 * milvus_answers. This is recommended especially when you have limited FAQ dataset and want to utilize both question and answer information.

    Milvus Ensemble Query stage

    Pipeline setup example - Milvus Ensemble Query stage

    Evaluate the query pipeline

    The Evaluate QnA Pipeline job evaluates the rankings of results from any Smart Answers pipeline and finds the best set of weights in the ensemble score. See Evaluate a Smart Answers Pipeline for setup instructions.

    Detailed pipeline setup

    Typically, you can use the default pipelines included with Fusion AI. These pipelines now utilize Milvus to store encoded vectors and to calculate vector similarity. This topic provides information you can use to customize the Smart Answers pipelines. See also Configure The Smart Answers Pipelines.

    "smart-answers" index pipeline

    smart-answers default index pipeline

    "smart-answers" query pipeline

    smart-answers default query pipeline

    Create the Milvus collection

    Prior to indexing data, the Create Collections in Milvus job can be used to create the Milvus collection(s) used by the Smart Answers pipelines (see Milvus overview).

    • Job ID. A unique identifier for the job.

    • Collection Name. A name for the Milvus collection you are creating. This name is used in both the Smart Answer Index and the Smart Answer Query pipelines.

    • Dimension. The dimension size of the vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encryption model. For example, if the model was created with either the Smart Answers Coldstart Training job or the Smart Answers Supervised Training job with the Model Base word_en_300d_2M, then the dimension would be 300.

    • Index file size. Files with more documents than this will cause Milvus to build an index on this collection.

    • Metric. The type of metric used to calculate vector similarity scores. Inner Product is recommended. It produces values between 0 and 1, where a higher value means higher similarity.

    Index pipeline setup

    Stages in the default "smart-answers" index pipeline

    smart-answers default index pipeline

    Only one custom index stage needs to be configured in your index pipeline, the Encode into Milvus index stage.

    The Encode into Milvus Index Stage

    If you are using a dynamic schema, make sure this stage is added after the Solr Dynamic Field Name Mapping stage.

    The Encode into Milvus index stage uses the specified model to encode the Field to Encode and store it in Milvus in the given Milvus collection. There are several required parameters:

    • Model ID. The ID of the model.

    • Encoder Output Vector. The name of the field that stores the compressed dense vectors output from the model. Default value: vector.

    • Field to Encode. The text field to encode into a dense vector, such as answer_t or body_t.

    • Milvus Collection Name. The name of the collection you created via the Create Milvus Collection job, which will store the dense vectors. When creating the collection you specify the type of Metric to use to calculate vector similarity. This stage can be used multiple times to encode additional fields, each into a different Milvus collection. See how to index and retrieve the question and answer together.

    Query pipeline setup

    The Query Fields stage

    The first stage is Query Fields. For more information see the Query Fields stage.

    The Milvus Query stage

    The Milvus Query stage encodes the query into a vector using the specified model. It then performs a vector similarity search against the specified Milvus collection and returns a list of the best document matches.

    • Model ID. The ID of the model used when configuring the model training job.

    • Encoder Output Vector. The name of the output vector from the specified model, which will contain the query encoded as a vector. Defaults to vector.

    • Milvus Collection Name. The name of the collection that you used in the Encode into Milvus index stage to store the encoded vectors.

    • Milvus Results Context Key. The name of the variable used to store the vector distances. It can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score for the document.

    • Number of Results. The number of highest scoring results returned from Milvus. This stage would typically be used the same number of times that the Encode into Milvus index stage is used, each with a different Milvus collection and a different Milvus Results Context Key.

    The Milvus Ensemble Query stage

    The Milvus Ensemble Query takes the results of the Milvus Query stage(s) and calculates the ensemble score, which is used to return the best matches.

    • Ensemble math expression. The mathematical expression used to calculate the ensemble score. It should reference the value(s) variable name specified in the Milvus Results Context Key parameter in the Milvus Query stage.

    • Result field name. The name of the field used to store the ensemble score. It defaults to ensemble_score.

    • Threshold- A parameter that filters the stage results to remove items that fall below the configured score. Items with a score at, or above, the threshold will be returned.

    The Threshold feature is only available in Fusion 5.4 and later.

    The Milvus Response Update Query stage

    The Milvus Response Update Query stage does not need to be configured and can be skipped if desired. It inserts the Milvus values, including the ensemble_score, into each of the returned documents, which is particularly useful when there is more than one Milvus Query Stage. This stage needs to come after the Solr Query stage.

    Short answer extraction

    By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question. See Extract Short Answers from Longer Documents.