Smart Answers - Lucidworks documentation

Managed Fusion Smart Answers brings the benefits of a versatile, scalable Semantic Search platform to provide cutting edge relevancy for your applications. This system makes use of advanced Deep Learning techniques to provide the power of Neural dense vectors search. Semantically rich models encode queries and documents into vectors in the same digital vector space in such way that query and the most relevant information are located near each other. It pushes search boundaries beyond classical token-based matching mechanisms by leveraging the contextual information and query understanding. Virtual assistants and Chatbots can incorporate Smart Answers to enable self-service for employees and customers and save cost on answering incoming queries. E-commerce can utilize Smart Answers to handle zero-result search queries and improve the overall relevancy by recommending products that are most likely of users interest. Training data can be explicitly provided as query/response pairs or constructed from the collected signals data. Even if you do not have an existing set of recorded interactions, you can rely on Smart Answers cold start solution. It uses various pre-trained models to get out-of-the-box (OOTB) semantic capabilities as well as provides a possibility to train unsupervised models for specific domains. These features bring traditional search relevancy development and Data Science together into an easy-to-use configuration framework that leverages Managed Fusion’s indexing and querying pipelines.

LucidAcademyLucidworks offers free training to help you get started.The Course for Smart Answers focuses on how to utilize Smart Answers to go beyond classical token-based matching mechanisms by leveraging contextual information and query understanding:

Visit the LucidAcademy to see the full training catalog.

Example business use cases

Call center or IT support

Embed this system as a self-help feature on your Help page or Contact Us page to reduce the call center load.
Make it available to your customer support team to find the answers to already-solved problems.

Zero-result search problem in E-commerce: Queries that lead to zero-results is a huge problem for E-commerce that leads to income loss and decrease of the overall users experience. Semantic search does not have this problem as it does not operate on the exact token matches as classical search. Instead search is done in the vector space which can find relevant products even if there is no token overlapping. Questions about products for E-commerce: E-commerce websites can use this system to search “how to” content, product user manuals, or existing product questions. For example, amazon.com provides a search function on questions about each product. Search in Slack, email conversations, or SharePoint FAQ docs You can achieve fast knowledge extraction by applying this solution to these types of knowledge repositories. Improve search for long queries As the solution utilizes state of the art Deep Learning techniques, it is able to capture semantic and contextual information into query understanding. This makes it very suitable to work with long queries or natural language questions.

Getting started

To get started, you need a trained machine learning model. There are two methods for building a model, depending on the kind of data you already have. Each method requires a slightly different model training procedure, but the model deployment procedure is the same for both. The Supervised solution: Use this solution when you already have a collection of query/response pairs or if you can construct such a dataset from the signals data. Workflow using input of a dataset of query/response pairs.

Train a Smart Answers Supervised Model

The Supervised solution for Smart Answers begins with training a model using your existing data and the Smart Answers Supervised Training job, as explained in this topic. The job includes an auto-tune feature that you can use instead of manually tuning the configuration.

Training job requirements

Storage150GB plus 2.5 times the total input data size.Processor and memoryThe memory requirements depend on whether you choose GPU or CPU processing:


GPU	CPU
one core 11GB RAM	32 cores 32GB RAM

If your training data contains more than 1 million entries, use GPU.

Prepare the input data

Format your input data as question/answer pairs, that is, a query and its corresponding response in each row. You can do this in any format that Managed Fusion supports. If there are multiple possible answers for a unique question, then repeat the questions and put the pair into different rows to make sure each row has one question and one answer, as in the example JSON below:
```
[{"question":"How to transfer personal auto lease to business auto lease?","answer":"I would approach the lender that you are getting the lease from..."}
 {"question":"How to transfer personal auto lease to business auto lease?","answer":"See what the contract says about transfers or subleases..."}]
```
Index the input data in Managed Fusion. If you wish to have the training data in Managed Fusion, index it into a separate collection for training data such as model_training_input. Otherwise you can use it directly from the cloud storage.

Configure the training job

In Managed Fusion, navigate to Collections > Jobs.
Select Add > Smart Answers Supervised Training:
In the Training Collection field, specify the input data collection that you created when you prepared the input data.
You can also configure this job to read from or write to cloud storage.
Enter the names of the Question Field and the Answer Field in the training collection.
Enter a Model Deployment Name. The new machine learning model will be saved in the blob store with this name. You will reference it later when you configure your pipelines.
Configure the Model base. There are several pre-trained word and BPE embeddings for different languages, as well as a few pre-trained BERT models. If you want to train custom embeddings, select word_custom or bpe_custom. This trains Word2vec on the provided data and specified fields. It might be useful in cases when your content includes unusual or domain-specific vocabulary. If you have content in addition to the query/response pairs that can be used to train the model, then specify it in the Texts Data Path. When you use the pre-trained embeddings, the log shows the percentage of processed vocabulary words. If this value is high, then try using custom embeddings. The job trains a few (configurable) RNN layers on top of word embeddings or fine-tunes a BERT model on the provided training data. The result model uses an attention mechanism to average word embeddings to obtain the final single dense vector for the content.
Dimension size of vectors for Transformer-based models is 768. For RNN-based models it is 2 times the number units of the last layer. To find the dimension size: download the model, expand the zip, open the log and search for Encoder output dim size: line. You might need this information when creating collections in Milvus.
Optional: Check Perform auto hyperparameter tuning to use auto-tune. Although training module tries to select the most optimal default parameters based on the training data statistics, auto-tune can extend it by automatically finding even better training configuration through hyper-parameter search. Although this is a resource-intensive operation, it can be useful to identify the best possible RNN-based configuration. Transformer-based models like BERT are not used during auto hyperparameter tuning as they usually perform better yet they are much more expensive on both training and inference time.
Click Save.
If using solr as the training data source ensure that the source collection contains the random_* dynamic field defined in its managed-schema.xml. This field is required for sampling the data. If it is not present, add the following entry to the managed-schema.xml alongside other dynamic fields <dynamicField name="random_*" type="random"/> and <fieldType class=“solr.RandomSortField” indexed=“true” name=“random”/> alongside other field types.
Click Run > Start.

After training is finished the model is deployed into the cluster and can be used in index and query pipelines.

Next steps

See A Smart Answers Supervised Job’s Status and Output
Configure The Smart Answers Pipelines
Evaluate a Smart Answers Query Pipeline

Configure the Smart Answers Pipelines (5.3 and later)

Before beginning this procedure, train a machine learning model using either the FAQ method or the cold start method.Regardless of how you set up your model, the deployment procedure is the same:

Create the Milvus collection.
Configure the smart-answers index pipeline.
Configure the smart-answers query pipeline.

See also Best Practices, Advanced Model Training Configuration for Smart Answers, and Smart Answers Detailed Pipeline Setup.

Create the Milvus collection

For complete details about job configuration options, see the Create Collections in Milvus job.

Navigate to Collections > Jobs > Add + and select Create Collections in Milvus.
Configure the job:
1. Enter an ID for this job.
2. Under Collections, click Add.
3. Enter a collection name.
4. In the Dimension field, enter the dimension size of vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encoding model. For example, the Smart Answers Pre-trained Coldstart models outputs vectors of 512 dimension size. Dimensionality of encoders trained by Smart Answers Supervised Training job depends on the provided parameters and printed in the training job logs.
Click Save. The Create Collections in Milvus job can be used to create multiple collections at once. In this image, the first collection is used in the indexing and query steps. The other two collections are used in the pipeline setup example.
Click Run > Start to run the job.

Configure the index pipeline

Open the Index Workbench.
Load or create your datasource using the default smart-answers index pipeline.
Configure the Encode into Milvus stage:
1. change the value of Model ID to match the model deployment name you chose when you configured the model training job.
2. Change Field to Encode to the document field name to be processed and encoded into dense vectors.
3. Ensure the Encoder Output Vector matches the output vector from the chosen model.
4. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
  To test out your settings, turn on Fail on Error in the Encode into Milvus stage and Apply the changes. This will cause an error message to display if any settings need to be changed.
Save the datasource.
Index your data.

Configure the query pipeline

Open the Query Workbench.
Load the default smart-answers query pipeline.
Configure the Milvus Query stage:
1. Change the Model ID value to match the model deployment name you chose when you configured the model training job.
2. Ensure the Encoder Output Vector matches the output vector from the chosen model.
3. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
4. Milvus Results Context Key can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score.
In the Milvus Ensemble Query stage, update the Ensemble math expression as needed based on your model and the name used in the prior stage for the storing the Milvus results. In versions 5.4 and later, you can also set the Threshold so that the Milvus Ensemble Query Stage will only return items with a score greater than or equal to the configured value.
Save the query pipeline.

Pipeline Setup Example

Index and retrieve the question and answer together

To show question and answer together in one document (that is, treat the question as the title and the answer as the description), you can index them together in the same document. You can still use the default smart-answers index and query pipelines with a few additional changes.Prior to configuring the Smart Answers pipelines, use the Create Milvus Collection job to create two collections, question_collection and answer_collection, to store the encoded “questions” and the encoded “answers”, respectively.

Index Pipeline

As shown in the pictures below, you will need two Encode into Milvus stages, named Encode Question and Encode Answer respectively.Encode Question (Encode Into Milvus) stage

Pipeline setup example - Encode Question stage

Encode Answer (Encode Into Milvus) stage

Pipeline setup example - Encode Answer stage

In the Encode Question stage, specify Field to Encode to be title_t and change the Milvus Collection Name to match the new Milvus collection, question_collection.In the Encode Answer stage, specify Field to Encode to be description_t and change the Milvus Collection Name to match the new Milvus collection, answer_collection.

Query Pipeline

Since we have two dense vectors generated during indexing, at query time we need to compute both query to question distance and query to answer distance. This can be set up as the pictures shown below with two Milvus Query Stages, one for each of the two Milvus collections. To store those two distances separately, the Milvus Results Context Key needs to be different in each of these two stages.In the Query Questions stage, we set the Milvus Results Context Key to milvus_questions and the Milvus collection name to question_collection.Query Questions (Milvus Query) stage:

Pipeline setup example - Query Questions stage

In the Query Answers stage, we set the Milvus Results Context Key to milvus_answers and the Milvus collection name to answer_collection.Query Answers (Milvus Query) stage:

Pipeline setup example - Query Answers stage

Now we can ensemble them together with the Milvus Ensemble Query Stage with the Ensemble math expression combining the results from the two query stages. If we want the question scores and answer scores weighted equally, we would use: 0.5 * milvus_questions + 0.5 * milvus_answers. This is recommended especially when you have limited FAQ dataset and want to utilize both question and answer information.Milvus Ensemble Query stage

Pipeline setup example - Milvus Ensemble Query stage

Evaluate the query pipeline

The Evaluate QnA Pipeline job evaluates the rankings of results from any Smart Answers pipeline and finds the best set of weights in the ensemble score.

Detailed pipeline setup

Typically, you can use the default pipelines included with Fusion AI. These pipelines now utilize Milvus to store encoded vectors and to calculate vector similarity. This topic provides information you can use to customize the Smart Answers pipelines.


”smart-answers” index pipeline		Encode into Milvus stage
”smart-answers” query pipeline		Query Fields Milvus Query Milvus Ensemble Query Milvus Response Update

Create the Milvus collection

Prior to indexing data, the Create Collections in Milvus job can be used to create the Milvus collection(s) used by the Smart Answers pipelines (see Milvus overview).

Job ID. A unique identifier for the job.
Collection Name. A name for the Milvus collection you are creating. This name is used in both the Smart Answer Index and the Smart Answer Query pipelines.
Dimension. The dimension size of the vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encryption model. For example, if the model was created with either the Smart Answers Coldstart Training job or the Smart Answers Supervised Training job with the Model Base word_en_300d_2M, then the dimension would be 300.
Index file size. Files with more documents than this will cause Milvus to build an index on this collection.
Metric. The type of metric used to calculate vector similarity scores. Inner Product is recommended. It produces values between 0 and 1, where a higher value means higher similarity.

Index pipeline setup

Stages in the default “smart-answers” index pipeline

Only one custom index stage needs to be configured in your index pipeline, the Encode into Milvus index stage.

The Encode into Milvus Index Stage

If you are using a dynamic schema, make sure this stage is added after the Solr Dynamic Field Name Mapping stage.

The Encode into Milvus index stage uses the specified model to encode the Field to Encode and store it in Milvus in the given Milvus collection. There are several required parameters:

Model ID. The ID of the model.
Encoder Output Vector. The name of the field that stores the compressed dense vectors output from the model. Default value: vector.
Field to Encode. The text field to encode into a dense vector, such as answer_t or body_t.
Milvus Collection Name. The name of the collection you created via the Create Milvus Collection job, which will store the dense vectors. When creating the collection you specify the type of Metric to use to calculate vector similarity. This stage can be used multiple times to encode additional fields, each into a different Milvus collection.

Query pipeline setup

The Query Fields stage

The first stage is Query Fields. For more information see the Query Fields stage.

The Milvus Query stage

The Milvus Query stage encodes the query into a vector using the specified model. It then performs a vector similarity search against the specified Milvus collection and returns a list of the best document matches.

Model ID. The ID of the model used when configuring the model training job.
Encoder Output Vector. The name of the output vector from the specified model, which will contain the query encoded as a vector. Defaults to vector.
Milvus Collection Name. The name of the collection that you used in the Encode into Milvus index stage to store the encoded vectors.
Milvus Results Context Key. The name of the variable used to store the vector distances. It can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score for the document.
Number of Results. The number of highest scoring results returned from Milvus. This stage would typically be used the same number of times that the Encode into Milvus index stage is used, each with a different Milvus collection and a different Milvus Results Context Key.

The Milvus Ensemble Query stage

The Milvus Ensemble Query takes the results of the Milvus Query stage(s) and calculates the ensemble score, which is used to return the best matches.

Ensemble math expression. The mathematical expression used to calculate the ensemble score. It should reference the value(s) variable name specified in the Milvus Results Context Key parameter in the Milvus Query stage.
Result field name. The name of the field used to store the ensemble score. It defaults to ensemble_score.
Threshold- A parameter that filters the stage results to remove items that fall below the configured score. Items with a score at, or above, the threshold will be returned.

The Threshold feature is only available in Fusion 5.4 and later.

The Milvus Response Update Query stage

The Milvus Response Update Query stage does not need to be configured and can be skipped if desired. It inserts the Milvus values, including the ensemble_score, into each of the returned documents, which is particularly useful when there is more than one Milvus Query Stage. This stage needs to come after the Solr Query stage.

Short answer extraction

By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question.

Evaluate a Smart Answers Query Pipeline

This topic explains how you can track the running steps of the following jobs:

Smart Answers Supervised Training
Smart Answers Coldstart Training
Smart Answers Evaluate Pipeline

The ML Model service logs provide information on data pre-processing, training steps, evaluations, and model generation for those jobs. Follow the instructions below to access the logs.

Find the name of the ML Model service pod:
```
kubectl get pods -n <your-namespace> | grep ml-model-service-ui
```
The pod name looks like <namespace>-ml-model-service-ui-<hash>-<random>, as in f5-ml-model-service-ui-547dd78d6-p9d6q.

Set up port forwarding:

kubectl port-forward <namespace>-ml-model-service-ui-<hash>-<random> 8001:8001 -n <your-namespace>

In the Fusion UI, navigate to Collections > Jobs.
Select your Smart Answers (QnA) model training job.
Click Job History.
Find the ML Model workflow ID.
In the ML Model service UI, locate instances of the ID in the logs.

The Cold Start solution: Use this solution when you have no historical training data or fewer than 200 query/response pairs. Workflow 1:

Set Up a Pre-Trained Cold Start Model for Smart Answers

Configure the Smart Answers Pipelines (5.3 and later)

Before beginning this procedure, train a machine learning model using either the FAQ method or the cold start method.Regardless of how you set up your model, the deployment procedure is the same:

Create the Milvus collection.
Configure the smart-answers index pipeline.
Configure the smart-answers query pipeline.

See also Best Practices, Advanced Model Training Configuration for Smart Answers, and Smart Answers Detailed Pipeline Setup.

Create the Milvus collection

For complete details about job configuration options, see the Create Collections in Milvus job.

Navigate to Collections > Jobs > Add + and select Create Collections in Milvus.
Configure the job:
1. Enter an ID for this job.
2. Under Collections, click Add.
3. Enter a collection name.
4. In the Dimension field, enter the dimension size of vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encoding model. For example, the Smart Answers Pre-trained Coldstart models outputs vectors of 512 dimension size. Dimensionality of encoders trained by Smart Answers Supervised Training job depends on the provided parameters and printed in the training job logs.
Click Save. The Create Collections in Milvus job can be used to create multiple collections at once. In this image, the first collection is used in the indexing and query steps. The other two collections are used in the pipeline setup example.
Click Run > Start to run the job.

Configure the index pipeline

Open the Index Workbench.
Load or create your datasource using the default smart-answers index pipeline.
Configure the Encode into Milvus stage:
1. change the value of Model ID to match the model deployment name you chose when you configured the model training job.
2. Change Field to Encode to the document field name to be processed and encoded into dense vectors.
3. Ensure the Encoder Output Vector matches the output vector from the chosen model.
4. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
  To test out your settings, turn on Fail on Error in the Encode into Milvus stage and Apply the changes. This will cause an error message to display if any settings need to be changed.
Save the datasource.
Index your data.

Configure the query pipeline

Open the Query Workbench.
Load the default smart-answers query pipeline.
Configure the Milvus Query stage:
1. Change the Model ID value to match the model deployment name you chose when you configured the model training job.
2. Ensure the Encoder Output Vector matches the output vector from the chosen model.
3. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
4. Milvus Results Context Key can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score.
In the Milvus Ensemble Query stage, update the Ensemble math expression as needed based on your model and the name used in the prior stage for the storing the Milvus results. In versions 5.4 and later, you can also set the Threshold so that the Milvus Ensemble Query Stage will only return items with a score greater than or equal to the configured value.
Save the query pipeline.

Pipeline Setup Example

Index and retrieve the question and answer together

Index Pipeline

As shown in the pictures below, you will need two Encode into Milvus stages, named Encode Question and Encode Answer respectively.Encode Question (Encode Into Milvus) stage

Encode Answer (Encode Into Milvus) stage

Query Pipeline

In the Query Answers stage, we set the Milvus Results Context Key to milvus_answers and the Milvus collection name to answer_collection.Query Answers (Milvus Query) stage:

Evaluate the query pipeline

The Evaluate QnA Pipeline job evaluates the rankings of results from any Smart Answers pipeline and finds the best set of weights in the ensemble score.

Detailed pipeline setup


”smart-answers” index pipeline		Encode into Milvus stage
”smart-answers” query pipeline		Query Fields Milvus Query Milvus Ensemble Query Milvus Response Update

Create the Milvus collection

Prior to indexing data, the Create Collections in Milvus job can be used to create the Milvus collection(s) used by the Smart Answers pipelines (see Milvus overview).

Job ID. A unique identifier for the job.
Collection Name. A name for the Milvus collection you are creating. This name is used in both the Smart Answer Index and the Smart Answer Query pipelines.
Dimension. The dimension size of the vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encryption model. For example, if the model was created with either the Smart Answers Coldstart Training job or the Smart Answers Supervised Training job with the Model Base word_en_300d_2M, then the dimension would be 300.
Index file size. Files with more documents than this will cause Milvus to build an index on this collection.
Metric. The type of metric used to calculate vector similarity scores. Inner Product is recommended. It produces values between 0 and 1, where a higher value means higher similarity.

Index pipeline setup

Stages in the default “smart-answers” index pipeline

Only one custom index stage needs to be configured in your index pipeline, the Encode into Milvus index stage.

The Encode into Milvus Index Stage

If you are using a dynamic schema, make sure this stage is added after the Solr Dynamic Field Name Mapping stage.

The Encode into Milvus index stage uses the specified model to encode the Field to Encode and store it in Milvus in the given Milvus collection. There are several required parameters:

Model ID. The ID of the model.
Encoder Output Vector. The name of the field that stores the compressed dense vectors output from the model. Default value: vector.
Field to Encode. The text field to encode into a dense vector, such as answer_t or body_t.
Milvus Collection Name. The name of the collection you created via the Create Milvus Collection job, which will store the dense vectors. When creating the collection you specify the type of Metric to use to calculate vector similarity. This stage can be used multiple times to encode additional fields, each into a different Milvus collection.

Query pipeline setup

The Query Fields stage

The first stage is Query Fields. For more information see the Query Fields stage.

The Milvus Query stage

Model ID. The ID of the model used when configuring the model training job.
Encoder Output Vector. The name of the output vector from the specified model, which will contain the query encoded as a vector. Defaults to vector.
Milvus Collection Name. The name of the collection that you used in the Encode into Milvus index stage to store the encoded vectors.
Milvus Results Context Key. The name of the variable used to store the vector distances. It can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score for the document.
Number of Results. The number of highest scoring results returned from Milvus. This stage would typically be used the same number of times that the Encode into Milvus index stage is used, each with a different Milvus collection and a different Milvus Results Context Key.

The Milvus Ensemble Query stage

The Milvus Ensemble Query takes the results of the Milvus Query stage(s) and calculates the ensemble score, which is used to return the best matches.

Ensemble math expression. The mathematical expression used to calculate the ensemble score. It should reference the value(s) variable name specified in the Milvus Results Context Key parameter in the Milvus Query stage.
Result field name. The name of the field used to store the ensemble score. It defaults to ensemble_score.
Threshold- A parameter that filters the stage results to remove items that fall below the configured score. Items with a score at, or above, the threshold will be returned.

The Threshold feature is only available in Fusion 5.4 and later.

The Milvus Response Update Query stage

Short answer extraction

By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question.

Evaluate a Smart Answers Query Pipeline

This topic explains how you can track the running steps of the following jobs:

Smart Answers Supervised Training
Smart Answers Coldstart Training
Smart Answers Evaluate Pipeline

The ML Model service logs provide information on data pre-processing, training steps, evaluations, and model generation for those jobs. Follow the instructions below to access the logs.

Find the name of the ML Model service pod:
```
kubectl get pods -n <your-namespace> | grep ml-model-service-ui
```
The pod name looks like <namespace>-ml-model-service-ui-<hash>-<random>, as in f5-ml-model-service-ui-547dd78d6-p9d6q.

Set up port forwarding:

kubectl port-forward <namespace>-ml-model-service-ui-<hash>-<random> 8001:8001 -n <your-namespace>

In the Fusion UI, navigate to Collections > Jobs.
Select your Smart Answers (QnA) model training job.
Click Job History.
Find the ML Model workflow ID.
In the ML Model service UI, locate instances of the ID in the logs.

Workflow 2 using input of a body of content that can be used for unsupervised training:

Train a Smart Answers Supervised Model

Training job requirements

Storage150GB plus 2.5 times the total input data size.Processor and memoryThe memory requirements depend on whether you choose GPU or CPU processing:


GPU	CPU
one core 11GB RAM	32 cores 32GB RAM

If your training data contains more than 1 million entries, use GPU.

Prepare the input data

Format your input data as question/answer pairs, that is, a query and its corresponding response in each row. You can do this in any format that Managed Fusion supports. If there are multiple possible answers for a unique question, then repeat the questions and put the pair into different rows to make sure each row has one question and one answer, as in the example JSON below:
```
[{"question":"How to transfer personal auto lease to business auto lease?","answer":"I would approach the lender that you are getting the lease from..."}
 {"question":"How to transfer personal auto lease to business auto lease?","answer":"See what the contract says about transfers or subleases..."}]
```
Index the input data in Managed Fusion. If you wish to have the training data in Managed Fusion, index it into a separate collection for training data such as model_training_input. Otherwise you can use it directly from the cloud storage.

Configure the training job

In Managed Fusion, navigate to Collections > Jobs.
Select Add > Smart Answers Supervised Training:
In the Training Collection field, specify the input data collection that you created when you prepared the input data.
You can also configure this job to read from or write to cloud storage.
Enter the names of the Question Field and the Answer Field in the training collection.
Enter a Model Deployment Name. The new machine learning model will be saved in the blob store with this name. You will reference it later when you configure your pipelines.
Configure the Model base. There are several pre-trained word and BPE embeddings for different languages, as well as a few pre-trained BERT models. If you want to train custom embeddings, select word_custom or bpe_custom. This trains Word2vec on the provided data and specified fields. It might be useful in cases when your content includes unusual or domain-specific vocabulary. If you have content in addition to the query/response pairs that can be used to train the model, then specify it in the Texts Data Path. When you use the pre-trained embeddings, the log shows the percentage of processed vocabulary words. If this value is high, then try using custom embeddings. The job trains a few (configurable) RNN layers on top of word embeddings or fine-tunes a BERT model on the provided training data. The result model uses an attention mechanism to average word embeddings to obtain the final single dense vector for the content.
Dimension size of vectors for Transformer-based models is 768. For RNN-based models it is 2 times the number units of the last layer. To find the dimension size: download the model, expand the zip, open the log and search for Encoder output dim size: line. You might need this information when creating collections in Milvus.
Optional: Check Perform auto hyperparameter tuning to use auto-tune. Although training module tries to select the most optimal default parameters based on the training data statistics, auto-tune can extend it by automatically finding even better training configuration through hyper-parameter search. Although this is a resource-intensive operation, it can be useful to identify the best possible RNN-based configuration. Transformer-based models like BERT are not used during auto hyperparameter tuning as they usually perform better yet they are much more expensive on both training and inference time.
Click Save.
If using solr as the training data source ensure that the source collection contains the random_* dynamic field defined in its managed-schema.xml. This field is required for sampling the data. If it is not present, add the following entry to the managed-schema.xml alongside other dynamic fields <dynamicField name="random_*" type="random"/> and <fieldType class=“solr.RandomSortField” indexed=“true” name=“random”/> alongside other field types.
Click Run > Start.

After training is finished the model is deployed into the cluster and can be used in index and query pipelines.

Next steps

See A Smart Answers Supervised Job’s Status and Output
Configure The Smart Answers Pipelines
Evaluate a Smart Answers Query Pipeline

Configure the Smart Answers Pipelines (5.3 and later)

Before beginning this procedure, train a machine learning model using either the FAQ method or the cold start method.Regardless of how you set up your model, the deployment procedure is the same:

Create the Milvus collection.
Configure the smart-answers index pipeline.
Configure the smart-answers query pipeline.

See also Best Practices, Advanced Model Training Configuration for Smart Answers, and Smart Answers Detailed Pipeline Setup.

Create the Milvus collection

For complete details about job configuration options, see the Create Collections in Milvus job.

Navigate to Collections > Jobs > Add + and select Create Collections in Milvus.
Configure the job:
1. Enter an ID for this job.
2. Under Collections, click Add.
3. Enter a collection name.
4. In the Dimension field, enter the dimension size of vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encoding model. For example, the Smart Answers Pre-trained Coldstart models outputs vectors of 512 dimension size. Dimensionality of encoders trained by Smart Answers Supervised Training job depends on the provided parameters and printed in the training job logs.
Click Save. The Create Collections in Milvus job can be used to create multiple collections at once. In this image, the first collection is used in the indexing and query steps. The other two collections are used in the pipeline setup example.
Click Run > Start to run the job.

Configure the index pipeline

Open the Index Workbench.
Load or create your datasource using the default smart-answers index pipeline.
Configure the Encode into Milvus stage:
1. change the value of Model ID to match the model deployment name you chose when you configured the model training job.
2. Change Field to Encode to the document field name to be processed and encoded into dense vectors.
3. Ensure the Encoder Output Vector matches the output vector from the chosen model.
4. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
  To test out your settings, turn on Fail on Error in the Encode into Milvus stage and Apply the changes. This will cause an error message to display if any settings need to be changed.
Save the datasource.
Index your data.

Configure the query pipeline

Open the Query Workbench.
Load the default smart-answers query pipeline.
Configure the Milvus Query stage:
1. Change the Model ID value to match the model deployment name you chose when you configured the model training job.
2. Ensure the Encoder Output Vector matches the output vector from the chosen model.
3. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
4. Milvus Results Context Key can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score.
In the Milvus Ensemble Query stage, update the Ensemble math expression as needed based on your model and the name used in the prior stage for the storing the Milvus results. In versions 5.4 and later, you can also set the Threshold so that the Milvus Ensemble Query Stage will only return items with a score greater than or equal to the configured value.
Save the query pipeline.

Pipeline Setup Example

Index and retrieve the question and answer together

Index Pipeline

As shown in the pictures below, you will need two Encode into Milvus stages, named Encode Question and Encode Answer respectively.Encode Question (Encode Into Milvus) stage

Encode Answer (Encode Into Milvus) stage

Query Pipeline

In the Query Answers stage, we set the Milvus Results Context Key to milvus_answers and the Milvus collection name to answer_collection.Query Answers (Milvus Query) stage:

Evaluate the query pipeline

The Evaluate QnA Pipeline job evaluates the rankings of results from any Smart Answers pipeline and finds the best set of weights in the ensemble score.

Detailed pipeline setup


”smart-answers” index pipeline		Encode into Milvus stage
”smart-answers” query pipeline		Query Fields Milvus Query Milvus Ensemble Query Milvus Response Update

Create the Milvus collection

Prior to indexing data, the Create Collections in Milvus job can be used to create the Milvus collection(s) used by the Smart Answers pipelines (see Milvus overview).

Job ID. A unique identifier for the job.
Collection Name. A name for the Milvus collection you are creating. This name is used in both the Smart Answer Index and the Smart Answer Query pipelines.
Dimension. The dimension size of the vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encryption model. For example, if the model was created with either the Smart Answers Coldstart Training job or the Smart Answers Supervised Training job with the Model Base word_en_300d_2M, then the dimension would be 300.
Index file size. Files with more documents than this will cause Milvus to build an index on this collection.
Metric. The type of metric used to calculate vector similarity scores. Inner Product is recommended. It produces values between 0 and 1, where a higher value means higher similarity.

Index pipeline setup

Stages in the default “smart-answers” index pipeline

Only one custom index stage needs to be configured in your index pipeline, the Encode into Milvus index stage.

The Encode into Milvus Index Stage

If you are using a dynamic schema, make sure this stage is added after the Solr Dynamic Field Name Mapping stage.

The Encode into Milvus index stage uses the specified model to encode the Field to Encode and store it in Milvus in the given Milvus collection. There are several required parameters:

Model ID. The ID of the model.
Encoder Output Vector. The name of the field that stores the compressed dense vectors output from the model. Default value: vector.
Field to Encode. The text field to encode into a dense vector, such as answer_t or body_t.
Milvus Collection Name. The name of the collection you created via the Create Milvus Collection job, which will store the dense vectors. When creating the collection you specify the type of Metric to use to calculate vector similarity. This stage can be used multiple times to encode additional fields, each into a different Milvus collection.

Query pipeline setup

The Query Fields stage

The first stage is Query Fields. For more information see the Query Fields stage.

The Milvus Query stage

Model ID. The ID of the model used when configuring the model training job.
Encoder Output Vector. The name of the output vector from the specified model, which will contain the query encoded as a vector. Defaults to vector.
Milvus Collection Name. The name of the collection that you used in the Encode into Milvus index stage to store the encoded vectors.
Milvus Results Context Key. The name of the variable used to store the vector distances. It can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score for the document.
Number of Results. The number of highest scoring results returned from Milvus. This stage would typically be used the same number of times that the Encode into Milvus index stage is used, each with a different Milvus collection and a different Milvus Results Context Key.

The Milvus Ensemble Query stage

The Milvus Ensemble Query takes the results of the Milvus Query stage(s) and calculates the ensemble score, which is used to return the best matches.

Ensemble math expression. The mathematical expression used to calculate the ensemble score. It should reference the value(s) variable name specified in the Milvus Results Context Key parameter in the Milvus Query stage.
Result field name. The name of the field used to store the ensemble score. It defaults to ensemble_score.
Threshold- A parameter that filters the stage results to remove items that fall below the configured score. Items with a score at, or above, the threshold will be returned.

The Threshold feature is only available in Fusion 5.4 and later.

The Milvus Response Update Query stage

Short answer extraction

By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question.

Evaluate a Smart Answers Query Pipeline

This topic explains how you can track the running steps of the following jobs:

Smart Answers Supervised Training
Smart Answers Coldstart Training
Smart Answers Evaluate Pipeline

The ML Model service logs provide information on data pre-processing, training steps, evaluations, and model generation for those jobs. Follow the instructions below to access the logs.

Find the name of the ML Model service pod:
```
kubectl get pods -n <your-namespace> | grep ml-model-service-ui
```
The pod name looks like <namespace>-ml-model-service-ui-<hash>-<random>, as in f5-ml-model-service-ui-547dd78d6-p9d6q.

Set up port forwarding:

kubectl port-forward <namespace>-ml-model-service-ui-<hash>-<random> 8001:8001 -n <your-namespace>

In the Fusion UI, navigate to Collections > Jobs.
Select your Smart Answers (QnA) model training job.
Click Job History.
Find the ML Model workflow ID.
In the ML Model service UI, locate instances of the ID in the logs.

Each method requires a slightly different model training procedure, but the model deployment procedure is the same for both.

Smart Answers workflow

To implement Smart Answers, you will follow this workflow:

Train or install a machine learning model.
This process differs depending on whether you use the Train a Smart Answers Supervised Model or the Train a Smart Answers Cold Start Model.
Configure the index and query pipelines.
Fusion includes default pipelines to get you started. See Configure the Smart Answers Pipelines (5.3 and later).
Evaluate the query pipeline’s effectiveness.
Fine tune your query pipeline configuration by running a job that analyzes its effectiveness. See Evaluate a Smart Answers Query Pipeline.

Train a Smart Answers Supervised Model

Training job requirements

Storage150GB plus 2.5 times the total input data size.Processor and memoryThe memory requirements depend on whether you choose GPU or CPU processing:


GPU	CPU
one core 11GB RAM	32 cores 32GB RAM

If your training data contains more than 1 million entries, use GPU.

Prepare the input data

Format your input data as question/answer pairs, that is, a query and its corresponding response in each row. You can do this in any format that Fusion supports. If there are multiple possible answers for a unique question, then repeat the questions and put the pair into different rows to make sure each row has one question and one answer, as in the example JSON below:
```
[{"question":"How to transfer personal auto lease to business auto lease?","answer":"I would approach the lender that you are getting the lease from..."}
 {"question":"How to transfer personal auto lease to business auto lease?","answer":"See what the contract says about transfers or subleases..."}]
```
Index the input data in Fusion. If you wish to have the training data in Fusion, index it into a separate collection for training data such as model_training_input. Otherwise you can use it directly from the cloud storage.

Configure the training job

In Fusion, navigate to Collections > Jobs.
Select Add > Smart Answers Supervised Training:
In the Training Collection field, specify the input data collection that you created when you prepared the input data.
In Fusion 5.3 and later, you can also configure this job to read from or write to cloud storage.
Enter the names of the Question Field and the Answer Field in the training collection.
Enter a Model Deployment Name. The new machine learning model will be saved in the blob store with this name. You will reference it later when you configure your pipelines.
Fusion 5.3 and later: Configure the Model base. There are several pre-trained word and BPE embeddings for different languages, as well as a few pre-trained BERT models. If you want to train custom embeddings, select word_custom or bpe_custom. This trains Word2vec on the provided data and specified fields. It might be useful in cases when your content includes unusual or domain-specific vocabulary. If you have content in addition to the query/response pairs that can be used to train the model, then specify it in the Texts Data Path. When you use the pre-trained embeddings, the log shows the percentage of processed vocabulary words. If this value is high, then try using custom embeddings. The job trains a few (configurable) RNN layers on top of word embeddings or fine-tunes a BERT model on the provided training data. The result model uses an attention mechanism to average word embeddings to obtain the final single dense vector for the content.
Dimension size of vectors for Transformer-based models is 768. For RNN-based models it is 2 times the number units of the last layer. To find the dimension size: download the model, expand the zip, open the log and search for Encoder output dim size: line. You might need this information when creating collections in Milvus.
Optional: Check Perform auto hyperparameter tuning to use auto-tune. Although training module tries to select the most optimal default parameters based on the training data statistics, auto-tune can extend it by automatically finding even better training configuration through hyper-parameter search. Although this is a resource-intensive operation, it can be useful to identify the best possible RNN-based configuration. Transformer-based models like BERT are not used during auto hyperparameter tuning as they usually perform better yet they are much more expensive on both training and inference time.
Click Save.
If using solr as the training data source ensure that the source collection contains the random_* dynamic field defined in its managed-schema.xml. This field is required for sampling the data. If it is not present, add the following entry to the managed-schema.xml alongside other dynamic fields <dynamicField name="random_*" type="random"/> and <fieldType class=“solr.RandomSortField” indexed=“true” name=“random”/> alongside other field types.
Click Run > Start.

After training is finished the model is deployed into the cluster and can be used in index and query pipelines.

Next steps

See A Smart Answers Supervised Job’s Status and Output
Configure The Smart Answers Pipelines
Evaluate a Smart Answers Query Pipeline

Train a Smart Answers Cold Start Model

Configure the Smart Answers Pipelines (5.3 and later)

Before beginning this procedure, train a machine learning model using either the FAQ method or the cold start method.Regardless of how you set up your model, the deployment procedure is the same:

Create the Milvus collection.
Configure the smart-answers index pipeline.
Configure the smart-answers query pipeline.

See also Best Practices, Advanced Model Training Configuration for Smart Answers, and Smart Answers Detailed Pipeline Setup.

Create the Milvus collection

For complete details about job configuration options, see the Create Collections in Milvus job.

Navigate to Collections > Jobs > Add + and select Create Collections in Milvus.
Configure the job:
1. Enter an ID for this job.
2. Under Collections, click Add.
3. Enter a collection name.
4. In the Dimension field, enter the dimension size of vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encoding model. For example, the Smart Answers Pre-trained Coldstart models outputs vectors of 512 dimension size. Dimensionality of encoders trained by Smart Answers Supervised Training job depends on the provided parameters and printed in the training job logs.
Click Save. The Create Collections in Milvus job can be used to create multiple collections at once. In this image, the first collection is used in the indexing and query steps. The other two collections are used in the pipeline setup example.
Click Run > Start to run the job.

Configure the index pipeline

Open the Index Workbench.
Load or create your datasource using the default smart-answers index pipeline.
Configure the Encode into Milvus stage:
1. change the value of Model ID to match the model deployment name you chose when you configured the model training job.
2. Change Field to Encode to the document field name to be processed and encoded into dense vectors.
3. Ensure the Encoder Output Vector matches the output vector from the chosen model.
4. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
  To test out your settings, turn on Fail on Error in the Encode into Milvus stage and Apply the changes. This will cause an error message to display if any settings need to be changed.
Save the datasource.
Index your data.

Configure the query pipeline

Open the Query Workbench.
Load the default smart-answers query pipeline.
Configure the Milvus Query stage:
1. Change the Model ID value to match the model deployment name you chose when you configured the model training job.
2. Ensure the Encoder Output Vector matches the output vector from the chosen model.
3. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
4. Milvus Results Context Key can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score.
In the Milvus Ensemble Query stage, update the Ensemble math expression as needed based on your model and the name used in the prior stage for the storing the Milvus results. In versions 5.4 and later, you can also set the Threshold so that the Milvus Ensemble Query Stage will only return items with a score greater than or equal to the configured value.
Save the query pipeline.

Pipeline Setup Example

Index and retrieve the question and answer together

Index Pipeline

As shown in the pictures below, you will need two Encode into Milvus stages, named Encode Question and Encode Answer respectively.Encode Question (Encode Into Milvus) stage

Encode Answer (Encode Into Milvus) stage

Query Pipeline

In the Query Answers stage, we set the Milvus Results Context Key to milvus_answers and the Milvus collection name to answer_collection.Query Answers (Milvus Query) stage:

Evaluate the query pipeline

The Evaluate QnA Pipeline job evaluates the rankings of results from any Smart Answers pipeline and finds the best set of weights in the ensemble score.

Detailed pipeline setup


”smart-answers” index pipeline		Encode into Milvus stage
”smart-answers” query pipeline		Query Fields Milvus Query Milvus Ensemble Query Milvus Response Update

Create the Milvus collection

Prior to indexing data, the Create Collections in Milvus job can be used to create the Milvus collection(s) used by the Smart Answers pipelines (see Milvus overview).

Job ID. A unique identifier for the job.
Collection Name. A name for the Milvus collection you are creating. This name is used in both the Smart Answer Index and the Smart Answer Query pipelines.
Dimension. The dimension size of the vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encryption model. For example, if the model was created with either the Smart Answers Coldstart Training job or the Smart Answers Supervised Training job with the Model Base word_en_300d_2M, then the dimension would be 300.
Index file size. Files with more documents than this will cause Milvus to build an index on this collection.
Metric. The type of metric used to calculate vector similarity scores. Inner Product is recommended. It produces values between 0 and 1, where a higher value means higher similarity.

Index pipeline setup

Stages in the default “smart-answers” index pipeline

Only one custom index stage needs to be configured in your index pipeline, the Encode into Milvus index stage.

The Encode into Milvus Index Stage

If you are using a dynamic schema, make sure this stage is added after the Solr Dynamic Field Name Mapping stage.

The Encode into Milvus index stage uses the specified model to encode the Field to Encode and store it in Milvus in the given Milvus collection. There are several required parameters:

Model ID. The ID of the model.
Encoder Output Vector. The name of the field that stores the compressed dense vectors output from the model. Default value: vector.
Field to Encode. The text field to encode into a dense vector, such as answer_t or body_t.
Milvus Collection Name. The name of the collection you created via the Create Milvus Collection job, which will store the dense vectors. When creating the collection you specify the type of Metric to use to calculate vector similarity. This stage can be used multiple times to encode additional fields, each into a different Milvus collection.

Query pipeline setup

The Query Fields stage

The first stage is Query Fields. For more information see the Query Fields stage.

The Milvus Query stage

Model ID. The ID of the model used when configuring the model training job.
Encoder Output Vector. The name of the output vector from the specified model, which will contain the query encoded as a vector. Defaults to vector.
Milvus Collection Name. The name of the collection that you used in the Encode into Milvus index stage to store the encoded vectors.
Milvus Results Context Key. The name of the variable used to store the vector distances. It can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score for the document.
Number of Results. The number of highest scoring results returned from Milvus. This stage would typically be used the same number of times that the Encode into Milvus index stage is used, each with a different Milvus collection and a different Milvus Results Context Key.

The Milvus Ensemble Query stage

The Milvus Ensemble Query takes the results of the Milvus Query stage(s) and calculates the ensemble score, which is used to return the best matches.

Ensemble math expression. The mathematical expression used to calculate the ensemble score. It should reference the value(s) variable name specified in the Milvus Results Context Key parameter in the Milvus Query stage.
Result field name. The name of the field used to store the ensemble score. It defaults to ensemble_score.
Threshold- A parameter that filters the stage results to remove items that fall below the configured score. Items with a score at, or above, the threshold will be returned.

The Threshold feature is only available in Fusion 5.4 and later.

The Milvus Response Update Query stage

Short answer extraction

By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question.

Evaluate a Smart Answers Query Pipeline

Smart Answers workflow

To implement Smart Answers, you will follow this workflow:

Train or install a machine learning model.
This process differs depending on whether you use the Train a Smart Answers Supervised Model or the Train a Smart Answers cold start model.
Configure the index and query pipelines.
Managed Fusion includes default pipelines to get you started. See Configure the Smart Answers pipelines.
Evaluate the query pipeline’s effectiveness.
Fine tune your query pipeline configuration by running a job that analyzes its effectiveness. See Evaluate a Smart Answers Query Pipeline.

Train a Smart Answers Supervised Model

Training job requirements

Storage150GB plus 2.5 times the total input data size.Processor and memoryThe memory requirements depend on whether you choose GPU or CPU processing:


GPU	CPU
one core 11GB RAM	32 cores 32GB RAM

If your training data contains more than 1 million entries, use GPU.

Prepare the input data

Format your input data as question/answer pairs, that is, a query and its corresponding response in each row. You can do this in any format that Managed Fusion supports. If there are multiple possible answers for a unique question, then repeat the questions and put the pair into different rows to make sure each row has one question and one answer, as in the example JSON below:
```
[{"question":"How to transfer personal auto lease to business auto lease?","answer":"I would approach the lender that you are getting the lease from..."}
 {"question":"How to transfer personal auto lease to business auto lease?","answer":"See what the contract says about transfers or subleases..."}]
```
Index the input data in Managed Fusion. If you wish to have the training data in Managed Fusion, index it into a separate collection for training data such as model_training_input. Otherwise you can use it directly from the cloud storage.

Configure the training job

In Managed Fusion, navigate to Collections > Jobs.
Select Add > Smart Answers Supervised Training:
In the Training Collection field, specify the input data collection that you created when you prepared the input data.
You can also configure this job to read from or write to cloud storage.
Enter the names of the Question Field and the Answer Field in the training collection.
Enter a Model Deployment Name. The new machine learning model will be saved in the blob store with this name. You will reference it later when you configure your pipelines.
Configure the Model base. There are several pre-trained word and BPE embeddings for different languages, as well as a few pre-trained BERT models. If you want to train custom embeddings, select word_custom or bpe_custom. This trains Word2vec on the provided data and specified fields. It might be useful in cases when your content includes unusual or domain-specific vocabulary. If you have content in addition to the query/response pairs that can be used to train the model, then specify it in the Texts Data Path. When you use the pre-trained embeddings, the log shows the percentage of processed vocabulary words. If this value is high, then try using custom embeddings. The job trains a few (configurable) RNN layers on top of word embeddings or fine-tunes a BERT model on the provided training data. The result model uses an attention mechanism to average word embeddings to obtain the final single dense vector for the content.
Dimension size of vectors for Transformer-based models is 768. For RNN-based models it is 2 times the number units of the last layer. To find the dimension size: download the model, expand the zip, open the log and search for Encoder output dim size: line. You might need this information when creating collections in Milvus.
Optional: Check Perform auto hyperparameter tuning to use auto-tune. Although training module tries to select the most optimal default parameters based on the training data statistics, auto-tune can extend it by automatically finding even better training configuration through hyper-parameter search. Although this is a resource-intensive operation, it can be useful to identify the best possible RNN-based configuration. Transformer-based models like BERT are not used during auto hyperparameter tuning as they usually perform better yet they are much more expensive on both training and inference time.
Click Save.
If using solr as the training data source ensure that the source collection contains the random_* dynamic field defined in its managed-schema.xml. This field is required for sampling the data. If it is not present, add the following entry to the managed-schema.xml alongside other dynamic fields <dynamicField name="random_*" type="random"/> and <fieldType class=“solr.RandomSortField” indexed=“true” name=“random”/> alongside other field types.
Click Run > Start.

After training is finished the model is deployed into the cluster and can be used in index and query pipelines.

Next steps

See A Smart Answers Supervised Job’s Status and Output
Configure The Smart Answers Pipelines
Evaluate a Smart Answers Query Pipeline

Train a Smart Answers cold start model

Configure the Smart Answers pipelines

Before beginning this procedure, train a machine learning model using either the FAQ method or the cold start method.Regardless of how you set up your model, the deployment procedure is the same:

Create the Milvus collection.
Configure the smart-answers index pipeline.
Configure the smart-answers query pipeline.

Create the Milvus collection

For complete details about job configuration options, see the Create Collections in Milvus job.

Navigate to Collections > Jobs > Add + and select Create Collections in Milvus.
Configure the job:
1. Enter an ID for this job.
2. Under Collections, click Add.
3. Enter a collection name.
4. In the Dimension field, enter the dimension size of vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encoding model. For example, the Smart Answers Pre-trained Coldstart models outputs vectors of 512 dimension size. Dimensionality of encoders trained by Smart Answers Supervised Training job depends on the provided parameters and printed in the training job logs.
Click Save. The Create Collections in Milvus job can be used to create multiple collections at once. In this image, the first collection is used in the indexing and query steps. The other two collections are used in the pipeline setup example.
Click Run > Start to run the job.

Configure the index pipeline

Open the Index Workbench.
Load or create your datasource using the default smart-answers index pipeline.
Configure the Encode into Milvus stage:
1. change the value of Model ID to match the model deployment name you chose when you configured the model training job.
2. Change Field to Encode to the document field name to be processed and encoded into dense vectors.
3. Ensure the Encoder Output Vector matches the output vector from the chosen model.
4. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
  To test out your settings, turn on Fail on Error in the Encode into Milvus stage and Apply the changes. This will cause an error message to display if any settings need to be changed.
Save the datasource.
Index your data.

Configure the query pipeline

Open the Query Workbench.
Load the default smart-answers query pipeline.
Configure the Milvus Query stage:
1. Change the Model ID value to match the model deployment name you chose when you configured the model training job.
2. Ensure the Encoder Output Vector matches the output vector from the chosen model.
3. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
4. Milvus Results Context Key can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score.
In the Milvus Ensemble Query stage, update the Ensemble math expression as needed based on your model and the name used in the prior stage for the storing the Milvus results. You can also set the Threshold so that the Milvus Ensemble Query Stage will only return items with a score greater than or equal to the configured value.
Save the query pipeline.

Pipeline Setup Example

Index and retrieve the question and answer together

Index Pipeline

As shown in the pictures below, you will need two Encode into Milvus stages, named Encode Question and Encode Answer respectively.Encode Question (Encode Into Milvus) stage

Encode Answer (Encode Into Milvus) stage

Query Pipeline

In the Query Answers stage, we set the Milvus Results Context Key to milvus_answers and the Milvus collection name to answer_collection.Query Answers (Milvus Query) stage:

Evaluate the query pipeline

The Evaluate QnA Pipeline job evaluates the rankings of results from any Smart Answers pipeline and finds the best set of weights in the ensemble score.

Detailed pipeline setup

Typically, you can use the default pipelines included with Managed Fusion. These pipelines now utilize Milvus to store encoded vectors and to calculate vector similarity. This topic provides information you can use to customize the Smart Answers pipelines. See also Configure The Smart Answers Pipelines.


”smart-answers” index pipeline		Encode into Milvus stage
”smart-answers” query pipeline		Query Fields Milvus Query Milvus Ensemble Query Milvus Response Update

Create the Milvus collection

Prior to indexing data, the Create Collections in Milvus job can be used to create the Milvus collection(s) used by the Smart Answers pipelines.

Job ID. A unique identifier for the job.
Collection Name. A name for the Milvus collection you are creating. This name is used in both the Smart Answer Index and the Smart Answer Query pipelines.
Dimension. The dimension size of the vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encryption model. For example, if the model was created with either the Smart Answers Coldstart Training job or the Smart Answers Supervised Training job with the Model Base word_en_300d_2M, then the dimension would be 300.
Index file size. Files with more documents than this will cause Milvus to build an index on this collection.
Metric. The type of metric used to calculate vector similarity scores. Inner Product is recommended. It produces values between 0 and 1, where a higher value means higher similarity.

Index pipeline setup

Stages in the default “smart-answers” index pipeline

Only one custom index stage needs to be configured in your index pipeline, the Encode into Milvus index stage.

The Encode into Milvus Index Stage

If you are using a dynamic schema, make sure this stage is added after the Solr Dynamic Field Name Mapping stage.

The Encode into Milvus index stage uses the specified model to encode the Field to Encode and store it in Milvus in the given Milvus collection. There are several required parameters:

Model ID. The ID of the model.
Encoder Output Vector. The name of the field that stores the compressed dense vectors output from the model. Default value: vector.
Field to Encode. The text field to encode into a dense vector, such as answer_t or body_t.
Milvus Collection Name. The name of the collection you created via the Create Milvus Collection job, which will store the dense vectors. When creating the collection you specify the type of Metric to use to calculate vector similarity. This stage can be used multiple times to encode additional fields, each into a different Milvus collection. See how to index and retrieve the question and answer together.

Query pipeline setup

The Query Fields stage

The first stage is Query Fields. For more information see the Query Fields stage.

The Milvus Query stage

Model ID. The ID of the model used when configuring the model training job.
Encoder Output Vector. The name of the output vector from the specified model, which will contain the query encoded as a vector. Defaults to vector.
Milvus Collection Name. The name of the collection that you used in the Encode into Milvus index stage to store the encoded vectors.
Milvus Results Context Key. The name of the variable used to store the vector distances. It can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score for the document.
Number of Results. The number of highest scoring results returned from Milvus. This stage would typically be used the same number of times that the Encode into Milvus index stage is used, each with a different Milvus collection and a different Milvus Results Context Key.

The Milvus Ensemble Query stage

The Milvus Ensemble Query takes the results of the Milvus Query stage(s) and calculates the ensemble score, which is used to return the best matches.

Ensemble math expression. The mathematical expression used to calculate the ensemble score. It should reference the value(s) variable name specified in the Milvus Results Context Key parameter in the Milvus Query stage.
Result field name. The name of the field used to store the ensemble score. It defaults to ensemble_score.
Threshold- A parameter that filters the stage results to remove items that fall below the configured score. Items with a score at, or above, the threshold will be returned.

The Milvus Response Update Query stage

Short answer extraction

By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question.

Evaluate a Smart Answers Query Pipeline

Smart Answers jobs

These jobs provide the machine learning features that drive Smart Answers:

Machine learning model training:
- Smart Answers Supervised Training job. Managed Fusion includes default pipelines to get you started. See Configure the Smart Answers pipelines.
- Smart Answers Coldstart Training job.
Machine learning model deployment:

Configure the Smart Answers pipelines

Before beginning this procedure, train a machine learning model using either the FAQ method or the cold start method.Regardless of how you set up your model, the deployment procedure is the same:

Create the Milvus collection.
Configure the smart-answers index pipeline.
Configure the smart-answers query pipeline.

Create the Milvus collection

For complete details about job configuration options, see the Create Collections in Milvus job.

Navigate to Collections > Jobs > Add + and select Create Collections in Milvus.
Configure the job:
1. Enter an ID for this job.
2. Under Collections, click Add.
3. Enter a collection name.
4. In the Dimension field, enter the dimension size of vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encoding model. For example, the Smart Answers Pre-trained Coldstart models outputs vectors of 512 dimension size. Dimensionality of encoders trained by Smart Answers Supervised Training job depends on the provided parameters and printed in the training job logs.
Click Save. The Create Collections in Milvus job can be used to create multiple collections at once. In this image, the first collection is used in the indexing and query steps. The other two collections are used in the pipeline setup example.
Click Run > Start to run the job.

Configure the index pipeline

Open the Index Workbench.
Load or create your datasource using the default smart-answers index pipeline.
Configure the Encode into Milvus stage:
1. change the value of Model ID to match the model deployment name you chose when you configured the model training job.
2. Change Field to Encode to the document field name to be processed and encoded into dense vectors.
3. Ensure the Encoder Output Vector matches the output vector from the chosen model.
4. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
  To test out your settings, turn on Fail on Error in the Encode into Milvus stage and Apply the changes. This will cause an error message to display if any settings need to be changed.
Save the datasource.
Index your data.

Configure the query pipeline

Open the Query Workbench.
Load the default smart-answers query pipeline.
Configure the Milvus Query stage:
1. Change the Model ID value to match the model deployment name you chose when you configured the model training job.
2. Ensure the Encoder Output Vector matches the output vector from the chosen model.
3. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
4. Milvus Results Context Key can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score.
In the Milvus Ensemble Query stage, update the Ensemble math expression as needed based on your model and the name used in the prior stage for the storing the Milvus results. You can also set the Threshold so that the Milvus Ensemble Query Stage will only return items with a score greater than or equal to the configured value.
Save the query pipeline.

Pipeline Setup Example

Index and retrieve the question and answer together

Index Pipeline

As shown in the pictures below, you will need two Encode into Milvus stages, named Encode Question and Encode Answer respectively.Encode Question (Encode Into Milvus) stage

Encode Answer (Encode Into Milvus) stage

Query Pipeline

In the Query Answers stage, we set the Milvus Results Context Key to milvus_answers and the Milvus collection name to answer_collection.Query Answers (Milvus Query) stage:

Evaluate the query pipeline

The Evaluate QnA Pipeline job evaluates the rankings of results from any Smart Answers pipeline and finds the best set of weights in the ensemble score.

Detailed pipeline setup


”smart-answers” index pipeline		Encode into Milvus stage
”smart-answers” query pipeline		Query Fields Milvus Query Milvus Ensemble Query Milvus Response Update

Create the Milvus collection

Prior to indexing data, the Create Collections in Milvus job can be used to create the Milvus collection(s) used by the Smart Answers pipelines.

Job ID. A unique identifier for the job.
Collection Name. A name for the Milvus collection you are creating. This name is used in both the Smart Answer Index and the Smart Answer Query pipelines.
Dimension. The dimension size of the vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encryption model. For example, if the model was created with either the Smart Answers Coldstart Training job or the Smart Answers Supervised Training job with the Model Base word_en_300d_2M, then the dimension would be 300.
Index file size. Files with more documents than this will cause Milvus to build an index on this collection.
Metric. The type of metric used to calculate vector similarity scores. Inner Product is recommended. It produces values between 0 and 1, where a higher value means higher similarity.

Index pipeline setup

Stages in the default “smart-answers” index pipeline

Only one custom index stage needs to be configured in your index pipeline, the Encode into Milvus index stage.

The Encode into Milvus Index Stage

If you are using a dynamic schema, make sure this stage is added after the Solr Dynamic Field Name Mapping stage.

The Encode into Milvus index stage uses the specified model to encode the Field to Encode and store it in Milvus in the given Milvus collection. There are several required parameters:

Model ID. The ID of the model.
Encoder Output Vector. The name of the field that stores the compressed dense vectors output from the model. Default value: vector.
Field to Encode. The text field to encode into a dense vector, such as answer_t or body_t.
Milvus Collection Name. The name of the collection you created via the Create Milvus Collection job, which will store the dense vectors. When creating the collection you specify the type of Metric to use to calculate vector similarity. This stage can be used multiple times to encode additional fields, each into a different Milvus collection. See how to index and retrieve the question and answer together.

Query pipeline setup

The Query Fields stage

The first stage is Query Fields. For more information see the Query Fields stage.

The Milvus Query stage

Model ID. The ID of the model used when configuring the model training job.
Encoder Output Vector. The name of the output vector from the specified model, which will contain the query encoded as a vector. Defaults to vector.
Milvus Collection Name. The name of the collection that you used in the Encode into Milvus index stage to store the encoded vectors.
Milvus Results Context Key. The name of the variable used to store the vector distances. It can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score for the document.
Number of Results. The number of highest scoring results returned from Milvus. This stage would typically be used the same number of times that the Encode into Milvus index stage is used, each with a different Milvus collection and a different Milvus Results Context Key.

The Milvus Ensemble Query stage

The Milvus Ensemble Query takes the results of the Milvus Query stage(s) and calculates the ensemble score, which is used to return the best matches.

Ensemble math expression. The mathematical expression used to calculate the ensemble score. It should reference the value(s) variable name specified in the Milvus Results Context Key parameter in the Milvus Query stage.
Result field name. The name of the field used to store the ensemble score. It defaults to ensemble_score.
Threshold- A parameter that filters the stage results to remove items that fall below the configured score. Items with a score at, or above, the threshold will be returned.

The Milvus Response Update Query stage

Short answer extraction

By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question.

Smart Answers Evaluate Pipeline
This job analyzes your configured Smart Answers query pipeline to provide insights about its effectiveness so that you can fine-tune your configuration for the best possible results.

”Smart-answers” pipelines and stages

Once you have trained and deployed your model, you can use one of the default pipelines that are automatically created with your Managed Fusion app. Both pipelines are called APP_NAME-smart-answers. See Configure the Smart Answers pipelines for more information.

Configure the Smart Answers pipelines

Before beginning this procedure, train a machine learning model using either the FAQ method or the cold start method.Regardless of how you set up your model, the deployment procedure is the same:

Create the Milvus collection.
Configure the smart-answers index pipeline.
Configure the smart-answers query pipeline.

Create the Milvus collection

For complete details about job configuration options, see the Create Collections in Milvus job.

Navigate to Collections > Jobs > Add + and select Create Collections in Milvus.
Configure the job:
1. Enter an ID for this job.
2. Under Collections, click Add.
3. Enter a collection name.
4. In the Dimension field, enter the dimension size of vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encoding model. For example, the Smart Answers Pre-trained Coldstart models outputs vectors of 512 dimension size. Dimensionality of encoders trained by Smart Answers Supervised Training job depends on the provided parameters and printed in the training job logs.
Click Save. The Create Collections in Milvus job can be used to create multiple collections at once. In this image, the first collection is used in the indexing and query steps. The other two collections are used in the pipeline setup example.
Click Run > Start to run the job.

Configure the index pipeline

Open the Index Workbench.
Load or create your datasource using the default smart-answers index pipeline.
Configure the Encode into Milvus stage:
1. change the value of Model ID to match the model deployment name you chose when you configured the model training job.
2. Change Field to Encode to the document field name to be processed and encoded into dense vectors.
3. Ensure the Encoder Output Vector matches the output vector from the chosen model.
4. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
  To test out your settings, turn on Fail on Error in the Encode into Milvus stage and Apply the changes. This will cause an error message to display if any settings need to be changed.
Save the datasource.
Index your data.

Configure the query pipeline

Open the Query Workbench.
Load the default smart-answers query pipeline.
Configure the Milvus Query stage:
1. Change the Model ID value to match the model deployment name you chose when you configured the model training job.
2. Ensure the Encoder Output Vector matches the output vector from the chosen model.
3. Ensure the Milvus Collection Name matches the collection name created via the Create Milvus Collection job.
4. Milvus Results Context Key can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score.
In the Milvus Ensemble Query stage, update the Ensemble math expression as needed based on your model and the name used in the prior stage for the storing the Milvus results. You can also set the Threshold so that the Milvus Ensemble Query Stage will only return items with a score greater than or equal to the configured value.
Save the query pipeline.

Pipeline Setup Example

Index and retrieve the question and answer together

Index Pipeline

As shown in the pictures below, you will need two Encode into Milvus stages, named Encode Question and Encode Answer respectively.Encode Question (Encode Into Milvus) stage

Encode Answer (Encode Into Milvus) stage

Query Pipeline

In the Query Answers stage, we set the Milvus Results Context Key to milvus_answers and the Milvus collection name to answer_collection.Query Answers (Milvus Query) stage:

Evaluate the query pipeline

The Evaluate QnA Pipeline job evaluates the rankings of results from any Smart Answers pipeline and finds the best set of weights in the ensemble score.

Detailed pipeline setup


”smart-answers” index pipeline		Encode into Milvus stage
”smart-answers” query pipeline		Query Fields Milvus Query Milvus Ensemble Query Milvus Response Update

Create the Milvus collection

Prior to indexing data, the Create Collections in Milvus job can be used to create the Milvus collection(s) used by the Smart Answers pipelines.

Job ID. A unique identifier for the job.
Collection Name. A name for the Milvus collection you are creating. This name is used in both the Smart Answer Index and the Smart Answer Query pipelines.
Dimension. The dimension size of the vectors to store in this Milvus collection. The Dimension should match the size of the vectors returned by the encryption model. For example, if the model was created with either the Smart Answers Coldstart Training job or the Smart Answers Supervised Training job with the Model Base word_en_300d_2M, then the dimension would be 300.
Index file size. Files with more documents than this will cause Milvus to build an index on this collection.
Metric. The type of metric used to calculate vector similarity scores. Inner Product is recommended. It produces values between 0 and 1, where a higher value means higher similarity.

Index pipeline setup

Stages in the default “smart-answers” index pipeline

Only one custom index stage needs to be configured in your index pipeline, the Encode into Milvus index stage.

The Encode into Milvus Index Stage

If you are using a dynamic schema, make sure this stage is added after the Solr Dynamic Field Name Mapping stage.

The Encode into Milvus index stage uses the specified model to encode the Field to Encode and store it in Milvus in the given Milvus collection. There are several required parameters:

Model ID. The ID of the model.
Encoder Output Vector. The name of the field that stores the compressed dense vectors output from the model. Default value: vector.
Field to Encode. The text field to encode into a dense vector, such as answer_t or body_t.
Milvus Collection Name. The name of the collection you created via the Create Milvus Collection job, which will store the dense vectors. When creating the collection you specify the type of Metric to use to calculate vector similarity. This stage can be used multiple times to encode additional fields, each into a different Milvus collection. See how to index and retrieve the question and answer together.

Query pipeline setup

The Query Fields stage

The first stage is Query Fields. For more information see the Query Fields stage.

The Milvus Query stage

Model ID. The ID of the model used when configuring the model training job.
Encoder Output Vector. The name of the output vector from the specified model, which will contain the query encoded as a vector. Defaults to vector.
Milvus Collection Name. The name of the collection that you used in the Encode into Milvus index stage to store the encoded vectors.
Milvus Results Context Key. The name of the variable used to store the vector distances. It can be changed as needed. It will be used in the Milvus Ensemble Query Stage to calculate the query score for the document.
Number of Results. The number of highest scoring results returned from Milvus. This stage would typically be used the same number of times that the Encode into Milvus index stage is used, each with a different Milvus collection and a different Milvus Results Context Key.

The Milvus Ensemble Query stage

The Milvus Ensemble Query takes the results of the Milvus Query stage(s) and calculates the ensemble score, which is used to return the best matches.

Ensemble math expression. The mathematical expression used to calculate the ensemble score. It should reference the value(s) variable name specified in the Milvus Results Context Key parameter in the Milvus Query stage.
Result field name. The name of the field used to store the ensemble score. It defaults to ensemble_score.
Threshold- A parameter that filters the stage results to remove items that fall below the configured score. Items with a score at, or above, the threshold will be returned.

The Milvus Response Update Query stage

Short answer extraction

By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question.

“-smart-answers” index pipeline
“-smart-answers” query pipeline

Short answer extraction

By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question. See Extract short answers from longer documents.

Extract short answers from longer documents

This topic explains how to deploy and configure the transformer-based deep learning model for short answer extraction with Smart Answers. This model is useful for analyzing long documents and extracting just a paragraph, a sentence, or a few words that answer the question.

This model is trained on the SQuAD2.0 dataset which consists of questions about Wikipedia articles and answers gleaned from those articles. Therefore, this model is most effective with Wikipedia-like content and may produce uneven results when applied to more informal content such as message boards.

The out-of-the-box (OOTB) model only supports English content.

Deploy the model in Fusion

Navigate to Collections > Jobs.
Select New > Create Seldon Core Model Deployment.
Configure the job as follows:
- Job ID. the ID for this job, such as deploy-answer-extractor.
- Model Name. model name of the Seldon Core deployment that will be referenced in the Machine Learning pipeline stage configurations, such as answer-extractor.
- Docker Repository. lucidworks
- Image Name. answer-extractor:v1.1
- Kubernetes Secret Name for Model Repo. (empty)
- Output Column Names for Model. [answer,score,start,end]
Click Save.
Click Run > Start.

Configure the Machine Learning query stage

This model provides the best results when used with one of the question-answering query pipelines. The default query pipeline is called APP_NAME-smart-answers.Starting with one of those pipelines, add a new Machine Learning stage to the end of the pipeline and configure it as described below.How to configure short answer extraction in the query pipeline

Make sure you have performed the basic configuration of your query pipeline.
In the query pipeline, click Add a Stage > Machine Learning.
In the Model ID field, enter the model name you configured above, such as answer-extractor.
In the Model input transformation script field, enter the following:
```
var textFieldToExtract = "answer_t"
var numDocsToExtract = 3
responses = new java.util.ArrayList();

var docs = response.get().getInnerResponse().getDocuments();
for (var i=0; i<numDocsToExtract; i++) {
  responses.add(docs[i].getField(textFieldToExtract))
}

var modelInput = new java.util.HashMap()
modelInput.put("question", request.getFirstParam("q"))
modelInput.put("context", responses)
modelInput.put("topk", 3)
modelInput.put("handle_impossible_answer", 'false')
modelInput
```
Configure the parameters in the script as follows:
- question (Required). The name of the field containing the questions.
  Make sure that the question is provided as it was originally entered by user. If you have previous stages that augments question (like stopwords removing or synonyms expansion), it is better to copy original question and use it for the answer extraction without additional modifications.
- context (Required). A string or list of contexts; by default this is the first num_docs_to_extract documents in the output of the previous stage in the pipeline. If only one question is present with multiple contexts, that question will be applied to every context and vice versa for 1 context and multiple questions. If a list of questions and contexts is passed, a 1:1 mapping of questions and contexts will be created in the order in which they’re passed.
- topk. The number of answers to return (will be chosen by order of likelihood). Default: 1
- handle_impossible_answer. Whether or not to deal with a question that has no answer in the context. If true, an empty string is returned. If false, the most probable (topk) answer(s) are returned regardless of how low the probability score is. Default: True
  Experiment with this parameter to see what value returns the most acceptable answers.
For advanced use cases, you can add the following parameters to the script to override their defaults:
- batch_size. How many samples to process at a time. Reducing this number will reduce memory usage but increase execution time, while increasing it will increase memory usage and decrease execution time to a certain extent. Default: 8
- max_context_len. If set to greater than 0, truncate contexts to this length in characters. Default: 5000
- max_answer_len. The maximum length of predicted answers (for example, only answers with a shorter length are considered). Default: 15
- max_question_len. The maximum length of the question after tokenization. It will be truncated if needed. Default: 64
- doc_stride. If the context is too long to fit with the question for the model, it will be split in several chunks with some overlap. This argument controls the size of that overlap. Default: 128
- max_seq_len. The maximum length of the total sentence (context + question) after tokenization. The context will be split in several chunks (using doc_stride) if needed. Default: 384

In the Model output transformation script field, enter the following:

{/* // Parse raw output from model */}
var jsonOutput = JSON.parse(modelOutput.get("_rawJsonResponse"))

var parsedOutput = {};
for (var i=0; i<jsonOutput["names"].length;i++){
  parsedOutput[jsonOutput["names"][i]] = jsonOutput["ndarray"][i]
}

{/* // Get response documents */}
var docs = response.get().getInnerResponse().getDocuments();
var ndocs = new java.util.ArrayList();

{/* // Add extracted answers to the response docs */}
for (var i=0; i < parsedOutput["answer"].length;i++){
  var doc = docs[i];
  doc.putField("extracted_answer", new java.util.ArrayList(parsedOutput["answer"][i]))
  doc.putField("extracted_score", new java.util.ArrayList(parsedOutput["score"][i]))
  doc.putField("extracted_start", new java.util.ArrayList(parsedOutput["start"][i]))
  doc.putField("extracted_end", new java.util.ArrayList(parsedOutput["end"][i]))
  ndocs.add(doc);
}
response.get().getInnerResponse().updateDocuments(ndocs);

Save the pipeline.

Model output

The model adds the following fields to the query pipeline output:

answer. The short answer extracted from the context. This may be blank if handle_impossible_answers=True and topk=1.
score. The score for the extracted answers.
start. The start index of the extracted answer in the provided context.
end. The end index of the extracted answer in the provided context.

UI tour

Index data

Query data

Metrics and analytics

Improve your queries

Administration

Developer documentation

Machine learning

Neural Hybrid Search

Release notes

FAQs

​Example business use cases

​Getting started

​Smart Answers workflow

​Smart Answers workflow

​Smart Answers jobs

Example business use cases

Getting started

Smart Answers workflow

Smart Answers workflow

Smart Answers jobs