Product Selector

Fusion 5.9
    Fusion 5.9

    Smart Answers Detailed Pipeline Setup

    Typically, you can use the default pipelines included with Fusion AI. This topic provides information you can use to customize these pipelines. See also Configure The Smart Answers Pipelines.

    "question-answering" index pipeline

    question-answering default index pipeline

    "question-answering" query pipeline

    question-answering default query pipeline

    Index pipeline setup

    Stages in the default "question-answering" index pipeline

    question-answering default index pipeline

    Typically, only one custom index stage needs to be configured in your index pipeline: the TensorFlow Deep Encoding index stage.

    The TensorFlow Deep Encoding stage encodes your documents to dense vectors by using a TensorFlow model.

    If you are using a dynamic schema, make sure this stage is added after the Solr Dynamic Field Name Mapping stage.

    There are several required parameters:

    • TensorFlow Deep Learning Encoder Model ID. The ID of the model for answers encoding which is uploaded to the blob store, with a name that ends with _a_fusion_model_bundle.zip.

    • Document Feature Field. The text field to encode to the dense vectors, such as answer_t or body_t (the default value).

    • Vector Field. The field that stores dense vectors. Default value: document_vector_ds.

      Fusion also compresses dense vectors to strings and stores then in a field with the prefix compressed_, such as compressed_document_vector_s. The compressed version is used as the default to get better query runtime performance.

    • Clusters Field. The field that stores answer clusters IDs. Default value: document_clusters_ss.

    • Distances Field. The field that stores distances between a document vector and its closest cluster centers. This can be used for eDiscovery purposes. For example, we can explore the clusters by sorting documents in ascending order based on this distance to clusters centers. Default value: document_distances_ds.

    • Number of clusters. This value denotes how many clusters each document can belong to. In most cases it’s better to use just one cluster, since we already provide a query pipeline stage to find multiple clusters at query time. Note: this value should be less than or equal to the number of clusters specified in the model training. Default value: 1.

    Query pipeline setup

    Stages in the default "question-answering" query pipeline

    question-answering default query pipeline

    In the "question-answering" query pipeline, all stages (except the Solr Query stage) can be tuned for better Smart Answers:

    1. Query Fields

    2. Rewrite Pagination Parameters for Ranking

    3. Filter Stop Words

    4. Tensorflow Deep Encoding

    5. Escape Query

    6. Query-Document Vectors Distance

    7. Computer Mathematical Expression

    8. Result Document Field Sorting

    You can also configure the number of clusters used in deep encoding.

    The Query Fields stage

    The first stage is Query Fields which should have one main parameter specified:

    • Return Fields. Since documents should be retrieved with vectors and clusters, make sure to include the documents’ Vector Field and Clusters Field, which are compressed_document_vector_s and document_clusters_ss by default.

    The Rewrite Pagination Parameters for Reranking stage

    The Rewrite Pagination Parameters for Reranking query stage is used to specify how many results from Solr should be used to create a pool of candidates for further re-ranking via model. It’s done under the hood, so users would still see the number of results controlled by "start" and "rows" query parameters.

    • Number of Results. Number of results to request from Solr, to be re-ranked by downstream stages. Default value: 500. This parameter affects both results accuracy and query time performance. Increasing the number can lead to better accuracy but worse query time performance and vice versa.

      This parameter can be dynamically overwritten by rowsFromSolrToRerank raw query parameter in the request.

    The Filter Stop Words stage

    When limited data is provided to train FAQ or coldstart models, the model may not be able to learn the weights of stop words accurately. Especially when using Solr to retrieve the top x results (if Number of Clusters is set to 0 in the TensorFlow Deep Encoding stage), stop words may have high impact on Solr. User can use this stage to provide customized stop words list by providing them in Words to filter part of the stage.

    See Filter Stop Words for reference information. See also Best Practices.

    The TensorFlow Deep Encoding stage

    The TensorFlow Deep Encoding query stage is very similar to the index stage.

    If you are using the cluster retrieval configuration in Example 1, then this stage should be located before the Solr Query stage.

    The important configuration keys are as follows:

    • TensorFlow Deep Learning Encoder Model ID. The ID of the model for query encoding which is uploaded to the blob store with a name that ends with _q_fusion_model_bundle.zip.

    • Query Feature Field. q (default value).

    • Vector Context Key. The context key that stores query dense vectors. Default value: query_vector.

    • Clusters Context Key. The context key that stores query clusters. Default value: query_clusters.

    • Distances Context Key. The context key that store distances between query vectors and cluster centers. Default value: query_distances.

    • Number of clusters. The number of clusters that should be used for each query. This value should be less than or equal to the number of clusters specified in the model training. Default value: 0

      When this is set to 0 (the default), Fusion uses Solr to return the top x results, then re-ranks them using vector cosine similarity, or combines the Solr score with the vector score using the Compute mathematical expression stage. If you choose this option, adjust the Number of Results parameter (Default value 500) in the Rewrite Pagination Parameters for Reranking stage in the query pipeline. This parameter controls how many documents are returned from Solr to Fusion and re-ranked under the hood. You can adjust this parameter to find a good balance between relevancy and query speed.

      If you set Number of clusters to a value x greater than 0, then when a query comes in, we transfer the query into a dense vector and find the closest x clusters to which the query belongs. The pipeline then obtains documents from the same clusters as a query and re-ranks them based on the vector cosine similarity between the query and the indexed answers/questions. Although this option provides faster query run time (clusters usually have less than 500 docs), results sometime might be worse than in the first option. Comprehensive evaluation should be done to decide which is best for you.

      Documents from the clusters might be obtained in the following ways:

      • By using Solr search. Then search space is narrowed by both clusters and Solr. And Solr score might be used in the ensemble in the same way as in the first option described above.

      • By using only clusters. Then all documents from certain clusters are retrieved for re-ranking. To do that you need to add Additional Query Parameters stage to the query pipeline right before Solr Query stage and configure it in the following way: Parameter Name: q, Parameter Value: *:*, Update Policy: replace In such way after query is encoded we ask Solr to return all documents from certain clusters (they are already added as facets)

      When set to 0, Fusion returns a pool of candidates based solely on the Solr score, then performs re-ranking. Moreover, we support overriding this configuration parameter by using the num_clusters raw query parameter. For example, if you expose this parameter on your website and allow your customer to change it, then different customers can view different levels of granularity in the results (that is, a higher num_clusters value provides more results).
      If the number of clusters is greater than 0 and the search term is *:*, then there a limited number of documents are shown in the result rather than showing the whole collection. This is because *:* is assigned to certain clusters.

    The Escape Query stage

    The Escape Query query stage escapes Lucene/Solr reserved characters in a query. In the Smart Answers use case, queries are natural-language questions. They tend to be longer than regular queries and with punctuation which we do not want to interpret as Lucene/Solr query operators.

    The Query-Document Vectors Distance stage

    The Query-Document Vectors Distance query stage computes the distance between a query vector and each candidate document vector. It should be added after the Solr Query stage.

    • Query Vector Context Key. The context key which stores dense vectors. It should be set to the same value as _Vector Context Key _in the TensorFlow Deep Encoding Query stage. Default value: query_vector.

    • Document Vector Field. The field which stores dense vectors or their compressed representations. It should be the same as _Vector Field _in TensorFlow Deep Encoding Query stage. Default: compressed_document_vector_s.

    • Keep document vector field. This option allows you to keep or drop returned document vectors. In most cases it makes sense to drop document vectors after computing vector distances, to improve run time performance. Default: False (unchecked).

    • Distance Type. Choose one of the supported distance types. For a FAQ solution, cosine_similarity is recommended, which produces values with a maximum value of 1. Higher value means higher similarity. Default: cosine_similarity.

    • Document Vectors Distance Field. The result document field that contains the distance between its vector and the query vector. Default: vectors_distance.

    The Compute Mathematical Expression stage

    We can use the Compute Mathematical Expression query stage to combine the Solr score with the vectors similarity score to borrow Solr’s keywords matching capabilities. This stage should be set before the Result Document Field Sorting stage. Result field name in this stage and Sort field in the Result Document Field Sorting stage should be set to the same value.

    • Math expression. Specify a mathematical formula for combining the Solr score and the vector similarity score. Note that Solr scores do not have an upper bound, so it is best to rescale it by using the max_score value, which is the max Solr score from all returned docs. vectors_distance is the cosine similarity in our case and it already has an upper bound of 1.0. The complete list of supported mathematical expressions is listed in the Oracle documentation.

      Default: 0.3 * score / max_score + 0.7 * vectors_distance

      We recommend adjusting the ratios 0.3 and 0.7 to other values based on query-time evaluation results (mentioned in the section above). Feel free to try other math expressions (such as log10) to scale the Solr score as well.

    • Result field name. The document field name that contains the combined score. The same value should be set in the Result Document Field Sorting stage, in the Sort Field parameter. Default: ensemble_score

    The Result Document Field Sorting stage

    Finally, the Result Document Field Sorting stage sorts results based on vector distances obtained from the previous Vectors distance per Query/Document stage or ensemble score from Compute mathematical expression stage.

    • Sort Field. The field to use for document sorting. Default value: ensemble_score

    • Sort order. The sorting order: (asc)ending or (desc)ending. If cosine_similarity is used with higher values meaning higher similarity, then this should be set to desc. Default: desc.

    Configuring the number of clusters

    Another important parameter to choose is the Number of clusters parameter in the TensorFlow Deep Encoding stage in the query pipeline. There are two options for utilizing the dense vectors from a query pipeline:

    • If set Number of clusters parameter to 0 (default option), then we use Solr to return the top x results, then re-rank using vector cosine similarity, or combine the Solr score with the vector score using our Compute mathematical expression stage. If you choose this option, then adjust Number of Results parameter (Default value 500) in the Rewrite Pagination Parameters for Reranking stage in the query pipeline. This parameter controls how many documents to be returned from Solr to Fusion and re-ranked under the hood. You can adjust this parameter to find a good balance between relevancy and query speed.

    • If you set the Number of clusters parameter to a value x greater than 0, when a query comes in, Fusion transfers the query into a dense vector and find the closest x clusters to which the query belongs. The pipeline then obtains documents from the same clusters as query and re-ranks them based on the vector cosine similarity between the query and the indexed answers/questions. The good default values can be 1 cluster per document and 10 clusters per query.

    Documents from the clusters might be obtained in the following ways: * By using Solr search. Then search space is narrowed by both clusters and Solr. And Solr score might be used in the ensemble in the same way as in the first option described above. * By using only clusters. Then all documents from certain clusters are retrieved for re-ranking. Solr doesn’t narrow the search space, but it’s also not used for ensemble.