Smart Answers
Managed Fusion Smart Answers brings the benefits of a versatile, scalable Semantic Search platform to provide cutting edge relevancy for your applications. This system makes use of advanced Deep Learning techniques to provide the power of Neural dense vectors search. Semantically rich models encode queries and documents into vectors in the same digital vector space in such way that query and the most relevant information are located near each other. It pushes search boundaries beyond classical token-based matching mechanisms by leveraging the contextual information and query understanding. Virtual assistants and Chatbots can incorporate Smart Answers to enable self-service for employees and customers and save cost on answering incoming queries.
E-commerce can utilize Smart Answers to handle zero-result search queries and improve the overall relevancy by recommending products that are most likely of users interest.
Training data can be explicitly provided as query/response pairs or constructed from the collected signals data. Even if you do not have an existing set of recorded interactions, you can rely on Smart Answers cold start solution. It uses various pre-trained models to get out-of-the-box (OOTB) semantic capabilities as well as provides a possibility to train unsupervised models for specific domains.
These features bring traditional search relevancy development and Data Science together into an easy-to-use configuration framework that leverages Managed Fusion’s indexing and querying pipelines.
|
Lucidworks offers free training to help you get started with Fusion. Check out the Smart Answers course, which focuses on how to utilize Smart Answers to go beyond classical token-based matching mechanisms by leveraging contextual information and query understanding:
Visit the LucidAcademy to see the full training catalog. |
Example business use cases
-
Embed this system as a self-help feature on your Help page or Contact Us page to reduce the call center load.
-
Make it available to your customer support team to find the answers to already-solved problems.
Queries that lead to zero-results is a huge problem for E-commerce that leads to income loss and decrease of the overall users experience. Semantic search does not have this problem as it does not operate on the exact token matches as classical search. Instead search is done in the vector space which can find relevant products even if there is no token overlapping.
E-commerce websites can use this system to search "how to" content, product user manuals, or existing product questions. For example, amazon.com provides a search function on questions about each product.
You can achieve fast knowledge extraction by applying this solution to these types of knowledge repositories.
As the solution utilizes state of the art Deep Learning techniques, it is able to capture semantic and contextual information into query understanding. This makes it very suitable to work with long queries or natural language questions.
Getting started
To get started, you need a trained machine learning model. There are two methods for building a model, depending on the kind of data you already have:
| The Supervised solution | The Cold Start solution |
|---|---|
Use this solution when you already have a collection of query/response pairs or if you can construct such a dataset from the signals data.
|
Use this solution when you have no historical training data or fewer than 200 query/response pairs.
|
Each method requires a slightly different model training procedure, but the model deployment procedure is the same for both.
Smart Answers workflow
To implement Smart Answers, you will follow this workflow:
-
Train or install a machine learning model.
This process differs depending on whether you use the Supervised Solution or the Cold Start Solution.
-
Configure the index and query pipelines.
Managed Fusion includes default pipelines to get you started. See Configure the Smart Answers Pipelines.
-
Evaluate the query pipeline’s effectiveness.
Fine tune your query pipeline configuration by running a job that analyzes its effectiveness. See Evaluate a Smart Answers Query Pipeline.
Smart Answers jobs
These jobs provide the machine learning features that drive Smart Answers:
-
Machine learning model training:
-
Smart Answers Supervised Training job. Managed Fusion includes default pipelines to get you started. See Configure the Smart Answers Pipelines.
-
-
Machine learning model deployment:
-
Smart Answers Evaluate Pipeline
This job analyzes your configured Smart Answers query pipeline to provide insights about its effectiveness so that you can fine-tune your configuration for the best possible results.
"Smart-answers" pipelines and stages
Once you have trained and deployed your model, you can use one of the default pipelines that are automatically created with your Managed Fusion app. Both pipelines are called APP_NAME-smart-answers.
See Configure the Smart Answers Pipeline for more information.
-
"-smart-answers" index pipeline
-
"-smart-answers" query pipeline
Short answer extraction
By default, the question-answering query pipelines return complete documents that answer questions. Optionally, you can extract just a paragraph, a sentence, or a few words that answer the question. See Extract Short Answers from Longer Documents.