Sentiment Analysis and Prediction
One major part of understanding intent is understanding sentiment. Fusion has done the heavy lifting by integrating state-of-the-art, pre-trained sentiment analysis models directly into the platform. Lucidworks makes these models readily available for search developers to embed sentiment scoring into the query and indexing pipelines as stages.
Sentiment analysis is the interpretation of queries or content to determine some of their subjective aspects, such as emotions, intentions, opinions, and so on, using machine learning. Sentiment analysis produces a sentiment model that Fusion can use to perform sentiment prediction.
Since stronger sentiments can be positive or negative while a weaker sentiment is more neutral, sentiment analysis uses the positive/negative polarity to measure a sentiment’s strength:
Strongly Strongly Negative Neutral Positive -2 -1 0 1 2
For example, a query on a customer support site can express negative emotions, as in "How can I fix this frustrating upgrade problem?" Similarly, a shopper may imply a positive or negative intention to purchase an item, or a positive or negative opinion of that item.
When you assign sentiment values to incoming queries or to documents at indexing time, you can leverage that data to deliver more insight to your business applications. Understanding the intent and sentiment of customers or employees can help determine which information you deliver or how you treat subsequent interactions in the workflow. This kind of intelligence is valuable for e-commerce, call centers, social media, or surveys.
Lucidworks provides two pre-trained sentiment models that you can deploy to get started:
A general-purpose model, trained on short sentences.
Suitable for short texts and for intent prediction.
See Deploy The sentiment-general:v1.0 Model for instructions.
A model trained on a variety of customer reviews.
Optimized for longer texts. It also supports highlighting the tokens that provide stronger sentiment.
See Deploy The sentiment-reviews:v1.0 Model for instructions.
Both of the pre-trained models output the following:
a score from
The sentiment-reviews:v1.0 model also optionally outputs tokens and their corresponding attention weights, that is, the weight that each token carries in the sentiment prediction. The total of all attention weights is always 1, that is, each value represents a percentage of the total weight. In the example below, "awesome" has the highest weight because it expresses the strongest sentiment compared to other tokens in the string:
// Input text = "That is awesome!" // Output sentiment_label = ‘positive’ sentiment_score = 1.998 sentiment_attention_tokens = ['That', "'", 's', 'awesome', '!'] sentiment_attention_weights = [0.154, 0.078, 0.069, 0.444, 0.255]