> ## Documentation Index
> Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Sentiment Analysis and Prediction

export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => {
  const [isLoaded, setIsLoaded] = useState(false);
  useEffect(() => {
    const timer = setTimeout(() => {
      setIsLoaded(true);
    }, 500);
    return () => clearTimeout(timer);
  }, []);
  return <div className="lw-template-container">
      <Card title={title} icon={icon}>
        {isLoaded && <span dangerouslySetInnerHTML={{
    __html: `<lw-template id="a029c1a9-28be-427e-b0e1-5d918920246a"></lw-template
            >`
  }} />}
        <Link href={linkHref} className="agent-studio-link text-left text-gray-600 gap-2 dark:text-gray-400 text-sm font-medium flex flex-row items-center hover:text-primary dark:hover:text-primary-light group-hover:text-primary group-hover:dark:text-primary-light">Powered by Lucidworks Agent Studio</Link>
      </Card>
    </div>;
};

[localhost link]: http://localhost:3000/docs/5/fusion/getting-data-out/advanced-query-enhancement/sentiment-analysis

[mintlify link]: https://doc.lucidworks.com/docs/5/fusion/getting-data-out/advanced-query-enhancement/sentiment-analysis

[old doc.lw link]: https://doc.lucidworks.com/fusion/5.9/453

One major part of understanding intent is understanding sentiment. Fusion has done the heavy lifting by integrating state-of-the-art, pre-trained sentiment analysis models directly into the platform. Lucidworks makes these models readily available for search developers to embed sentiment scoring into the query and indexing pipelines as stages.

<LwTemplate />

## What are sentiment analysis and sentiment prediction?

Sentiment analysis is the interpretation of queries or content to determine some of their subjective aspects, such as emotions, intentions, opinions, and so on, using machine learning. Sentiment analysis produces a sentiment model that Fusion can use to perform sentiment prediction.

Since stronger sentiments can be positive or negative while a weaker sentiment is more neutral, sentiment analysis uses the positive/negative polarity to measure a sentiment’s strength:

```txt theme={"dark"}
Strongly                Strongly
Negative    Neutral     Positive
  -2    -1     0     1     2
```

For example, a query on a customer support site can express negative emotions, as in "How can I fix this frustrating upgrade problem?" Similarly, a shopper may imply a positive or negative intention to purchase an item, or a positive or negative opinion of that item.

## Intelligent applications with sentiment analysis and prediction

When you assign sentiment values to incoming queries or to documents at indexing time, you can leverage that data to deliver more insight to your business applications. Understanding the intent and sentiment of customers or employees can help determine which information you deliver or how you treat subsequent interactions in the workflow. This kind of intelligence is valuable for e-commerce, call centers, social media, or surveys.

## Pre-trained sentiment models

Lucidworks provides two pre-trained sentiment models that you can deploy to get started:

| sentiment-general:v1.0                                                                                                                                                                      | sentiment-reviews:v1.0                                                                                                                                                                                                                     |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| A general-purpose model, trained on short sentences.<br /><br />Suitable for short texts and for intent prediction.<br /><br />See **Deploy the sentiment-general Model** for instructions. | A model trained on a variety of customer reviews.<br /><br />Optimized for longer texts. It also supports highlighting the tokens that provide stronger sentiment.<br /><br />See **Deploy the sentiment-reviews Model** for instructions. |

<AccordionGroup>
  <Accordion title="Deploy the sentiment-general Model">
    This topic explains how to deploy the sentiment-general pre-trained [sentiment prediction](/docs/5/fusion/getting-data-out/advanced-query-enhancement/sentiment-analysis) model.  This is a general-purpose sentiment prediction model, trained on short sentences.  It is suitable for short texts and for intent prediction.

    ## Install the model in Fusion

    1. Navigate to **Collections** > **Jobs**.
    2. Select **New** > **Create Seldon Core Model Deployment**.
    3. Configure the job as follows:

       * **Job ID.** The ID for this job, such as `deploy-sentiment-general`.
       * **Model Name.** The model name of the Seldon Core deployment that will be referenced in the Machine Learning pipeline stage configurations, such as `sentiment-general`.
       * **Docker Repository.** The value is `lucidworks`.
       * **Image Name.** The value is `sentiment-general:v1.0`.
       * **Kubernetes Secret Name for Model Repo.** This value is left empty.
       * **Output Column Names for Model.** The value is `[label, score]`.
    4. Click **Save**.
    5. Click **Run > Start**.

    <Note>If you are running a private Docker repository that virtualizes external images, enter your private registry into the **Docker Repository** field and `lucidworks/sentiment-general:v1.0` into the **Image Name** field.</Note>

    ## Configure the Machine Learning pipeline stages

    In your index or query pipelines add Machine Learning stage and specify sentiment-general in the Model ID field (or a custom model name that was used during deployment).

    ### Configure the Machine Learning index stage

    1. In your index pipeline, click **Add a Stage** > **Machine Learning**.
    2. In the **Model ID** field, enter the model name you configured above, such as `sentiment-general`.
    3. In the **Model input transformation script** field, enter the following:

       ```js theme={"dark"}
       var modelInput = new java.util.HashMap()
       modelInput.put("text", doc.getFirstFieldValue("text"))
       modelInput
       ```
    4. In the **Model output transformation script** field, enter the following:

       ```js theme={"dark"}
       doc.addField("sentiment_label_s", modelOutput.get("label")[0])
       doc.addField("sentiment_score_d", modelOutput.get("score")[0])
       ```
    5. Save the pipeline.

    ### Configure the Machine Learning query stage

    1. In your query pipeline, click **Add a Stage** > **Machine Learning**.
    2. In the **Model ID** field, enter the model name you configured above, such as sentiment-general.
    3. In the **Model input transformation script** field, enter the following:

       ```js theme={"dark"}
       var modelInput = new java.util.HashMap()
       modelInput.put("text", request.getFirstParam("q"))
       modelInput
       ```
    4. In the **Model output transformation script** field, enter the following:

       ```js theme={"dark"}
       {/* // To put into request */}
       request.putSingleParam("sentiment_label", modelOutput.get("label")[0])
       request.putSingleParam("sentiment_score", modelOutput.get("score")[0])

       {/* // To put into query context */}
       context.put("sentiment_label", modelOutput.get("label")[0])
       context.put("sentiment_score", modelOutput.get("score")[0])

       {/* // To put into response documents. NOTE: This can be done only after Solr Query stage */}
       var docs = response.get().getInnerResponse().getDocuments();
       var ndocs = new java.util.ArrayList();

       for (var i=0; i<docs.length;i++){
         var doc = docs[i];
         doc.putField("query_sentiment_label", modelOutput.get("label")[0])
         doc.putField("query_sentiment_score", modelOutput.get("score")[0])
         ndocs.add(doc);
       }

       response.get().getInnerResponse().updateDocuments(ndocs);
       ```
    5. Save the pipeline.

    ## Model output

    Both of the pre-trained models output the following:

    * a **label**, `negative` or `positive`
    * a **score** from `-2` to `2`

    The sentiment-reviews:v1.0 model also optionally outputs tokens and their corresponding attention weights, that is, the weight that each token carries in the sentiment prediction.  The total of all attention weights is always 1, that is, each value represents a percentage of the total weight.  In the example below, "awesome" has the highest weight because it expresses the strongest sentiment compared to other tokens in the string:

    ```cfg theme={"dark"}
    {/* // Input */}
    text = "That is awesome!"

    {/* // Output */}
    sentiment_label = ‘positive’
    sentiment_score = 1.998
    sentiment_attention_tokens = ['That', "'", 's', 'awesome', '!']
    sentiment_attention_weights = [0.154, 0.078, 0.069, 0.444, 0.255]
    ```
  </Accordion>

  <Accordion title="Deploy the sentiment-reviews Model">
    This topic explains how to deploy the sentiment-reviews pre-trained [sentiment prediction](/docs/5/fusion/getting-data-out/advanced-query-enhancement/sentiment-analysis) model.  This model is trained on a variety of customer reviews and optimized for longer texts.  It also supports attention weights output that can be used for highlighting the tokens that provide stronger sentiment; see [Model output](#model-output) below for an example.

    ## Install the model in Fusion

    1. Navigate to **Collections** > **Jobs**.
    2. Select **New** > **Create Seldon Core Model Deployment**.
    3. Configure the job as follows:
       * **Job ID.** The ID for this job, such as `deploy-sentiment-reviews`
       * **Model Name.** The model name of the Seldon Core deployment that will be referenced in the Machine Learning pipeline stage configurations, such as `sentiment-reviews`.
       * **Docker Repository.** The value is `lucidworks`.
       * **Image Name.** The value is `sentiment-reviews:v1.0`.
       * **Kubernetes Secret Name for Model Repo.** The value is left empty.
       * **Output Column Names for Model.** The value is `[label, score, tokens, attention_weights]`.
    4. Click **Save**.
    5. Click **Run** > **Start**.

    ## Configure the Machine Learning pipeline stages

    You can put your sentiment prediction model to work using the Machine Learning index stage or Machine Learning query stage.  You will specify the same **Model Name** that you used when you installed the model above.

    Generally, you only need to apply the model in the index pipeline, in order to perform sentiment prediction on your content.  Optionally, you can configure the query pipeline in a similar way, to perform sentiment prediction on incoming queries and outgoing responses and apply special treatment depending on the prediction.

    ### Configure the Machine Learning *index* stage

    1. In your index pipeline, click **Add a Stage** > **Machine Learning**.
    2. In the **Model ID** field, enter the model name you configured above, such as `sentiment-reviews`.
    3. In the **Model input transformation script** field, enter one of the following, depending on whether you want to output attention weights:\
       Without attention weights:
       ```js theme={"dark"}
       var modelInput = new java.util.HashMap()
       modelInput.put("text", doc.getFirstFieldValue("text"))
       modelInput
       ```
       With attention weights:
       ```js theme={"dark"}
       var modelInput = new java.util.HashMap()
       modelInput.put("text", doc.getFirstFieldValue("text"))
       modelInput.put("attention_output", "true")
       modelInput
       ```
    4. In the **Model output transformation script** field, enter the following:\
       Without attention weights:
       ```js theme={"dark"}
       doc.addField("sentiment_label_s", modelOutput.get("label")[0])
       doc.addField("sentiment_score_d", modelOutput.get("score")[0])
       ```
       With attention weights:
       ```js theme={"dark"}
       doc.addField("sentiment_label_s", modelOutput.get("label")[0])
       doc.addField("sentiment_score_d", modelOutput.get("score")[0])
       doc.addField("sentiment_attention_tokens_ss", modelOutput.get("tokens"))
       doc.addField("sentiment_attention_weights_ds", modelOutput.get("attention_weights"))
       ```
    5. Save the pipeline.

    ### Optional: Configure the Machine Learning *query* stage

    1. In your query pipeline, click **Add a Stage** > **Machine Learning**.
    2. In the **Model ID** field, enter the model name you configured above, such as sentiment-reviews.
    3. In the **Model input transformation script** field, enter the following:\
       Without attention weights:
       ```js theme={"dark"}
       var modelInput = new java.util.HashMap()
       modelInput.put("text", request.getFirstParam("q"))
       modelInput
       ```
       With attention weights:
       ```js theme={"dark"}
       var modelInput = new java.util.HashMap()
       modelInput.put("text", request.getFirstParam("q"))
       modelInput.put("attention_output", "true")
       modelInput
       ```
    4. In the **Model output transformation script** field, enter the following, noting the sections that need to be uncommented if you are using attention weights:
       ```js theme={"dark"}
       {/* // To put into request */}
       request.putSingleParam("sentiment_label", modelOutput.get("label")[0])
       request.putSingleParam("sentiment_score", modelOutput.get("score")[0])

       {/* // With attention output also uncomment this */}
       {/* // request.putSingleParam("sentiment_attention_tokens", modelOutput.get("tokens")) */}
       {/* // request.putSingleParam("sentiment_attention_weights", modelOutput.get("attention_weights")) */}

       {/* // To put into query context */}
       context.put("sentiment_label", modelOutput.get("label")[0])
       context.put("sentiment_score", modelOutput.get("score")[0])

       {/* // With attention output also uncomment this */}
       {/* // context.put("sentiment_attention_tokens", modelOutput.get("tokens")) */}
       {/* // context.put("sentiment_attention_weights", modelOutput.get("attention_weights")) */}

       {/* // To put into response documents (can be done only after Solr Query stage) */}
       var docs = response.get().getInnerResponse().getDocuments();
       var ndocs = new java.util.ArrayList();

       var attention_tokens = modelOutput.get("tokens")
       var attention_weights = modelOutput.get("attention_weights")
       var attention_tokens_arr = new java.util.ArrayList(attention_tokens.size());
       var attention_weights_arr = new java.util.ArrayList(attention_weights.size());
       for ( i = 0; i < attention_tokens.size(); i++) {
        attention_tokens_arr.add(attention_tokens[i])
        attention_weights_arr.add(attention_weights[i])
       }

       for (var i=0; i<docs.length;i++){
        var doc = docs[i];
        doc.putField("query_sentiment_label", modelOutput.get("label")[0])
        doc.putField("query_sentiment_score", modelOutput.get("score")[0])
       {/*   // With attention output also uncomment this */}
       {/*   // doc.putField("query_sentiment_attention_tokens", attention_tokens_arr) */}
       {/*   // doc.putField("query_sentiment_attention_weights", attention_weights_arr) */}
        ndocs.add(doc);
       }

       response.get().getInnerResponse().updateDocuments(ndocs);
       ```
    5. Save the pipeline.

    ## Model output

    Both of the pre-trained models output the following:

    * a **label**, `negative` or `positive`
    * a **score** from `-2` to `2`

    The sentiment-reviews:v1.0 model also optionally outputs tokens and their corresponding attention weights, that is, the weight that each token carries in the sentiment prediction.  The total of all attention weights is always 1, that is, each value represents a percentage of the total weight.  In the example below, "awesome" has the highest weight because it expresses the strongest sentiment compared to other tokens in the string:

    ```cfg theme={"dark"}
    {/* // Input */}
    text = "That is awesome!"

    {/* // Output */}
    sentiment_label = ‘positive’
    sentiment_score = 1.998
    sentiment_attention_tokens = ['That', "'", 's', 'awesome', '!']
    sentiment_attention_weights = [0.154, 0.078, 0.069, 0.444, 0.255]
    ```
  </Accordion>
</AccordionGroup>

<Note>
  These models support the English language only.
</Note>

## Model output

Both of the pre-trained models output the following:

* a **label**, `negative` or `positive`
* a **score** from `-2` to `2`

The sentiment-reviews:v1.0 model also optionally outputs tokens and their corresponding attention weights, that is, the weight that each token carries in the sentiment prediction. The total of all attention weights is always 1, that is, each value represents a percentage of the total weight. In the example below, "awesome" has the highest weight because it expresses the strongest sentiment compared to other tokens in the string:

```js theme={"dark"}
// Input
text = "That is awesome!"

// Output
sentiment_label = ‘positive’
sentiment_score = 1.998
sentiment_attention_tokens = ['That', "'", 's', 'awesome', '!']
sentiment_attention_weights = [0.154, 0.078, 0.069, 0.444, 0.255]
```