> ## Documentation Index
> Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Keyword extraction use case

export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => {
  const [isLoaded, setIsLoaded] = useState(false);
  useEffect(() => {
    const timer = setTimeout(() => {
      setIsLoaded(true);
    }, 500);
    return () => clearTimeout(timer);
  }, []);
  return <div className="lw-template-container">
      <Card title={title} icon={icon}>
        {isLoaded && <span dangerouslySetInnerHTML={{
    __html: `<lw-template id="a029c1a9-28be-427e-b0e1-5d918920246a"></lw-template
            >`
  }} />}
        <Link href={linkHref} className="agent-studio-link text-left text-gray-600 gap-2 dark:text-gray-400 text-sm font-medium flex flex-row items-center hover:text-primary dark:hover:text-primary-light group-hover:text-primary group-hover:dark:text-primary-light">Powered by Lucidworks Agent Studio</Link>
      </Card>
    </div>;
};

[old doc.lw link]: https://doc.lucidworks.com/lw-platform/ai/937meo

[localhost link]: http://localhost:3000/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/keyword-extraction-async

[mintlify link]: https://doc.lucidworks.com/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/keyword-extraction-async

In the Keyword extraction use case of the [Lucidworks AI Async Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/overview), the LLM ingests text and returns a JSON response that contains a list of keywords extracted from the text. No options can be configured.

This use case can be used:

* To extract keywords to create new facets
* To extract keywords from each document:
  * To write the keyword into a new field that subsequently searches large content documents more efficiently
  * To boost specific keyword content
  * For document clustering

The Keyword extraction use case contains two requests:

* POST request - submits a prediction task for a specific `useCase` and `modelId`. The API responds with the following information:
  * `predictionId`. A unique UUID for the submitted prediction task that can be used later to retrieve the results.
  * `status`. The current state of the prediction task.
* GET request - uses the `predictionId` you submit from a previously-submitted POST request and returns the results associated with that previous request.

<Note>
  For detailed API specifications in Swagger/OpenAPI format, see [Platform APIs](/api-reference/get-predictions/keyword-extraction-use-case).
</Note>

<LwTemplate />

## Prerequisites

To use this API, you need:

* The unique `APPLICATION_ID` for your Lucidworks AI application, which is provided by Lucidworks.
* A bearer token generated with a scope value of `machinelearning.predict`. For more information, see [Authentication API](/docs/lw-platform/lw-platform/authentication-api).
* The `USE_CASE` and `MODEL_ID` fields in the `/async-prediction` for the POST request. The path is `/ai/async-prediction/USE_CASE/MODEL_ID`. A list of supported modes is returned in the [Lucidworks AI Use Case API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-use-case-api). For more information about supported models, see [Generative AI models](/docs/lw-platform/lw-ai/lw-ai-generative-ai#generative-ai-models).

## Common POST request parameters and fields

Some parameters in the `/ai/async-prediction/USE_CASE/MODEL_ID` POST request are common to all of the generative AI (Gen-AI) use cases, such as the `modelConfig` parameter.
Also referred to as hyperparameters, these fields set certain controls on the response.
Refer to the [API spec](/api-reference/get-predictions/keyword-extraction-use-case) for more information.

## Unique values for the keyword extraction use case

Some parameter values available in the `keyword extraction` use case are unique to this use case, such as the `useCaseConfig` parameter.
Refer to the [API spec](/api-reference/get-predictions/keyword-extraction-use-case) for more information.

## Example POST request

This example does not include `modelConfig` parameters, but you can submit requests that include parameters described in [Common POST request parameters and fields](#common-post-request-parameters-and-fields).

<CodeGroup>
  ```json wrap Request theme={"dark"}
  curl --request POST \
    --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/keyword-extraction/MODEL_ID \
    --header 'Authorization: Bearer ACCESS_TOKEN' \
    --header 'Content-type: application/json' \
    --data '{
    "batch": [
      {
        "text": "Joseph Robinette Biden Jr.is an American politician who is the 46th and current president of the United States. Ideologically a moderate member of the Democratic Party, he previously served as the 47th vice president from 2009 to 2017 under President Barack Obama and represented Delaware in the United States Senate from 1973 to 2009.Born in Scranton, Pennsylvania, Biden moved with his family to Delaware in 1953. He studied at the University of Delaware before earning his law degree from Syracuse University. He was elected to the New Castle County Council in 1970 and to the U.S. Senate in 1972. As a senator, Biden drafted and led the effort to pass the Violent Crime Control and Law Enforcement Act and the Violence Against Women Act. He also oversaw six U.S. Supreme Court confirmation hearings, including the contentious hearings for Robert Bork and Clarence Thomas. Biden ran unsuccessfully for the Democratic presidential nomination in 1988 and 2008. In 2008, Obama chose Biden as his running mate, and Biden was a close counselor to Obama during his two terms as vice president. In the 2020 presidential election, Biden and his running mate, Kamala Harris, defeated incumbents Donald Trump and Mike Pence. Biden is the second Catholic president in U.S. history (after John F. Kennedy), and his politics have been widely described as profoundly influenced by Catholic social teaching."
        }
      ],
      "useCaseConfig": {
        "maxKeywords": 100
      }
    }'
  ```

  ```json wrap Success theme={"dark"}
  {
  	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  	"status": "SUBMITTED"
  }
  ```

  ```json wrap Error theme={"dark"}
  {
  	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  	"status": "ERROR",
  	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
  }
  ```
</CodeGroup>

## Example GET request

<CodeGroup>
  ```bash wrap Request theme={"dark"}
  curl --request GET
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
  --header 'Authorization: Bearer Auth '
  ```

  ```json wrap Response theme={"dark"}
  {
    "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
    "status": "READY",
    "predictions": [
      {
        "response": "Joseph Robinette Biden Jr., 46th president of the United States, Democratic Party, Vice President, Barack Obama, Delaware, University of Delaware, Syracuse University, Violent Crime Control and Law Enforcement Act, Violence Against Women Act.",
        "tokensUsed": {
          "promptTokens": 148,
          "completionTokens": 27,
          "totalTokens": 175
        },
        "keywords": [
          "Joseph Robinette Biden Jr.",
          "46th president of the United States",
          "Democratic Party",
          "Vice President",
          "Barack Obama",
          "Delaware",
          "University of Delaware",
          "Syracuse University",
          "Violent Crime Control and Law Enforcement Act",
          "Violence Against Women Act"
        ]
      }
    ]
  }
  ```
</CodeGroup>

## Learn more

<Accordion title="LWAI pipeline with async sum keyword extraction">
  This pipeline uses the following stages:

  * Field Mapping
  * Solr Dynamic Field Name Mapping
  * LWAI: Summarization
  * LWAI: Keyword Extraction
  * Solr Partial Update Indexer

  ## Add the pipeline

  1. Navigate to **Indexing > Index Pipelines**.
  2. Click **Add+**.
  3. Enter the **Pipeline ID**, for example `LWAI-async-summarization-and-keyword-extraction`.
  4. Select the **Solr Indexer** stage and click on **Remove stage**.

  ## Field Mapping

  Configure the Field Mapping stage as follows.

  1. Make sure that **Unmapped Fields** is checked.
  2. Make sure that **Keep** is checked.
  3. Save the pipeline.

  ## Solr Dynamic Field Name Mapping

  Configure the Solr Dynamic Field Name Mapping stage as follows.

  1. Enter a **Max Length for Advanced Indexing of Text Fields**, such as `100`.
  2. Save the pipeline.

  ## LWAI: Summarization

  Summarization is set up via the LWAI Prediction stage.

  1. Click **Add a new pipeline stage > LWAI Prediction**.
  2. Enter a **Label**, such as `LWAI: Summarization`.
  3. Select the **Account Name** as defined in Lucidworks AI Gateway.
  4. Select the **Use Case** as `summarization`.
  5. Select the **Model** to use.
  6. Enter the **Input context variable**, for example, `<doc.body_t>`.
  7. Enter the **Destination Field Name & Context Output**, for example, `body_summarization`.
  8. Enter values for the **Use Case Configuration**:
     1. Enter the **Parameter Name**, such as `maxWords`.
     2. Enter a **Parameter Value**, for example, `250`.
  9. Make sure **Call Asynchronously?** is checked.
  10. Enter the **Maximum Asynchronous Call Tries**, such as `1`.
  11. Save the pipeline.

  ## LWAI: Keyword Extraction

  Keyword extraction is set up using the LWAI Prediction stage.

  1. Click **Add a new pipeline stage > LWAI Prediction**.
  2. Enter a **Label**, such as `LWAI: Keyword Extraction`.
  3. Select the **Account Name** as defined in Lucidworks AI Gateway.
  4. Select the **Use Case** as `keyword-extraction`.
  5. Select the **Model** to use.
  6. Enter the **Input context variable**, for example, `<doc.body_t>`.
  7. Enter the **Destination Field Name & Context Output**, for example, `body_extracted_keywords`.
  8. Enter values for the **Use Case Configuration**:
     1. Enter the **Parameter Name**, such as maxKeywo\`rds.
     2. Enter the **Parameter Value**, for example, `50`.
  9. Make sure **Call Asynchronously?** is checked.
  10. Enter the **Maximum Asynchronous Call Tries**, such as `1`.
  11. Save the pipeline.

  ## Solr Partial Update Indexer

  Configure the Solr Partial Update Indexer stage as follows.

  1. Click **Add a new pipeline stage > Solr Partial Update Indexer**.
  2. Make sure **Map to Solr Schema** is checked.
  3. Make sure **Process All Pipeline Doc Fields** is checked.
  4. Enter the **Solr Document ID Field Value** as `<doc.id>`.
  5. Enter values under **Updates**:
     1. Enter the **Update Type** as `increment`.
     2. Enter the **Field Name** as `increment_i`.
     3. Enter a **Value**, such as `1`.
  6. Make sure **Allow reserved fields** is checked.
  7. Save the pipeline.

  ## Order the stages

  1. Make sure the stages are in the following order:
     1. Field Mapping
     2. Solr Dynamic Field Name Mapping
     3. LWAI: Summarization
     4. LWAI: Keyword Extraction
     5. Solr Partial Update Indexer
  2. Save the pipeline.
</Accordion>