Skip to main content
In the Keyword extraction use case of the Lucidworks AI Async Prediction API, the LLM ingests text and returns a JSON response that contains a list of keywords extracted from the text. No options can be configured. This use case can be used:
  • To extract keywords to create new facets
  • To extract keywords from each document:
    • To write the keyword into a new field that subsequently searches large content documents more efficiently
    • To boost specific keyword content
    • For document clustering
The Keyword extraction use case contains two requests:
  • POST request - submits a prediction task for a specific useCase and modelId. The API responds with the following information:
    • predictionId. A unique UUID for the submitted prediction task that can be used later to retrieve the results.
    • status. The current state of the prediction task.
  • GET request - uses the predictionId you submit from a previously-submitted POST request and returns the results associated with that previous request.
For detailed API specifications in Swagger/OpenAPI format, see Platform APIs.

​
Prerequisites

To use this API, you need:
  • The unique APPLICATION_ID for your Lucidworks AI application. For more information, see credentials to use APIs.
  • A bearer token generated with a scope value of machinelearning.predict. For more information, see Authentication API.
  • The USE_CASE and MODEL_ID fields in the /async-prediction for the POST request. The path is /ai/async-prediction/USE_CASE/MODEL_ID. A list of supported modes is returned in the Lucidworks AI Use Case API. For more information about supported models, see Generative AI models.

​
Common POST request parameters and fields

Some parameters in the /ai/async-prediction/USE_CASE/MODEL_ID POST request are common to all of the generative AI (Gen-AI) use cases, such as the modelConfig parameter. Also referred to as hyperparameters, these fields set certain controls on the response. Refer to the API spec for more information.

​
Unique values for the keyword extraction use case

Some parameter values available in the keyword extraction use case are unique to this use case, such as the useCaseConfig parameter. Refer to the API spec for more information.

​
Example POST request

This example does not include modelConfig parameters, but you can submit requests that include parameters described in Common POST request parameters and fields. 
curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/keyword-extraction/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
  "batch": [
    {
      "text": "Joseph Robinette Biden Jr.is an American politician who is the 46th and current president of the United States. Ideologically a moderate member of the Democratic Party, he previously served as the 47th vice president from 2009 to 2017 under President Barack Obama and represented Delaware in the United States Senate from 1973 to 2009.Born in Scranton, Pennsylvania, Biden moved with his family to Delaware in 1953. He studied at the University of Delaware before earning his law degree from Syracuse University. He was elected to the New Castle County Council in 1970 and to the U.S. Senate in 1972. As a senator, Biden drafted and led the effort to pass the Violent Crime Control and Law Enforcement Act and the Violence Against Women Act. He also oversaw six U.S. Supreme Court confirmation hearings, including the contentious hearings for Robert Bork and Clarence Thomas. Biden ran unsuccessfully for the Democratic presidential nomination in 1988 and 2008. In 2008, Obama chose Biden as his running mate, and Biden was a close counselor to Obama during his two terms as vice president. In the 2020 presidential election, Biden and his running mate, Kamala Harris, defeated incumbents Donald Trump and Mike Pence. Biden is the second Catholic president in U.S. history (after John F. Kennedy), and his politics have been widely described as profoundly influenced by Catholic social teaching."
      }
    ],
    "useCaseConfig": {
      "maxKeywords": 100
    }
  }'

​
Example GET request

curl --request GET
--url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
--header 'Authorization: Bearer Auth '

​
Learn more

This pipeline uses the following stages:
  • Field Mapping
  • Solr Dynamic Field Name Mapping
  • LWAI: Summarization
  • LWAI: Keyword Extraction
  • Solr Partial Update Indexer

​
Add the pipeline

  1. Navigate to Indexing > Index Pipelines.
  2. Click Add+.
  3. Enter the Pipeline ID, for example LWAI-async-summarization-and-keyword-extraction.
  4. Select the Solr Indexer stage and click on Remove stage.

​
Field Mapping

Configure the Field Mapping stage as follows.
  1. Make sure that Unmapped Fields is checked.
  2. Make sure that Keep is checked.
  3. Save the pipeline.

​
Solr Dynamic Field Name Mapping

Configure the Solr Dynamic Field Name Mapping stage as follows.
  1. Enter a Max Length for Advanced Indexing of Text Fields, such as 100.
  2. Save the pipeline.

​
LWAI: Summarization

Summarization is set up via the LWAI Prediction stage.
  1. Click Add a new pipeline stage > LWAI Prediction.
  2. Enter a Label, such as LWAI: Summarization.
  3. Select the Account Name as defined in Lucidworks AI Gateway.
  4. Select the Use Case as summarization.
  5. Select the Model to use.
  6. Enter the Input context variable, for example, <doc.body_t>.
  7. Enter the Destination Field Name & Context Output, for example, body_summarization.
  8. Enter values for the Use Case Configuration:
    1. Enter the Parameter Name, such as maxWords.
    2. Enter a Parameter Value, for example, 250.
  9. Make sure Call Asynchronously? is checked.
  10. Enter the Maximum Asynchronous Call Tries, such as 1.
  11. Save the pipeline.

​
LWAI: Keyword Extraction

Keyword extraction is set up using the LWAI Prediction stage.
  1. Click Add a new pipeline stage > LWAI Prediction.
  2. Enter a Label, such as LWAI: Keyword Extraction.
  3. Select the Account Name as defined in Lucidworks AI Gateway.
  4. Select the Use Case as keyword-extraction.
  5. Select the Model to use.
  6. Enter the Input context variable, for example, <doc.body_t>.
  7. Enter the Destination Field Name & Context Output, for example, body_extracted_keywords.
  8. Enter values for the Use Case Configuration:
    1. Enter the Parameter Name, such as maxKeywo`rds.
    2. Enter the Parameter Value, for example, 50.
  9. Make sure Call Asynchronously? is checked.
  10. Enter the Maximum Asynchronous Call Tries, such as 1.
  11. Save the pipeline.

​
Solr Partial Update Indexer

Configure the Solr Partial Update Indexer stage as follows.
  1. Click Add a new pipeline stage > Solr Partial Update Indexer.
  2. Make sure Map to Solr Schema is checked.
  3. Make sure Process All Pipeline Doc Fields is checked.
  4. Enter the Solr Document ID Field Value as <doc.id>.
  5. Enter values under Updates:
    1. Enter the Update Type as increment.
    2. Enter the Field Name as increment_i.
    3. Enter a Value, such as 1.
  6. Make sure Allow reserved fields is checked.
  7. Save the pipeline.

​
Order the stages

  1. Make sure the stages are in the following order:
    1. Field Mapping
    2. Solr Dynamic Field Name Mapping
    3. LWAI: Summarization
    4. LWAI: Keyword Extraction
    5. Solr Partial Update Indexer
  2. Save the pipeline.