LWAI async summarization and keyword extraction pipelineLucidworks AI

Table of Contents

This pipeline uses the following stages:

Add the pipeline

Navigate to Indexing > Index Pipelines.
Click Add+.
Enter the Pipeline ID, for example LWAI-async-summarization-and-keyword-extraction.
Select the Solr Indexer stage and click on Remove stage.

Configure the Field Mapping stage as follows.

Configure the Solr Dynamic Field Name Mapping stage as follows.

Summarization is set up via the LWAI Prediction stage.

Click Add a new pipeline stage > LWAI Prediction.
Enter a Label, such as LWAI: Summarization.
Select the Account Name as defined in Lucidworks AI Gateway.
Select the Use Case as summarization.
Select the Model to use.
Enter the Input context variable, for example, <doc.body_t>.
Enter the Destination Field Name & Context Output, for example, body_summarization.
Enter values for the Use Case Configuration:
1. Enter the Parameter Name, such as maxWords.
2. Enter a Parameter Value, for example, 250.
Make sure Call Asynchronously? is checked.
Enter the Maximum Asynchronous Call Tries, such as 1.
Save the pipeline.

Keyword extraction is set up using the LWAI Prediction stage.

Click Add a new pipeline stage > LWAI Prediction.
Enter a Label, such as LWAI: Keyword Extraction.
Select the Account Name as defined in Lucidworks AI Gateway.
Select the Use Case as keyword_extraction.
Select the Model to use.
Enter the Input context variable, for example, <doc.body_t>.
Enter the Destination Field Name & Context Output, for example, body_extracted_keywords.
Enter values for the Use Case Configuration:
1. Enter the Parameter Name, such as maxKeywo`rds.
2. Enter the Parameter Value, for example, 50.
Make sure Call Asynchronously? is checked.
Enter the Maximum Asynchronous Call Tries, such as 1.
Save the pipeline.

Configure the Solr Partial Update Indexer stage as follows.

Click Add a new pipeline stage > Solr Partial Update Indexer.
Make sure Map to Solr Schema is checked.
Make sure Process All Pipeline Doc Fields is checked.
Enter the Solr Document ID Field Value as <doc.id>.
Enter values under Updates:
1. Enter the Update Type as increment.
2. Enter the Field Name as increment_i.
3. Enter a Value, such as 1.
Make sure Allow reserved fields is checked.
Save the pipeline.