> ## Documentation Index
> Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Index Pipeline Stages

> Configuration specifications

export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => {
  const [isLoaded, setIsLoaded] = useState(false);
  useEffect(() => {
    const timer = setTimeout(() => {
      setIsLoaded(true);
    }, 500);
    return () => clearTimeout(timer);
  }, []);
  return <div className="lw-template-container">
      <Card title={title} icon={icon}>
        {isLoaded && <span dangerouslySetInnerHTML={{
    __html: `<lw-template id="a029c1a9-28be-427e-b0e1-5d918920246a"></lw-template
            >`
  }} />}
        <Link href={linkHref} className="agent-studio-link text-left text-gray-600 gap-2 dark:text-gray-400 text-sm font-medium flex flex-row items-center hover:text-primary dark:hover:text-primary-light group-hover:text-primary group-hover:dark:text-primary-light">Powered by Lucidworks Agent Studio</Link>
      </Card>
    </div>;
};

[localhost link]: http://localhost:3000/docs/lucidworks-search/09-developer-documentation/config-specs/index-pipeline-stages/overview

[mintlify link]: https://doc.lucidworks.com/docs/lucidworks-search/09-developer-documentation/config-specs/index-pipeline-stages/overview

[old doc.lw link]: https://doc.lucidworks.com/managed-fusion/5.9/0m6vl6

Index Pipeline stages are used to create and modify [PipelineDocument objects](https://javadoc.lucidworks.com/fusion-pipeline-javadocs/5.9/com/lucidworks/apollo/common/pipeline/PipelineDocument.html). Use the [Index Workbench](/docs/lucidworks-search/03-ui-tour/index-workbench) to configure stages in a pipeline and preview the results.

The detailed configuration article about each Lucidworks Search index pipeline stage is provided in this section.

Configuration details are found for each pipeline stage in this section.

To configure an index pipeline stage, sign in to Lucidworks Search and click **Indexing > Index Workbench**. Then click **Load** to load a datasource, click **Add a Stage**, and select the stage you want to configure. The stages are displayed under these categories:

* **Document Transformation.** This section displays options that let you configure stages such as Asynchronous Field Parser and XML Transformation.
* **Document Filtering and Enrichment.** This section displays options that let you configure stages such as Detect Language, Exclude Documents, Include Documents, and Format Signals.
* **Field Transformation.** This section displays options that let you configure stages such as Date Parsing, Field Mapping, and Geoip Lookup.
* **Natural Language Processing.** This section displays the option that lets you configure the Gazetteer Lookup Extraction stage.
* **Indexing.** This section displays options that let you configure stages such as Solr Indexer and Update Related Document.
* **Troubleshooting.** This section displays the option that lets you configure the Logging stage.
* **Advanced.** This section displays options that let you configure stages such as Call Data Model Pipeline, Data Model Mapping, Exclusion Filter, and Set Property.
* **Custom.** This section displays the option that lets you configure the JavaScript stage.
* **AI.** This section displays options that let you configure stages such as LWAI Prediction, LWAI Vectorize Field, and Machine Learning.

For information about how to configure stages in an index pipeline, see [Index Workbench](/docs/lucidworks-search/03-ui-tour/index-workbench).

For conceptual information, see [Index Pipeline Stages](/docs/lucidworks-search/04-move-data-in/index-pipeline/index-pipeline-stages).

<LwTemplate />

## Global pipeline stage properties

**Required**

* **type:** The type of the pipeline stage. This must be one of the defined Lucidworks Search pipeline stage types, such as `index-logging`. If the Lucidworks Search UI is used to define the stage, this property is filled in automatically.

**Optional**

* **label:** A string field with a maximum length of 255 characters. The label is displayed on the Lucidworks Search UI.
* **skip:** A boolean value. If true, pipeline processing bypasses this stage. The default is false.
* **condition:** A JavaScript expression that evaluates to true (1) or false (0). If this condition evaluates to false, this stage is skipped. The default is true.

## Pipeline Condition Expression

The JavaScript expression specified in the condition property of a pipeline stage has access the pipeline objects.

### Index Pipeline Stage Condition Example

An index PipelineDocument has two available variables: `doc` and `ctx`.

To check whether pipeline document contains a named field:

```js wrap  theme={"dark"}
doc.hasField("acl_ss")
```

### Query pipeline stage Condition Example

To process a query based the query request object:

```js wrap  theme={"dark"}
request.hasParam("fusion-user-name") && request.getFirstParam("fusion-user-name").equals("SuperUser");
!request.hasParam("isFusionPluginQuery")
```

The first condition checks that the request parameter `fusion-user-name` is present and has the value `SuperUser`. The second condition checks that the request parameter `isFusionPluginQuery` is not present.

## Run stages asynchronously

Some stages allow for asynchronous processing for work that may take a while, keeping the overall pipeline moving.
Lucidworks Search does not block the request thread while the stage is waiting.
Instead, it waits for the stage's result right before it needs it to finish the response.

### When to use asynchronous processing

Use asynchronous processing for the following:

* A stage calls out to a remote system.
* The stage reads or writes large files or blobs.
* The stage can run in parallel with other stages.

Use synchronous processing for the following:

* The stage does only local work and typically finishes in under a couple of milliseconds.
* The stage coordinates tasks that must happen one at a time.

### Benefits of asynchronous processing

Asynchronous processing allows:

* Lower latency by overlapping slower calls with other work.
* Greater resilience because if one asynchronous call is slow or errors out, Lucidworks Search can time it out without stalling the whole pipeline.
* Easier error recognition and resolution because timeouts and cancellations are scoped to the stage.
