Pipeline Stages Reference

Index Pipeline stages are used to create and modify PipelineDocument objects to control how incoming data is indexed in Fusion’s Solr core.

Query Pipeline stages are used to modify Request objects and Response objects to control how query results are returned to the end user.

Pipeline Stage Properties

Fusion associates a pipeline stage definition with a unique ID and stores the definition so that stages can be reused across pipelines and applications. In addition to an ID, all stages have the following properties:

  • type (required): an enumeration, one of the defined Fusion pipeline stage types, e.g., "index-logging". If the Fusion UI is used to define the stage, this property is filled in automatically.

  • label (optional) : a string field with a maximum length of 255 characters. The label is displayed on the Fusion UI.

  • skip (optional): a boolean value, If true, pipeline processing bypasses this stage altogether. The default is false.

  • condition (optional): a JavaScript expression that evaluates to true (1) or false (0). If this condition evaluates to false, this stage is skipped. The default is true.

Conditional Processing

Each Fusion pipeline stage contains a "Condition" field at the beginning of the stage configuration found in the Fusion UI. A conditional script can be used to dynamically turn a stage on/off. For example, in the index pipeline you can create a field mapping stage and create a condition to only execute the stage if a document contains the field 'sample_field', and to skip the stage otherwise.

doc.hasField("sample_field");

For the example above, if the pipeline document has the field 'sample_field', then the conditional script returns 'true' and the pipeline stage is executed. If the pipeline document does not have the 'sample_field', the condition returns 'false' and the pipeline stage is skipped.

The JavaScript expression specified in the condition property of a pipeline stage has access to the pipeline objects. For example, query processing can be conditional on information in the query Request object:

request.hasParam("fusion-user-name") && request.getFirstParam("fusion-user-name").equals("SuperUser");

The above condition first checks that the property "fusion-user-name" is specified, then checks for a particular value.