Query Pipeline Stages

A query pipeline is made up of a series of query stages that process incoming search queries.

A pipeline stage definition associates a unique ID with a set of properties. These definitions are registered with the Fusion API service and stored in ZooKeeper for re-use across pipelines and search applications.

Fusion includes a number of specialized query stages as well as a JavaScript stage that allows advanced processing via a JavaScript program.

Configuring query pipeline stages

Asynchronous query pipeline processing

Query pipeline processing performance can be improved by enabling asynchronous processing for certain stages that make requests to secondary collections, external databases, and so on. The following stages support asynchronous processing:

This feature uses the fork-and-join model, where any of the stages above can create a fork on the pipeline. The parallel processes are joined again using the Merge Async Results stage at a later point in the pipeline.

How to enable asynchronous query pipeline processing
  1. In the Query Workbench, open the query pipeline you want to modify.

  2. Click Add a Stage and select a stage that supports asynchronous processing to your pipeline, or select the stage in your pipeline if it has already been added.

    Stages that support this are listed above.

  3. In the stage configuration panel, select Asynchronous Execution Config.

  4. Select Enable Async Execution.

    Enabling asynchronous processing

    Fusion automatically assigns an Async ID value to this stage. Change this to a more memorable string that describes the asynchronous stages you are merging, such as signals or access_control.
  5. Copy the Async ID value.

  6. Click Apply.

    Verify that the stage is in the correct position in your pipeline.

  7. Click Add a Stage and select the Merge Async Results stage.

    Add the Merge Async Results stage

  8. In the stage configuration panel, click the Add add icon button next to Async IDs.

  9. Paste the ID from the asynchronous query stage into the new ID field.

    When using multiple asynchronous stages, the order in which IDs are entered here is the order in which they are joined back into the pipeline.
  10. Click Apply.

    The Merge Async Results stage is now the first stage in the pipeline.

  11. Drag the Merge Async Results stage down so that it appears immediately before the Solr Query stage:

    Move the Merge Async Results

  12. Click Save.

Conditional query processing

Query Pipeline stages are used to modify Request objects and Response objects. Each stage can include a conditional JavaScript expression (the condition property in its configuration) that can access these objects.

For example, this condition first checks that the property fusion-user-name is specified in the Request object, then checks for a particular value:

request.hasParam("fusion-user-name") && request.getFirstParam("fusion-user-name").equals("SuperUser");

Reference topics

See these reference topics for complete details about each query pipeline stage:

Query rewriting