Product Selector

Fusion 5.9
    Fusion 5.9

    Query Pipeline Stages

    A query pipeline is made up of a series of query stages that process incoming search queries.

    A pipeline stage definition associates a unique ID with a set of properties. These definitions are registered with the Fusion API service and stored in ZooKeeper for re-use across pipelines and search applications.

    Fusion includes a number of specialized query stages as well as a JavaScript stage that allows advanced processing via a JavaScript program.

    The Query Pipeline Stages Reference section contains articles that detail the configuration for each of the Fusion query pipeline stages. The configuration information includes field names and values, as well as examples and other key reference information.

    Configuring query pipeline stages

    Pipeline stage JSON editor

    The pipeline stage JSON editor gives the ability to create and copy pipeline stages by pasting JSON objects in the Fusion UI. Only JSON is supported, and JSON validation is included to prevent the user from saving an invalid object.

    Navigate to your pipelines, select a pipeline stage, and click the JSON View button to open the editor:

    JSON View button

    Existing stages are considered READ-ONLY. They will have only a copy button which can be used to create stages. New stages will have the option to copy or paste valid JSON. Changes are applied to the stage with the Apply button.

    Users will still need to Save the stage for the changes made in the editor to be saved.

    The editor has several elements worth noting:

    Elements Description

    Copy

    Copies the JSON to the clipboard.

    Paste

    Pastes the clipboard into the editor. Only valid JSON is accepted.

    Expand

    Expands the editor to a fullscreen view.

    Condense

    Condenses the editor to a compact view.

    Apply

    Applies the JSON in the editor and updates the stage. The stage must be saved to preserve the changes.

    Reset

    Resets changes made since the last time changes were applied.

    For instructions, see Use the Pipeline Stage JSON Editor.

    JavaScript selector

    Now your pipeline definitions can include your choice of JavaScript engine, either Nashorn or OpenJDK Nashorn. While Nashorn is the default option, it is in the process of being deprecated and will eventually be removed, so it is recommended to use OpenJDK Nashorn when possible. You can select the JavaScript engine in the pipeline views or in the workbenches. Your JavaScript pipeline stages are interpreted by the selected engine.

    JavaScript engine selector

    Asynchronous query pipeline processing

    Query pipeline processing performance can be improved by enabling asynchronous processing for certain stages that make requests to secondary collections, external databases, and so on. The following stages support asynchronous processing:

    This feature uses the fork-and-join model, where any of the stages above can create a fork on the pipeline. The parallel processes are joined again using the Merge Async Results stage at a later point in the pipeline.

    Conditional query processing

    Query Pipeline stages are used to modify Request objects and Response objects. Each stage can include a conditional JavaScript expression (the condition property in its configuration) that can access these objects.

    For example, this condition first checks that the property fusion-user-name is specified in the Request object, then checks for a particular value:

    request.hasParam("fusion-user-name") && request.getFirstParam("fusion-user-name").equals("SuperUser");