Query Pipelines API

The Query Pipelines API allows you to define parameters that should be applied to all queries processed by the system. A query pipeline consists of one or more stages. Query pipeline stages can be defined with the Query Stages API. See the Query Language Cheat Sheet for help constructing queries to send through a query pipeline.

Manage Query Pipelines: List, Create, Delete

The path for this request is:

/api/apollo/query-pipelines/<id>

where <id> is the name of an specific pipeline.

  • GET - returns the definition of a specific index pipeline, or all defined pipelines, if the pipeline name is omitted.

  • POST - creates a new pipeline. The pipeline name is included in the definition in the request body.

  • DELETE - removes the pipeline.

Manage Query Pipelines: Update

A PUT request to endpoint:

/api/apollo/query-pipelines/<id>

will update an existing query pipeline. A followup PUT request to endpoint:

/api/apollo/query-pipelines/<id>/refresh

is required to make this change take effect.

List a Query Pipeline

The path for this request is:

/api/apollo/query-pipelines/<id>

where <id> is the name of an specific pipeline.

A GET request returns the definition of a specific query pipeline, or all defined pipelines, if the pipeline name is omitted.

The output is a representation of each pipeline requested, created, or modified, including all properties of each stage.

Output

The output is a JSON object or list of JSON objects for each pipeline.

In the case of DELETE, no output is returned.

Note:

If you update a stage with a PUT request, you must reload the stage to the system with a second PUT request:

curl -u user:password-X PUT http://localhost:8764/api/apollo/query-pipelines/id/refresh

Send a Query through a Pipeline

The path for this request is:

/api/apollo/query-pipelines/<id>/collections/<collectionName>/<handler>?<solrQuery>

where <id> is the name of an specific pipeline, <collectionName> is the name of an specific collection, and <handler> is a Solr request handler, e.g. "select" and <solrQuery> is any valid Solr query.

Either a GET or POST method can be used to submit this request. Use the POST method if the length of your query exceeds normal GET limitations.

The output will be very similar to a standard Solr JSON response to a query. The response header will include the query parameters, and the response will include the documents and any other requested features such as facets, highlighting, etc.

Query Pipeline Definition Properties

Property Description

id
Required

Required only when creating a new pipeline.

stages
Required

A JSON map of the stages to include with the pipeline. The stages can exist already or they can be defined while defining the pipeline.

The stage must include the stage definitions:

  • id: the ID of the stage to include.

  • stageParams: Any parameters for the stage. These are only required for a stage that has been pre-defined and you wish to modify the properties.

  • type: The stage type. You can define a stage directly on a pipeline, or you can use a pre-existing stage. If you use a pre-existing stage, you must use "ref", as a reference to an existing stage.

  • skip: If the stage should be skipped during pipeline processing. By default, this is 'false', and does not need to be defined unless you would like to skip a stage in the future.

If the stage supports setting other properties, as defined in the section Query Pipeline Stages, then those can be defined while defining the stage within the pipeline.

Examples

Get the definition for the default query pipeline:

REQUEST

curl -u user:pass http://localhost:8764/api/apollo/query-pipelines/default

RESPONSE

{
  "id" : "default",
  "stages" : [ {
    "type" : "recommendation",
    "id" : "recommendation",
    "numRecommendations" : 10,
    "useFq" : true,
    "numSignals" : 100,
    "aggrType" : "click@doc_id_s-query_s-filters_s",
    "skip" : false
  }, {
    "type" : "solr-query",
    "id" : "solr",
    "skip" : false
  } ]
}

Create a new query pipeline, specifying a facet stage that is new and a solr-query stage that was previously defined:

REQUEST

curl -u user:pass -X POST -H 'Content-Type: application/json' -d '{"id":"my-pipeline", "stages":[{"type":"facet", "id":"myFacets", "fieldFacets":[{"field":"data_source_s"},{"field":"author_s", "minCount":5, "missing":true},{"field":"isRetweet_b"}], "skip":false}, {"id":"mySolr", "type":"ref", "skip":false}]}' http://localhost:8764/api/apollo/query-pipelines

RESPONSE

{
  "id" : "my-pipeline",
  "stages" : [ {
    "type" : "facet",
    "id" : null,
    "fieldFacets" : [ {
      "field" : "data_source_s",
      "prefix" : null,
      "sort" : null,
      "limit" : null,
      "offset" : null,
      "missing" : null,
      "method" : null,
      "threads" : null,
      "minCount" : null,
      "enumCacheMinDf" : null
    }, {
      "field" : "author_s",
      "prefix" : null,
      "sort" : null,
      "limit" : null,
      "offset" : null,
      "missing" : true,
      "method" : null,
      "threads" : null,
      "minCount" : 5,
      "enumCacheMinDf" : null
    }, {
      "field" : "isRetweet_b",
      "prefix" : null,
      "sort" : null,
      "limit" : null,
      "offset" : null,
      "missing" : null,
      "method" : null,
      "threads" : null,
      "minCount" : null,
      "enumCacheMinDf" : null
    } ],
    "skip" : false
  }, {
    "type" : "solr-query",
    "id" : null,
    "skip" : false
  } ]
}

Query a pipeline named 'default' and a collection named "docs" for the term "solr". Also limit the response to only document 'titles' and return it in JSON format:

REQUEST

curl -u user:pass http://localhost:8764/api/apollo/query-pipelines/default/collections/docs/select?q=solr&fl=title&wt=json

RESPONSE

{
    "response": {
        "numFound": 106,
        "start": 0,
        "docs": [
            {
                "title": [
                    "Solr and SolrAdmin APIs - Fusion Documentation - Lucidworks"
                ]
            },
            {
                "title": [
                    "Solr and SolrAdmin APIs - Fusion Documentation - Lucidworks"
                ]
            },
            {
                "title": [
                    "Search Clusters - Fusion Documentation - Lucidworks"
                ]
            },
            {
                "title": [
                    "Search Clusters - Fusion Documentation - Lucidworks"
                ]
            },
            {
                "title": [
                    "Searching - Fusion Documentation - Lucidworks"
                ]
            }
        ]
    },
    "responseHeader": {
        "status": 0,
        "QTime": 1,
        "params": {
            "fl": "title",
            "lw.pipelineId": "default",
            "q": "solr",
            "wt": "json",
            "rows": "5",
            "defType": "edismax"
        }
    }
}