Domain Specific Language (DSL)

Fusion Domain Specific Language (DSL) provides expected search results as a JSON response in a way that reduces search query complexity for the user. Previously, users needed to understand complex syntax to express certain search queries in Fusion (for example, the best way to express a facet filter). DSL gives Fusion control over how to execute a query by transforming a structured query input into a Solr request, where we can add intelligence around the index and the user’s intent.

Fusion Search DSL provides:

  1. Standardized feature implementation with predefined syntax, abstracting away underlying SOLR semantics

  2. Future optionality to serve features from other platforms

  3. Revamped critical DSL object-based Query Rewrites, rather than arbitrary and unpredictable string manipulation

Request format

The query request includes parameters for relevant information, such as facets, number of results, and how to sort the response.

Example Request

{
 "query": {
  "type": "matchAll"
 },
 "filter": {
  "type": "matchAll"
 },
 "facets": [
  {
   "type": "field",
   "field": "glass_s",
   "limit": 3,
   "countMissing": true
  }
 ],
 "results": {
  "size": 10,
  "fields": [
   "glass_s",
   "instructions_t",
   "ingredients_ss"
  ],
  "sort": [
   {
    "field": "size_l",
    "order": "descending"
   }
  ]
 }
}

Response Format

A key part of DSL is defining what the search engine returns.

Example Response

Here is an example response, followed by descriptions for the response components.

{
  "meta": {
    "timing": {
      "total": 100,
      "mainQuery": 50,
      "pipeline": {
        "stage1": 10,
        "stage2": 20
      }
    },
    "debug": {
      "solrParams": {
        "f.alcoholic_s.facet.limit": "3",
        "f.glass_s.facet.limit": "3",
        "facet.field": [
          "alcoholic_s",
          "glass_s"
        ],
        "df": "_text_",
        "json.nl": "arrarr",
        "echoParams": "all",
        "lw.pipelineId": "cocktail",
        "fl": [
          "*",
          "score"
        ],
        "start": "0",
        "isFusionQuery": "true",
        "sort": "",
        "rows": "10",
        "fusionQueryId": "Mp2XLX0VCB",
        "q": "glass",
        "defType": "edismax",
        "f.glass_s.facet.mincount": "1",
        "f.alcoholic_s.facet.missing": "true",
        "context": "app:cocktail",
        "f.alcoholic_s.facet.mincount": "1",
        "wt": "json",
        "facet": "true",
        "f.glass_s.facet.missing": "true"
      },
      "solrDebug": {}
    }
  },
  "facets": {
    "field": {
      "glass_s": {
        "missing": 1,
        "counts": [
          {
            "name": "Cocktail glass",
            "count": 122
          },
          {
            "name": "Highball glass",
            "count": 83
          },
          {
            "name": "Collins glass",
            "count": 61
          }
        ],
        "displayName": "Type of glass"
      },
      "alcoholic_s": {
        "missing": 2,
        "displayName": "Alcoholic?",
        "counts": [
          {
            "name": "Alcoholic",
            "count": 478
          },
          {
            "name": "Non alcoholic",
            "count": 57
          },
          {
            "name": "Optional alcohol",
            "count": 9
          }
        ]
      }
    }
  },
  "results": {
    "hits": 493,
    "maxScore": 3.1299,
    "pagination": {
      "cursorMark": "YV9MaXF1b3JAoE/A3F1aWNrc3RhcnQvSW93fU2FsZXMuY3N2IzA="
    },
    "docs": [
      {
        "collection": "cocktails",
        "type": "generic",
        "id": "15549",
        "score": 3.1299,
        "fields": {
          "glass_s": "Old-fashioned glass",
          "instructions_t": "Stir into glass over ice, garnish and serve.",
          "alcoholic_s": "Alcoholic",
          "ingredients_ss": [
            "Gin",
            "Sweet Vermouth",
            "Campari"
          ],
          "ibaList_s": "Unforgettables",
          "category_s": "Ordinary Drink",
          "drink_t": "Negroni"
        },
        "highlighting": {
          "instructions_t": [
            "Stir into <em>glass</em> over ice"
          ]
        }
      }
    ]
  }
}

Response sections

Meta

The meta section holds response parts not displayed by the search interface, but that may be helpful in debugging or profiling the Fusion application.

Meta can include:

  1. Search engine Debug information, such as Solr debug output or echoParams

  2. Fusion debug information, such as applied query rewrite rules

  3. Timing information, such as Solr QTime, total time spent in pipeline, or timing breakdown per stage in the pipeline

Facets

The facets section holds response data keyed facet type. Unlike Solr’s standard facet response format, facet values are expressed as an object list, regardless of the facet type (for example, field or range). The facet object is made extensible and can contain additional metadata, such as how many docs are missing the field value.

Results

The results section holds the main part of the response, including metadata.

Docs

The results section also contains a docs section with the list of objects representing documents.

Document objects in DSL differ from the Solr document response format:

  • docs fields contain the actual key-value pairs for document data stored in Solr

  • The document object is only a part of the response document, rather than it being the whole document

  • Other top-level keys:

    • score computed for this document on this query is stored here, rather than as a synthetic field within the document

    • unique key id is kept here rather than in the regular fields data

    • type contains information if the document belongs to a data model type

  • highlighting contains per-document emphasis based on the search