Signals API

The Signals API accepts a set of signals, encoded as JSON objects, for indexing into a signals collection.

Normally, signals are indexed just like ordinary documents, through a configured datasource and index pipeline. This API is provided for cases where it is more convenient to index signals directly.

To aggregate the signals, see the Signals Aggregator API.

Signal document structure

A raw signal is stored as a Solr document with the following fields, which are derived from the raw signal as follows:

Field Description

id
Optional

The signal ID. If no ID is supplied, one will be automatically generated.

type
Required

The signal type that is being sent. This value is used during aggregation to filter events of the same type. Types can be mixed in aggregation jobs, if needed.

The type can consist of any string you choose. For consistency, always send events of the same type with the same type value.

During indexing, type values will be moved to a field named type_s.

params
Optional

The params allow flexible field definition of the fields you care about and will use later for signal aggregation:

  • docId - A unique document ID

    This is stored in the Solr raw signal document as field doc_id_s.

  • query - A query string; for example, a user’s search

    This is copied to the Solr raw signal document as both fields query_s and query_t. Some cleanup occurs to convert the string to lowercase, decode URL encoding, and replace white space with single space characters. The original query is saved in field query_orig_s.

  • filterQueries - A list of strings, such as filters on the search query

    This is copied to the Solr raw signal document as both filters_s and filters_orig_ss.

  • collection - The primary collection name

  • weight - A float value representing the relative weight of this signal

    This is saved in the field weight_d.

  • count - A positive integer value representing the incremented count of signals

    This is saved in the field count_i.

timestamp

The timestamp of the signal event.

  • When using the Signals API, this property is optional; it defaults to the current server time.

  • When using the Signal Formatter index stage, one of the following fields must be present: timestamp, timestamp_tdt, timestamp_dt, or epoch.

Here is the JSON representation of one click signal, taken from an example dataset of synthetic clickstream data:

{ "params": {
      "docId": "2125233",
      "filterQueries": ["cat00000","abcat0100000", "abcat0101000", "abcat0101001"],
      "query": "Televisiones Panasonic  50 pulgadas" }
 "type":"click",
 "timestamp": "2011-09-01T23:44:52.533000Z",
}

Examples

Send two signal events to record user clicks:

REQUEST

curl \
 -u admin:password123 -X POST -H 'Content-type:application/json' -d @- \
http://localhost:8764/api/apollo/signals/docs?commit=true \
<<EOF
[
 {"params": {
      "query": "Televisiones Panasonic  50 pulgadas",
      "filterQueries": ["cat00000","abcat0100000", "abcat0101000", "abcat0101001"],
      "docId": "2125233" },
 "type":"click",
 "timestamp": "2011-09-01T23:44:52.533000Z"
 },
 {"params": {
       "query": "Sharp",
       "filterQueries": ["cat00000", "abcat0100000", "abcat0101000", "abcat0101001"],
       "docId": "2009324" },
  "type":"click",
  "timestamp": "2011-09-05T12:25:37.420000Z"
 }
]
EOF

A successful request results in events being added to the signals collection. For the above example, the events will be represented as follows:

 {
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "indent":"true",
      "q":"doc_id_s: 2125233",
      "wt":"json"}
 },
  "response":{"numFound":1198,"start":0,"docs":[
      {
        "id": "7aee7b1f-5cde-4957-b73c-c15881f559ec",
        "filters_s": "abcat0100000 $ abcat0101000 $ abcat0101001 $ cat00000",
        "query_orig_s": "Televisiones Panasonic  50 pulgadas",
        "params.user_s": "000000df17cd56a5df4a94074e133c9d4739fae3",
        "doc_id_s": "2125233",
        "params.query_time__s": "2011-09-01T23:43:59.752Z",
        "query_t": "televisiones panasonic 50 pulgadas",
        "query_s": "televisiones panasonic 50 pulgadas",
        "filters_orig_ss": [
          "abcat0100000",
          "abcat0101000",
          "abcat0101001",
          "cat00000"
        ],
        "flag_s": "EVENT",
        "type_s": "click",
        "attr_params.filterQueries_": [
          "cat00000",
          "abcat0100000",
          "abcat0101000",
          "abcat0101001"
        ],
        "timestamp_dt": "2011-09-01T23:44:52.533Z",
        "_version_": 1478892846857584600
      },
      {
        "id": "6789a209-f5b5-457e-9df6-8033b8f7f317",
        "filters_s": "abcat0100000 $ abcat0101000 $ abcat0101001 $ cat00000",
        "query_orig_s": "Sharp",
        "params.user_s": "000001928162247ffaf63185cd8b2a244c78e7c6",
        "doc_id_s": "2009324",
        "params.query_time__s": "2011-09-05T12:25:01.187Z",
        "query_t": "sharp",
        "query_s": "sharp",
        "filters_orig_ss": [
          "abcat0100000",
          "abcat0101000",
          "abcat0101001",
          "cat00000"
        ],
        "flag_s": "EVENT",
        "type_s": "click",
        "attr_params.filterQueries_": [
          "cat00000",
          "abcat0100000",
          "abcat0101000",
          "abcat0101001"
        ],
        "timestamp_dt": "2011-09-05T12:25:37.42Z",
        "_version_": 1478892846859681800
      }
    ]
  }
}