System API

The System REST API allows you to monitor the system performance.

The 'metrics' endpoint provides statistics on the performance of each service, with several available counters, gauges, histograms, etc. More information about the available metrics is available in the section System Metrics.

The 'ping' endpoint allows you to ping the system for a response, to be sure it is up.

The 'threads' endpoint reports on the running threads.

The 'buffers' endpoint allows flushing buffers.

The 'version' endpoint shows the versions of included components.

Get Metric Names

The request path is:

/api/apollo/system/metricNames

A GET request shows available metric names, for use with the 'metric' endpoint (described in the following section). This request takes the following optional parameters:

Parameter Description

pattern

The pattern allows using a regular expression to find all configuration items that contain the specified string.

prefix

The prefix allows finding all configuration items that start with the specified string.

The response will include all matching metrics, separated by type of metric:

  • Gauges: single values, valid for the point in time at which the metrics are collected.

  • Counters: values that are incremented or decremented over time.

  • Histograms: the distribution of values. They will report the minimum, maximum, mean, and the values at the 50th, 75th, 95th, 98th, 99th, and 99.9th percentiles.

  • Meters: rate of events over time. They include a mean rate, as well as a 1-, 5- and 15-minute moving average. Most of these moving averages are exponentially weighted, so that more recent values contribute more heavily than older values; exceptions to this rule have the word "unweighted" in their name.

  • Timers: a meter combined with a histogram; it measures the length of time that a particular operation takes (both mean duration and moving averages) as well as the distribution of those durations.

When filtering on metric names, matching names will be shown in the applicable section.

Get System Metrics

The request path is:

/api/apollo/system/metrics

A GET request will return the system metrics.

Because there are nearly 600 metrics, the metricNames endpoint can help you identify a specific metric to return. This request takes the following optional query parameters which can also be used to filter the outputs:

Parameter Description

pattern

The pattern allows using a regular expression to find all configuration items that contain the specified string.

prefix

The prefix allows finding all configuration items that start with the specified string.

pretty

If true, the response will be formatted for easier reading.

rateUnit

The time unit (i.e., "seconds", "minutes", etc.) to use to show rates.

durationUnit

The time unit (i.e., "seconds", "minutes", etc.) to use to show durations.

showSamples

If true, the data values used in calculations will be shown.

The output will include matching metrics, separated by type of metric, and any data for the metrics.

Many of the metrics include percentile and mean rates. Some are simply counts.

See the section System Metrics for more details about metrics that may be of interest.

Threads

The request path is:

/api/apollo/system/threads

A GET request shows all active system threads, as Java thread dump in JSON. The output is a standard Java thread dump, formatted for JSON.

Buffers

The request path is:

/api/apollo/system/buffers

A PUT request with this endpoint will flush the currently active buffers, if needed.

Ping

The request path is:

/api/apollo/system/ping

A GET request queries the system to see if it is up. If the system is up, the response will be 'pong'.

Examples

Metric names

Get metric names that start with 'mem.heap':

REQUEST

curl -u user:pass http://localhost:8764/api/apollo/system/metricNames?prefix=mem.heap

RESPONSE

{
  "gauges" : [ "mem.heap.committed", "mem.heap.init", "mem.heap.max", "mem.heap.usage", "mem.heap.used" ],
  "counters" : [ ],
  "histograms" : [ ],
  "meters" : [ ],
  "timers" : [ ]
}

Metrics

Find all metrics that match the regular expression pattern 'com.lucidworks.apollo.pipeline.index.', format the response so it’s readable, and show the sample data, if any:

REQUEST

curl -u user:pass http://localhost:8764/api/apollo/system/metrics?pattern=com.lucidworks.apollo.*pipeline.index.*&pretty=true&showSamples=true

RESPONSE

{
  "version" : "3.0.0",
  "gauges" : { },
  "counters" : { },
  "histograms" : { },
  "meters" : {
    "com.lucidworks.apollo.pipeline.index.IndexPipelineCache.cacheHit" : {
      "count" : 4775,
      "m15_rate" : 0.03604340402401043,
      "m1_rate" : 0.04985610410800882,
      "m5_rate" : 0.04753263154077047,
      "mean_rate" : 0.05028487069705915,
      "units" : "events/second"
    }
  },
  "timers" : {
    "com.lucidworks.apollo.pipeline.index.IndexPipelineCache.deserialize" : {
      "count" : 7,
      "max" : 0.078783,
      "mean" : 0.028265285714285715,
      "min" : 9.800000000000001E-5,
      "p50" : 1.94E-4,
      "p75" : 0.06996100000000001,
      "p95" : 0.078783,
      "p98" : 0.078783,
      "p99" : 0.078783,
      "p999" : 0.078783,
      "values" : [ 9.800000000000001E-5, 1.09E-4, 1.62E-4, 1.94E-4, 0.04855, 0.06996100000000001, 0.078783 ],
      "stddev" : 0.03620774742010466,
      "m15_rate" : 2.964393875E-314,
      "m1_rate" : 2.128434034679706E-46,
      "m5_rate" : 4.9195401948202935E-138,
      "mean_rate" : 7.371603924510614E-5,
      "duration_units" : "seconds",
      "rate_units" : "calls/second"
    },
    "com.lucidworks.apollo.pipeline.index.IndexStageConfigCache.deserialize" : {
      "count" : 7,
      "max" : 0.002377,
      "mean" : 4.6642857142857147E-4,
      "min" : 8.7E-5,
      "p50" : 1.16E-4,
      "p75" : 3.46E-4,
      "p95" : 0.002377,
      "p98" : 0.002377,
      "p99" : 0.002377,
      "p999" : 0.002377,
      "values" : [ 8.7E-5, 9.300000000000001E-5, 1.11E-4, 1.16E-4, 1.35E-4, 3.46E-4, 0.002377 ],
      "stddev" : 8.47267737523163E-4,
      "m15_rate" : 2.964393875E-314,
      "m1_rate" : 2.1522151745137967E-46,
      "m5_rate" : 5.086288568158318E-138,
      "mean_rate" : 7.372680363461613E-5,
      "duration_units" : "seconds",
      "rate_units" : "calls/second"
    },
    "com.lucidworks.apollo.pipeline.index.IndexStageConfigStore.deserialize" : {
      "count" : 6,
      "max" : 0.0019760000000000003,
      "mean" : 5.606666666666667E-4,
      "min" : 1.0800000000000001E-4,
      "p50" : 1.4250000000000002E-4,
      "p75" : 0.00114875,
      "p95" : 0.0019760000000000003,
      "p98" : 0.0019760000000000003,
      "p99" : 0.0019760000000000003,
      "p999" : 0.0019760000000000003,
      "values" : [ 1.0800000000000001E-4, 1.2200000000000001E-4, 1.35E-4, 1.5000000000000001E-4, 8.730000000000001E-4, 0.0019760000000000003 ],
      "stddev" : 7.54704622131511E-4,
      "m15_rate" : 2.964393875E-314,
      "m1_rate" : 1.0220227879692082E-48,
      "m5_rate" : 7.20591046931865E-140,
      "mean_rate" : 6.318494243486903E-5,
      "duration_units" : "seconds",
      "rate_units" : "calls/second"
    },
    "com.lucidworks.apollo.pipeline.index.IndexStageConfigStore.getItem" : {
      "count" : 6,
      "max" : 0.003128,
      "mean" : 0.002295,
      "min" : 0.0017770000000000002,
      "p50" : 0.0020715,
      "p75" : 0.002774,
      "p95" : 0.003128,
      "p98" : 0.003128,
      "p99" : 0.003128,
      "p999" : 0.003128,
      "values" : [ 0.0017770000000000002, 0.002066, 0.002066, 0.0020770000000000003, 0.0026560000000000004, 0.003128 ],
      "stddev" : 4.989869737778733E-4,
      "m15_rate" : 2.964393875E-314,
      "m1_rate" : 1.0220227879692082E-48,
      "m5_rate" : 7.20591046931865E-140,
      "mean_rate" : 6.318494244418448E-5,
      "duration_units" : "seconds",
      "rate_units" : "calls/second"
    }
  }
}

The above output has been truncated for space to remove metrics with no data or with very long value lists.

Threads

REQUEST

curl -u user:pass http://localhost:8764/api/apollo/system/threads

RESPONSE (truncated to a single thread)

[ {
  "id" : 2,
  "native" : false,
  "name" : "Reference Handler",
  "locks" : {
    "waiting" : {
      "identity" : "0x7257d934",
      "class" : "java.lang.ref.Reference$Lock"
    },
    "locking" : {
      "identity" : "0x7257d934",
      "class" : "java.lang.ref.Reference$Lock"
    }
  },
  "state" : "WAITING",
  "suspended" : false,
  "stackTrace" : [ {
    "methodName" : "wait",
    "fileName" : "Object.java",
    "lineNumber" : -2,
    "className" : "java.lang.Object",
    "nativeMethod" : true
  }, {
    "methodName" : "wait",
    "fileName" : "Object.java",
    "lineNumber" : 503,
    "className" : "java.lang.Object",
    "nativeMethod" : false
  }, {
    "methodName" : "run",
    "fileName" : "Reference.java",
    "lineNumber" : 133,
    "className" : "java.lang.ref.Reference$ReferenceHandler",
    "nativeMethod" : false
  } ]
} ]

Buffers

REQUEST

curl -u user:pass -X PUT http://localhost:8764/api/apollo/system/buffers

Ping

REQUEST

curl -u user:pass http://localhost:8764/api/apollo/system/ping

RESPONSE

pong