Manual Scaling and Autoscaling

Managed search supports both manual scaling of the number of Solr nodes in a Managed Search cluster and autoscaling, for which Managed Search scales the number of Solr nodes automatically based on specific conditions.

What scaling accomplishes

Scaling the number of Solr nodes (that is, increasing or decreasing the number of Solr nodes) accomplishes this:

  • Scaling up – Handle higher search-request loads by adding Solr nodes. This keeps search-request latencies at a desirable level, instead of letting them increase.

  • Scaling down – When search-request loads decrease, free up resources that aren’t needed to provide desirable search-request latencies.

Manual scaling

Scale Solr nodes in Managed Search clusters to meet the search needs of your organization. You can specify these scale-related attributes of a cluster when you create it, and when you manually scale the cluster later.

Attribute Description

nodeType

Solr node type. Different Solr node types make available different numbers of CPUs and different amounts of memory. Different node types also have different default amounts of disk storage, if you create a cluster without specifying storageSizeGB.

nodeCount

Number of Solr nodes. This top-level nodeCount serves two purposes. First, with manual scaling, it is a number you set when creating a cluster and that you can change. It is also the current node count.

Note
By default, Managed Search supports clusters of between 3 and 30 nodes. If you need more nodes, Contact Customer Support.

storageSizeGB

Amount of disk space to use per Solr node, in gigabytes.

Important
You can’t decrease storageSizeGB; you can only increase it.

Note
During both manual scaling and autoscaling, Managed Search ensures that no data is lost.

Additionally, you can enable autoscaling to ensure that the performance of Solr nodes in your Managed Search clusters scales up and down automatically to meet varying levels of demand.

Node type (nodeType) and storage size (storageSizeGB) settings apply both when autoscaling is enabled and disabled.

Example JSON request body

Use this JSON request body in a POST request to the /customers/customerId/clusters endpoint (to create a cluster) or in a PUT or PATCH request to the /customers/customerId/clusters/clusterId endpoint (to modify a cluster). The request sets the Solr node count to use when autoscaling isn’t enabled to 5.

{
  "id": "test-1",
  "nodeCount": 5,
  "nodeType": "standard-2",
  "storageSizeGB": 20,
  "description": "test cluster",
  "maintenanceWindow": "SUNDAY,21:45"
}

Introduction to autoscaling

Enable autoscaling to ensure that your Managed Search clusters handle varying levels of searching demand. Autoscaling automatically increases and decreases the number of Solr nodes in a Managed Search cluster.

Managed Search supports two types of autoscaling.

Type Description

Scheduled

Summary: Managed Search automatically scales the number of Solr nodes in a cluster to specified numbers at specified times on specified days. If no time is specified in a schedule entry, then the default time 00:00 UTC (Coordinated Universal Time) is used.

For example, this schedule scales the number of nodes up to 10 at 15:00 UTC (7:00 PM Pacific Standard Time) on Monday and back down to 5 nodes at 02:00 UTC on Friday (6:00 AM Pacific Standard Time):

"schedule": [
  {
    "MON 15:00": {
      "nodeCount": 10
    }
  },
  {
    "FRI 02:00": {
      "nodeCount": 5
    }
  }
]

Scale-up: When a scheduled time for autoscaling arrives and the number of Solr nodes specified in the schedule entry exceeds the current number of Solr nodes

Scale-down: When a scheduled time for autoscaling arrives and the number of Solr nodes specified in the schedule entry is fewer than the current number of Solr nodes

Search rate

Summary: Managed Search compares the actual average search rate across Solr nodes to the target average search rate (targetSearchRate). Managed Search adds and removes Solr nodes to keep the actual average search rate in the vicinity of the target search rate. The number of Solr nodes can range between a minumum (minNodes) and maximum (maxNodes).

Important
Managed Search calculates the search rate for the /select request handler only. If you use a different or custom request handler, then searches that use the other request handler aren’t considered for autoscaling. If you use search rate autoscaling, we caution against using request handlers other than /select.

Scale-up: When the average search rate equals or exceeds the target search rate

Scale-down: When the average search rate falls below the target search rate

Note
During both manual scaling and autoscaling, Managed Search ensures that no data is lost.

Scheduled autoscaling

Scheduled autoscaling scales the number of Solr nodes in a Managed Search cluster based on the day of the week and the time of day. For example, the level of demand for searching might increase at the end of the work day and then decrease at night.

This diagram illustrates the simple case that is used as an example in Introduction to autoscaling: autoscaling up (to 10 nodes) for the work week and down (to 5 nodes) for the weekend.

Scheduled autoscaling

A schedule contains two or more schedule entries. Each schedule entry gives the time of the scale transition and the number of Solr nodes to use after the transition. See Attributes for scheduled autoscaling for more information.

Note
The autoscaling schedule applies every week, until you change the schedule or disable autoscaling.

Attributes for scheduled autoscaling

Specify these attributes when enabling scheduled autoscaling.

Attribute Description

enabled

Specify true to enable autoscaling.

schedule

JSON array of two or more schedule-entry JSON objects. In each schedule entry, specify:

  • Autoscaling-time strings. These consist of a comma-separated list day-of-week strings, optionally followed by a space and then a comma-separated list of time-of-day strings). These are the times of scale transitions from the current number of Solr nodes to the nodeCount specified in the schedule entry. Use three-letter abbreviations for the days of the week. Specify 24-hour UTC (Coordinated Universal Time) times. If an autoscaling-time string omits the time, then the time 00:00 (UTC) is used.

  • nodeCount, the number of Solr nodes to use at this scale. The smallest permitted nodeCount is 3.

Example autoscaling-time strings are:

  • Trigger autoscaling to the number of nodes specified in nodeCount on Sunday at 00:00 UTC:

    SUN
  • Trigger autoscaling to the number of nodes specified in nodeCount at 07:45 UTC on both Saturday and Monday:

    SAT,MON 7:45
  • Trigger autoscaling to the number of nodes specified in nodeCount at both 16:30 and 21:00 UTC on both Thursday and Saturday:

    THU,SAT 16:30,21:00

Example JSON request body

Use this JSON request body in a PUT or PATCH request to the /customers/customerId/clusters/clusterId endpoint. The request enables scheduled autoscaling. Solr node counts are changed among 3, 5, and 15 on the schedules shown.

{
  "id": "test-cluster",
  "nodeCount": 3,
  "nodeType": "standard-4",
  "storageSizeGB": 10,
  "description": "test cluster",
  "maintenanceTimeFrame": "SUN 21:45",
  "autoscaling": {
      "enabled":true,
        "schedule": [
          {
            "SUN": {
              "nodeCount": 5
            }
          },
          {
            "MON,TUE 11:45": {
              "nodeCount": 15
            }
          },
          {
            "THU,SAT 16:30": {
              "nodeCount": 3
            }
          }
        ]
  }
}

Search rate autoscaling

Search rate autoscaling scales the number of Solr nodes in a Managed Search cluster up and down based on comparisons of the actual average of search rates for all nodes and a target average search rate for all nodes:

  • Up – When search load increases, Managed Search increases the number of Solr nodes to better meet the demand.

    When the average of the trailing one-minute search rates across current Solr nodes equals or exceeds the target search rate, Managed Search increases the number of nodes by one, until the maximum number of nodes maxNodes is reached.

  • Down – When search load decreases, Managed Search decreases the number of Solr nodes to reduce resource consumption.

    When the average of the trailing one-minute search rates across current Solr nodes is less than the target search rate, Managed Search decreases the number of nodes by one, until the minimum number of nodes minNodes is reached.

This diagram illustrates search rate autoscaling. "QPN" is queries per node per time interval, that is, the average search rate across nodes.

Search rate autoscaling

Search rate autoscaling keeps the actual average search rate closer to the target average search rate

Attributes for search rate autoscaling

Specify these attributes when enabling search rate autoscaling.

Attribute Description

enabled

Specify true to enable autoscaling.

minNodes

The minimum number of Solr nodes to use. The smallest permitted minNodes is 3.

maxNodes

The maximum number of Solr nodes to use.

targetSearchRate

Target average of per-node search rates. This rate is used as a criterion for autoscaling both upward and downward. Looked at another way, Managed Search keeps the actual average search rate in the vicinity of targetSearchRate by adding Solr nodes to and subtracting Solr nodes from a Managed Search cluster.

Example JSON request body

Use this JSON request body in a PUT or PATCH request to the /customers/customerId/clusters/clusterId endpoint. The request enables search rate autoscaling. When the trailing one-minute search rate averaged across current Solr nodes equals or exceeds 10000, then Managed Search increases the number of Solr nodes from 3 to 10. When the trailing one-minute search rate averaged across current Solr nodes falls below 10000, then Managed Search decreases the number of Solr nodes from 10 to 3.

{
  "id": "test-cluster",
  "nodeCount": 3,
  "nodeType": "standard-4",
  "storageSizeGB": 10,
  "description": "test cluster",
  "maintenanceTimeFrame": "SUN 21:45",
  "autoscaling": {
      "enabled":true,
      "minNodes":3,
      "maxNodes":10,
      "targetSearchRate":10000
  }
}

Changes in node counts when switching approaches

These rules apply when enabling and disabling autoscaling, and when switching between different types of autoscaling:

Enabling autoscaling:

  • When you enable scheduled or search rate autoscaling, the node count initially doesn’t change.

  • With scheduled autoscaling, the first adjustment in node count occurs when the first transition time in the schedule arrives.

  • With search rate autoscaling, the first adjustment in node count occurs when the search-rate comparison algorithm determines that an adjustment is needed.

Switching between autoscaling types:

  • When you switch between scheduled and search-rate autoscaling, the node count doesn’t change.

  • When you switch to scheduled autoscaling, the first adjustment in node count occurs when the first transition time in the schedule arrives.

  • When you switch to search rate autoscaling, the first adjustment in node count occurs when the search-rate comparison algorithm determines that an adjustment is needed.

Disabling autoscaling:

  • When you disable autoscaling, the node count doesn’t change, with one exception: when you disable autoscaling and specify a nodeCount in the request.