Solr Upgrade Notes

The following notes describe changes to Solr in recent releases that you should be aware of before upgrading.

These notes highlight the biggest changes that may impact the largest number of implementations. It is not a comprehensive list of all changes to Solr in any release.

When planning your Solr upgrade, consider the customizations you have made to your system and review the {solr-javadocs}/changes/Changes.html[CHANGES.txt] file found in your Solr package. That file includes all the changes and updates that may effect your existing implementation.

Detailed steps for upgrading a Solr cluster are in the section Upgrading a Solr Cluster.

Upgrading to 7.x Releases

Solr 7.7

See the 7.7 Release Notes for an overview of the main new features in Solr 7.7.

When upgrading to Solr 7.7.x, users should be aware of the following major changes from v7.6:

Admin UI

  • The Admin UI now presents a login screen for any users with authentication enabled on their cluster. Clusters with Basic Authentication will prompt users to enter a username and password. On clusters configured to use Kerberos Authentication, users will be directed to configure their browser to provide an appropriate Kerberos ticket.

    The login screen’s purpose is cosmetic only - Admin UI-triggered Solr requests were subject to authentication prior to 7.7 and still are today. The login screen changes only the user experience of providing this authentication.

Distributed Requests

  • The shards parameter, used to manually select the shards and replicas that receive distributed requests, now checks nodes against a whitelist of acceptable values for security reasons.

    In SolrCloud mode this whitelist is automatically configured to contain all live nodes. In standalone mode the whitelist is empty by default. Upgrading users who use the shards parameter in standalone mode can correct this value by setting the shardsWhitelist property in any shardHandler configurations in their solrconfig.xml file.

    For more information, see the Distributed Request documentation.

Solr 7.6

See the 7.6 Release Notes for an overview of the main new features in Solr 7.6.

When upgrading to Solr 7.6, users should be aware of the following major changes from v7.5:

Collections

  • The JSON parameter to set cluster-wide default cluster properties with the CLUSTERPROP command has changed.

    The old syntax nested the defaults into a property named clusterDefaults. The new syntax uses only defaults. The command to use is still set-obj-property.

    An example of the new syntax is:

    {
      "set-obj-property": {
        "defaults" : {
          "collection": {
            "numShards": 2,
            "nrtReplicas": 1,
            "tlogReplicas": 1,
            "pullReplicas": 1
          }
        }
      }
    }

    The old syntax will be supported until at least Solr 9, but users are advised to begin using the new syntax as soon as possible.

  • The parameter min_rf has been deprecated and no longer needs to be provided in order to see the achieved replication factor. This information will now always be returned to the client with the response.

Autoscaling

  • An autoscaling policy is now used as the default strategy for selecting nodes on which new replicas or replicas of new collections are created.

    A default policy is now in place for all users, which will sort nodes by the number of cores and available freedisk, which means by default a node with the fewest number of cores already on it and the highest available freedisk will be selected for new core creation.

  • The change described above has two additional impacts on the maxShardsPerNode parameter:

    1. It removes the restriction against using maxShardsPerNode when an autoscaling policy is in place. This parameter can now always be set when creating a collection.

    2. It removes the default setting of maxShardsPerNode=1 when an autoscaling policy is in place. It will be set correctly (if required) regardless of whether an autoscaling policy is in place or not.

      The default value of maxShardsPerNode is still 1. It can be set to -1 if the old behavior of unlimited maxSharedsPerNode is desired.

DirectoryFactory

  • Lucene has introduced the ByteBuffersDirectoryFactory as a replacement for the RAMDirectoryFactory, which will be removed in Solr 9.

    While most users are still encouraged to use the NRTCachingDirectoryFactory, which allows Lucene to select the best directory factory to use, if you have explicitly configured Solr to use the RAMDirectoryFactory, you are encouraged to switch to the new implementation as soon as possible before Solr 9 is released.

    For more information about the new directory factory, see the Jira issue LUCENE-8438.

    For more information about the directory factory configuration in Solr, see the section DataDir and DirectoryFactory in SolrConfig.

Solr 7.5

See the 7.5 Release Notes for an overview of the main new features in Solr 7.5.

When upgrading to Solr 7.5, users should be aware of the following major changes from v7.4:

Schema Changes

  • Since Solr 7.0, Solr’s schema field-guessing has created _str fields for all _txt fields, and returned those by default with queries. As of 7.5, _str fields will no longer be returned by default. They will still be available and can be requested with the fl parameter on queries. See also the section on field guessing for more information about how schema field guessing works.

  • The Standard Filter, which has been non-operational since at least Solr v4, has been removed.

Index Merge Policy

  • When using the TieredMergePolicy, the default merge policy for Solr, optimize and expungeDeletes now respect the maxMergedSegmentMB configuration parameter, which defaults to 5000 (5GB).

    If it is absolutely necessary to control the number of segments present after optimize, specify maxSegments as a positive integer. Setting maxSegments higher than 1 are honored on a "best effort" basis.

    The TieredMergePolicy will also reclaim resources from segments that exceed maxMergedSegmentMB more aggressively than earlier.

UIMA Removed

  • The UIMA contrib has been removed from Solr and is no longer available.

Logging

  • Solr’s logging configuration file is now located in server/resources/log4j2.xml by default.

  • A bug for Windows users has been corrected. When using Solr’s examples (bin/solr start -e) log files will now be put in the correct location (example/ instead of server). See also Solr Examples and Solr Control Script Reference for more information.

Solr 7.4

See the 7.4 Release Notes for an overview of the main new features in Solr 7.4.

When upgrading to Solr 7.4, users should be aware of the following major changes from v7.3:

Logging

  • Solr now uses Log4j v2.11. The Log4j configuration is now in log4j2.xml rather than log4j.properties files. This is a server side change only and clients using SolrJ won’t need any changes. Clients can still use any logging implementation which is compatible with SLF4J. We now let Log4j handle rotation of Solr logs at startup, and bin/solr start scripts will no longer attempt this nor move existing console or garbage collection logs into logs/archived either. See Configuring Logging for more details about Solr logging.

  • Configuring slowQueryThresholdMillis now logs slow requests to a separate file named solr_slow_requests.log. Previously they would get logged in the solr.log file.

Legacy Scaling (non-SolrCloud)

  • In the master-slave model of scaling Solr, a slave no longer commits an empty index when a completely new index is detected on master during replication. To return to the previous behavior pass false to skipCommitOnMasterVersionZero in the slave section of replication handler configuration, or pass it to the fetchindex command.

If you are upgrading from a version earlier than Solr 7.3, please see previous version notes below.

Solr 7.3

See the 7.3 Release Notes for an overview of the main new features in Solr 7.3.

When upgrading to Solr 7.3, users should be aware of the following major changes from v7.2:

ConfigSets

  • Collections created without specifying a configset name have used a copy of the _default configset since Solr 7.0. Before 7.3, the copied configset was named the same as the collection name, but from 7.3 onwards it will be named with a new ".AUTOCREATED" suffix. This is to prevent overwriting custom configset names.

Learning to Rank

  • The rq parameter used with Learning to Rank rerank query parsing no longer considers the defType parameter. See Running a Rerank Query for more information about this parameter.

Autoscaling & AutoAddReplicas

  • The behaviour of the autoscaling system will now pause all triggers from execution between the start of actions and the end of a cool down period. The triggers will resume after the cool down period expires. Previously, the cool down period was a fixed period started after actions for a trigger event completed and during this time all triggers continued to run but any events were rejected and tried later.

  • The throttling mechanism used to limit the rate of autoscaling events processed has been removed. This deprecates the actionThrottlePeriodSeconds setting in the set-properties Autoscaling API which is now non-operational. Use the triggerCooldownPeriodSeconds parameter instead to pause event processing.

  • The default value of autoReplicaFailoverWaitAfterExpiration, used with the AutoAddReplicas feature, has increased to 120 seconds from the previous default of 30 seconds. This affects how soon Solr adds new replicas to replace the replicas on nodes which have either crashed or shutdown.

Logging

  • The default Solr log file size and number of backups have been raised to 32MB and 10 respectively. See the section Configuring Logging for more information about how to configure logging.

SolrCloud

  • The old Leader-In-Recovery implementation (implemented in Solr 4.9) is now deprecated and replaced. Solr will support rolling upgrades from old 7.x versions of Solr to future 7.x releases until the last release of the 7.x major version.

    This means to upgrade to Solr 8 in the future, you will need to be on Solr 7.3 or higher.

  • Replicas which are not up-to-date are no longer allowed to become leader. Use the FORCELEADER command of the Collections API to allow these replicas become leader.

Spatial

  • If you are using the spatial JTS library with Solr, you must upgrade to 1.15.0. This new version of JTS is now dual-licensed to include a BSD style license. See the section on Spatial Search for more information.

Highlighting

  • The top-level <highlighting> element in solrconfig.xml is now officially deprecated in favour of the equivalent <searchComponent> syntax. This element has been out of use in default Solr installations for several releases already.

If you are upgrading from a version earlier than Solr 7.2, please see previous version notes below.

Solr 7.2

See the 7.2 Release Notes for an overview of the main new features in Solr 7.2.

When upgrading to Solr 7.2, users should be aware of the following major changes from v7.1:

Local Parameters

  • Starting a query string with local parameters {!myparser …​} is used to switch from one query parser to another, and is intended for use by Solr system developers, not end users doing searches. To reduce negative side-effects of unintended hack-ability, Solr now limits the cases when local parameters will be parsed to only contexts in which the default parser is "lucene" or "func".

    So, if defType=edismax then q={!myparser …​} won’t work. In that example, put the desired query parser into the defType parameter.

    Another example is if deftype=edismax then hl.q={!myparser …​} won’t work for the same reason. In this example, either put the desired query parser into the hl.qparser parameter or set hl.qparser=lucene. Most users won’t run into these cases but some will need to change.

    If you must have full backwards compatibility, use luceneMatchVersion=7.1.0 or an earlier version.

eDisMax Parser

  • The eDisMax parser by default no longer allows subqueries that specify a Solr parser using either local parameters, or the older _query_ magic field trick.

    For example, {!prefix f=myfield v=enterp} or _query_:"{!prefix f=myfield v=enterp}" are not supported by default any longer. If you want to allow power-users to do this, set uf=* query or some other value that includes _query_.

    If you need full backwards compatibility for the time being, use luceneMatchVersion=7.1.0 or something earlier.

If you are upgrading from a version earlier than Solr 7.1, please see previous version notes below.

Solr 7.1

See the 7.1 Release Notes for an overview of the main new features of Solr 7.1.

When upgrading to Solr 7.1, users should be aware of the following major changes from v7.0:

AutoAddReplicas

  • The feature to automatically add replicas if a replica goes down, previously available only when storing indexes in HDFS, has been ported to the autoscaling framework. Due to this, autoAddReplicas is now available to all users even if their indexes are on local disks.

    Existing users of this feature should not have to change anything. However, they should note these changes:

    • Behavior: Changing the autoAddReplicas property from disabled (false) to enabled (true) using MODIFYCOLLECTION API no longer replaces down replicas for the collection immediately. Instead, replicas are only added if a node containing them went down while autoAddReplicas was enabled. The parameters autoReplicaFailoverBadNodeExpiration and autoReplicaFailoverWorkLoopDelay are no longer used.

    • Deprecations: Enabling/disabling autoAddReplicas cluster-wide with the API will be deprecated; use suspend/resume trigger APIs with name=".auto_add_replicas" instead.

      More information about the changes to this feature can be found in the section SolrCloud Automatically Adding Replicas.

Metrics Reporters

  • Shard and cluster metric reporter configuration now require a class attribute.

    • If a reporter configures the group="shard" attribute then please also configure the class="org.apache.solr.metrics.reporters.solr.SolrShardReporter" attribute.

    • If a reporter configures the group="cluster" attribute then please also configure the class="org.apache.solr.metrics.reporters.solr.SolrClusterReporter" attribute.

      See the section Shard and Cluster Reporters for more information.

Streaming Expressions

  • All Stream Evaluators in solrj.io.eval have been refactored to have a simpler and more robust structure. This simplifies and condenses the code required to implement a new Evaluator and makes it much easier for evaluators to handle differing data types (primitives, objects, arrays, lists, and so forth).

ReplicationHandler

  • In the ReplicationHandler, the master.commitReserveDuration sub-element is deprecated. Instead please configure a direct commitReserveDuration element for use in all modes (master, slave, cloud).

RunExecutableListener

  • The RunExecutableListener was removed for security reasons. If you want to listen to events caused by updates, commits, or optimize, write your own listener as native Java class as part of a Solr plugin.

XML Query Parser

  • In the XML query parser (defType=xmlparser or {!xmlparser …​ }) the resolving of external entities is now disallowed by default.

If you are upgrading from a version earlier than Solr 7.0, please see Major Changes in Solr 7 before starting your upgrade.

Upgrading to 7.x from Any 6.x Release

The upgrade from Solr 6.x to Solr 7.0 introduces several major changes that you should be aware of before upgrading. Please do a thorough review of the section Major Changes in Solr 7 before starting your upgrade.

Upgrading to 7.x from pre-6.x Versions of Solr

Users upgrading from versions of Solr prior to 6.x are strongly encouraged to consult {solr-javadocs}/changes/Changes.html[CHANGES.txt] for the details of all changes since the version they are upgrading from.

A summary of the significant changes between Solr 5.x and Solr 6.0 is in the section Major Changes from Solr 5 to Solr 6.