Upgrade Fusion 3.x to a Newer Release

A migrator is available for upgrading Fusion 3.x to any later release. The migrator migrates all Fusion objects for a seamless upgrade. Migration will involve down time, and the process will involve multiple starts and stops of Fusion services. Please plan accordingly, especially in terms of disabling external load balancers or monitors that might react adversely to the starts and stops.

Important
The newer Fusion instance must be newly untarred and never started.

When upgrading from Fusion 3.0 to 3.1.0 and higher, some objects are transformed for compatibility with the new set of object types:

  • New 3.1 object types are group, link, task, job, and spark.

  • The aggregation object type has been removed. Aggregation objects are migrated to the spark object type.

  • The schedule object type has been removed. Schedule objects are migrated to the job object type.

  • The new HTML and XML parser stages are added to all default parsers.

  • Any installed JDBC drivers from Fusion 3.0 are uploaded to the 3.1 blob store.

In Fusion 3.1.0 and above, only a subset of connectors are included by default. The migrator detects which connectors were used in the older version of Fusion, and installs them automatically in Fusion 3.1. This step requires an internet connection. If no connection is available, then connectors must be downloaded at http://lucidworks.com/connectors/ and installed separately. If this is necessary, then a message in fusion/old_fusion_version/var/upgrade/tmp/migrator.log will indicate so.

Important
Be sure to download the correct migrator package for your platform (Unix or Windows).

Upgrade on Unix (Single Node)

How to upgrade Fusion 3.x on a single Unix node
  1. Download the latest version of Fusion and the migrator.

  2. Install the newer version of Fusion:

    $ cd /fusion-parent/
    $ mv ~/Downloads/fusion-new_fusion_version.tar.gz ./
    $ tar -xf fusion-new_fusion_version.tar.gz

    For example, if Fusion is currently installed in /opt/fusion/old_fusion_version, then extract the file in the /opt/ directory. Don’t run Fusion yet.

  3. Create this folder:

    $ mkdir fusion/old_fusion_version/var/upgrade
  4. Install the migrator:

    $ tar -zxv -C ./fusion/old_fusion_version/var/upgrade --strip-components=1 -f ~/Downloads/fusion-migrator.tar.gz
  5. Stop Fusion if it is running:

    $ fusion/old_fusion_version/bin/fusion stop
  6. Run migrator.jar, using absolute paths:

    $ java -jar /path/to/fusion/old_fusion_version/var/upgrade/migrator.jar -f /path/to/fusion/old_fusion_version -t /path/to/fusion/new_fusion_version -p properties-filename

    The filename for the correct properties file can be found in the fusion/old_fusion_version/var/upgrade/ directory. For example, to upgrade 3.1.0 to 3.1.2, use the 3.1.x-3.1.x.properties file.

    The end of the output from a successful migration looks like this:

    2017-07-12T13:35:38,654 - INFO  [main:Main@220] - Stopping new Fusion services
    2017-07-12T13:35:38,654 - INFO  [main:ServicesManager@194] - Starting process: java -jar /Users/username/Fusion/fusion/new_fusion_version/apps/lucidworks-agent.jar stop --no-agent zookeeper solr api connectors
    2017-07-12T13:35:43,430 - INFO  [main:Iterator@116] - Successfully stopped connectors (process ID 7748)
    2017-07-12T13:35:45,948 - INFO  [main:Iterator@116] - Successfully stopped api (process ID 7598)
    2017-07-12T13:35:50,237 - INFO  [main:Iterator@116] - Successfully stopped solr (process ID 7595)
    2017-07-12T13:35:51,375 - INFO  [main:Iterator@116] - Successfully stopped zookeeper (process ID 7575)
    Tip
    After migration, you can find details about the results in the fusion/old_fusion_version/var/upgrade/tmp directory. If the migration produces unexpected results, the files in this directory are helpful for troubleshooting.
  7. Set the FUSION_HOME environment variable to be the full path of the new Fusion folder:

    $ export FUSION_HOME=fusion/new_fusion_version
  8. Run the new version of Fusion (all services defined in fusion.properties):

    $ cd $FUSION_HOME
    $ ./bin/fusion start
  9. Validate the installation:

    1. Clear your browser’s cache.

      Otherwise, you might inadvertently access a cached version of the old Fusion UI and see inconsistent behavior.

    2. Access the Fusion UI at http://localhost:8764/.

    3. Log in.

    4. Navigate to admin > About Fusion.

      The About Fusion panel should display the newer Fusion release number.

Tip
When you are satisfied with the migration and you have backed up the fusion/old_fusion_version/ directory, you can rm -fr fusion/old_fusion_version/ to remove the older version of Fusion.

Upgrade on Unix (Multiple Nodes)

How to upgrade Fusion 3.x on multiple Unix nodes

Important: For every step on multiple nodes, ensure that the step completes on all nodes before going to the next step. There is also the notion of a "main node" during the migration process. This node will be used for certain centralized migration activities that do not need to be done on every node, such as downloading connectors that are then uploaded to blob storage that is shared by all, etc. Just pick one of your Fusion nodes to be the "main node"; there’s no special requirement as to which one you pick.

  1. Download the latest version of Fusion and the migrator.

  2. On all nodes, install the newer version of Fusion:

    $ cd /fusion-parent/
    $ mv ~/Downloads/fusion-new_fusion_version.tar.gz ./
    $ tar -xf fusion-new_fusion_version.tar.gz

    For example, if Fusion is currently installed in /opt/fusion/old_fusion_version, then extract the file in the /opt/ directory. Don’t run Fusion yet.

  3. Create this folder:

    $ mkdir fusion/old_fusion_version/var/upgrade
  4. On all nodes, install the migrator:

    $ tar -zxv -C ./fusion/old_fusion_version/var/upgrade --strip-components=1 -f ~/Downloads/fusion-migrator.tar.gz
  5. On the main node, run the migrator with the argument -1:

    $ java -jar $FUSION_HOME/var/upgrade/migrator.jar -f /var/lib/fusion/binary/fusion/old_fusion_version -t /var/lib/fusion/binary/fusion/new_fusion_version -p old_fusion_version-new_fusion_version.properties -1

    This message indicates that the step finished successfully:

    Step 1 finished successfully.
  6. On all nodes, stop the Fusion UI, Fusion connectors, Fusion API, and Solr; but not ZooKeeper:

    $ ./fusion/bin/ui stop && ./fusion/bin/connectors stop && ./fusion/bin/api stop && ./fusion/bin/solr stop
  7. On all nodes, run the migrator with the argument -2:

    $ java -jar $FUSION_HOME/var/upgrade/migrator.jar -f /var/lib/fusion/binary/fusion/old_fusion_version -t /var/lib/fusion/binary/fusion/new_fusion_version -p old_fusion_version-new_fusion_version.properties -2

    This message indicates that the step finished successfully:

    Step 2 finished successfully.
  8. On all nodes, stop ZooKeeper (unless you are using an external ZooKeeper instance, in which case you can ignore this step):

    $ ./fusion/bin/zookeeper stop
  9. On the main node, run the migrator with the argument -3:

    $ java -jar $FUSION_HOME/var/upgrade/migrator.jar -f /var/lib/fusion/binary/fusion/old_fusion_version -t /var/lib/fusion/binary/fusion/new_fusion_version -p old_fusion_version-new_fusion_version.properties -3

    This message indicates that the step finished successfully:

    Step 3 finished successfully.
  10. On all nodes except the main node, run the migrator with the argument -7:

    java -jar $FUSION_HOME/var/upgrade/migrator.jar -f /var/lib/fusion/binary/fusion/old_fusion_version -t /var/lib/fusion/binary/fusion/new_fusion_version -p old_fusion_version-new_fusion_version.properties -7

    This message indicates that the step finished successfully:

    Step 7 finished successfully.
  11. On all nodes, start ZooKeeper (unless you are using an external ZooKeeper instance, in which case you can ignore this step):

    ./fusion/new_fusion_version/bin/zookeeper start
  12. On the main node, run the migrator with the argument -4:

    $ java -jar /var/lib/fusion/binary/fusion/old_fusion_version/var/upgrade/migrator.jar -f /var/lib/fusion/binary/fusion/old_fusion_version -t /var/lib/fusion/binary/fusion/new_fusion_version -p old_fusion_version-new_fusion_version.properties -4

    This message indicates that the step finished successfully:

    Step 4 finished successfully.
  13. On all nodes, start the Fusion UI, Fusion connectors, Fusion API, and Solr:

    $ ./fusion/new_fusion_version/bin/fusion start
  14. On the main node, run the migrator with the argument -5:

    java -jar /var/lib/fusion/binary/fusion/old_fusion_version/var/upgrade/migrator.jar -f /var/lib/fusion/binary/fusion/old_fusion_version -t /var/lib/fusion/binary/fusion/new_fusion_version -p old_fusion_version-new_fusion_version.properties -5

    This message indicates that the step finished successfully:

    Step 5 finished successfully.
  15. On the main node, run the migrator with the argument -6:

    java -jar /var/lib/fusion/binary/fusion/old_fusion_version/var/upgrade/migrator.jar -f /var/lib/fusion/binary/fusion/old_fusion_version -t /var/lib/fusion/binary/fusion/new_fusion_version -p old_fusion_version-new_fusion_version.properties -6

    This message indicates that the step finished successfully:

    Step 6 finished successfully.
    Tip
    After migration, you can find details about the results in the fusion/old_fusion_version/var/upgrade/tmp directory. If the migration produces unexpected results, the files in this directory are helpful for troubleshooting.
  16. Set the FUSION_HOME environment variable to be the full path of the new Fusion 3.1.x folder:

    # export FUSION_HOME=fusion/new_fusion_version
  17. Run Fusion 3.1.x (all services defined in fusion.properties):

    # cd $FUSION_HOME
    # ./bin/fusion start
  18. Validate the installation:

    1. Clear your browser’s cache.

      Otherwise, you might inadvertently access a cached version of the old Fusion UI and see inconsistent behavior.

    2. Access the Fusion UI at http://localhost:8764/.

    3. Log in.

    4. Navigate to admin > About Fusion.

      The About Fusion panel should display "Version: 3.1.x".

Tip
When you are satisfied with the migration and you have backed up the fusion/old_fusion_version/ directory, you can rm -fr fusion/old_fusion_version/ to remove Fusion 3.0.

Upgrade on Windows (Single Node)

How to upgrade Fusion 3.x on a single Windows node
  1. Download Fusion 3.1 and the migrator zip file for Windows.

  2. Copy the fusion-new_fusion_version.zip file to the directory that contains the fusion\ directory.

    For example, if Fusion is installed at C:\Users\admin\fusion\3.0.1, then copy the file to C:\Users\admin\fusion-new_fusion_version.zip.

  3. Unzip the fusion-new_fusion_version.zip file, for example with 7zip. Don’t run Fusion yet.

  4. Create a fusion\old_fusion_version\var\upgrade directory.

  5. Unzip the migrator zip file, and move the contents of the extracted folder to fusion\old_fusion_version\var\upgrade.

  6. Make sure Fusion is stopped.

  7. Run migrator.jar.

    Tip
    After migration, you can find details about the results in the fusion\old_fusion_version\var\upgrade\tmp directory. If the migration produces unexpected results, the files in this directory are helpful for troubleshooting.
  8. Run the new version of Fusion.

  9. Validate the installation:

    1. Clear your browser’s cache.

      Otherwise, you might inadvertently access a cached version of the old Fusion UI and see inconsistent behavior.

    2. Access the Fusion UI at http://localhost:8764/.

    3. Log in.

    4. Navigate to admin > About Fusion.

      The About Fusion panel should display the new Fusion release number.

Tip
When you are satisfied with the migration and you have backed up the fusion\old_fusion_version directory, you can remove that directory.

Upgrade on Windows (Multiple Nodes)

How to upgrade Fusion 3.0.x to 3.1.x on multiple Windows nodes

Important: For every step on multiple nodes, ensure that the step completes on all nodes before going to the next step.

Perform the following tasks on all nodes:

  1. Download Fusion 3.1 and the 3.1 migrator zip file for Windows.

  2. Copy the fusion-new_fusion_version.zip file to the directory that contains the fusion\ directory.

    For example, if Fusion 3.0 is installed at C:\Users\admin\fusion\3.0.1, then copy the file to C:\Users\admin\fusion-new_fusion_version.zip.

  3. Unzip the fusion-new_fusion_version.zip file, for example with 7zip. Don’t run Fusion 3.1.x.

  4. Create a fusion*old_fusion_version*\var\upgrade directory.

  5. Unzip the migrator zip file, and move the contents of the extracted folder to fusion*old_fusion_version*\var\upgrade.

  6. Make sure Fusion is stopped.

  7. Set the FUSION_HOME environment variable to be the full path of the new Fusion 3.1.x folder.

Perform the following tasks on the indicated nodes:

  1. Run the migrator with the argument -1 (on the main node):

    $ java -jar $FUSION_HOME\var\upgrade\migrator.jar -f \var\lib\fusion\binary\fusion*old_fusion_version* -t \var\lib\fusion\binary\fusion*new_fusion_version* -p old_fusion_version-new_fusion_version.properties -1

    Note: In the above command and in what follows, each x in the name of the properties file is just an x. It isn’t a placeholder for a third-level version number.

    This message indicates that the step finished successfully:

    Step 1 finished successfully.
  2. On all nodes, stop the Fusion UI, Fusion connectors, Fusion API, and Solr:

    $ .\fusion\bin\ui stop && .\fusion\bin\connectors stop && .\fusion\bin\api stop && .\fusion\bin\solr stop
  3. Run the migrator with the argument -2 (on all nodes):

    $ java -jar $FUSION_HOME\var\upgrade\migrator.jar -f \var\lib\fusion\binary\fusion*old_fusion_version* -t \var\lib\fusion\binary\fusion*new_fusion_version* -p old_fusion_version-new_fusion_version.properties -2

    This message indicates that the step finished successfully:

    Step 2 finished successfully.
  4. On all nodes, Stop ZooKeeper (unless you are using an external ZooKeeper instance, in which case you can ignore this step):

    $ .\fusion\bin\zookeeper stop
  5. Run the migrator with the argument -3 (on the main node):

    $ java -jar $FUSION_HOME\var\upgrade\migrator.jar -f \var\lib\fusion\binary\fusion*old_fusion_version* -t \var\lib\fusion\binary\fusion*new_fusion_version* -p old_fusion_version-new_fusion_version.properties -3

    This message indicates that the step finished successfully:

    Step 3 finished successfully.
  6. Run the migrator with the argument -7 (on all nodes except the main node):

    java -jar $FUSION_HOME\var\upgrade\migrator.jar -f \var\lib\fusion\binary\fusion*old_fusion_version* -t \var\lib\fusion\binary\fusion*new_fusion_version* -p old_fusion_version-new_fusion_version.properties -7

    This message indicates that the step finished successfully:

    Step 7 finished successfully.
  7. On all nodes, start ZooKeeper (unless you are only using an external ZooKeeper instance, in which case you can ignore this step):

    .\fusion\*_new_fusion_version_*\bin\zookeeper start
  8. Run the migrator with the argument -4 (on the main node):

    $ java -jar \var\lib\fusion\binary\fusion\3.0.1\var\upgrade\migrator.jar -f \var\lib\fusion\binary\fusion*old_fusion_version* -t \var\lib\fusion\binary\fusion*new_fusion_version* -p old_fusion_version-new_fusion_version.properties -4

    This message indicates that the step finished successfully:

    Step 4 finished successfully.
  9. Start the Fusion UI, Fusion connectors, Fusion API, and Solr (on all nodes):

    $ .\fusion\*_new_fusion_version_*\bin\fusion start
  10. Run the migrator with the argument -5 (on the main node):

    java -jar \var\lib\fusion\binary\fusion*old_fusion_version*\var\upgrade\migrator.jar -f \var\lib\fusion\binary\fusion*old_fusion_version* -t \var\lib\fusion\binary\fusion*new_fusion_version* -p old_fusion_version-new_fusion_version.properties -5

    This message indicates that the step finished successfully:

    Step 5 finished successfully.
  11. Run the migrator with the argument -6 (on the main node):

    java -jar \var\lib\fusion\binary\fusion*old_fusion_version*\var\upgrade\migrator.jar -f \var\lib\fusion\binary\fusion*old_fusion_version* -t \var\lib\fusion\binary\fusion*new_fusion_version* -p old_fusion_version-new_fusion_version.properties -6

    This message indicates that the step finished successfully:

    Step 6 finished successfully.
    Tip
    After migration, you can find details about the results in the fusion*old_fusion_version*\var\upgrade\tmp directory. If the migration produces unexpected results, the files in this directory are helpful for troubleshooting.
  12. Run Fusion 3.1.x.

  13. Validate the installation:

    1. Clear your browser’s cache.

      Otherwise, you might inadvertently access a cached version of the old Fusion UI and see inconsistent behavior.

    2. Access the Fusion UI at http://localhost:8764/.

    3. Log in.

    4. Navigate to admin > About Fusion.

      The About Fusion panel should display "Version: 3.1.x".