Upgrade Fusion 2.1.4 or 2.4.x to Fusion 3.0.0
This article describes how to upgrade from Fusion 2.1.4 or 2.4.x to Fusion 3.0.0 for Linux and Windows.
Note
|
To upgrade to Fusion 3.0.0 from an earlier version of Fusion, first follow the steps to upgrade from 1.2 to 2.4; then follow the steps below. |
Linux
The steps in this section describe how to upgrade from Fusion 2.1.4 or 2.4.x to Fusion 3.0 on Linux.
Ensure that these prerequisites are met before upgrading Fusion.
Note
|
If you have multi-node Fusion, this software is required on all nodes. |
The upgrade scripts require Python 2.7, and that the following Python libraries are already installed:
These steps describe how to upgrade from Fusion 2.1.4 or 2.4.x to Fusion 3.0.0 on Linux.
Note
|
If you have a multi-node installation and unless otherwise stated, you must perform each step on each node as you go. For example, if you have 3 nodes, perform step 1 on node 1, node 2, and then node 3. Next, perform step 2 on node 1, node 2, and then node 3. Run steps that are only needed on a single node on your first node; not on the other nodes. |
-
(On all nodes) Download Fusion 3.0.0.
-
(On all nodes) Extract Fusion to a folder. Here we’ll call it
fusion-new
, as an example:mkdir fusion-new tar -C fusion-new --strip-components=1 -xf fusion-3.0.0.tar.gz
-
(On all nodes) Set the
FUSION_OLD_HOME
environment variable to the location of the old version of Fusion (2.x.y):export FUSION_OLD_HOME=/path/to/fusion/2.x.y
-
(On all nodes) Set the
FUSION_HOME
environment variable to the location of the new version of Fusion (3.0):export FUSION_HOME=fusion-new/3.0.0
-
(On all nodes) Copy data from the older fusion instance to the new fusion instance:
cp -R $FUSION_OLD_HOME/data/* $FUSION_HOME/data/
-
(On all nodes) Clone the fusion-upgrade-scripts repository, if you haven’t already:
git clone https://github.com/lucidworks/fusion-upgrade-scripts
-
(On all nodes) Upgrade the configuration. This upgrades the customized properties in
$FUSION_OLD_HOME/conf/config.sh
to the new properties file in 3.0.0 ($FUSION_HOME/conf/fusion.properties
).cd fusion-upgrade-scripts/src python upgrade-to-3.0.py --upgrade config
Alternatively, if all of the Fusion nodes use the same configuration, then you can just run this on one node, and then copy the new
fusion.properties
file to all of the other nodes. -
(On all nodes) Check the upgrade configuration in
$FUSION_HOME/conf/config.sh
and ensure that all of the modified properties inconfig.sh
are reflected. Pay particular attention to the ZK connection strings if your existing Fusion installation is connected to an external ZooKeeper cluster. -
(On all nodes) If you are running an external Solr or Zookeeper, then modify the
group.default
property infusion.properties
to reflect the services that should be started when thebin/fusion
script is executed. -
(On all nodes) Start the ZooKeeper service that the new Fusion installation will use. If you are using the ZooKeeper bundled within Fusion, that would be:
cd $FUSION_HOME ./bin/zookeeper start
-
(On all nodes) Copy your ZooKeeper configuration file over from the old installation to the new one:
cp $FUSION_OLD_HOME\conf\zookeeper\zoo.cfg $FUSION_NEW_HOME\conf\zookeeper\zoo.cfg
-
(For multi-node Fusion, only run this step on the first node) Upgrade ZooKeeper:
cd fusion-upgrade-scripts-internal/src python upgrade-to-3.0.py --upgrade zk
-
(On all nodes) Run Fusion 3.0.0 (start all services defined in
fusion.properties
):cd $FUSION_HOME ./bin/fusion start
Ensure that all services have started on all nodes before proceeding.
-
(For multi-node Fusion, only run this step on the first node) Upgrade your custom dashboards, which are stored in the Solr collection
system_banana
(after the upgrade, they are stored in the Solr collectionsystem_blobs
):cd fusion-upgrade-scripts-internal/src python upgrade-to-3.0.py --upgrade banana
-
(On all nodes) Use the Fusion UI to validate the deployment.
TipBefore opening the Fusion UI, clear your browser’s cache. Otherwise, you might inadvertently access a cached version of the old Fusion UI and see inconsistent behavior.
Windows
The steps in this section describe how to upgrade from Fusion 2.1.4 or 2.4.x to Fusion 3.0 on Windows.
Ensure that these prerequisites are met before upgrading Fusion.
Note
|
If you have multi-node Fusion, perform these steps on all nodes. |
-
Install python 2.7.13 from here: https://www.python.org/ftp/python/2.7.13/python-2.7.13.amd64.msi
-
Open a normal Command Prompt window and set a
PYTHON_HOME
environment variable to the Python installation directory you just created:set PYTHON_HOME=C:\path\to\python\directory
Example:
set PYTHON_HOME=C:\Python27
Keep this command prompt window open.
-
Install the
kazoo
andrequests
python packages"%PYTHON_HOME%\Scripts\pip" install kazoo "%PYTHON_HOME%\Scripts\pip" install requests
These steps describe how to upgrade from Fusion 2.1.4 or 2.4.x to Fusion 3.0.0 on Windows.
Note
|
If you have a multi-node installation and unless otherwise stated, you must perform each step on each node as you go. For example, if you have 3 nodes, perform step 1 on node 1, node 2, and then node 3. Next, perform step 2 on node 1, node 2, and then node 3. Run steps that are only needed on a single node on your first node; not on the other nodes. |
-
(On all nodes) Create a
fusion-new
folder somewhere on your computer, and download and extract Fusion 3.0.0 to it. -
(On all nodes) Set the
FUSION_HOME
environment variable to the full path of thefusion\3.0.0
directory:Example:
SET FUSION_HOME=C:\lucidworks\fusion-3.0.0\fusion\3.0.0
At this point,
%FUSION_HOME%\bin\fusion.cmd
should point to the Fusion 3.0.0fusion.cmd
file. -
(On all nodes) Set the
FUSION_OLD_HOME
environment variable to the full path of the old Fusion home directory; for example:SET FUSION_OLD_HOME=C:\lucidworks\fusion
At this point,
%FUSION_OLD_HOME%\bin\fusion.cmd
should point to the Fusion 2.x.yfusion.cmd
file. -
(On all nodes) Stop the old Fusion services:
cd "%FUSION_OLD_HOME%" "%FUSION_OLD_HOME%\bin\fusion" stop
-
(On all nodes) In Windows Explorer, copy the
%FUSION_OLD_HOME%\data
directory on top of the%FUSION_HOME%\data
directory; respond with Overwrite All when prompted. -
(On all nodes) Edit
%FUSION_HOME%\conf\fusion.properties
and apply all custom JVM settings that you had applied to your%FUSION_OLD_HOME%\conf\config.cmd
and%FUSION_OLD_HOME%\bin\*.cmd
files into the newjvmOptions
properties. If you have multi-node Fusion, make sure you uncommentdefault.zk.connect
and set it to what you haveFUSION_ZK
set to.For example, many customers customize the max heap size for
%FUSION_OLD_HOME%\bin\connectors.cmd
. Here is an example for someone who changed it to use a max heap size of 4 GB:set "JAVA_OPTIONS=-Djava.net.preferIPv4Stack=true -Xmx4g -Dapple.awt.UIElement=true"
If this were the case, you would need to use this entry in
%FUSION_HOME%\conf\fusion.properties
for theconnectors.jvmOptions
variable:connectors.jvmOptions=-Xmx4g -Xss256k -Dcom.lucidworks.connectors.pipelines.embedded=false
-
(On all nodes) Copy your ZooKeeper configuration file over from the old installation to the new one, that is, copy
%FUSION_OLD_HOME%\conf\zookeeper\zoo.cfg
to%FUSION_NEW_HOME%\conf\zookeeper\zoo.cfg
. -
(On all nodes) Start the new ZooKeeper service:
"%FUSION_HOME%\bin\zookeeper" start
Ensure that ZooKeeper services have started on all nodes before proceeding.
-
(For multi-node Fusion, only run this step on the first node) Download the
fusion-upgrade-scripts
project from https://github.com/lucidworks/fusion-upgrade-scripts. Click Download > Download Zip.Extract this zip file somewhere on your local computer, for example, to
C:\lucidworks
. When the zip file is extracted, you will have the directoryC:\lucidworks\fusion-upgrade-scripts-master
. -
(For multi-node Fusion, only run this step on the first node) Run the ZooKeeper upgrade script:
cd "C:\lucidworks\fusion-upgrade-scripts-master\src" "%PYTHON_HOME%\python" upgrade-to-3.0.py --upgrade zk
Example output:
C:\lucidworks\fusion-upgrade-scripts-master\src>"%PYTHON_HOME%\python" upgrade-to-3.0.py --upgrade zk 2017-10-04 08:45:25,517 - root - load_fusion_3x_config.py:36 - INFO - Creating config file using agent 2017-10-04 08:45:27,563 - root - load_fusion_3x_config.py:45 - INFO - Generated config file at path 'ui.config.json' 2017-10-04 08:45:27,563 - root - zookeeper_client.py:12 - INFO - Starting zookeeper client 2017-10-04 08:45:27,565 - kazoo.client - connection.py:599 - INFO - Connecting to 127.0.0.1:9983 2017-10-04 08:45:27,596 - kazoo.client - client.py:465 - INFO - Zookeeper connection established, state: CONNECTED 2017-10-04 08:45:27,612 - root - upgrade-to-3.0.py:64 - INFO - Migrating from fusion version '2.4.5' to '3.0.0' 2017-10-04 08:45:27,612 - root - upgrade-to-3.0.py:67 - INFO - Copying znodes from old fusion paths to new paths 2017-10-04 08:45:27,612 - root - znodes_migration.py:15 - INFO - Migrating Solr data to new ZK namespace /lwfusion/3.0.0/solr 2017-10-04 08:45:47,562 - root - znodes_migration.py:21 - INFO - Migrating api data to new ZK namespace lwfusion/3.0.0/core 2017-10-04 08:45:58,421 - root - znodes_migration.py:28 - INFO - Migrating proxy data to new ZK namespace lwfusion/3.0.0/proxy 2017-10-04 08:45:59,338 - root - upgrade-to-3.0.py:69 - INFO - Migration from old znode paths to new paths complete 2017-10-04 08:45:59,352 - root - api_pojo_migrator.py:16 - INFO - Updating search-cluster payload at path 'lwfusion/3.0.0/core/search-clusters/default' 2017-10-04 08:45:59,405 - root - proxy_pojo_migrator.py:15 - INFO - Updating init-meta payload at path 'lwfusion/3.0.0/proxy/sys/init-meta' 2017-10-04 08:45:59,490 - root - resource_manager.py:17 - INFO - Loading file from path C:\lucidworks\fusion-upgrade-scripts-master\src\utils\../resources/migrators.json 2017-10-04 08:45:59,490 - root - connectors_migrator.py:126 - INFO - Trying to migrate datasource: test2352352, type: jdbc 2017-10-04 08:45:59,490 - root - upgrade-to-3.0.py:78 - INFO - Performing splitter migrator 2017-10-04 08:45:59,523 - kazoo.client - connection.py:566 - INFO - Closing connection to 127.0.0.1:9983 2017-10-04 08:45:59,523 - kazoo.client - client.py:469 - INFO - Zookeeper session lost, state: CLOSED
-
(On all nodes) Run Fusion 3.0.0 (start all services defined in
fusion.properties
), and wait for all services to start:"%FUSION_HOME%\bin\fusion" start
-
(For multi-node Fusion, only run this step on the first node) Upgrade your custom dashboards, which are stored in the Solr collection
system_banana
(after the upgrade, they are stored in the Solr collectionsystem_blobs
):cd "C:\lucidworks\fusion-upgrade-scripts-master\src" "%PYTHON_HOME%\python" upgrade-to-3.0.py --upgrade banana
If you get an error containing
Unknown collection system_banana
, you can safely ignore it. That just means that you had no dashboards to migrate in the first place. -
(On all nodes) Use the Fusion UI to validate the deployment.
TipBefore opening the Fusion UI, clear your browser’s cache. Otherwise, you might inadvertently access a cached version of the old Fusion UI and see inconsistent behavior.