> ## Documentation Index > Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt > Use this file to discover all available pages before exploring further. # Fusion Server 4.2.2 Release Notes [localhost link]: http://localhost:3000/docs/4/fusion-server/release-notes/4.2.2-release-notes [mintlify link]: https://doc.lucidworks.com/docs/4/fusion-server/release-notes/4.2.2-release-notes [old doc.lw link]: https://doc.lucidworks.com/fusion-server/4.2/91 **Release date:** 17 May 2019 **Component versions:** | | | | | | | -------- | ---------------- | ----------- | ---------------------- | ------------ | | Solr 7.5 | ZooKeeper 3.4.13 | Spark 2.3.2 | Jetty 9.4.12.v20180830 | Ignite 2.6.0 | **More information about support dates can be found at [Lucidworks Fusion Product Lifecycle](/docs/policies/lifecycle-policies/lw-fusion-product-lifecycle).** ## New features No new features were introduced in this release. ## Improvements * [Web V1 connector](/docs/fusion-connectors/connectors/v1/web) improvements: * A new `f.allowCircularRedirects`/**Allow circular redirects** configuration key allows a request to be redirected to the same URL multiple times. * Fixed an issue that prevented crawling sitemaps that include `` tags. * [MongoDB connector](/docs/fusion-connectors/connectors/v1/mongodb) improvements: * The connector now supports SSL * The Java driver was updated to eliminate a driver-related error when crawling. * In query pipelines, security trimming can now be configured with multiple [security trimming stages](/docs/4/fusion-server/reference/pipeline-stages/query/security-trimming-query-stage). * The [Box connector](/docs/fusion-connectors/connectors/v1/box.com) now includes an option to keep security cache local to a connector node. * It is no longer necessary to repeat the entire **Upgrade to Fusion 4.x** to recover from an error. During upgrades to later versions of Fusion Server, the migrator script now permits recovery from errors by correcting the error, repeating the step that failed, and continuing with the upgrade procedure. When you have a Fusion-based search application running, at some point it might be necessary to upgrade to a later version of Fusion. We provide a migrator tool to simplify the upgrade process. See the [release history](/docs/4/fusion-server/release-notes/4.2.0-release-notes) to find out what is new, including which versions of Solr, Spark, and ZooKeeper are bundled with each Fusion release. The migrator transfers over *most* of the objects that make up your search application, all configurations and customizations for your application, and all data in collections in the application. In some cases, manual steps are required for objects that the migrator cannot handle automatically. We give you instructions and guidance about what might be required. You should also review the log of the upgrade in `/opt/fusion/x.y.z/var/upgrade/tmp/migrator.log` (on Unix) or `C:\lucidworks\var\fusion\x.y.z\upgrade\tmp\migrator.log` (on Windows). The x.y.z directory is for the Fusion version that you are migrating *from*. ## Key points Following are some key points about upgrading Fusion: * **Migration involves down time.** The upgrade process involves multiple starts and stops of Fusion services. Please plan accordingly, especially in terms of disabling external load balancers or monitors that might react adversely to the starts and stops. * **Current deployment is preserved.** Upgrades preserve the current Fusion deployment, copying information over from the current deployment to the new one. This provides a rapid roll-back option if you encounter problems during the upgrade process. * **If the upgrade fails.** If an upgrade fails, there is a procedure for dealing with that. ## Supported upgrade sequences Only specific version-to-version upgrade sequences are supported. Some upgrades require multiple steps. These upgrade sequences are supported. ### Upgrades to the current version * **3.1.x to 4.2.y.** From any 3.1.x version to 4.2.6 SP1 (one step, using the migrator) * **4.0.x to 4.2.y.** From any 4.0.x version to 4.2.6 SP1 (one step, using the migrator) * **4.1.x to 4.2.y.** From any 4.1.x version to 4.2.6 SP1 (one step, using the migrator) For links to these procedures, see [Per-version instruction sets](#per-version-instruction-sets). ### Upgrades to prior versions Using the migrator: * **3.1.x to 4.0.y.** From 3.1.5 directly to 4.0.2 (one step) For more information, see Upgrade Fusion 3.1.x to 4.0.y. * **4.0.x to 4.0.y.** From 4.0.0 or 4.0.1 to 4.0.2 (one step) For more information, see Upgrade Fusion Server 4.0.x to 4.0.y. * **3.1.x to 4.1.y.** From any 3.1.x version to 4.1.3 (one step, using the migrator) For more information, see Upgrade Fusion Server 3.1.x to 4.1.y. * **4.0.x to 4.1.y.** From 4.0.2 to 4.1.3 (one step, using the migrator) For more information, see Upgrade Fusion Server 4.0.x to 4.1.y. * **4.1.x to 4.1.y.** From 4.1.0 to 4.1.3 (one step, using the migrator) For more information, see Upgrade Fusion Server 4.1.x to 4.1.y. ### Example For example, to upgrade from Fusion 3.0.1 to Fusion Server 4.2.5, you would perform the following upgrades (both of them using the migrator): 1. Upgrade from Fusion 3.0.1 to Fusion 3.1.5 2. Upgrade from Fusion 3.1.5 to Fusion Server 4.2.5 ## Per-version instruction sets To upgrade to a later version of Fusion from an existing installation requires transferring over all configurations and data from your existing Fusion installation to the new version. **How to upgrade from Fusion 3.1.x to Fusion Server 4.2.y** Perform the steps in this article: **Upgrade from Fusion Server 3.1.x to 4.2.y** - Run a migrator to upgrade from Fusion Server 3.1.x to 4.2.y. **How to upgrade from Fusion 4.0.x to Fusion Server 4.2.y** Perform the steps in this article: **Upgrade from Fusion Server 4.0.x to 4.2.y** - Run a migrator to upgrade from Fusion Server 4.0.x to 4.2.y. **How to upgrade from Fusion 4.1.x to Fusion Server 4.2.y** Perform the steps in this article: **Upgrade from Fusion Server 4.1.x to 4.2.y** - Run a migrator to upgrade from Fusion Server 4.1.x to 4.2.y. **How to upgrade from Fusion 4.2.x to Fusion Server 4.2.y** Perform the steps in this article: **Upgrade from Fusion Server 4.2.x to 4.2.y** - Run a migrator to upgrade from Fusion Server 4.2.x to 4.2.y. * [Jobs API](/docs/4/fusion-server/reference/api/jobs-api) performance is improved when working with large numbers of datasources. * [Parallel Bulk Loader (PBL) jobs](/docs/4/fusion-ai/reference/jobs/parallel-bulk-loader) (Fusion AI only) and [Script jobs](/docs/4/fusion-server/reference/jobs/script) can now be configured to set environment variables. In the Fusion UI, click **Advanced** to see this option. * Fusion now supports running multiple Parallel Bulk Loader (PBL), Script, or Custom jobs in parallel.\ This feature can be enabled via the Configurations API, as follows: ```bash wrap theme={"dark"} curl -X POST -H "Content-type: application/json" -d "2" http://localhost:8765/api/v1/configurations/fusion.spark.max.launcher.jobs ``` This is an experimental feature. Note that running multiple jobs in parallel is memory-intensive. * Performance is improved in v2 and custom SDK-based connectors. ## Other changes * In the [SQL service](/docs/4/fusion-server/concepts/sql/overview), basic select SQL queries on `{collection}_signals`, `{collection}_query_rewrite`, or `{collection}_query_rewrite_staging` no longer fail when an ORDER BY clause is not included. * The `install.sh` script for `systemd` no longer starts services after installing them. Services must be started manually. * Fixed a document sampling issue which prevented sample documents from appearing in the Index Workbench when previewing a pipeline that used a v2 connector or custom SDK-based connector. * Fixed an issue that allowed users to delete pipelines without the correct permissions. * Fixed an issue that affected the accuracy of the start times displayed in the [Scheduler](/docs/4/fusion-server/concepts/jobs/overview). * Fixed some display issues in the [Log Viewer](/docs/4/fusion-server/concepts/system/devops-center). * Fixed the ownership settings on files in the Fusion tarball so that both the UID and GID are 0 when the package is unpacked by a user with root privileges. * Fixed a bug that caused an infinite loop after logging out of Fusion on Chrome. * Fixed a bug that prevented the RPC service from starting after connectors were deleted from the blob store. * [Query Workbench](/docs/4/fusion-server/concepts/querying/query-workbench) bug fixes: * Fixed an issue that changed the display in both panels instead of only the working panel when using Compare mode. * Fixed an issue that prevented some document fields from appearing in the dropdown list of fields for faceting. * Fixed a bug that caused a "Search Error Unknown Pipeline" error after logging out and logging in again. * A few display issues were fixed. * [Confluence connector](/docs/fusion-connectors/connectors/v1/confluence) bug fixes: * Added parent permission ACL values. * Fixed a bug that caused a null pointer exception when `f.includePrivateContent`/**Include private content** and `f.enableSecurityTrimming`/**Enable Security Trimming** were both set to "false". * Fixed an issue in the [Local Filesystem connector](/docs/fusion-connectors/connectors/local-filesystem-v2) which cause crawls to continue after the configured **Maximum Output Limit**/`maximumItemLimitConfig` had been reached. ## Addressed Security Vulnerabilities * [Apache Commons Email](https://commons.apache.org/proper/commons-email/) was upgraded to version 1.5.0 to resolve a [security vulnerability](https://www.cvedetails.com/cve/CVE-2018-1294/). ## Known issues * When under load, the Fusion proxy service can occasionally become stuck, causing user authentication to fail. This is the result of the proxy `InputStream` failing to close properly.\ An upgrade to Fusion 4.2.4 is required to fix this issue. See **Upgrade Fusion Server 4.2.x to 4.2.y**. ## Introduction This article describes how to perform the following upgrade: * From version: Fusion `4.2.x` * To version: Fusion `4.2.y` Only specific version-to-version upgrade sequences are supported. Some upgrades require multiple steps. For more information, see Fusion 4.x. For Fusion 3.1 and later releases, a migrator is available for upgrading Fusion. During the upgrade process, the migrator uses a properties file. After downloading and installing the migrator, the properties files is in the `/opt/lucidworks/fusion/4.2.x/var/upgrade` directory (on Unix or MacOS) or the `C:\lucidworks\fusion\4.2.*x\var\upgrade\` directory (on Windows). The file names reference the versions you are upgrading from and to. For example: * To upgrade `4.2.3` to `4.2.5`, the migrator uses the `4.2.x-4.2.x.properties` file. Migration entails down time and multiple starts and stops of Fusion services. Plan accordingly, especially in terms of disabling external load balancers or monitors that might react adversely to the starts and stops. Download the latest migrator immediately before upgrading. This helps ensure that the upgrade goes smoothly. The newer Fusion instance must be newly untarred and *never started*. ## About the upgrade This section describes how connectors, object migrations, and signals are migrated during an upgrade. ### Connectors In Fusion 3.1.0 and later, only a subset of [connectors](/docs/fusion-connectors/connectors/overview) are included by default. The migrator detects which connectors were used in the older version of Fusion, and installs them automatically in Fusion `4.0.y`. This step requires an Internet connection. If no connection is available, then download the connectors from [Fusion 4.x Connector Downloads](/docs/fusion-connectors/downloads/fusion-4-x-connector-downloads) and install them as bootstrap plugins. If a connector to be upgraded was not available during the upgrade, then a message in `/opt/lucidworks/fusion/3.1.x/var/upgrade/tmp/migrator.log` (on Unix) or `C:\lucidworks\fusion\3.1.*x\var\upgrade\tmp\migrator.log` (on Windows) indicates this. Only datasources for connectors that are supported in the new Fusion version are upgraded. Datasources for custom connectors are not upgraded. #### If no Internet connection is available If no Internet connection is available during an upgrade, the migrator cannot automatically download the connectors it needs. Using the Fusion UI or API later to install the connectors also might not be an option. In this case, download the connectors from [Fusion 4.x Connector Downloads](/docs/fusion-connectors/downloads/fusion-4-x-connector-downloads) for all existing connectors and place them in `apps/connectors/bootstrap-plugins` for the new deployment (on all Fusion nodes). Do so at the time indicated in the procedures that follow. #### Adding connectors during an upgrade You can *add* connectors during an upgrade (that is, add connectors that are not in the old deployment). Download the connectors from [Fusion 4.x Connector Downloads](/docs/fusion-connectors/downloads/fusion-4-x-connector-downloads) and place them in `apps/connectors/bootstrap-plugins` for the new version (on all Fusion nodes). ### Object migrations and transformations The migrator automatically migrates these Fusion 4.2 object types, transforming them as needed: * Collections * Index pipelines * Query pipelines * Search cluster configurations * Datasources * Parsing configurations * Object groups * Links * Tasks * Jobs * Spark objects * Apps * Appkit apps * Index profiles * Query profiles * Blobs In Fusion Server 4.0 and later, most objects exist in the context of apps. When you upgrade from Fusion Server 4.2.x to 4.2.y, the migrator upgrades app objects, all objects in or linked to objects in apps, and objects that are not linked to apps. You can explore the objects in [Object Explorer](/docs/4/fusion-server/concepts/object-explorer). ### Access control migration The migrator upgrades all access control configurations: * Security realms * Roles * Users None of these should require adjustments after migration. ## Review known issues Before upgrading, review the known issues to see whether any of them apply to the circumstances of your upgrade. Some known issues might require actions *before* upgrading. That article also contains instructions regarding what to do if an upgrade step fails. ## Upgrade on Unix Use this procedure to upgrade Fusion on a single Unix node or on multiple Unix nodes. Perform the steps in this procedure on the indicated nodes *on which Fusion is running* ("Fusion nodes"). To perform an upgrade, Fusion nodes must have at least these services running: * API service (`api`) * Proxy service (`proxy`) For every step on multiple nodes, ensure that the step completes *on all Fusion nodes* before going to the next step. There is the notion of a "main node" during the migration process. This node will be used for certain centralized migration activities that do not need to be done on every node, such as downloading connectors that are then uploaded to blob storage that is shared by all, etc. Just pick one of your Fusion nodes to be the "main node"; there is no special requirement as to which one you pick. ### Ensure that your current version of Fusion has a valid license Ensure that your *current* version of Fusion has a valid permanent Fusion license before proceeding with the upgrade. Place a valid `license.properties` file in the `/opt/lucidworks/fusion/4.2.x/conf` directory. ### Download and install the newer version of Fusion **Perform these tasks *on all Fusion nodes*:** 1. Select the Fusion release to which you are upgrading from [Fusion Server 4.x File Download Links](/docs/4/fusion-server/reference/fusion-server-4-x-file-download-links). 2. Extract the newer version of Fusion: ```bash theme={"dark"} cd /opt/lucidworks mv ~/Downloads/fusion-4.2.y.tar.gz ./ tar -xf fusion-4.2.y.tar.gz ``` For example, if Fusion is currently installed in `/opt/lucidworks/fusion/4.2.x`, then change your working directory to `/opt/lucidworks/` and extract the file there. *do not run the new version of Fusion yet.* 3. Ensure that the *new* version of Fusion has a valid permanent Fusion license before proceeding with the upgrade. Place a valid `license.properties` file in the `/opt/lucidworks/fusion/4.2.y/conf` directory. 4. (*If there are custom* `jar` *files*) If your deployment has custom `jar` files, add them to the new Fusion deployment. 5. (*If you are performing an upgrade without Internet access*) Without Internet access, the migrator cannot download new versions of connectors automatically. Download the *new versions* of connector zip files *for your current connectors* from [Fusion 4.x Connector Downloads](/docs/fusion-connectors/downloads/fusion-4-x-connector-downloads) and place them in `apps/connectors/bootstrap-plugins` for the new deployment. 6. (*If you are adding* new *connectors*) If you want your new deployment to use connectors that *are not* in the current deployment, you can add them now. Download the connector zip files from [Fusion 4.x Connector Downloads](/docs/fusion-connectors/downloads/fusion-4-x-connector-downloads) and place them in `apps/connectors/bootstrap-plugins` for the new deployment. 7. Verify that there is sufficient disk space for a second copy of the Solr index directory, `fusion/4.2.x/data/solr`. If there is not sufficient disk space, free up space before proceeding. ### Download and install the Fusion migrator **Perform these tasks *on all Fusion nodes*:** 1. [Download the latest migrator zip file for Unix](https://download.lucidworks.com/fusion-migrator.tar.gz). (Do this now, even if you have downloaded the migrator before, to ensure that you have the latest version.) 2. Create `FUSION_OLD` and `FUSION_NEW` environment variables that point to the old and new Fusion installation directories respectively (using the full path). ```bash theme={"dark"} export FUSION_OLD="/opt/lucidworks/fusion/4.2.x" export FUSION_NEW="/opt/lucidworks/fusion/4.2.y" ``` For example, when upgrading from Fusion 4.2.0 to 4.2.6: ```bash theme={"dark"} export FUSION_OLD="/opt/lucidworks/fusion/4.2.0" export FUSION_NEW="/opt/lucidworks/fusion/4.2.6" ``` 3. Create this directory: ```bash theme={"dark"} mkdir $FUSION_OLD/var/upgrade ``` 4. Install the migrator: ```bash wrap theme={"dark"} tar -zxv -C $FUSION_OLD/var/upgrade --strip-components=1 -f fusion-migrator.tar.gz ``` ### Run the migrator **Perform these tasks on the indicated nodes:** 1. (*On all Fusion nodes*) Start all Fusion services for the old version of Fusion: ```bash theme={"dark"} $FUSION_OLD/bin/fusion start ``` 2. (*Only on the main Fusion node*) Run the migrator to export the configuration data from the old version of Fusion: ```bash theme={"dark"} java -jar $FUSION_OLD/var/upgrade/migrator.jar --export ``` This message indicates that the step finished successfully: ```bash theme={"dark"} Old Fusion configuration export (--export) finished successfully. ``` 3. (*On all Fusion nodes*) Stop the old versions of Fusion services and Solr; but not ZooKeeper: ```bash theme={"dark"} $FUSION_OLD/bin/log-shipper stop $FUSION_OLD/bin/sql stop $FUSION_OLD/bin/admin-ui stop $FUSION_OLD/bin/webapps stop $FUSION_OLD/bin/proxy stop $FUSION_OLD/bin/connectors-rpc stop $FUSION_OLD/bin/connectors-classic stop $FUSION_OLD/bin/api stop $FUSION_OLD/bin/solr stop ``` If Spark services are running, also stop those: ```bash theme={"dark"} $FUSION_OLD/bin/spark-master stop $FUSION_OLD/bin/spark-worker stop ``` You can see what is running with `$FUSION_OLD/bin/fusion status`. 4. (*Only on secondary Fusion nodes*) Prepare secondary nodes: ```bash wrap theme={"dark"} java -jar $FUSION_OLD/var/upgrade/migrator.jar --prepare-secondary ``` This message indicates that the step finished successfully: ```bash wrap theme={"dark"} Prepare secondary nodes (--prepare-secondary) finished successfully. ``` 5. (*On all Fusion nodes*) Stop ZooKeeper for the old version of Fusion (*unless you are using an external ZooKeeper instance, in which case you can ignore this step*): ```bash theme={"dark"} $FUSION_OLD/bin/zookeeper stop ``` 6. (*Only on the main Fusion node*) Transform configuration data on the main Fusion node: ```bash theme={"dark"} java -jar $FUSION_OLD/var/upgrade/migrator.jar --main-transform ``` Depending on the size of your Solr index, this step can take a long time (for example, multiple tens of minutes). This message indicates that the step finished successfully: ```bash wrap theme={"dark"} Fusion data transformations on main node (--main-transform) finished successfully. ``` 7. (*On all Fusion nodes*) Start ZooKeeper for the new version of Fusion (*unless you are using an external ZooKeeper instance, in which case you can ignore this step*): ```bash theme={"dark"} $FUSION_NEW/bin/zookeeper start ``` 8. (*Only on the main Fusion node*) Import the first part of configuration data into the new version of Fusion: ```bash theme={"dark"} java -jar $FUSION_OLD/var/upgrade/migrator.jar --zookeeper-import ``` This message indicates that the step finished successfully: ```bash wrap theme={"dark"} New Fusion Zookeeper import (--zookeeper-import) finished successfully. ``` 9. (*On all Fusion nodes*) Start all Fusion services for the new version of Fusion: ```bash theme={"dark"} $FUSION_NEW/bin/fusion start ``` 10. (*Only on the main Fusion node*) Import the second part of configuration data into the new version of Fusion: ```bash theme={"dark"} java -jar $FUSION_OLD/var/upgrade/migrator.jar --fusion-import ``` This message indicates that the step finished successfully: ```bash theme={"dark"} New Fusion object import (--fusion-import) finished successfully. ``` After migration, you can find details about the results in the `fusion/4.2.x/var/upgrade/tmp` directory. If the migration produces unexpected results, the files in this directory are helpful for troubleshooting. ### Validate the new version of Fusion **How to validate the new version of Fusion** 1. (*Only on the main Fusion node*) Restart the new version of Fusion (all services defined in `fusion.properties`): ```bash theme={"dark"} $FUSION_NEW/bin/fusion restart ``` 2. Log into the Fusion UI (your `admin` password is the same as for the old installation), and confirm the release number of the new version of Fusion: 1. Clear your browser’s cache.\ Otherwise, you might inadvertently access a cached version of the old Fusion UI and see inconsistent behavior. 2. In a browser, open the Fusion UI at `http://localhost:8764/` (replace `localhost` with your server name or IP address if necessary). 3. Log in. 4. Navigate to **Admin** > **About Fusion**.\ The **About Fusion** panel should display the newer Fusion release number. 3. Ensure that all connectors were installed automatically during the upgrade. * For Fusion 4.x from the Fusion launcher, click the tile for a migrated app. Click **System** > **Blobs**. If any connectors are missing from the list, click **Add** > **Connector Plugin** and install them manually. * For Fusion 3.x from the Fusion launcher, click **Devops** > **Home** Home > **Blobs**. If any connectors are missing from the list, click **Add** > **Connector Plugin** and install them manually. 4. Ensure that all customizations you made in the former version of Fusion are present in the new one. 5. When you are satisfied with the migration and you have backed up the `fusion/4.2.x/` directory, you can `rm -fr fusion/4.2.*x/` to remove the older version of Fusion (*on all Fusion nodes*). ### Add support for business rules to existing apps Fusion AI 4.2 introduces functionality for using business rules, that is, manually created formulas for rewriting queries and responses. You can add support for business rules to apps that were created in versions of Fusion AI prior to version 4.2. To do so, perform the steps in this section. The `add-rule-objects-xyz.zip` file (where `xyz` is a version number) specifies the objects to add to an app. It is supplied in the Fusion migrator zip file at the top level. After installing the migrator, the location is `$FUSION_OLD/var/upgrade/import-files`. Adding support for business rules has a costs. Additional collections and objects are created. Only add support for business rules to apps in which you plan to use them. You have a choice. You can update each app using the Fusion UI or the Fusion API. #### Fusion UI For each app in which you plan to use business rules, import the objects in the `add-rule-objects-xyz.zip` file into the app. **How to import business rule objects** 1. In the Fusion launcher, click the app into which you want to import objects.\ The Fusion workspace appears. 2. In the upper left, click **System** System > **Import Fusion Objects**.\ The Import Fusion Objects window opens. 3. For the data file, select `add-rule-objects-xyz.zip` from your local filesystem. The location in the extracted migrator files is `$FUSION_OLD/var/upgrade/import-files`. 4. Click **Import**. 5. Edit the `Application ID` parameter value to use the app name. If the app name contains spaces, replace those with underscore characters. For example, `Lucene Revolution` would become `Lucene_Revolution`. 6. Click **Import**. 7. If there are conflicts, Fusion prompts you to specify an import policy. Click **Merge** to skip all conflicting objects and import only the non-conflicting objects.\ Fusion confirms that the import was successful. 8. Click **Close** to close the Import Fusion Objects window. #### Fusion API For each app in which you plan to use business rules, import the objects in the `add-rule-objects-xyz.zip` file into the app. **How to import business rule objects** 1. Create an `app-name.txt` file with the following content: ```json theme={"dark"} {"app.id" : "_name-of-migrated-app_"} ``` For example, for the app `Lucene Revolution`: ```json theme={"dark"} {"app.id" : "Lucene_Revolution"} ``` Here, we assume that you create the files in your home directory, for which the `$HOME` environment variable is defined. 2. Import the business rule objects: ```bash wrap theme={"dark"} curl -u USERNAME:PASSWORD -H "Content-Type:multipart/form-data" -X POST -F 'variableValues=path-to-txt-file/app-name.txt' -F "importData=path-to-zip-file/add-rule-objects-xyz.zip" "http:///api/objects/import?importPolicy=merge" ``` For example: ``` curl -u admin:Very-SecurePW8 -H "Content-Type:multipart/form-data" -X POST -F 'variableValues=$HOME/Lucene_Revolution.txt' -F "importData=$FUSION_OLD/var/upgrade/import-files/add-rule-objects-001.zip" "http:///api/objects/import?importPolicy=merge" ``` ## Upgrade on Windows Use this procedure to upgrade Fusion on a single Windows node or multiple Windows nodes. Perform the steps in this procedure on the indicated nodes *on which Fusion is running* ("Fusion nodes"). To perform an upgrade, Fusion nodes must have at least these services running: * API service (`api`) * Proxy service (`proxy`) If you are upgrading Fusion on multiple nodes, then, for every step on multiple nodes, ensure that the step completes *on all Fusion nodes* before going to the next step. There is the notion of a "main node" during the migration process. This node will be used for certain centralized migration activities that do not need to be done on every node, such as downloading connectors that are then uploaded to blob storage that is shared by all, etc. Just pick one of your Fusion nodes to be the "main node"; there is no special requirement as to which one you pick. ### Ensure that your current version of Fusion has a valid license Ensure that your current version of Fusion has a valid permanent Fusion license before proceeding with the upgrade. Place a valid `license.properties` file in the `C:\lucidworks\fusion\4.2.x\conf` directory. ### Download and install the newer version of Fusion Perform these tasks *on all Fusion nodes*: 1. Select the Fusion release to which you are upgrading from [Fusion Server 4.x File Download Links](/docs/4/fusion-server/reference/fusion-server-4-x-file-download-links). 2. Move the `fusion-4.2.y.zip` file to the directory that contains the `fusion\` directory.\ For example, if Fusion is installed in `C:\lucidworks\fusion\4.2.x`, then move the file to `C:\lucidworks`. 3. Unzip the `fusion-4.2.y.zip` file. *do not run the new version of Fusion yet.* 4. For Fusion 4.x, ensure that the *new* version of Fusion has a valid permanent Fusion license before proceeding with the upgrade. Place a valid `license.properties` file in the `C:\lucidworks\fusion\4.2.y\conf` directory. 5. (*If there are custom* `jar` *files*) If your deployment has custom `jar` files, add them to the new Fusion deployment. 6. (*If you are performing an upgrade without Internet access*) Without Internet access, the migrator cannot download new versions of connectors automatically. Download the *new versions* of connector zip files *for your current connectors* from [Fusion 4.x Connector Downloads](/docs/fusion-connectors/downloads/fusion-4-x-connector-downloads) and place them in `apps\connectors\bootstrap-plugins` for the new deployment. 7. (*If you are adding* new *connectors*) If you want your new deployment to use connectors that *are not* in the current deployment, you can add them now. Download the connector zip files from [Fusion 4.x Connector Downloads](/docs/fusion-connectors/downloads/fusion-4-x-connector-downloads) and place them in `apps\connectors\bootstrap-plugins` for the new deployment. 8. Verify that there is sufficient disk space for a second copy of the Solr index directory, `fusion\4.2.x\data\solr`. If there is not sufficient disk space, free up space before proceeding. ### Download and install the Fusion migrator **Perform these tasks *on all Fusion nodes*:** 1. [Download the latest migrator zip file for Windows](https://download.lucidworks.com/fusion-migrator.zip). (Do this now, even if you have downloaded the migrator before, to ensure that you have the latest version.) 2. Open a Command Prompt window and create `FUSION_OLD` and `FUSION_NEW` environment variables that point to the old and new Fusion installation directories respectively. For example: ```bash theme={"dark"} set FUSION_OLD=C:\lucidworks\fusion\4.2.0 set FUSION_NEW=C:\lucidworks\fusion\4.2.6 ``` 3. Create a `fusion\4.2.x\var\upgrade` directory. 4. Unzip the migrator zip file, and move the contents of the extracted folder to `+fusion\+4.2.x\var\upgrade`. ### Run the migrator **Perform these tasks on the indicated nodes:** 1. (*On all Fusion nodes*) Start all Fusion services for the old version of Fusion: ```bash theme={"dark"} %FUSION_OLD%\bin\fusion.cmd start ``` 2. (*Only on the main Fusion node*) Run the migrator to export the configuration data from the old version of Fusion: ```bash theme={"dark"} java -jar "%FUSION_OLD%\var\upgrade\migrator.jar" --export ``` This message indicates that the step finished successfully: ```bash theme={"dark"} Old Fusion configuration export (--export) finished successfully. ``` 3. (*On all Fusion nodes*) Stop the old versions of Fusion services and Solr; but not ZooKeeper: ```bash theme={"dark"} %FUSION_OLD%\bin\log-shipper.cmd stop %FUSION_OLD%\bin\sql.cmd stop %FUSION_OLD%\bin\admin-ui.cmd stop %FUSION_OLD%\bin\webapps.cmd stop %FUSION_OLD%\bin\proxy.cmd stop %FUSION_OLD%\bin\connectors-rpc.cmd stop %FUSION_OLD%\bin\connectors-classic.cmd stop %FUSION_OLD%\bin\api.cmd stop %FUSION_OLD%\bin\solr.cmd stop ``` If Spark and SQL services are running, also stop those: ```bash theme={"dark"} %FUSION_OLD%\bin\spark-master.cmd stop %FUSION_OLD%\bin\spark-worker.cmd stop ``` You can see what is running with `%FUSION_OLD%\bin\fusion status`. 4. (*Only on secondary Fusion nodes*) Prepare secondary nodes: ```bash wrap theme={"dark"} java -jar "%FUSION_OLD%\var\upgrade\migrator.jar" --prepare-secondary ``` This message indicates that the step finished successfully: ```bash wrap theme={"dark"} Prepare secondary nodes (--prepare-secondary) finished successfully. ``` 5. (*On all Fusion nodes*) Stop ZooKeeper for the old version of Fusion (*unless you are using an external ZooKeeper instance, in which case you can ignore this step*): ```bash theme={"dark"} %FUSION_OLD%\bin\zookeeper.cmd stop ``` 6. (*Only on the main Fusion node*) Transform configuration data on the main Fusion node: ```bash wrap theme={"dark"} java -jar "%FUSION_OLD%\var\upgrade\migrator.jar" --main-transform ``` Depending on the size of your Solr index, this step can take a long time (for example, multiple tens of minutes). This message indicates that the step finished successfully: ```bash wrap theme={"dark"} Fusion data transformations on main node (--main-transform) finished successfully. ``` 7. (*On all Fusion nodes*) Start ZooKeeper for the new version of Fusion (*unless you are only using an external ZooKeeper instance, in which case you can ignore this step*): ```bash theme={"dark"} %FUSION_NEW%\bin\zookeeper.cmd start ``` 8. (*Only on the main Fusion node*) Import the first part of configuration data into the new version of Fusion: ```bash wrap theme={"dark"} java -jar "%FUSION_OLD%\var\upgrade\migrator.jar" --zookeeper-import ``` This message indicates that the step finished successfully: ```bash wrap theme={"dark"} New Fusion Zookeeper import (--zookeeper-import) finished successfully. ``` 9. (*On all Fusion nodes*) Start Solr for the new Fusion version: ```bash theme={"dark"} %FUSION_NEW%\bin\solr.cmd start ``` 10. (*Only on the main Fusion node*) Run a script to remove all old plugins from the blob store. Replace `solr-address` and `solr-port` as appropriate (as shown in the example): ```bash wrap theme={"dark"} java -cp "%FUSION_OLD%\var\upgrade\jython-standalone-2.7.1.jar;%FUSION_OLD%\var\upgrade\migrator.jar" org.python.util.jython "%FUSION_OLD%\var\upgrade\transformations\manual_delete_old_plugin_blobs.py" --solr-address solr-address --solr-port solr-port ``` For example: ```bash wrap theme={"dark"} java -cp "%FUSION_OLD%\var\upgrade\jython-standalone-2.7.1.jar;%FUSION_OLD%\var\upgrade\migrator.jar" org.python.util.jython "%FUSION_OLD%\var\upgrade\transformations\manual_delete_old_plugin_blobs.py" --solr-address localhost --solr-port 8983 ``` This message indicates that plugins were deleted successfully: ```bash wrap theme={"dark"} Deleted old plugin blobs from solr 0 246 Old connector plugin blobs were deleted successfully. ``` 11. (*On all Fusion nodes*) Start all Fusion services for the new version of Fusion: ```bash theme={"dark"} %FUSION_NEW%\bin\fusion.cmd start ``` 12. (*Only on the main Fusion node*) Import the second part of configuration data into the new version of Fusion: ```bash theme={"dark"} java -jar "%FUSION_OLD%\var\upgrade\migrator.jar" --fusion-import ``` This message indicates that the step finished successfully: ```bash theme={"dark"} New Fusion object import (--fusion-import) finished successfully. ``` After migration, you can find details about the results in the `+fusion\+4.2.x\var\upgrade\tmp` directory. If the migration produces unexpected results, the files in this directory are helpful for troubleshooting. ### Validate the new version of Fusion **How to validate the new version of Fusion** 1. (*On all Fusion nodes*) Restart all Fusion services for the new version of Fusion: ```bash theme={"dark"} %FUSION_NEW%\bin\fusion.cmd restart ``` 2. Log into the Fusion UI (your `admin` password is the same as for the old installation), and confirm the release number of the new version of Fusion: 1. Clear your browser’s cache.\ Otherwise, you might inadvertently access a cached version of the old Fusion UI and see inconsistent behavior. 2. In a browser, open the Fusion UI at `http://localhost:8764/` (replace `localhost` with your server name or IP address if necessary). 3. Log in. 4. Navigate to **Admin** > **About Fusion**.\ The **About Fusion** panel should display the newer Fusion release number. 3. Ensure that all connectors were installed automatically during the upgrade. 1. For Fusion 4.x from the Fusion launcher, click the tile for a migrated app. Click **System** > **Blobs**. If any connectors are missing from the list, click **Add** > **Connector Plugin** and install them manually. 2. For Fusion 3.x from the Fusion launcher, click **Devops** > **Home** Home > **Blobs**. If any connectors are missing from the list, click **Add** > **Connector Plugin** and install them manually. 4. Ensure that all customizations you made in the former version of Fusion are present in the new one. 5. When you are satisfied with the migration and you have backed up the `+fusion\+4.2.x` directory, you can remove the older version of Fusion by removing that directory (*on all Fusion nodes*). ### Add support for business rules to existing apps Fusion AI 4.2 introduces functionality for using business rules, that is, manually created formulas for rewriting queries and responses. You can add support for business rules to apps that were created in versions of Fusion AI prior to version 4.2. To do so, perform the steps in this section. The `add-rule-objects-xyz.zip` file (where `xyz` is a version number) specifies the objects to add to an app. It is supplied in the Fusion migrator zip file at the top level. After installing the migrator, the location is `%FUSION_OLD%\var\upgrade\import-files\`. You have a choice. You can update each app using the Fusion UI or the Fusion API. #### Fusion UI For each app in which you plan to use business rules, import the objects in the `add-rule-objects-xyz.zip` file into the app. **How to import business rule objects** 1. In the Fusion launcher, click the app into which you want to import objects. The Fusion workspace appears. 2. In the upper left, click **System** System > **Import Fusion Objects**. The Import Fusion Objects window opens. 3. For the data file, select `add-rule-objects-xyz.zip` from your local filesystem. The location in the extracted migrator files is `%FUSION_OLD%\var\upgrade\import-files\`. 4. Click **Import**. 5. Edit the `Application ID` parameter value to use the app name. If the app name contains spaces, replace those with underscore characters. For example, `Lucene Revolution` would become `Lucene_Revolution`. 6. Click **Import**. 7. If there are conflicts, Fusion prompts you to specify an import policy. Click **Merge** to skip all conflicting objects and import only the non-conflicting objects. Fusion confirms that the import was successful. 8. Click **Close** to close the Import Fusion Objects window. #### Fusion API For each app in which you plan to use business rules, import the objects in the `add-rule-objects-xyz.zip` file into the app. **How to import business rule objects** 1. Create an `app-name.txt` file with the following content: ```json theme={"dark"} {"app.id" : "_name-of-migrated-app_"} ``` For example, for the app `Lucene Revolution`: ```json theme={"dark"} {"app.id" : "Lucene_Revolution"} ``` Here, we assume that you create the files in your home directory, for which the `%HOMEPATH%` environment variable is defined. 2. Import the business rule objects: ```bash wrap theme={"dark"} curl -u USERNAME:PASSWORD -H "Content-Type:multipart/form-data" -X POST -F 'variableValues=path-to-txt-file\app-name.txt' -F "importData=path-to-zip-file\add-rule-objects-xyz.zip" "http:///api/objects/import?importPolicy=merge" ``` For example: ```bash wrap theme={"dark"} curl -u admin:Very-SecurePW8 -H "Content-Type:multipart/form-data" -X POST -F 'variableValues=%HOMEPATH%\Lucene_Revolution.txt' -F "importData=%FUSION_OLD%\import-files\add-rule-objects-001.zip" "http:///api/objects/import?importPolicy=merge" ``` * Fusion Admin may become sluggish with extended use over a period of several hours. To alleviate this issue, reload the page in the browser while bypassing the cache using the Shift+F5 shortcut. * Omitting the port from the `Origin` and `Host` headers from a cURL request incorrectly triggers the CORS filter, resulting in a `400` bad request error. For example: ``` $ curl -s -o /dev/null -w "%{http_code}\n" -H "Origin: https://example.com:443" -H "Host: example.com:443" http://localhost:8764/api 200 $ curl -s -o /dev/null -w "%{http_code}\n" -H "Origin: https://example.com" -H "Host: example.com" http://localhost:8764/api 400 ``` * If a user has no assigned groups, conducting SharePoint security trimming on that user will result in a `NullPointerException` error. * An `IOException` error is sometimes logged while attempting to fetch documents from SharePoint. * Entering a number value only in the Query Workbench finds stages as a result. No result should be found. * A `Job execution error` is produced while trying to run a Phrase Detection job with an empty output collection. * When creating a new collection within a Fusion AI app, collections previously deleted from other apps may appear in the new app as collections or sub-collections. * After starting and stopping a job in a new Fusion AI app, the job’s `start` time value may be listed as being older than the job’s `stop` time value. Example: ``` start: Wed, April 17, 2019 at 06:15:55 PM +0300 stop: Wed, April 17, 2019 at 06:14:57 PM +0300 ```