Migration guide

​

Prerequisites

• Admin privileges to a Fusion environment running Fusion 5.9.8 or later.
• A valid Predictive Merchandiser license.
• Workspace Owner permissions in a Lucidworks Platform environment.
• Privileges to access and use Lucidworks Commerce Studio in a Lucidworks platform workspace.

​

Migration process

​

Prepare Predictive Merchandiser for migration

1. Download the provided precheck script to check for staged (unpublished) rules and rewrites: Click to download precheck.sh
2. Run the script, replacing the placeholder values with your values.
   Copy

   ./precheck.sh FUSION_USERNAME FUSION_PASSWORD FUSION_BASE_URL FUSION_APP_ID
   

   Placeholder definitions

   If you receive an error stating bash: ./precheck.sh: Permission denied: 1. Make the script executable: chmod +x precheck.sh. 2. Run the script again.
3. Check the output to see if any issues were detected. If there are no issues, the output returns All pre-checks passed. No issues found. Continue to retrieve list of objects in Predictive Merchandiser. If there are issues, the output returns a list of issues like the following:
   Copy

   The following pre-checks failed:
     Found published rules with staged changes. Publish or revert the changes to fix this issue.
     - 2fRtM1GEsU
     - E1YkUTX9r6
     - Eb2A6WARag
     - FNS7Whciwx
     - LlIsMfK4kq
     - OidpIUrVv3
     - P39TDQ2X1K
     - RYbY2XPMOI
     - ZHvW2LiQpe
     - aCseNzc4Xc
     ... and 6 more.
   See precheck_results.json for the full list of invalid content
   

4. Review precheck_results.json and inspect each reported rule. If you created the rules, sign into Predictive Merchandiser and publish or revert the staged changes. If the unpublished rules were created by a different user, you cannot publish or revert the rules in the Predictive Merchandiser UI. There are a few options for resolving unpublished rules owned by another user:
   • Proceed with migration. These rules are skipped and you can re-create them in Commerce Studio.
   • Use the Fusion Query Rewrite and Custom Rule APIs to manually publish or revert the unpublished rules.
   • Use the Fusion Query Rewrite and Custom Rule APIs, or Solr, to manually change the edit session ID of the content in Solr to your current user. Then, manually fix the content in the Predictive Merchandiser UI.
   • Have the user that owns the content in Predictive Merchandiser sign in and either publish or revert the changes.
5. After resolving issues, re-run the script. If there are no issues, continue to retrieve list of objects in Predictive Merchandiser.

​

Retrieve lists of objects in Predictive Merchandiser

1. Export a list of your rules and rewrites:
   Copy

   curl --request GET \
     --url  USER:PASSWORD \
     'https://FUSION_HOST/api/apps/APP_NAME/query-rewrite/instances' \
     --header 'accept: application/json' \
     > exported-rules.json
   

2. Create an inventory of your rules, noting how many of each type and tag appear in the list.
3. Export a list of your zones:
   Copy

   curl --request GET \
   -u USER:PASSWORD \
     --url 'https://FUSION_HOST/api/templating/zones?context=app:APP_NAME' \
     --header 'Accept: application/json' \
     > zones.json
   

4. Export a list of your templates:
   Copy

   curl --request GET \
   -u USER:PASSWORD \
     --url 'https://FUSION_HOST/api/templating/templates?context=app:APP_NAME' \
     --header 'Accept: application/json' \
     > templates.json
   

​

Set up the Commerce Studio integration

1. Copy the ID of the Fusion instance you’re migrating from and save in a secure location.
   
2. Copy the Lucidworks Platform workspace ID and save in a secure location. You can find this in the JWKS and Issuer URLs.
   

​

Create a Commerce Studio instance using the API

1. Obtain a bearer token from your browser session:
   1. Open your browser’s developer console and navigate to the Network tab.
   2. Select Fetch/XHR, then navigate within the Platform UI.
   3. From one of the recorded requests, find the Authorization header and copy the Bearer token value. Save this in a secure place. This token expires after one hour, so repeat this step as needed.
      
2. Using the token copied in the previous step, run the following command in a terminal, replacing the placeholder values with the appropriate values:
   Copy

   curl --location "https://api.lucidworks.com/customers/WORKSPACE_ID/experience-manager-instances" \
   --header 'Content-Type: application/json' \
   --header "Authorization: Bearer BEARER_TOKEN" \
   --data '{
       "name": "EM_INSTANCE_NAME",
       "fusionInstance": "FUSION_INSTANCE_ID",
       "fusionAppId": "FUSION_APP",
       "region": "REGION",
       "migrate": true
   }'
   

   Placeholder definitions

   This process may take a few minutes.
   You may encounter an error that states Fusion app APP_NAME is not ready for Experience Manager. If you do, try running the command again. You may need to retry several times.
   Copy the Commerce Studio instance ID that appears in the response and save it in a secure location. You can also view the ID by navigating to the Commerce Studio instance in the UI. The ID appears in the URL after /em/, for example, platform.lucidworks.com/applications/em/12a34567-b890-1245-cd67-8efg901234h5. This is your EM_INSTANCE_ID value.
3. Optional: Check the status of the instance creation:
   Copy

   curl --location "https://api.lucidworks.com/customers/WORKSPACE_ID/experience-manager-instances/EM_INSTANCE_ID" \
   --header "Authorization: Bearer BEARER_TOKEN"
   

   The EM_INSTANCE_ID is the ID of your Commerce Studio instance. Your Commerce Studio instance is successfully created when the state field contains the value COMPLETED.
4. Verify your instance is ready for migration:
   Copy

   curl --location "https://EM_INSTANCE_ID.experiencemanager.lucidworks.com/pm-migration" \
   --header "Authorization: Bearer BEARER_TOKEN"
   

   Your Commerce Studio instance is ready when you see a 200 HTTP response code, and the state field contains the value READY.

​

Migrate data from Predictive Merchandiser to Commerce Studio

1. To start the migration, send a PUT request to set the instance state to MIGRATING:
   Copy

   curl --location --request PUT "https://EM_INSTANCE_ID.experiencemanager.lucidworks.com/pm-migration" \
   --header "Content-Type: application/json" \
   --header "Authorization: Bearer BEARER_TOKEN" \
   --data '{
       "state": "MIGRATING"
   }'
   

   The migration process may take an extended period to complete.
2. Optional: Check the migration status:
   Copy

   curl --location "https://EM_INSTANCE_ID.experiencemanager.lucidworks.com/pm-migration" \
   --header "Authorization: Bearer BEARER_TOKEN"
   

   The response includes the count of successfully migrated objects by type. The process includes backoff and retry logic for handling import errors. If the number of errors exceeds a threshold, the migration stops and is marked as a failure.
3. When the migration completes, the state value changes to COMPLETED or FAILED. Check if the skippedContents array is populated. This array contains a list of all contents that were skipped during migration due to unsupported or invalid content. For example, see the following:
   Response

   Copy

   {
     "skippedContents": [
       {
         "contentType": "rules",
         "fusionId": "8otY20teB5",
         "reason": "Published rule with staged changes will be skipped. To fix, revert or publish the changes. Or, manually re-create this content in the destination instance."
       },
       {
         "contentType": "rules",
         "fusionId": "eWvjz3B2UX",
         "reason": "Unsupported or invalid content will be skipped. Attempted to import content .... EM rejected this content with status 400 BAD_REQUEST and error message: {\"statusCode\":500,\"message\":\"conditions.queries[0].<list element>: size must be between 0 and 8192\",\"fusionErrorMessage\":null,\"violations\":{\"conditions.queries[0].<list element>\":\"size must be between 0 and 8192\"}} ."
       },
       {
         "contentType": "rules",
         "fusionId": "yIOvDet9JY",
         "reason": "Published rule with staged changes will be skipped. To fix, revert or publish the changes. Or, manually re-create this content in the destination instance."
       }
     ],
     "migratedContentCounts": {
       "rules": 6,
       "zones": 0,
       "templates": 0,
       "rewrites": 0
     }
   }
   

   See all 25 lines
4. Review the completion status of the migration to determine how to proceed.
   • If your migration completed successfully and you don’t have any skipped rules, continue to validate migration data.
   • If your migration completed successfully but you have skipped rules and rewrites, follow the steps in prepare Predictive Merchandiser for migration, then attempt the migration again.
   • If your migration failed, review the error message and complete the following steps to restart the migration process:
     • Delete your Commerce Studio instance.
     • Clear any Fusion collections related to your Commerce Studio instance. These collections have names ending in _em.
     Ensure you are clearing the correct collections!Do not inadvertently delete other Fusion collections. This may result in permanent data loss. Lucidworks recommends backing up Predictive Merchandiser collections APP_NAME_query_rewrite and APP_NAME_query_rewrite_staging prior to clearing.
     • Restart the process at preparing Predictive Merchandiser for migration.

​

Validate migration data

1. Sign in to Lucidworks Platform as a workspace owner and navigate to Commerce Studio.
2. Navigate to Rules, then compare your rule count in Commerce Studio against the JSON you exported from Predictive Merchandiser.
3. Navigate to Rewrite, then validate that your query rewrites in Commerce Studio match the exported JSON.
4. Navigate to Pages, then validate your templates have become pages in Commerce Studio, and that the count matches the exported JSON.
5. Open each page, then validate your Predictive Merchandiser zones have become sections in Commerce Studio, and they are associated with the correct page.
6. Navigate to the Editor and check that rules and rewrites are applied as expected. Some rules and rewrites only fire under specific conditions:
   1. For rules that fire under specific templates, change your page to match the required template.
   2. For rules that fire only when specific tags are applied, verify the tags are displayed on the Rules page for the rule.
7. Perform additional rules and rewrites validations as needed, using the exported rules JSON as a reference. For example, check that the direction of synonym rewrites matches the expected direction.

​

Launch for live traffic

curl --location --request PUT "https://EM_INSTANCE_ID.experiencemanager.lucidworks.com/pm-migration" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer BEARER_TOKEN" \
--data '{
    "state": "CONNECTED"
}'

​

Revert back to Predictive Merchandiser

​

Revert to Predictive Merchandiser using the UI

1. Sign in to Lucidworks Platform as a workspace owner and navigate to Commerce Studio.
2. Hold the pointer over the instance you want to delete, click the three dots, and then click Delete.
3. In the Delete dialog, click Delete to confirm.

​

Revert to Predictive Merchandiser manually

​

Key differences and limitations

• Migration cannot be retried. You must delete and re-create the Commerce Studio instance.
• Lucidworks Platform does not automatically clear Commerce Studio collections in Fusion after a failed migration. You must manually clear them using the Fusion UI.

​

Frequently asked questions

​

Troubleshooting

​

Symptoms

• No results or facets returned in the Editor
• Error messages when performing searches
• Rules not firing or not appearing after creation
• Rewrites not triggering or not affecting live traffic
• Mismatch between facets and results list
• “Targeted documents” column in the rules table is empty or incorrect
• No field value suggestions when configuring rule conditions
• Missing stage data in the Ranking Factors UI

​

General troubleshooting steps

1. Open your browser’s developer tools and go to the Network tab.
   1. Retry the failing action.
   2. Look for requests with a non-200 status or check render endpoint responses for errorMessage fields.
   3. Use the fusionRequestUrl to replicate the request in Fusion Query Workbench.
2. Verify that the query pipeline behaves as expected in Fusion Query Workbench.
   1. Confirm there are no errors.
   2. Verify expected results are returned.
   3. In the JSON view, check for fusion.applicable_rules or fusion.tagger values.
3. Confirm that rules and rewrites are saved to the correct Solr collections.
   1. In Solr Admin, check APP_NAME_query_rewrite_staging_em and APP_NAME_query_rewrite_em.
   2. Use the rule ID to query and confirm the entry exists.
4. Ensure the staging collection has only one replica.
   1. If multiple replicas exist, set shards.preference=replica.leader in the Apply Rules and Text Tagger stage parameters.
5. Verify that the pipeline supports the fl parameter.
   1. Do not overwrite fl. Add required fields only when needed.
   2. Do not use fl=*.

​

Required pipeline configuration

1. Text Tagger stage. Leave Tagger Collection blank.
2. Apply Rules stage. Leave Collection blank.
3. Solr Query stage.
4. Modify Response with Rules stage.
5. Response Diagnostics stage.

• Use the edismax query parser (defType=edismax)
• Preserve the following query parameters: tags, lw.rules.simulate, lw.app.emStatus, context, lw.tagger.debug, lw.rules.debug, lw.em.staging
• Preserve required response fields: response.docs, response.numFound, fusion.tagger, fusion.applicable_rules, fusion.applicable_stages, response.facet_counts, responseHeader.params

​

Feature-specific checks

​

Targeted document lookup

• The Fusion app and query profile must have the same name.
• The query profile must support q=**:** and standard fq parameters.
• Avoid using grouping, collapse/expand, or custom logic that may interfere with standard filter behavior.

​

Field value suggestions

• The Solr collection must match the Fusion app name.
• The collection must support the /terms request handler with standard parameters.
• Suggestions will fail if /terms is missing or misconfigured.

​

Rewrites and rules interaction

• If the Text Tagger stage appears before the Apply Rules stage, rewrites apply first.
• If the Apply Rules stage appears before the Text Tagger stage, only the original terms are used for rule evaluation.
• Rules must target the rewritten query term, not the original term.

​

Grouping via collapse/expand

• The query pipeline must set an fq with the collapse field specified, indicating what field identifies members of a group. For example, set fq={!collapse field=product_id_s} when expand=true is passed as a query param.
• Set the following query parameters:
  Copy

  expand=true
  expand.rows=5
  

• Relevant response fields include the following:
  Copy

  expanded
  response.docs
  

​

Query Workbench debugging tips

• Use the correct query profile and associated pipeline.
• Set the following query parameters:
  Copy

  lw.rules.debug=1
  lw.tagger.debug=1
  lw.em.staging=1=<FUSION_APP_NAME>_query_rewrite_staging_em
  lw.app.emStatus=CONNECTED
  queryProfileID=<ID>
  tags=<comma-separated values>
  lw.rules.target_segment=<segment>
  

• Use View As: JSON to verify fusion.applicable_rules and the final query string.
• Use custom JavaScript stages to inspect ctx or log intermediate values.

​

Known configuration issues

• Hardcoded tags in pipelines may prevent rules from matching if users forget to apply corresponding tags in the Editor.
• Query pipelines that use grouping may break document targeting. Only collapse/expand-based grouping is suported. Grouping via Solr group params or blockjoin query parser is not supported.
• If a boost stage and a rule both adjust relevancy, only one may take effect, depending on rule priority and creation date.