Migration guideMigrate from Predictive Merchandiser to Commerce Studio
This guide explains how to migrate from Predictive Merchandiser to Commerce Studio in Lucidworks Platform. It covers all required setup steps, including prerequisites, AI gateway configuration, and API-based instance creation. You’ll learn how to transfer rules and rewrites, verify the success of the migration, and launch Commerce Studio for live traffic.
The migration is one-way. You can revert back to Predictive Merchandiser, but any changes made in Commerce Studio after switching will be lost.
See Commerce Studio versus Predictive Merchandiser for a feature comparison. |
Prerequisites
Before beginning the migration, ensure that your environment meets all required conditions.
-
Admin privileges to a Fusion environment running Fusion 5.9.8 or later.
-
A valid Predictive Merchandiser license.
-
Workspace Owner permissions in a Lucidworks Platform environment.
-
Privileges to access and use Lucidworks Commerce Studio in a Lucidworks platform workspace.
Fusion environments 5.9.8-5.9.11 require a patch to support migration. Contact your Lucidworks client support representative for more information. |
Migration process
Prepare Predictive Merchandiser for migration
Before starting the migration, you must publish or revert all staged changes to rules and rewrites. This includes newly created items, any updates to existing ones, and rule deletions.
Unpublished rules will not migrate to Commerce Studio. |
You can only view staged changes in Predictive Merchandiser that belong to your user. To check for staged changes that belong to other users in your organization, do the following:
-
Download the provided precheck script to check for staged (unpublished) rules and rewrites: Click to download precheck.sh
-
Run the script, replacing the placeholder values with your values.
./precheck.sh FUSION_USERNAME FUSION_PASSWORD FUSION_BASE_URL FUSION_APP_ID
Placeholder Description FUSION_USERNAME
Your Fusion username.
FUSION_PASSWORD
The password for your Fusion user.
FUSION_BASE_URL
The URL of your Fusion instance including
https://
. For example,https://FUSION_HOST.com
.FUSION_APP_ID
The application ID of the Fusion app that contains your Predictive Merchandiser.
This is the value of the
id
field, and may or may not match thename
of the Fusion app. To obtain the ID, sign in to your Fusion instance. For example,FUSION_INSTANCE.com
. Then also open a different browser window and enterhttps://FUSION_INSTANCE.com/api/apps
. Theid
andname
display for each valid Fusion app. The value in theid
field is the value that needs to be used in this field.If you receive an error stating
bash: ./precheck.sh: Permission denied
:-
Make the script executable:
chmod +x precheck.sh
. -
Run the script again.
-
-
Check the output to see if any issues were detected. If there are no issues, the output returns
All pre-checks passed. No issues found.
Continue to retrieve list of objects in Predictive Merchandiser. -
If there are issues, the output returns a list of issues like the following:
The following pre-checks failed: Found published rules with staged changes. Publish or revert the changes to fix this issue. - 2fRtM1GEsU - E1YkUTX9r6 - Eb2A6WARag - FNS7Whciwx - LlIsMfK4kq - OidpIUrVv3 - P39TDQ2X1K - RYbY2XPMOI - ZHvW2LiQpe - aCseNzc4Xc ... and 6 more. See precheck_results.json for the full list of invalid content
-
Review
precheck_results.json
and inspect each reported rule. If you created the rules, sign into Predictive Merchandiser and publish or revert the staged changes.If the unpublished rules were created by a different user, you cannot publish or revert the rules in the Predictive Merchandiser UI. There are a few options for resolving unpublished rules owned by another user:
-
Proceed with migration. These rules are skipped and you can re-create them in Commerce Studio.
-
Use the Fusion Query Rewrite and Custom Rule APIs to manually publish or revert the unpublished rules.
-
Use the Fusion Query Rewrite and Custom Rule APIs, or Solr, to manually change the edit session ID of the content in Solr to your current user. Then, manually fix the content in the Predictive Merchandiser UI.
-
Have the user that owns the content in Predictive Merchandiser sign in and either publish or revert the changes.
-
-
After resolving issues, re-run the script. If there are no issues, continue to retrieve list of objects in Predictive Merchandiser.
You can run the precheck.sh script at any time, even after a migration if desired. It makes no changes to contents and works even if Commerce Studio is enabled. |
Retrieve lists of objects in Predictive Merchandiser
After preparing Predictive Merchandiser for migration, Lucidworks recommends that you export lists of rules, rewrites, templates, and zones from your Predictive Merchandiser instance to compare against Commerce Studio later. This will help serve as a reference when performing your post-migration validation checks.
-
Export a list of your rules and rewrites:
curl --request GET \ --url USER:PASSWORD \ 'https://FUSION_HOST/api/apps/APP_NAME/query-rewrite/instances' \ --header 'accept: application/json' \ > exported-rules.json
-
Create an inventory of your rules, noting how many of each type and tag appear in the list.
-
Export a list of your zones:
curl --request GET \ -u USER:PASSWORD \ --url 'https://FUSION_HOST/api/templating/zones?context=app:APP_NAME' \ --header 'Accept: application/json' \ > zones.json
-
Export a list of your templates:
curl --request GET \ -u USER:PASSWORD \ --url 'https://FUSION_HOST/api/templating/templates?context=app:APP_NAME' \ --header 'Accept: application/json' \ > templates.json
Set up the Commerce Studio integration
Follow the instructions in Commerce Studio integration with Fusion to establish the integration. However, do not create the instance using the UI. Lucidworks will create the Commerce Studio instance using the API.
While completing the process:
-
Copy the ID of the Fusion instance you’re migrating from and save in a secure location.
-
Copy the Lucidworks Platform workspace ID and save in a secure location. You can find this in the JWKS and Issuer URLs.
Migrate data from Predictive Merchandiser to Commerce Studio
-
To start the migration, send a
PUT
request to set the instance state toMIGRATING
:curl --location --request PUT "https://EM_INSTANCE_ID.experiencemanager.lucidworks.com/pm-migration" \ --header "Content-Type: application/json" \ --header "Authorization: Bearer BEARER_TOKEN" \ --data '{ "state": "MIGRATING" }'
The migration process may take an extended period to complete.
-
Optional: Check the migration status:
curl --location "https://EM_INSTANCE_ID.experiencemanager.lucidworks.com/pm-migration" \ --header "Authorization: Bearer BEARER_TOKEN"
The response includes the count of successfully migrated objects by type. The process includes backoff and retry logic for handling import errors. If the number of errors exceeds a threshold, the migration stops and is marked as a failure.
-
When the migration completes, the
state
value changes toCOMPLETED
orFAILED
. Check if theskippedContents
array is populated. This array contains a list of all contents that were skipped during migration due to unsupported or invalid content. For example, see the following:{ "skippedContents": [ { "contentType": "rules", "fusionId": "8otY20teB5", "reason": "Published rule with staged changes will be skipped. To fix, revert or publish the changes. Or, manually re-create this content in the destination instance." }, { "contentType": "rules", "fusionId": "eWvjz3B2UX", "reason": "Unsupported or invalid content will be skipped. Attempted to import content .... EM rejected this content with status 400 BAD_REQUEST and error message: {\"statusCode\":500,\"message\":\"conditions.queries[0].<list element>: size must be between 0 and 8192\",\"fusionErrorMessage\":null,\"violations\":{\"conditions.queries[0].<list element>\":\"size must be between 0 and 8192\"}} ." }, { "contentType": "rules", "fusionId": "yIOvDet9JY", "reason": "Published rule with staged changes will be skipped. To fix, revert or publish the changes. Or, manually re-create this content in the destination instance." } ], "migratedContentCounts": { "rules": 6, "zones": 0, "templates": 0, "rewrites": 0 } }
-
Review the completion status of the migration to determine how to proceed.
-
If your migration completed successfully and you don’t have any skipped rules, continue to validate migration data.
-
If your migration completed successfully but you have skipped rules and rewrites, follow the steps in prepare Predictive Merchandiser for migration, then attempt the migration again.
-
If your migration failed, review the error message and complete the following steps to restart the migration process:
-
Delete your Commerce Studio instance.
-
Clear any Fusion collections related to your Commerce Studio instance. These collections have names ending in
_em
.Ensure you are clearing the correct collections!Do not inadvertently delete other Fusion collections. This may result in permanent data loss. Lucidworks recommends backing up Predictive Merchandiser collections
APP_NAME_query_rewrite
andAPP_NAME_query_rewrite_staging
prior to clearing. -
Restart the process at preparing Predictive Merchandiser for migration.
-
-
Validate migration data
After successfully migrating, validate that all Predictive Merchandiser objects appear in Commerce Studio.
When you create your Commerce Studio using the API, you are locked out of Predictive Merchandiser, so you must consult the exported JSON for validation checks.
-
Sign in to Lucidworks Platform as a workspace owner and navigate to Commerce Studio.
-
Navigate to Rules, then compare your rule count in Commerce Studio against the JSON you exported from Predictive Merchandiser.
-
Navigate to Rewrite, then validate that your query rewrites in Commerce Studio match the exported JSON.
-
Navigate to Pages, then validate your templates have become pages in Commerce Studio, and that the count matches the exported JSON.
-
Open each page, then validate your Predictive Merchandiser zones have become sections in Commerce Studio, and they are associated with the correct page.
-
Navigate to the Editor and check that rules and rewrites are applied as expected. Some rules and rewrites only fire under specific conditions:
-
For rules that fire under specific templates, change your page to match the required template.
-
For rules that fire only when specific tags are applied, verify the tags are displayed on the Rules page for the rule.
-
-
Perform additional rules and rewrites validations as needed, using the exported rules JSON as a reference. For example, check that the direction of synonym rewrites matches the expected direction.
If you encounter any issues, check the Troubleshooting section of this guide.
Web traffic continues to use rules and rewrites from Predictive Merchandiser. Changes made in Commerce Studio do not affect live traffic. When you’ve finished validating and are ready to switch, complete the launch process.
Changes made in Commerce Studio are not reflected in Predictive Merchandiser. If you choose to revert later, there is no migration path from Commerce Studio back to Predictive Merchandiser. |
Launch for live traffic
If you are ready to switch live traffic to Commerce Studio rules and rewrites, send a PUT
request to set the instance state to CONNECTED
:
curl --location --request PUT "https://EM_INSTANCE_ID.experiencemanager.lucidworks.com/pm-migration" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer BEARER_TOKEN" \
--data '{
"state": "CONNECTED"
}'
The change takes effect immediately.
Revert back to Predictive Merchandiser
If you want to revert back to using Predictive Merchandiser, Lucidworks must delete the Commerce Studio instance.
If the Commerce Studio instance is deleted, changes you made to Commerce Studio after the initial migration from Predictive Merchandiser are not populated back into Predictive Merchandiser.
When the Commerce Studio instance is deleted, the system should automatically revert the Fusion app back to Predictive Merchandiser. This restores access to Predictive Merchandiser and routes live traffic back to its rules and rewrites, if traffic had previously been switched to Commerce Studio.
In rare cases, the automatic reversion can fail. You can identify this by visiting the Fusion home page. If the “Commerce Studio enabled” badge still appears on the Fusion app after deleting the Commerce Studio instance and refreshing the page, the reversion did not complete. You will also be unable to access Predictive Merchandiser.
Key differences and limitations
Commerce Studio and Predictive Merchandiser are built on different frameworks with distinct architectures and tooling. As a result, there are key differences and limitations you will encounter after migrating.
-
Migration cannot be retried. You must delete and re-create the Commerce Studio instance.
-
Lucidworks Platform does not automatically clear Commerce Studio collections in Fusion after a failed migration. You must manually clear them using the Fusion UI.
Frequently asked questions
Can I retry a failed migration from Predictive Merchandiser to Commerce Studio?
No. Migration cannot be retried. If the migration fails, you must delete the Commerce Studio instance, manually clear related Fusion collections, and restart the process from the beginning.
Can I revert back to Predictive Merchandiser after switching to Commerce Studio?
Yes. You can revert by deleting the Commerce Studio instance. However, any changes made in Commerce Studio after switching will be lost. In rare cases, the reversion may fail and must be completed manually by setting the instance state to DISABLED
.
Does switching to Commerce Studio immediately affect live web traffic?
No. After a successful migration, web traffic continues to use rules and rewrites from Predictive Merchandiser. To switch live traffic to Commerce Studio, you must explicitly launch by setting the instance state to CONNECTED
.
What happens if I forget to publish changes in Predictive Merchandiser before migrating?
If you fail to publish pending rules and rewrites before starting the migration, the migration may fail or result in incomplete or inaccurate data. All changes must be published before proceeding.
Troubleshooting
This section provides troubleshooting steps for resolving issues with the Commerce Studio Editor when integrated with Fusion. Use this guidance to debug unexpected behavior that may be caused by Fusion query pipeline or query profile configuration.
This content is intended for users with experience in Fusion and search engineering.
Symptoms
You may be experiencing a configuration issue if you observe any of the following:
-
No results or facets returned in the Editor
-
Error messages when performing searches
-
Rules not firing or not appearing after creation
-
Rewrites not triggering or not affecting live traffic
-
Mismatch between facets and results list
-
“Targeted documents” column in the rules table is empty or incorrect
-
No field value suggestions when configuring rule conditions
-
Missing stage data in the Ranking Factors UI
General troubleshooting steps
-
Open your browser’s developer tools and go to the Network tab.
-
Retry the failing action.
-
Look for requests with a non-200 status or check
render
endpoint responses forerrorMessage
fields. -
Use the
fusionRequestUrl
to replicate the request in Fusion Query Workbench.
-
-
Verify that the query pipeline behaves as expected in Fusion Query Workbench.
-
Confirm there are no errors.
-
Verify expected results are returned.
-
In the JSON view, check for
fusion.applicable_rules
orfusion.tagger
values.
-
-
Confirm that rules and rewrites are saved to the correct Solr collections.
-
In Solr Admin, check
APP_NAME_query_rewrite_staging_em
andAPP_NAME_query_rewrite_em
. -
Use the rule ID to query and confirm the entry exists.
-
-
Ensure the staging collection has only one replica.
-
If multiple replicas exist, set
shards.preference=replica.leader
in the Apply Rules and Text Tagger stage parameters.
-
-
Verify that the pipeline supports the
fl
parameter.-
Do not overwrite
fl
. Add required fields only when needed. -
Do not use
fl=*
.
-
Required pipeline configuration
Your Fusion query pipeline must include the following stages in this order:
-
Text Tagger stage. Leave Tagger Collection blank.
-
Apply Rules stage. Leave Collection blank.
-
Solr Query stage.
-
Modify Response with Rules stage.
-
Response Diagnostics stage.
In addition, the pipeline must:
-
Use the
edismax
query parser (defType=edismax
) -
Preserve the following query parameters:
tags
,lw.rules.simulate
,lw.app.emStatus
,context
,lw.tagger.debug
,lw.rules.debug
,lw.em.staging
-
Preserve required response fields:
response.docs
,response.numFound
,fusion.tagger
,fusion.applicable_rules
,fusion.applicable_stages
,response.facet_counts
,responseHeader.params
Feature-specific checks
Targeted document lookup
-
The Fusion app and query profile must have the same name.
-
The query profile must support
q=*:*
and standardfq
parameters. -
Avoid using grouping, collapse/expand, or custom logic that may interfere with standard filter behavior.
Field value suggestions
-
The Solr collection must match the Fusion app name.
-
The collection must support the
/terms
request handler with standard parameters. -
Suggestions will fail if
/terms
is missing or misconfigured.
Rewrites and rules interaction
-
If the Text Tagger stage appears before the Apply Rules stage, rewrites apply first.
-
If the Apply Rules stage appears before the Text Tagger stage, only the original terms are used for rule evaluation.
-
Rules must target the rewritten query term, not the original term.
Grouping via collapse/expand
-
The query pipeline must set an
fq
with the collapse field specified, indicating what field identifies members of a group. For example, setfq={!collapse field=product_id_s}
whenexpand=true
is passed as a query param. -
Set the following query parameters:
expand=true expand.rows=5
-
Relevant response fields include the following:
expanded response.docs
Query Workbench debugging tips
-
Use the correct query profile and associated pipeline.
-
Set the following query parameters:
lw.rules.debug=1 lw.tagger.debug=1 lw.em.staging=1=<FUSION_APP_NAME>_query_rewrite_staging_em lw.app.emStatus=CONNECTED queryProfileID=<ID> tags=<comma-separated values> lw.rules.target_segment=<segment>
-
Use View As: JSON to verify
fusion.applicable_rules
and the final query string. -
Use custom JavaScript stages to inspect
ctx
or log intermediate values.
Known configuration issues
-
Hardcoded
tags
in pipelines may prevent rules from matching if users forget to apply corresponding tags in the Editor. -
Query pipelines that use grouping may break document targeting. Only collapse/expand-based grouping is suported. Grouping via Solr
group
params or blockjoin query parser is not supported. -
If a boost stage and a rule both adjust relevancy, only one may take effect, depending on rule priority and creation date.