Fusion Server 5.1.2 Release Notes

Subscribe

Release date: 14 May 2020

Component versions:

Solr 8.4.1

ZooKeeper 3.5.6

Spark 2.4.3

New Features

Predictive Merchandiser

  • Rules can now be created for product groups in Predictive Merchandiser. Product groups can be boosted or blocked. If you want to promote a product group that’s on sale, for example, you can boost the group to the top of the results. You can still apply rules individually to products within a group. This allows you to boost a product based on a specific user’s search behavior. If a user typically favors the color red, you can boost the red variant of the product to the top of the group.

    Read more about product grouping in Predictive Merchandiser.

  • Predictive Merchandiser allows search engineers to ensure users don’t encounter a zero results page by creating rules to display select products when a zero results page is reached. Read more about how to resolve zero result queries with Predictive Merchandiser.

  • The UI now shows potentially conflicting or similar business rules when creating or editing rules. For example, if you have 3 rules that boost, bury, and block the same product, Predictive Merchandiser highlights these conflicting rules so you can address them.

    Rules conflict resolution

    Read more about the feature in the Predictive Merchandiser documentation.

Improvements

Fusion

  • The Query Rewriting API adds a new endpoint, /targeted-docs, that lists which rules affect a particular document.

  • Custom Index Stages now let user know they cannot be deleted if they are in the blob store and being used by the Fusion index pipeline.

  • Added row control and Enabled/Disabled column toggle functionality to Affected Rules table.

  • Tail rewrites based on reasons of 'number', 'spelling', 'rare-term', or 'others' are removed from the staging collection, but are retained for the reports collection.

  • Kubernetes has a 63-character limit on pod names due to DNS naming. Spark job names are now truncated to comply with Kubernetes limits during pod creation.

  • You can now add annotations to Solr pods on a per-node pool basis by setting annotations on the Solr statefulset.

  • Rule tables rows now have a conflict indicator that pops out on hover notifying the user there is another rule that potentially conflicts with this one.

  • Performance for signals boosting via boost parameter has been improved.

  • Unpublished rules list loads 25 items upon opening and fetches another 25 from API when user scrolls to the end of the list to imitate infinite scroll.

  • Write option for configuring commit_within added to the ALS recommender config to prevent overwhelming Solr when running a large recommendation job. We suggest setting this to {{commit_within=-1}}.

Fusion AI

  • The Synonym Rules UI previously showed synonym pairs by default. The job now creates new grouped rules based on whether the job determines pair or context. If the job predicts context, then synonym pairs will display as context pairs.

  • Synonym context will be shown as default if the job predicts context. UI will allow users to switch between suggestions per synonym.

Predictive Merchandiser

  • Hovering over rules in the list now shows rules affected.

  • Users can now see a list of applied rules for a product to help determine which rules are affected when creating a new rule or editing an existing one.

Other changes

Fusion

  • The Rewrites dashboard defaults have been updated to remove misspelling and head/tail columns and hide phrase and synonym columns.

  • When classic/rpc are started for the first time, the plugins from bootstrap path are installed by default. Now it is possible to decide whether to install the plugins from the bootstrap. Before deploying classic or rpc, modify via configMap: from bootstrapEnabled: true to bootstrapEnabled: false.

  • The helm chart variable solrAutocalingPolicyJson for fusion-admin has been deprecated in favour of solrAutoscalingPolicyJson and will be removed in a future release. Please update any custom values files that reference this variable.

  • The Pulsar client option to block the message queue if full is now configurable. By default, this boolean is set to true. In the query service this value is set to false.

  • Messaging Stage Configs are now deprecated.

  • Upgraded Jackson to 2.9.10 and Jackson bind to 2.9.10.4.

  • Upgraded Fusion to use Tika 1.24.

Fusion AI

  • Synonym detection confidence scores are rescaled so the final output appears more significant than the default threshold.

  • Misspelling confidence scores are changed from numerical values of 0, 0.5, and 1 to categorical values of low, medium, and high, respectively.

Predictive Merchandiser

  • Licensing for Predictive Merchandiser is updated. With a valid license, users have access to the Dashboard, Merchandiser, Rules, and Rewrites screens in the Rules Editor. Without a license, users have access to the Rules, Rewrites, and Simulator screens.

    Note
    The Rules Editor requires an Advanced tier license. See Pricing Plans for more information.
  • Product details can now truncate values for multiple purchases in the bar display.

  • Fusion now has two additional rule types, in_group_boost and in_group_bury, which are designed to boost and bury documents within a particular "group" in Solr requests that use standard grouping. These rule types should be chosen over the traditional "Boost List" and "Bury List" rules when users want to reorder documents within a group without affecting the ordering of the groups themselves.

  • Users can now see a list of affected rules for each product.

  • There is now an additional active date facet.

  • Conflict resolution added inside rules manager edit modal to enable/disable conflicting rules.

Connectors

  • Connectors that support security filtering previously used separate fetchers for content and access control. One fetcher type is now used for both content and security fetching. AccessControlFetcher has been deprecated. Fetcher implementations that use AccessControlFetcher should instead use ContentFetcher.

Bug fixes

Fusion

  • Index Workbench search now shows Custom Stages uploaded through the blob store without requiring a refresh.

  • New Query Workbench stages set to OFF will now stay off when applied.

  • Fixed a bug where the collection picker in Query Workbench would retain a new collection even when canceling the operation.

  • Fixed a bug where Query Workbench would sometimes require typing the full field name before selecting it.

  • The column tick icon in the Rules UI no longer moves up when adding multiple synonyms.

  • Quick start message window will now be shown after initial login even when licensing is missing.

  • Fixed issue in the Helm QnA chart where having no defined secrets would break the Argo workflow.

  • Added GCS and Azure Hadoop fs libraries to the Spark Kubernetes configuration to successfully run associated data loading scripts.

  • Fixed a bug where aggregting signals from external catalog tables would ignore timestamps and cache all data.

  • Added support for Query Pipeline operations on grouped responses with multiple groups. Old API methods that worked with single group fields, but would return unpredictable results if the response contained more than one field, are marked as deprecated.

  • Updated Chrome driver dependencies versions to fix Web connector crashes.

  • The job launcher now includes untracked Argo jobs to correctly report Argo status for situations where the job-launcher pod is killed and restarted.

  • Reduced Spark pod to 30 characters to prevent running over the Kubernetes DNS pod name limits.

  • Fixed a job service bug where multiple same-request interceptors executed and caused multiple internal redirects.

  • Upgraded several services to the latest version of Jersey libraries.

  • Error messages in the datasources panel will now show the error icon.

  • Improved form autofill to display new values instead of default after saving.

  • Collections for an app no longer show _recommender_models collections from other apps.

  • Fixed console error when closing the Parsers pane in the UI.

  • Improved Index Pipelines in the UI to not allow adding stages with duplicate IDs.

  • The Messaging Service dropdown will now show an actual fallback value instead of 'undefined'.

  • QnA pipeline dropdowns now include tooltips to account for long names.

  • Cold Start Training job field is now marked as required.

  • Pipelines dropdown is now sorted on datasource page.

  • Removed false console warning when multiple IDs exist in the scheduler.

  • Updated facet label management to allow creating and editing blobs from the Query Workbench.

  • Updated links to documentation within the Admin UI.

  • Query Workbench rows parameter override feature fixed so QWB results in Firefox reflect JavaScript stage changes.

  • Metrics for individual stages are no longer lost when those stages are asynchronous.

  • Added liveness probe for Pulsar broker deployments that attempts to detect brokers in the bad state and restart them automatically.

  • Collections that are not associated with the current app are now searchable in Object Explorer.

  • Fixed issue where system autocomplete collection was not created without manual intervention if an error occurred during the first attempt to create it.

  • Fixed an issue with the fusion-notebooks service chart where specifying resource requests and limits would cause the install to fail due to invalid YAML syntax.

  • Azure connector now restarts if initially missing Storage Container field.

  • Restored 4.x ability to send Arrays to Solr as multivalued objects while also allowing translation of nested JSON structures.

  • Access table now sorts usernames in descending alphabetical order.

Fusion AI

  • Fixed a bug where the tooltip would cover the delete option for individual synonyms.

  • Fixed a bug where the synonyms cell would not display newly created synonyms.

  • Updated liveness/readiness probes in ml-model service to be configurable.

Predictive Merchandiser

  • Fixed rule sorting to be consistent across rules with the same created date so that adding synonyms to a rule using search no longer resets the UI.

  • Renamed facet labels in the Fusion Query Workbench stage now display the new name instead of the default field name.

  • Fixed a bug where the 'created on' date was not fully visible.

  • Improved speed when publishing a rule on a deployed cluster using the Publish button. A new parameter was added to specify the time interval for skipping Solr optimize for the {{publish}} end point. The makes the API call performance 6-7 times faster.

  • Ability added to toggle which rules are fired via tags. Search rules are now visible in merchandiser views with associated tags.

  • Fixed a bug where pagination controls were sometimes unavailable when total doc count was less than 20.

  • Fixed a bug where the field typeahead value would sometimes show an empty suggestion list.

  • Rules table now refreshes after creating a clone rule so cloning triggers a re-search.

  • Item-to-item recommendations now display on the PDP in PM.

  • Enabled entering field names and values which do not exist in the schema while editing a rule.

  • Range facets will now display properly in PM

  • Tag tooltip description now appropriately reflects the feature’s functionality.

  • In previous releases, triggered BuryList rules would add the buried docs to the resultset even if they weren’t matched by the user’s original query. This change alters this behavior to be more intuitive - buried documents have their score reduced if they are in the result set but are not otherwise added.

  • Delete button is hidden when a rule doesn’t exist.

  • Phrase Word Counter is now updated on the fly to account for edited phrases.

Connectors

  • Fixed a problem with the V2 connector service downscaling during a running job.

  • Fixed a bug which caused the V2 Connectors service to crash while crawling FetchInputs that produce many (that is, millions) of documents. Crawling large files now finishes indexing the documents as expected.

  • Index profile panel can now open when connectors-classic service is not running.

  • The UI no longer reports an error if connectors services are not deployed in the cluster.