Fusion AI Release Notes

Current Fusion AI Release: 5.0.0

Release date: 11 September 2019

Past Fusion AI releases

Version Release Date Major Changes Release Notes


17 May 2019

  • Synonym pairs are now grouped and treated as one term when for better compatibility with Solr’s "minimum match" (mm) query parameter.

  • Query rewriting accuracy is improved in this release.

  • Parallel Bulk Loader (PBL) jobs and Script jobs can now be configured to set environment variables. In the Fusion UI, click Advanced to see this option.

  • Recommender jobs are no longer scheduled by default; they must now be scheduled manually or run on demand. (Previously, default recommender jobs were automatically scheduled to run after the _user_item_preferences_aggregation job.)

  • The Spark driver now writes error.log and output.log files to var/log/api instead of var/api/work.

  • Query Rewriting UI bug fixes:

    • Fixed an issue in Fusion Server 4.2.0 and 4.2.1 that caused the UI’s search feature to return a Problem with underlying storage when a query was : followed by an additional query term.

    • Fixed a bug that caused the Query Rewriting UI to display an error window that could not be closed.

  • Signals from the Simulator are no longer sent to the signals collection.

  • Fixed an issue in the Text Tagger query stage which prevented query rewrites from working after Save Tags in Context had been enabled.

  • Business rules defined with tags are no longer triggered by queries that contain no tags parameters.


5 April 2019

  • In the _user_query_history_aggregation SQL aggregation job, the default value of the signalTypeWeights SQL parameter has been changed from click:1.0,add-to-cart:10.0,purchase:25.0 to request:1.0,click:5.0,cart:10.0,purchase:25.0 to add request signals so that all signal types are included in these aggregations.

  • Query Rewriting UI improvements:

    • Facet groups can now be collapsed and expanded for easier viewing.

    • Query rewrites are now faceted by tag.

    • The Apply Rules query pipeline stage has a new parameter, Partially Matched Filter Queries Will Trigger the Rule/matchPartialFilterQueries, which allows a rule to fire when it is configured with multiple Field Value conditions and only some of those conditions are matched.

    • The Misspelling Detection page now has a Published column.

    • Manually-created query rewrites are now automatically assigned a review value of "Approved" and the Review field is no longer editable in the query rewrite creation interface.

    • Fixed an issue which broken rules with banner actions when the Banner Zone field value was an integer.

    • Better checks for duplicate rewrites.

    • Uni-directional synonyms are now working correctly in the query_rewrite_staging collection and the Simulator.

    • A manually-created head/tail query improvement now has its action field correctly populated.

  • A variety of minor UI issues were fixed.


28 February 2019


7 December 2018

No changes to AI features.


7 November 2018

  • Phrase Extraction job improvements

    • The job now trims low-confidence phrases based on likelihood.

    • The job adds a review tag on the result to facilitate the review process based on beginning and ending POS and likelihood.

    • The output now connects phrase tokens with an underscore (_) to make a single token per phrase so that complete phrases can be used as facets.

    • New metadata fields:

      • phrases_count shows how many times the phrases appear in the documents.

      • word_num shows how many words are in the phrase.

    • This release includes optimizations to the default Lucene analyzer configuration.

    • Output names are updated for clarity, such as llr to likelihood and ngram to phrases.

  • The Head/Tail Analysis job now processes large data sets twice as fast.

  • The Ranking Metrics job now correctly accepts values for the queryPipelines property in the Fusion UI.

  • The Solr Query pipeline stage has two new properties to configure signals:

    • responseSignalsEnabled

      Disable this option to prevent the stage from generating a response signal containing metadata about the response from Solr.

      Response signals are used by App Insights and experiments.

      In auto-complete pipelines, disable this option to avoid generating a response signal for each keystroke.
    • excludeResponseSignalMatchRules

      If responseSignalsEnabled is "true", then you can prevent generating a response signal based on specific parameters in the query, such as to enable response signals in general but to disable them for auto-complete queries.

  • The Spark driver now cleans up $FUSION_HOME/var/spark/Spark-workDir-* directories and shaded jars correctly on Windows to prevent excessive disk consumption.

  • For installations that were upgraded from 3.1.x to 4.1.0, upgrading to 4.1.1 resolves an issue that prevented successful signal aggregations with the Parameterized SQL Aggregation job.


17 July 2018

  • SparkSQL datasource loader job

    • SQL Engine Improvements

  • Added virtual table join support to the Solr/SQL pushdown strategy (SQL 360 Collections)

  • Added Kerberos authentication and SSO authentication support

  • Fixed scaling bugs with Tableau

  • Added robust Tableau support (better than MySQL and PostGres on cross table joins)

    • Experiment management improvements

  • New "default response time in ms" default metric

  • Custom stopword lists can now be uploaded to the blob store and utilized by the Head/Tail Analysis, Document Clustering (formerly the Bisecting KMeans Clustering job), Cluster Labeling, Phrase Extraction (formerly the Statistically Interesting Phrases job), and Token and Phrase Spell Correction jobs.

  • The Levenshtein Spell Checking job has been removed. Use the Token and Phrase Spell Correction job instead.

  • Add schema groups and categories by default on all Spark Jobs

  • Some jobs have been renamed:

4.0 Job 4.1 Job

Co-occurrence Similarity

Legacy Item Similarity

Item Similarity Recommender

Legacy Item Recommender

Matrix Decomposition Based Query - Query Similarity

Query to Query Similarity Computation

Statistically Interesting Phrases

Phrase Extraction

  • When you disable signals for a collection, the associated jobs are now deleted.

  • The Boost with Signals query pipeline stage can now be configured to point to an alternative collection for aggregated signals.


22 May 2018


28 February 2018

  • Session rollup job improvements

    Session rollup jobs now include the app_id, host, ip_address, and user_token fields, for use in App Insights.


21 February 2018

  • App Insights

    App Insights provides detailed, real-time, searchable reports and visualizations derived from your signals data. It also provides alerts and triggers to notify you when specific events occur.

    Pre-4.0 signals data will not produce useful visualization in App Insights. New signals generated with Fusion 4.0 will produce the best results.
  • New jobs

    • Head/Tail Analysis

      Perform head/tail analysis of queries from collections of raw or aggregated signals, to identify underperforming queries and the reasons. This information is valuable for configuring better synonyms, auto-suggest, recommendations, and so on, in order to improve conversion rates.

    • Token and Phrase Spell Correction

      Extract tail tokens (one word) and phrases (two words) and find similarly-spelled head tokens and phrases. If several matching heads are found for each tail, the job can compare and pick the best correction using multiple configurable criteria.

  • Experiment management features

    The Fusion UI now includes interfaces for running and managing experiments.

  • SQL for aggregations

    By default, aggregation jobs are now defined using Spark SQL. To use the old aggregation configuration scheme in the Fusion UI, click Advanced in the Aggregation job configuration panel.