1.2.3 Release Notes

The following changes have been made for the 1.2.3 version of Fusion.

System Changes

Improved integration with enterprise security systems:

  • Fusion can be run as a Kerberos-protected service, allowing for secure single-sign on via Kerberos for authentication and providing fine-grained access to Fusion roles for individual Kerberos users.

  • LDAP integration has been enhanced so that LDAP group memberships can be used to determine Fusion roles. Existing LDAP integration provides authentication and document-level access protection based on LDAP permissions.

  • Fusion provides default system settings and resource configurations for each service.

  • Fusion adds collection audit_logs, which indexes all HTTP requests received by the backend services of Fusion.

Pipelines

  • New Alerting index and query pipeline stages which provide notification via email or Slack based on document properties.

  • JDBC index and query pipeline stages provide the same ability to create documents and queries by running SQL queries against a relational database. Can be used with any relational database that can be accessed by a JDBC4 database driver.

  • Fusion adds two new index stages for Natural Language Processing (NLP):

    • Part of Speech tagging

    • Sentence Detection

  • Fusion adds a find-replace index stage for wholesale replacement of a set of known boilerplate text blocks.

  • New index stages for ingesting CSV and JSON data

  • New Query stage Active Directory Security Trimming stage applies Active Directory access restrictions to query results.

  • New Query stage which returns query params for use in downstream stages, allowing for richer, more powerful query pipelines.

Connectors

New Connectors:

  • The Logstash connector provides direct access to log data compiled by Logstash. A Fusion Logstash connector datasource manages all Logstash configurations and plugins and all this can be done in the Fusion UI Admin Tool. In consequence, Fusion ships with an embedded Logstash installation.

  • Drupal

  • Salesforce

  • Github

Changes & Bug Fixes:

All Anda-based datasources now:

  • automatically clear the crawldb when clearing the datasource via the Fusion Admin UI.

  • have an option to use an on-disk or in-memory crawl db.

Web datasources now:

  • handle non-standard HTTP/HTTPS ports when evaluating restrictToTree rules for Web crawls.

  • properly handle redirects from a start link that does not include a 'www' prefix (such as, if the start link "http://lucidworks.com" was provided but redirects to "http://www.lucidworks.com").

  • changes the default for the option f.discardLinkURLQueries to "false". This will prevent discarding the query portions of URLs by default (such as "http://www.example.com/list?product=shirt").

The Subversion connector now includes SVN client binaries for Linux and Windows.

The FTP datasource type now supports SFTP.

The SharePoint connector has been improved for better handling of SSL sites.

MongoDB no longer throws an NullPointerException at the end of crawling (CONN-696).

The JDBC connector will now:

  • properly remove documents for the right data source ID when cleaning before a recrawl (CONN-709).

  • use a manually modified dataconfig.xml file (such as, if it has been modified to index BLOBs, or for other reasons) (CONN-446).

Signals and Aggregations

  • Signal extraction and event aggregations can run in Apache Spark for faster processing. In consequence, Fusion includes Apache Spark.

  • Aggregating Index stage aggregates documents on the fly.

  • Aggregate Query Results query stage allows use of aggregators framework over query results.

  • New query stage User Recommendation Boosting uses user recommendations for search boosting.

Solr 5.x

  • Fusion supports Solr 5.x clusters.