Release date: 7 November 2018
Component versions:
Solr 7.4.0 |
ZooKeeper 3.4.13 |
Spark 2.3.1 |
Jetty 9.3.25.v20180904 |
Ignite 2.3.0 |
See also the Fusion AI 4.1.1 release notes.
Note
|
As of Fusion 4.1.1, the SSL configuration procedure has changed. See SSL security for Unix and SSL security for Windows for updated instructions. |
New features
-
You can now upload Javascript files to the blob store and use them in these new pipeline stages:
Improvements
-
ZooKeeper has been upgraded from version 3.4.11 to 3.4.13.
See the ZooKeeper 3.4.13 release notes.
-
Jetty has been upgraded from version 9.3.8.v20160314 to 9.3.25.v20180904.
See the list of changes in 9.3.25.v20180904.
-
Two new blob types were added, to support the Managed Javascript index stage and query stage:
-
file:js-index
-
file:js-query
-
-
Web connector improvements:
-
The connector can now send custom headers with a new
addedHeaders
property. -
When the Web connector crawls a Web site and Server Name Indication (SNI) is enabled but the Web site doesn’t support it, you may receive an
unrecognized name
error. A new property,useIpAddressForSslConnections
configures the connector to use the IP address instead of the hostname. -
Kerberos is now supported with these new properties:
-
kerberosEnabled
- Boolean; default false. If true, the connector attempts to perform Kerberos/Spnego authentication when a Web request returns a401 WWW-Negotiate
challenge. If false, the Web connector refuses to attempt Kerberos/Spnego authentication. -
kerberosPrincipalName
- Optional - Use this principal name as the logged-in Kerberos user instead of the environment’s default. If set, you must also specify a keytab with one eitherkerberosKeytabFilePath
orkerberosKeytabBase64
(see below). -
kerberosKeytabFile
- Optional - The path to the Kerberos keytab file that contains the credentials. -
kerberosKeytabBase64
- Optional - A base64-encoded Kerberos keytab file that contains the credentials.
See the main topic for additional configuration details.
-
-
-
The ServiceNow connector now supports proxying.
-
The JDBC Connector has a new
convert_type
parameter for binary streaming data from a SQL ServerBy default, a JDBC column will automatically use the Solr field type that matches the underlying database’s type of the column. If checked, this will use the field name of the column to choose the Solr field type.
-
The Slack connector can now filter by channel names using a new
channel_filters
property.
-
When you import objects using the Objects API, a new
context
parameter specifies the name of an existing app which will be the new context for the imported objects. To support this, the Links API has two newlinkType
values:-
inContextOf
-
hasContext
-
-
Performance improvements in proxy.
The proxy service now reuses HTTP client instances for better throughput. Customers with high QPS expectations (~ 1000 QPS) are strongly encouraged to upgrade to 4.1.1.
Other changes
-
The Fusion SQL Service (
bin/sql
) has been added to the default group inconf/fusion.properties
. If you do not intend to run the SQL service for self-service analytics, you can remove it from the default group. See Fusion SQL service for more information on using the SQL service.
-
When exporting and importing objects with secret keys, the format is now
secret.{object_type}.{object_id}.{number}.password
. In the case of data source objects, the format is nowsecret.{object_type}.{object_id}.{datasource_type}.{number}.password
.
-
The Index Profiles API no longer sets
bufferDocsForSolr
by default.
-
The Query Workbench now correctly displays all metadata fields.
-
JavaScript is now thread-safe in pipeline stages.
This resolves an issue in previous releases where variables in Javascript stages that were not declared with
var
were shared between threads and other stages using variables with the same name.
-
The JWT security realm now works with multiple JWTs.
-
Headers missing from
request.headers
in the trusted-http security realm are available again. This includes:-
fusion-user-realm-name
-
fusion-user-realm-type
-
fusion-user-name
-
fusion-user-permissions
now has the role restriction assigned with the security realm.
-
Known issues
-
The Tika parser and Tika stage Return parsed content as XML option works only if the input document is also HTML/XML.
-
Vulnerability scanners are reporting false positives for security vulnerabilities.
-
Fusion may not redirect to the login page for users of Google Chrome release version 73.0.3683.75.
-
When exporting configuration data during a Fusion upgrade, the migrator may fail to log the complete process.
-
Fusion UI may not display the complete job history, despite the history being complete in the jobs endpoint.
-
When using High Performance JavaScript Evaluation mode in Google Chrome, the proxy must be used only for HTTP traffic, not HTTPS.
-
In Red Hat Enterprise Linux (RHEL) 7.6, running the
install-high-perf-web-deps.sh
script fails to install Google Chrome.
-
Fusion Proxy service’s SAML library does not fully support Windows.
-
The
responseHeaders_ss
field may be missing from PDF documents acquired by a web crawl search.
-
In Fusion UI, the job history status icons may not accurately show the job status.
-
Running
count(distinct query)
in the Fusion SQL service results in an inaccurate number of results. This can be mitigated by using a query similar toselect count(1) from (select distinct query from table) a
.
-
When using ZooKeeper Import/Export API to export an app, the process may fail if the app contains large blobs.
-
Using JavaScript dedupe may result in the creation of a ScriptRunner with a large default cache size. This may cause the connectors to run out of heap space during large crawls.
-
Attempting to use JSONPath
$.id
in the JSON Parser to parse all document IDs only processes the first document in the list.
-
Some Fusion SQL queries do not work with time-based partitions of collections.
-
Clicking "Ok" while Fusion UI is in the process of deleting an app will cause a
503 Service Unavailable
error.
-
In Fusion UI, deleting an app with a datasource and creating a new app with the same name may result in
404 Not Found
and500 Server Error
errors when uploading a datasource. Reloading the page prior to uploading the datasource will resolve this issue.
-
Updating a datasource may result in the loss of unsaved changes to pipeline or parser stages.
-
User-created index pipeline or parser fields are not included in the Index Profile search’s autocomplete function.
-
When deleting a collection, selecting the option "Switch to another collection", not making a choice, and re-selecting "Or return to the launcher to pick a different app" results in the "Proceed to Delete Collection" button being disabled.
-
In Fusion UI’s Collection Manager, unchecking "Enable Search Logs" in Manage All Collections > Collection Features before saving results in a
500 Server Error
error.
-
In Fusion UI’s Collection Manager, unchecking "Enable Signals" in Manage All Collections > Collection Features when creating a collection does not result in signals being disabled.
-
Starting and stopping Fusion services while in the Fusion UI may result in the components and fields being inaccessible.
-
Setting a parser’s
maxParserRecursionDepth
field to a negative value will not result in an error until attempting to implement it in the Index Workbench.
-
The SQL service will not initialize by default unless the API is located on the same host.
-
Indexing an Oracle JDBC datasource may fail due to an inability to update ConvertType.
-
The Hadoop Authentication Plugin
hadoop-auth-3.1.1.jar
is incompatible with Jetty9.4.12.v20180830.jar
. Reverting tohadoop-auth-2.7.1.jar
inside ROOT.war fixes the issue.
-
Users in a JSON Web Token (JWT) realm in Fusion may fail to authenticate. Use of a base64 encoded token may fix this issue.
-
Stopping Fusion with a
TERM
signal fails to stop all Fusion services.
-
Deleting folders from the
fusion/var/log/
directory causes Fusion services to fail during start.
-
Creating an app with the same name as a previously deleted app may result in the error "Datasource ID doesn’t exist".
-
Fusion UI’s Index Workbench may fail to display the correct job status until the page is reloaded.
-
When performing a Head/Tail Analysis Job, the user may receive a
Job execution error: Error during job execution: empty collection
error, even if there are documents in the app collections.
-
Some connectors are not visible in the datasource panel until after they have been installed.
-
In Windows, SDK connectors may time out during the first run. Subsequent runs are successful.
-
Blob types
file:js-index
andfile:js-query
cannot be added in Fusion UI but can be added with the API.
-
If an incorrectly configured stage is disabled, results will not generate despite active stages being valid.
-
No error message is produced during a signal aggregation job if the field name value being searched is inaccurate or non-existent.