Release NotesFusion Connectors

Incorporates OAuth for compatibility with Ping Identity and Azure.

Fixes a bug where links listed under BULK START LINKS were not being indexed.

A new property, retry count, has been introduced to the S3 connector. This property value is passed in the AWS SDK, determining the number of retry attempts for retrieving the file from the AWS S3 bucket. The configurability of this property enhances the effectiveness of the connector, especially in situations of network instability.

Adds validation to establish a connection with Kaltura before commencing the crawling process.

Updates the plugin to replace static Security Trimming with Graph Security Trimming, improving performance.

Adds an Enable Stray Content Deletion property in the Fusion UI for the S3 connector to toggle stray content deletion on or off. When stray content deletion is enabled, content that was removed from the datasource is deleted from the index in Fusion. When stray content deletion is disabled, content that was removed from the datasource is not deleted from the index in Fusion. This property is enabled by default.

Updates the plugin to replace static Security Trimming with Graph Security Trimming, improving performance.

Fixed an issue where security trimming was enabled by default.

Provides a convenient way to quickly ingest data from your local filesystem.

Constructed using the V2 framework and serves as a replacement for the classic version.

Updates the plugin to replace static Security Trimming with Graph Security Trimming, improving performance.

Fixes the invalid RegEx implementation that adjusts the Box URL to retrieve data from the Box datasource so the connector now accepts dashes (-) in Box start URLs.

Corrects relative path field data when indexing documents that are of type 'File' in nested folders.

Fixes a bug where new documents were not being indexed upon incremental crawls.

Migrates the Box-Java-SDK to version 4.8.0.

When the maximum number of referrals is reached, an exception is now thrown to handle the situation while ensuring the connector does not stop functioning.

Upgrades the plugin to use the latest SDK version.

Updates the plugin to replace static Security Trimming with Graph Security Trimming, improving performance.

Fixes a bug where an "Error validating datasource" message displayed after trying to save a datasource with the Enable DFS connection property.

Now you can configure the AEM connector to include child paths when indexing fields. This option is off by default; enable it by selecting Index metadata by child path in your AEM datasource configuration.

Fixes a bug that prevented running the connector on Windows.

The basic authentication username and password fields have moved under Authentication Settings > Login Settings.

Fixes Graph security trimming not working with Everyone except external access.

Allows connector to reach maximum referral. The connector will throw the exception but will not stop.

Added the Events Reducer feature, which prevents performing multiple requests to the same SharePoint object and ensures index integrity. Specifically, if multiple change events are made over the same SharePoint object, the connector detects and requests the most relevant event and crawls it correctly.

Fixed Graph Security Trimming when using custom-level permissions.

Fixed an issue to reindex restored lists.

Errors from item/attachment are deleted from crawlDB after being successfully retried.

Fixed a pagination issue that limited the number of records returned.

Added support for pagination of IBM Db2 (version 11 and earlier) by using a template sub-query:

-- add any WHERE, ORDER BY clauses as needed following your_table
SELECT * FROM ( SELECT *, ROW_NUMBER() OVER () AS row_num FROM your_table ) AS subquery WHERE row_num BETWEEN :offset + 1 AND (:offset + :limit)

Fixed an issue where UTF-8 characters stored as CLOB data types didn’t index properly.

LIMIT - specifies the number of rows returned in the results.

OFFSET - dictates the number of rows to skip from the beginning of the returned data before presenting the results.

disableAutomaticPagination - disables automatic pagination to ignore limit and offset fields.

Indexes the all-users acl document when indexing from AzureAD.

Fixes Graph Security Trimming with Everyone except external access in SharePoint Online.

Exposes configuration options for the validation timeout.

Resolves an issue where documents were deleted on incremental crawls when stray delete was enabled.

This adds back in the Kaltura External Media Entry Compare Attribute fields.

The connector supports remote configurations.

Supports remote configurations.

Uses the SDK that correctly processes the NOT-EQUAL phrase in delete-by-query.

Supports unauthenticated proxy.

The connector allows OAuth 2 support for JWT token.

This release expands the types of character large objects (CLOBs) which can be indexed to include IBM CLOBs.

For SharePoint Online connector OAuth authentication and NTLM authentication, an issue with the SharePoint Optimized V2 connector crawl was fixed. Now the crawl fails if the data source is configured with incorrect credentials.

When the moderation status of a published document moves to unpublished, the document is deleted from the index.

The URL that requests folder metadata is now properly encoded and retrieves requested files correctly.

The datasource indexing job will now return an error when invalid access credentials are provided instead of reporting a success.

The Security parameter is now set to enabled by default.

Fixed an issue that occurred when reaching Kaltura’s API limit of 10,000 documents. Indexing over 10,000 documents should now work as expected.

The JDBC V2 connector now supports indexing character large objects (CLOBs).

A NoClassDefFound error in the connector has been fixed.

MapDB tables that are no longer needed after indexing are now automatically removed when the indexing job completes.

The connector no longer uses the index pipeline client when the onlyMetadata property is set to true or when Tika async parsing is enabled. Instead, documents are processed using the Connectors SDK’s .newDocument method.

A bug with the Rewrite URI configuration option that prevented the configuration from being applied during the indexing job has been fixed.

The SAML, Form, and NTML authorization form fields are updated to match those present in the Web V1 connector.

A bug that affected JDBC datasources with large document sizes has been fixed. Previously, the indexing job would fail and produce a Solr error message similar to the following:

Document contains at least one immense term in field="FIELD_NAME"

Added support for the graph security trimming stage for Active Directory in Azure.

A bug that prevented the connector from retrying after a connection error has been fixed.

The statement.setMaxRows() field was added to resolve a timeout error for large queries on a JDBC data source.

A new field, _lw_document_type_s, has been added to differentiate between document types. SharePoint content documents use the value contentDocument. ACL documents use the value aclDocument.

A bug that caused a NullPointerException error when the listDefaultDisplayFormUrl value is null has been fixed.

A bug that prevented the connector from indexing SharePoint Online URLs containing certain special characters, such as +, &, and #, has been fixed.

A bug that stopped the indexing job unexpectedly when the digestValue request fails has been fixed.

Fixed a memory leak that resulted in an OutOfMemoryError runtime error, causing recrawls to quickly fail.

Fixed an issue where sometimes the connector did not index all attachments in full crawls.

Fixed an error related to OAuth authentication where the body would not return the expected value.

Folders with unique permissions no longer inherit list permissions instead of their own.

Fixed a bug that sometimes resulted in an OutOfMemory: Java Heap Space error when the connector was run remotely.

Fixed a dependency issue that sometimes resulted in a Error starting controllers error that caused the job to fail.

Fixed a bug that removed access control lists (ACLs) during incremental crawls in Azure Active Directory. This bug resulted from a deviation from the LDAP ACLs V2 connector’s unique incremental crawl behavior for Azure Active Directory.

Fixed a bug that sometimes deleted ACLs that the indexing job failed to update during its previous run.

Known issue: The job may complete with a "Successful" status even when a network communication error occurred.

Fixed a bug that sometimes generated a "User cannot be found" error statement and prevented the connector from indexing files.

Updated the description of the Item size limit property to clarify its behavior.

Fixed a bug that sometimes prevented the connector from parsing PDF, Word, Excel, and other file types.

Some conflicting dependencies were removed to resolve errors.

A missing dependency, guava-retrying, was added to resolve a NoClassDefFoundError error.

A bug was fixed that resulted in the connector performing full crawls instead of recrawling as expected.

Known issue: JavaScript evaluation is not functional at this time.

Known issue: The counter values in the Datasources > DATASOURCE_NAME > Job History view do not reflect actual values, when the datasource uses the Web V2 connector.

A bug that sometimes prevented files from being deleted as expected when the datasource was recrawled was fixed.

A bug that prevented the connector from indexing data within certain lists was fixed.

When indexing data to the MetaInfo_s field, an error could occur that prevented indexing data when the data was too large. Now, data larger than 32,766 bytes is indexed to the MetaInfo_txt field instead.

The connector now accepts URLs with spaces.