Develop a Custom Connector

Java SDK configuration

To build a valid connector configuration, you must:

  • Define an interface.

  • Extend ConnectorConfig.

  • Apply a few annotations.

  • Define connector methods and annotations.

All methods that are annotated with @Property are considered to be configuration properties. For example, @Property() String name(); results in a String property called name. This property would then be present in the generated schema.

Here is an example of the most basic configuration, along with required annotations:

@RootSchema(
    title = "My Connector",
    description = "My Connector description",
    category = "My Category"
)
public interface MyConfig extends ConnectorConfig<MyConfig.Properties> {

  @Property(
      title = "Properties",
      required = true
  )
  public Properties properties();

  /**
   * Connector specific settings
   */
  interface Properties extends FetcherProperties {

    @Property(
        title = "My custom property",
        description = "My custom property description"
    )
    public Integer myCustomProperty();

  }

}

The metadata defined by @RootSchema is used by Fusion when showing the list of available connectors. The ConnectorConfig base interface represents common, top-level settings required by all connectors. The type parameter of the ConnectorConfig class indicates the interface to use for custom properties.

Once a connector configuration has been defined, it can be associated with the ConnectorPlugin class. From that point, the framework takes care of providing the configuration instances to your connector. It also generates the schema, and sends it along to Fusion when it connects to Fusion.

Schema metadata can be applied to properties using additional annotations. For example, applying limits to the min/max length of a string, or describing the types of items in an array.

Nested schema metadata can also be applied to a single field by using "stacked" schema based annotations:

interface MySetConfig extends Model {
    @SchemaAnnotations.Property(title = "My Set")
    @SchemaAnnotations.ArraySchema(defaultValue = "[\"a\"]")
    @SchemaAnnotations.StringSchema(defaultValue = "some-set-value", minLength = 1, maxLength = 1)
    Set<String> mySet();
  }

Plugin client

The Fusion connector plugin client provides a wrapper for the Fusion Java plugin-sdk so that plugins do not need to directly talk with gRPC code. Instead, they can use high-level interfaces and base classes, like Connector and Fetcher.

The plugin client also provides a standalone "runner" that can host a plugin that was built from the Fusion Java Connector SDK. It does this by loading the plugin zip file, then calling on the wrapper to provide the gRPC interactions.

Java SDK / gRPC Wrapper

One of the primary goals of the plugin-client is to isolate plugin code from the underlying framework details. Specifically, the underlying message formats (protocol buffers) and native gRPC code. This makes it possible to make some changes to the base support layer, without having to make changes to the Java plugin implementation.

Standalone Connector Plugin Application

The second goal of the plugin-client is to allow Java SDK plugins to run remotely. The instructions for deploying a connector using this method are provided below.

Locating the UberJar

The uberjar is located in this location in the Fusion file system:

$FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar

where $FUSION_HOME is your Fusion installation directory and <version> is your Fusion version number.

Starting the Host

To start the host app, you need a Fusion SDK-based connector, built into the standard packaging format as a .zip file. This zip must contain only one connector plugin.

Here is an example of how to start up using the web connector:

java -jar $FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar fusion-connectors/build/plugins/connector-web-4.0.0-SNAPSHOT.zip

To run the client with remote debugging enabled:

java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5010 -jar $FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar fusion-connectors/build/plugins/connector-web-4.0.0-SNAPSHOT.zip

Java SDK security

See the main custom connector security overview for instructions on how to configure SSL/TLS support for a Fusion custom connector.

Java Plugin Client

The information here is specific to running a Java SDK plugin outside of Fusion.

The plugin-client supports several variations of SSL/TLS auth. The examples below show the relevant Java properties.

Example with Mutual TLS auth and private key passwords

-Dcom.lucidworks.apollo.app.hostname=myhost
-Dcom.lucidworks.fusion.tls.trustCertCollection=./sslcerts/ca.crt
-Dcom.lucidworks.fusion.tls.client.certChain=./sslcerts/client.crt
-Dcom.lucidworks.fusion.tls.client.privateKey=./sslcerts/client.pem
-Dcom.lucidworks.fusion.tls.client.privateKeyPassword=password123
-Dcom.lucidworks.fusion.tls.enabled=true</td>

Example without TLS auth and no private key passwords:

-Dcom.lucidworks.apollo.app.hostname=myhost
-Dcom.lucidworks.fusion.tls.trustCertCollection=./sslcerts/ca.crt
-Dcom.lucidworks.fusion.tls.enabled=true