Product Selector

Fusion 5.9
    Fusion 5.9

    Develop a Custom Connector

    Java SDK configuration

    To build a valid connector configuration, you must:

    • Define an interface.

    • Extend ConnectorConfig.

    • Apply a few annotations.

    • Define connector methods and annotations.

    All methods that are annotated with @Property are considered to be configuration properties. For example, @Property() String name(); results in a String property called name. This property would then be present in the generated schema.

    Here is an example of the most basic configuration, along with required annotations:

    @RootSchema(
    title = "My Connector",
    description = "My Connector description",
    category = "My Category"
)
public interface MyConfig extends ConnectorConfig<MyConfig.Properties> {

  @Property(
      title = "Properties",
      required = true
  )
  public Properties properties();

  /**
   * Connector specific settings
   */
  interface Properties extends FetcherProperties {

    @Property(
        title = "My custom property",
        description = "My custom property description"
    )
    public Integer myCustomProperty();

  }

}

    The metadata defined by @RootSchema is used by Fusion when showing the list of available connectors. The ConnectorConfig base interface represents common, top-level settings required by all connectors. The type parameter of the ConnectorConfig class indicates the interface to use for custom properties.

    Once a connector configuration has been defined, it can be associated with the ConnectorPlugin class. From that point, the framework takes care of providing the configuration instances to your connector. It also generates the schema, and sends it along to Fusion when it connects to Fusion.

    Schema metadata can be applied to properties using additional annotations. For example, applying limits to the min/max length of a string, or describing the types of items in an array.

    Nested schema metadata can also be applied to a single field by using "stacked" schema based annotations:

    interface MySetConfig extends Model {
    @SchemaAnnotations.Property(title = "My Set")
    @SchemaAnnotations.ArraySchema(defaultValue = "[\"a\"]")
    @SchemaAnnotations.StringSchema(defaultValue = "some-set-value", minLength = 1, maxLength = 1)
    Set<String> mySet();
  }

    Plugin client

    The Fusion connector plugin client provides a wrapper for the Fusion Java plugin-sdk so that plugins do not need to directly talk with gRPC code. Instead, they can use high-level interfaces and base classes, like Connector and Fetcher.

    The plugin client also provides a standalone "runner" that can host a plugin that was built from the Fusion Java Connector SDK. It does this by loading the plugin zip file, then calling on the wrapper to provide the framework interactions.

    Standalone Connector Plugin Application

    The second goal of the plugin-client is to allow Java SDK plugins to run remotely. The instructions for deploying a connector using this method are provided below.

    Locating the UberJar

    The uberjar is located in this location in the Fusion file system:

    $FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar

    where $FUSION_HOME is your Fusion installation directory and <version> is your Fusion version number.

    Starting the Host

    To start the host app, you need a Fusion SDK-based connector, built into the standard packaging format as a .zip file. This zip must contain only one connector plugin.

    Here is an example of how to start up using the web connector:

    java -jar $FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar fusion-connectors/build/plugins/connector-web-4.0.0-SNAPSHOT.zip

    To run the client with remote debugging enabled:

    java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5010 -jar $FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar fusion-connectors/build/plugins/connector-web-4.0.0-SNAPSHOT.zip

    Java SDK security

    Fusion connectors support SSL/TLS.

    Java Plugin Client

    The information here is specific to running a Java SDK plugin outside of Fusion.

    The Fusion Connector Plugin Client supports several variations of SSL/TLS auth. The examples below show the relevant Java properties.

    Example with Mutual TLS auth and private key passwords

    -Dcom.lucidworks.apollo.app.hostname=myhost
-Dcom.lucidworks.fusion.tls.trustCertCollection=./sslcerts/ca.crt
-Dcom.lucidworks.fusion.tls.client.certChain=./sslcerts/client.crt
-Dcom.lucidworks.fusion.tls.client.privateKey=./sslcerts/client.pem
-Dcom.lucidworks.fusion.tls.client.privateKeyPassword=password123
-Dcom.lucidworks.fusion.tls.enabled=true</td>

    Example without TLS auth and no private key passwords:

    -Dcom.lucidworks.apollo.app.hostname=myhost
-Dcom.lucidworks.fusion.tls.trustCertCollection=./sslcerts/ca.crt
-Dcom.lucidworks.fusion.tls.enabled=true

    Simple Connector

    Connector Description

    This connector generates a configurable number of documents, all with random titles and body fields.

    Quick start

    1. Clone the repo:

      git clone https://github.com/lucidworks/connectors-sdk-resources.git
cd connectors-sdk-resources/java-sdk/connectors/
./gradlew assemblePlugins

    1. This produces one zip file, named simple-connector.zip, located in the build/plugins directory. This artifact is now ready to be uploaded directly to Fusion as a connector plugin.

    2. See the following instructions on how to build, deploy, and run the plugin.

    Connector properties

    Random content generator properties

    Property Name Property description

    Total

    The total number of documents to generate

    Minimum number of sentences

    The minimum number of sentences to generate per document, the random generator will use this value as lower bound to calculate a random number of sentences

    maximum number of sentences

    The maximum number of sentences to generate per document, the random generator will use this value as upper bound to calculate a random number of sentences

    How to use the connector

    • Create a configuration with the properties listed above.

    • After the first job is completed, the connector will index the same number of documents as defined in the Random content generator properties.Total property.