Develop a Custom Connector
Java SDK configuration
To build a valid connector configuration, you must:
-
Define an interface.
-
Extend
ConnectorConfig. -
Apply a few annotations.
-
Define connector methods and annotations.
All methods that are annotated with @Property are considered to be configuration properties.
For example, @Property() String name(); results in a String property called name.
This property would then be present in the generated schema.
Here is an example of the most basic configuration, along with required annotations:
@RootSchema(
title = "My Connector",
description = "My Connector description",
category = "My Category"
)
public interface MyConfig extends ConnectorConfig<MyConfig.Properties> {
@Property(
title = "Properties",
required = true
)
public Properties properties();
/**
* Connector specific settings
*/
interface Properties extends FetcherProperties {
@Property(
title = "My custom property",
description = "My custom property description"
)
public Integer myCustomProperty();
}
}The metadata defined by @RootSchema is used by Fusion when showing the list of available connectors.
The ConnectorConfig base interface represents common, top-level settings required by all connectors.
The type parameter of the ConnectorConfig class indicates the interface to use for custom properties.
Once a connector configuration has been defined, it can be associated with the ConnectorPlugin class.
From that point, the framework takes care of providing the configuration instances to your connector.
It also generates the schema, and sends it along to Fusion when it connects to Fusion.
Schema metadata can be applied to properties using additional annotations. For example, applying limits to the min/max length of a string, or describing the types of items in an array.
Nested schema metadata can also be applied to a single field by using "stacked" schema based annotations:
interface MySetConfig extends Model {
@SchemaAnnotations.Property(title = "My Set")
@SchemaAnnotations.ArraySchema(defaultValue = "[\"a\"]")
@SchemaAnnotations.StringSchema(defaultValue = "some-set-value", minLength = 1, maxLength = 1)
Set<String> mySet();
}Plugin client
The Fusion connector plugin client provides a wrapper for the Fusion Java plugin-sdk so that plugins do not need to directly talk with gRPC code. Instead, they can use high-level interfaces and base classes, like Connector and Fetcher.
The plugin client also provides a standalone "runner" that can host a plugin that was built from the Fusion Java Connector SDK. It does this by loading the plugin zip file, then calling on the wrapper to provide the framework interactions.
Standalone Connector Plugin Application
The second goal of the plugin-client is to allow Java SDK plugins to run remotely. The instructions for deploying a connector using this method are provided below.
Locating the UberJar
The uberjar is located in this location in the Fusion file system:
$FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jarwhere $FUSION_HOME is your Fusion installation directory and <version> is your Fusion version number.
Starting the Host
To start the host app, you need a Fusion SDK-based connector, built into the standard packaging format as a .zip file. This zip must contain only one connector plugin.
Here is an example of how to start up using the web connector:
java -jar $FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar fusion-connectors/build/plugins/connector-web-4.0.0-SNAPSHOT.zipTo run the client with remote debugging enabled:
java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5010 -jar $FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar fusion-connectors/build/plugins/connector-web-4.0.0-SNAPSHOT.zipJava SDK security
Fusion connectors support SSL/TLS.
Java Plugin Client
The information here is specific to running a Java SDK plugin outside of Fusion.
The Fusion Connector Plugin Client supports several variations of SSL/TLS auth. The examples below show the relevant Java properties.
Example with Mutual TLS auth and private key passwords
-Dcom.lucidworks.apollo.app.hostname=myhost
-Dcom.lucidworks.fusion.tls.trustCertCollection=./sslcerts/ca.crt
-Dcom.lucidworks.fusion.tls.client.certChain=./sslcerts/client.crt
-Dcom.lucidworks.fusion.tls.client.privateKey=./sslcerts/client.pem
-Dcom.lucidworks.fusion.tls.client.privateKeyPassword=password123
-Dcom.lucidworks.fusion.tls.enabled=true</td>Example without TLS auth and no private key passwords:
-Dcom.lucidworks.apollo.app.hostname=myhost
-Dcom.lucidworks.fusion.tls.trustCertCollection=./sslcerts/ca.crt
-Dcom.lucidworks.fusion.tls.enabled=trueSimple Connector
Connector Description
This connector generates a configurable number of documents, all with random titles and body fields.
Quick start
-
Clone the repo:
git clone https://github.com/lucidworks/connectors-sdk-resources.git cd connectors-sdk-resources/java-sdk/connectors/ ./gradlew assemblePlugins
-
This produces one zip file, named
simple-connector.zip, located in thebuild/pluginsdirectory. This artifact is now ready to be uploaded directly to Fusion as a connector plugin. -
See the following instructions on how to build, deploy, and run the plugin.
Connector properties
Random content generator properties
| Property Name | Property description |
|---|---|
Total |
The total number of documents to generate |
Minimum number of sentences |
The minimum number of sentences to generate per document, the random generator will use this value as lower bound to calculate a random number of sentences |
maximum number of sentences |
The maximum number of sentences to generate per document, the random generator will use this value as upper bound to calculate a random number of sentences |
How to use the connector
-
Create a configuration with the properties listed above.
-
After the first job is completed, the connector will index the same number of documents as defined in the
Random content generator properties.Totalproperty.