Fusion Connectors

Note
The steps for installing a connector have changed in newer releases; see Install A Connector - 5.2 and Above or Install A Connector - Fusion 5.1.

In Fusion 4.x, connectors are installed by uploading them to the blob store. You can use any of these methods to install a connector:

  • By installing connectors as "bootstrap plugins", that is, by putting them in the bootstrap-plugins directory during initial installation or an upgrade

  • By using the Fusion UI after installation or an upgrade

  • By using the Blob Store API after installation or an upgrade.

Note
During upgrades, the migrator handles some aspects of installing connectors. Depending on the target version and the presence or absence of an Internet connection, there might be manual steps. Installing connectors during upgrades is explained where needed in the upgrade procedures.

After you install a connector, you can Configure A New Datasource.

Tip
You can view and download all current and previous V2 connector releases at plugins.lucidworks.com.

Installing a connector as a bootstrap plugin

Fusion can install connectors as "bootstrap plugins." All this means is that you put the connector zip files in a specific directory named bootstrap-plugins, and Fusion installs the connectors the first time it starts during initial installation or an upgrade.

How to install a connector as a bootstrap plugin
  1. Download the connector zip file.

    Don’t expand the archive; Fusion consumes it as-is. Also, don’t start Fusion until instructed to do so by the installation or upgrade instructions.

  2. Under the version-numbered Fusion directory, place the connector in the directory apps/connectors/bootstrap-plugins/ (on Unix) or \apps\connectors\bootstrap-plugins\ (on Windows).

  3. Start Fusion when instructed to do so in the installation or upgrade procedure.

Installing a connector using the Fusion UI

  1. Download the connector zip file.

    Do not expand the archive; Fusion consumes it as-is.

  2. In the Fusion workspace, navigate to System > Blobs.

  3. Click Add.

  4. Select Connector Plugin.

    Add a connector

    The "New Connector Plugin Upload" panel appears.

  5. Click Choose File and select the downloaded zip file from your file system.

    Upload a connector

  6. Click Upload.

    The new connector’s blob manifest appears.

    Uploaded connector

    From this screen you can also delete or replace the connector.

Installing a connector using the Blob Store API

  1. Download the connector zip file.

    Do not expand the archive; Fusion consumes it as-is.

  2. Upload the connector zip file to Fusion’s blob store.

    Specify an arbitrary blob ID, and a resourceType value of plugin:connector, as in this example:

    curl -H 'content-type:application/zip' -X PUT 'localhost:8764/api/blobs/myplugin?resourceType=plugin:connector' --data-binary @myplugin.zip

    Fusion automatically publishes the event to the cluster, and the listeners perform the connector installation process on each node.

    Tip
    If the blob ID is identical to an existing one, the old connector will be uninstalled and the new connector will installed in its place. To get the list of existing blob IDs, run: curl -u user:pass localhost:8764/api/blobs
  3. Look in fusion/5.0.x/apps/connectors/plugins/ to verify that the new connector is installed.

  1. In your Fusion app, navigate to Indexing > Datasources.

  2. Click Add.

  3. Select your connector.

    The connector configuration panel appears. The specific configuration options vary depending on the connector.

    Create a new datasource

    If you don’t see your connector in the list, you may need to install it:

  4. Configure your connector’s options:

    1. Enter a useful name for your datasource in the Datasource ID field.

    2. You can leave the Pipeline ID and Parser fields as-is to start with the defaults, or select different ones if you have them.

    3. Configure the connector-specific options.

      Refer to your connector’s configuration reference topic for details about each option.

  5. Click Save.

  6. Test your datasource configuration:

    1. Navigate to the Index Workbench at Indexing > Index Workbench.

    2. Click Load…​.

    3. Select the datasource ID you specified when you created the datasource.

      Now you can see your datasource configuration and a simulation of the results you can expect when you run this datasource job to index your data.

      Open a datasource in the Index Workbench
    4. Adjust the configurations of your datasource, parsers, and index pipeline until the simulated results are satisfactory.

    5. Click Save.

  7. Index your data:

    1. In the Index Workbench, click Start Job.

      Start Job

    2. When the job status is Finished, navigate to the Query Workbench at Querying > Query Workbench to view the indexed documents and configure your query pipeline.

Lucidworks provides a Connectors SDK in a public repository on GitHub with all the resources you need to develop custom connectors. Clone the repository to get started:

git clone https://github.com/lucidworks/connectors-sdk-resources

It includes the Random Content connector, which generates random documents and serves as an example project that you can use as a model for your own custom connector projects.

Currently, only Java is supported. See the Javadocs.

These topics provide details about developing custom connectors for Fusion: