Managing Connectors

Installing a connector

Connectors are installed by uploading them to the blob store. You can do this using the Fusion UI or the Blob Store API. Both methods are explained below.

Installing a connector using the Fusion UI

  1. Download the connector zip file from http://lucidworks.com/connectors/.

    Do not expand the archive; Fusion consumes it as-is.

  2. In the Fusion workspace, navigate to System > Blobs.

  3. Click Add.

  4. Select Connector Plugin.

    Add a connector

    The "New Connector Plugin Upload" panel appears.

  5. Click Choose File and select the downloaded zip file from your file system.

    Upload a connector

  6. Click Upload.

    The new connector’s blob manifest appears.

    Uploaded connector

    From this screen you can also delete or replace the connector.

Installing a connector using the API

  1. Download the connector zip file from http://lucidworks.com/connectors/.

    Do not expand the archive; Fusion consumes it as-is.

  2. Upload the connector zip file to Fusion’s blob store.

    Specify an arbitrary blob ID, and a resourceType value of plugin:connector, as in this example:

    curl -H 'content-type:application/zip' -X PUT 'localhost:8764/api/blobs/myplugin?resourceType=plugin:connector' —data-binary @myplugin.zip

    Fusion automatically publishes the event to the cluster, and the listeners perform the connector installation process on each node.

    Tip
    If the blob ID is identical to an existing one, the old connector will be uninstalled and the new connector will installed in its place. To get the list of existing blob IDs, run: curl -u user:pass localhost:8764/api/blobs
  3. Look in fusion/4.1.x/apps/connectors/plugins/ to verify that the new connector is installed.

Updating a connector

On Unix, you can update a connector by simply uploading the new one. Fusion overwrites the old one, and no restart is needed.

On Windows, a different procedure is needed:

How to update a Fusion connector on Windows
  1. Delete the old connector, as explained below.

  2. Restart Fusion.

  3. Upload the new connector.

Deleting a connector

You can delete a connector using the Fusion UI or the Blob Store API.

Deleting a connector using the Fusion UI

  1. In the Fusion UI, navigate to System > Blobs.

  2. Under Connector Plugin, select the connector to delete.

  3. Click Delete Blob.

    Delete a connector

    Fusion prompts you to confirm that you want to delete the blob.

  4. Click Yes, Delete.

    The connector disappears from the blob list.

Deleting a connector using the REST API

  1. Get the list of blobs of the connector plugin type:

    curl -u user:pass http://localhost:8764/api/blobs?resouType=plugin:connector
  2. Locate the connector you want to delete, and copy its ID.

    For example, the Jive connector ID is lucid.jive:

    {
      "name" : "lucid.jive",
      "contentType" : "application/zip",
      "size" : 125302,
      "modifiedTime" : "2017-06-13T17:49:20.171Z",
      "version" : 1570112704530612224,
      "md5" : "7032bf2c038bb2d1e27aee82c056c0fb",
      "metadata" : {
        "connectorBootstrapPluginName" : "lucid.jive",
        "resourceType" : "plugin:connector"
      }
    }
  3. Delete the connector as follows:

    curl -u user:pass -X DELETE http://localhost:8764/api/blobs/<id>

    For example

    curl -u user:pass -X DELETE http://localhost:8764/api/blobs/lucid.jive

    A null response indicates success. You can verify that the connector is deleted like this:

    curl -u user:pass http://localhost:8764/api/blobs | grep lucid.jive

Connector configuration

When you add a datasource to a collection, you select a connector and configure it. There are two ways to do this:

  • Using the API

  • Using the UI

Configuring Connectors Using The API

You can create or update a datasource with the Connector Datasources API, specifying the connector, its properties, and their values.

Example: Create and configure a datasource to index Solr-formatted XML files
curl -u user:pass -X POST -H 'Content-type: application/json' -d '{"id":"SolrXML", "connector":"lucid.solrxml", "type":"solrxml", "properties":{"path":"/Applications/solr-4.10.2/example/exampledocs", "generate_unique_key":false, "collection":"MyCollection"}}' http://localhost:8764/api/connectors/datasources

See the Connectors and Datasources Reference for details about configuration options.

Tip
Be sure the include the collection property; otherwise the datasource will not be available in the Fusion UI.
Example: Change the max_docs value for the above datasource
curl -u user:pass -X PUT -H 'Content-type: application/json' -d '{"id":"SolrXML", "connector":"lucid.solrxml", "type":"solrxml", "properties":{"path":"/Applications/solr-4.10.2/example/exampledocs", "max_docs":10}}' http://localhost:8764/api/connectors/datasources/SolrXML

Configuring Connectors Using The Fusion UI

  • To create and configure a new datasource and its connector:

    1. In the Fusion workspace, navigate to Indexing > Index Workbench.

    2. Click New.

    3. Select Or, if you know the location of your data.

    4. In the dropdown list, select the datasource type, which corresponds to a Fusion connector.

    5. Click Add New Datasource.

    6. Edit the configuration fields in the datasource panel that appears.

      See the Connectors and Datasources Reference for details about configuration options.

    7. Click Apply to test your configuration.

      Make changes as needed until the preview panel displays your data as it should be indexed.

    8. Click Save.

  • To change the connector configuration for an existing datasource:

    1. In the Fusion workspace, navigate to Indexing > Index Workbench.

    2. Click Load and select the datasource you want to change.

    3. Edit the configuration fields as needed.

    4. Click Apply to test your configuration.

      Make changes as needed until the preview panel displays your data as it should be indexed.

    5. Click Save.

Connector logs

You can find connector logs in fusion/4.1.x/var/log/connectors.