Importing Data with Connectors

Connectors are the built-in mechanism for pulling your data into Fusion. Fusion comes with a wide variety of connectors, each specialized for a particular data type. When you add a datasource to a collection, you specify the connector to use for ingesting data. See Connector Types for a complete list of connectors, with links to configuration reference information for each one.

Connector configuration

When you add a datasource to a collection, you select a connector and configure it. There are two ways to do this:

  • Using the API

  • Using the UI

Configuring Connectors Using The API

You can create or update a datasource with the Connector Datasources API, specifying the connector, its properties, and their values.

Example: Create and configure a datasource to index Solr-formatted XML files 
curl -u user:pass -X POST -H 'Content-type: application/json' -d '{"id":"SolrXML", "connector":"lucid.solrxml", "type":"solrxml", "properties":{"path":"/Applications/solr-4.10.2/example/exampledocs", "generate_unique_key":false, "collection":"MyCollection"}}' http://localhost:8764/api/apollo/connectors/datasources

See the Connectors and Datasources Reference for details about configuration options.

Tip
 Be sure the include the collection property; otherwise the datasource will not be available in the Fusion UI.
Example: Change the max_docs value for the above datasource 
curl -u user:pass -X PUT -H 'Content-type: application/json' -d '{"id":"SolrXML", "connector":"lucid.solrxml", "type":"solrxml", "properties":{"path":"/Applications/solr-4.10.2/example/exampledocs", "max_docs":10}}' http://localhost:8764/api/apollo/connectors/datasources/SolrXML

Configuring Connectors Using The Fusion UI

  • To create and configure a new datasource and its connector:

    1. Click Applications > Collections.

    2. Select a Collection, or click Add a Collection to create a new one.

    3. Click Add a Datasource.

    4. Select a datasource type; these correspond to Fusion’s Connectors.

    5. Edit the configuration fields in the datasource panel that appears.

      See the Connectors and Datasources Reference for details about configuration options.

    6. Click Save.

  • To change the connector configuration for an existing datasource:

    1. Navigate to Applications > Collections.

    2. Click your collection name.

    3. Click the datasource you want to change.

    4. Edit the configuration fields as needed.

    5. Click Save.