Connector Datasources API

The connector datasources API is used to create and configure datasources. See Connectors and Datasources for general information, Connectors and Datasource Reference for details on configuring specific datasources.

List a Datasource or All Datasources

The path for this request is:

/api/apollo/connectors/datasources?collection=<collectionName>

A GET request will return the definitions for all datasources for that collection. If the collection parameter is omitted, all datasource definitions will be returned.

Input

None.

Output

The output will include all datasources that match the request, with all properties.

Examples

Note
Use port 8765 in local development environments only. In production, use port 8764.

Get datasources assigned to the "demo" collection:

REQUEST

curl -u user:pass http://localhost:8764/api/apollo/connectors/datasources?collection=demo

RESPONSE

[ {
  "id" : "database",
  "created" : "2014-05-04T19:47:22.867Z",
  "modified" : "2014-05-04T19:47:22.867Z",
  "connector" : "lucid.jdbc",
  "type" : "jdbc",
  "description" : null,
  "pipeline" : "conn_solr",
  "properties" : {
    "db" : null,
    "commit_on_finish" : true,
    "verify_access" : true,
    "sql_select_statement" : "select CONTENTID as id from CONTENT;",
    "debug" : false,
    "collection" : "demo",
    "password" : "password",
    "url" : "jdbc:postgresql://localhost:5432/db",
    "nested_queries" : null,
    "clean_in_full_import_mode" : true,
    "username" : "user",
    "delta_sql_query" : null,
    "commit_within" : 900000,
    "primary_key" : null,
    "driver" : "org.postgresql.Driver",
    "max_docs" : -1
  }
} ]

Create or Update a Datasource

A POST request will create a new datasource. The path for this request is:

/api/apollo/connectors/datasources

A PUT request will update an existing datasource. The path for this request is:

/api/apollo/connectors/datasources/<id>

where <id> is the name of the datasource to be updated.

Input

Property Description

id
Required

A unique identifier for this datasource. When creating a datasource, this property is required.

connector
Required

The connector that will be used for this datasource.

type
Required

The type of datasource, which must be supported by the defined connector.

description
Optional

A description of the datasource.

pipeline
Optional

The index pipeline that should be used for this datasource.

properties
Required

The properties for the datasource. The available properties will depend on the connector and type chosen.

Not all available properties are required, but all datasources have at least one property that must be set to create the datasource.

Output

A successful request returns the datasource definition. A failed request returns a JSON-formatted error message.

Examples

Create a datasource to index Solr-formatted XML files:

REQUEST

curl -u user:pass -X POST -H 'Content-type: application/json' -d '{"id":"SolrXML", "connector":"lucid.solrxml", "type":"solrxml", "properties":{"path":"/Applications/solr-4.10.2/example/exampledocs", "generate_unique_key":false, "collection":"MyCollection"}}' http://localhost:8764/api/apollo/connectors/datasources

RESPONSE

{
  "id" : "SolrXML",
  "created" : "2015-05-18T15:47:51.199Z",
  "modified" : "2015-05-18T15:47:51.199Z",
  "connector" : "lucid.solrxml",
  "type" : "solrxml",
  "properties" : {
    "commit_on_finish" : true,
    "verify_access" : true,
    "generate_unique_key" : false,
    "collection" : "MyCollection",
    "include_datasource_metadata" : true,
    "include_paths" : [ ".*\\.xml" ],
    "initial_mapping" : {
      "id" : "a35c9ff3-dbb6-434b-af40-597722c2986a",
      "skip" : false,
      "label" : "field-mapping",
      "type" : "field-mapping"
    },
    "path" : "/Applications/apache-repos/solr-4.10.2/example/exampledocs",
    "exclude_paths" : [ ],
    "url" : "file:/Applications/apache-repos/solr-4.10.2/example/exampledocs/",
    "max_docs" : -1
  }
}

Change the max_docs value for the above datasource:

REQUEST

curl -u user:pass -X PUT -H 'Content-type: application/json' -d '{"id":"SolrXML", "connector":"lucid.solrxml", "type":"solrxml", "properties":{"path":"/Applications/solr-4.10.2/example/exampledocs", "max_docs":10}}' http://localhost:8764/api/apollo/connectors/datasources/SolrXML

RESPONSE

true

Delete a Datasource or All Datasources

The path for a DELETE request is:

/api/apollo/connectors/datasources/<id>

where <id> is the name of the datasource to be deleted. If the id is omitted, all datasources will be deleted. The delete request takes one parameter:

Parameter Description

force

If false, the default, the delete will wait for any running datasource jobs to complete before deleting the datasource. If you want to abort jobs before deleting, you can change this to true.

Input

None.

Output

The request succeeds silently. A failed request returns a JSON-formatted error message.

Examples

Delete the datasource named 'database':

REQUEST

curl -u user:pass -X DELETE http://localhost:8764/api/apollo/connectors/datasources/database

RESPONSE

If successful, no response.