REST V2Connector Configuration Reference
The REST V2 connector is a way to use one connector for multiple datasources. It works with recipes, which are premade JSON configurations.
An API call is made to the REST V2 connector containing a recipe as the body. API calls to set up the REST V2 connector go through the Connector Datasources API.
This article lists and describes the Fusion properties used by REST V2. The recommended method of populating these fields is through the use of a recipe. For information on how to get a recipe from the public GitHub repository into Fusion, see How to use REST V2 recipes.
The REST V2 connector performs a full crawl each time the connectors runs, as incremental crawling is not supported. When changes are made to the source, such as adding, deleting, or modifying content, those changes will reflect in the index after the next crawl is performed. You can check for new documents in the index by navigating to Querying > Query Workbench in the Fusion UI.
Below is an example configuration showing how to specify the file system to index under the connector-plugins entry in your values.yaml file:
additionalVolumes:
- name: fusion-data1-pvc
persistentVolumeClaim:
claimName: fusion-data1-pvc
- name: fusion-data2-pvc
persistentVolumeClaim:
claimName: fusion-data2-pvc
additionalVolumeMounts:
- name: fusion-data1-pvc
mountPath: "/connector/data1"
- name: fusion-data2-pvc
mountPath: "/connector/data2"
You may also need to specify the user that is authorized to access the file system, as in this example:
securityContext:
fsGroup: 1002100000
runAsUser: 1002100000
Connector to index data from Rest APIs
properties - Rest Connector properties
Plugin specific properties.
serviceURL - string
URL to the service from where extract data
authenticationMode - Authentication
Authentication to the Rest API Service.
basicAuth - Basic Auth
Basic Auth Properties
user - string
user
>= 1 characters
password - string
password
>= 1 characters
customAuth - Custom Auth
Custom Auth Properties
authUrl - string
URL to use for the auth request definition
httpMethod - string
Http Method to use for the request
Default: POST
Allowed values: GETPOST
secureVariables - array[object]
Variables to use within the auth request definition
object attributes:{keyValue : {
display name: Key Name
type: string
}secretValue : {
display name: Secret Value
type: string
}}
headers - array[object]
Headers to use within the auth request
object attributes:{headerKey : {
display name: Header Key
type: string
}headerValue : {
display name: Header Value
type: string
}}
queries - array[object]
Queries to use within the auth request
object attributes:{queryKey : {
display name: Query Key
type: string
}queryValue : {
display name: Query Value
type: string
}}
bodyAuthRequest - string
Body Auth Request in Json Format
serviceEndpoints - array[object]
List of Service Endpoints
object attributes:{rootRequest required : {
display name: Root Endpoint Request
type: object
}childrenRequests : {
display name:
Child Endpoint Request
type: array
}}
httpClientProperties - Http Client Settings
A set of options for configuring the http client.
readTimeoutMs - number
<= 600000
exclusiveMinimum: false
exclusiveMaximum: false
Default: 300000
Multiple of: 1
connectTimeoutMs - number
<= 300000
exclusiveMinimum: false
exclusiveMaximum: false
Default: 60000
Multiple of: 1
id - stringrequired
A unique identifier for this Configuration.
>= 1 characters
Match pattern: ^[a-zA-Z0-9_-]+$
pipelineId - stringrequired
Name of the IndexPipeline used for processing output.
>= 1 characters
Match pattern: ^[a-zA-Z0-9_-]+$
parserId - string
The Parser to use in the associated IndexPipeline.
Match pattern: ^[a-zA-Z0-9_-]+$
description - string
Optional description
<= 125 characters
diagnosticLogging - boolean
Enable diagnostic logging; disabled by default
Default: false
coreProperties - Core Properties
Common behavior and performance settings.
fetchSettings - Fetch Settings
System level settings for controlling fetch behavior and performance.
numFetchThreads - number
Maximum number of fetch threads; defaults to 5. This setting controls the number of threads that call the Connectors fetch method. Higher values can, but not always, help with overall fetch performance.
>= 1
<= 500
exclusiveMinimum: false
exclusiveMaximum: false
Default: 5
Multiple of: 1
indexingThreads - number
Maximum number of indexing threads; defaults to 4. This setting controls the number of threads in the indexing service used for processing content documents emitted by this datasource. Higher values can sometimes help with overall fetch performance.
>= 1
<= 10
exclusiveMinimum: false
exclusiveMaximum: false
Default: 4
Multiple of: 1
pluginInstances - number
Maximum number of plugin instances for distributed fetching. Only specified number of plugin instances will do fetching. This is useful for distributing load between different instances.
<= 500
exclusiveMinimum: false
exclusiveMaximum: false
Default: 0
Multiple of: 1
fetchResponseScheduledTimeout - number
The maximum amount of time for a response to be scheduled. The task will be canceled if this setting is exceeded.
>= 1000
<= 500000
exclusiveMinimum: false
exclusiveMaximum: false
Default: 300000
Multiple of: 1
indexingInactivityTimeout - number
The maximum amount of time to wait for indexing results (in seconds). If exceeded, the job will fail with an indexing inactivity timeout.
>= 60
<= 691200
exclusiveMinimum: false
exclusiveMaximum: false
Default: 86400
Multiple of: 1
pluginInactivityTimeout - number
The maximum amount of time to wait for plugin activity (in seconds). If exceeded, the job will fail with a plugin inactivity timeout.
>= 60
<= 691200
exclusiveMinimum: false
exclusiveMaximum: false
Default: 600
Multiple of: 1
indexMetadata - boolean
When enabled the metadata of skipped items will be indexed to the content collection.
Default: false
indexContentFields - boolean
When enabled, content fields will be indexed to the crawl-db collection.
Default: false
asyncParsing - boolean
When enabled, content will be indexed asynchronously.
Default: false