Twitter Search Connector Configuration Reference

Configuration

The Twitter Search connector uses Twitter’s search API to query Twitter for tweets that match specific parameters. It allows querying for any keyword, location or other query terms.

Configuration

Tip	When entering configuration values in the UI, use unescaped characters, such as `\t` for the tab character. When entering configuration values in the API, use escaped characters, such as `\\t` for the tab character.

Property Description

access_token

Access Token

required

The OAuth Access Token is provided by Twitter when registering the application.

type: string

minLength: 1

collection

Collection

Collection documents will be indexed to.

type: string

pattern: ^[a-zA-Z0-9_-]+$

commit_on_finish

Solr commit on finish

Set to true for a request to be sent to Solr after the last batch has been fetched to commit the documents to the index.

type: boolean

default value: 'true'

consumer_key

Consumer Key

required

The OAuth Consumer Key is provided by Twitter when registering the application.

type: string

minLength: 1

consumer_secret

Consumer Secret

required

The OAuth Consumer Secret is provided by Twitter when registering the application.

type: string

minLength: 1

lang

Language

Restrict Tweets to the given language

type: string

default value: 'en'

enum: { aa ab ae af ak am an ar as av ay az ba be bg bh bi bm bn bo br bs ca ce ch co cr cs cu cv cy da de dv dz ee el en eo es et eu fa ff fi fj fo fr fy ga gd gl gn gu gv ha he hi ho hr ht hu hy hz ia id ie ig ii ik in io is it iu iw ja ji jv ka kg ki kj kk kl km kn ko kr ks ku kv kw ky la lb lg li ln lo lt lu lv mg mh mi mk ml mn mo mr ms mt my na nb nd ne ng nl nn no nr nv ny oc oj om or os pa pi pl ps pt qu rm rn ro ru rw sa sc sd se sg si sk sl sm sn so sq sr ss st su sv sw ta te tg th ti tk tl tn to tr ts tt tw ty ug uk ur uz ve vi vo wa wo xh yi yo za zh zu }

max_docs

Max Documents

The maximum number of documents to pull down, as a long. -1 for no limit

type: integer

default value: '-1'

queries

Queries

A list of queries to perform

type: array of string

sleep

Sleep

The amount of time, in milliseconds, to sleep when listening so as to not get throttled

type: integer

default value: '10000'

token_secret

Token Secret

required

The OAuth Token Secret is provided by Twitter when registering the application.

type: string

minLength: 1

Field Mapping

Initial Mappings Description

mappings

Field Mappings

List of mapping rules

type: array of object

object attributes: {
  operation : {
    display name: Operation
    type: string
    default value: 'copy'
    description : The type of mapping to perform: move, copy, delete, add, set, or keep.
    enum: { copy move delete set add keep     }

    }
  source (required) : {
    display name: Source Field
    type: string
    description : The name of the field to be mapped.
    }
  target : {
    display name: Target Field
    type: string
    description : The name of the field to be mapped to.
    }
  }

reservedFieldsMappingAllowed

Allow System Fields Mapping?

type: boolean

default value: 'false'

unmapped

Unmapped Fields

If fields do not match any of the field mapping rules, these rules will apply.

type: object

    }
  source : {
    display name: Source Field
    type: string
    description : The name of the field to be mapped.
    }
  target : {
    display name: Target Field
    type: string
    description : The name of the field to be mapped to.
    }
  }

ConnectorDb Configuration

Property Description

aliases

Process Aliases?

Keep track of original URI-s that resolved to the current URI. This negatively impacts performance and size of DB.

type: boolean

default value: 'false'

inlinks

Process Inlinks?

Keep track of incoming links. This negatively impacts performance and size of DB.

type: boolean

default value: 'false'

inv_aliases

Process Inverted Aliases?

Keep track of target URI-s that the current URI resolves to. This negatively impacts performance and size of DB.

type: boolean

default value: 'false'

type

Implementation Class Name

Fully qualified class name of ConnectorDb implementation.

type: string

default value: 'com.lucidworks.connectors.db.impl.MapDbConnectorDb'

minLength: 1

General Configuration

Property

Description

description

Description

Optional description for datasource instance.

Datasource ID

Unique name for datasource instance.

parserId

Parser

The parser used to process raw content.

pipeline

Pipeline ID

The index pipeline used to process documents.