Apache Tika Parser Stage

Configuration
- General parser stage properties
- Apache Tika stage-specific properties

Apache Tika is a versatile parser that supports many types of unstructured document formats, such as HTML, PDF, Microsoft Office documents, OpenOffice, RTF, audio, video, images, and more. A complete list of supported formats is available at http://tika.apache.org/.

Tip	When entering configuration values in the UI, use unescaped characters, such as `\t` for the tab character. When entering configuration values in the API, use escaped characters, such as `\\t` for the tab character.

Configuration

General parser stage properties

Property Description, Type

enableMediaTypeDetection

Enable automatic media type detection

Automatically detect the Content-Type of each document; disable this to use `application/octet-stream`.

type: boolean

default value: 'true'

hidden

Hidden

Objects marked as hidden will only be returned in the API with hidden=true

type: boolean

Parser ID

type: string

default value: '5ab1464a-9445-4f20-bc4d-8c1e4685c867'

maxLength: 128

pattern: ^[A-Za-z0-9_\-]+$

idField

Document ID Source Field

A document field to use as the document ID

type: string

maxFieldLength

Maximum Document Field Length

Maximum allowed document field length in bytes. Field values exceeding this limit will be truncated.

type: integer

default value: '1048576'

exclusiveMaximum: false

exclusiveMinimum: false

maximum: 2147483647

minimum: -1

maxParserDepth

Maximum Parser Recursion Depth

Maximum number of times a parser may recurse over any document before proceeding to the next parser.

type: integer

default value: '16'

exclusiveMinimum: false

minimum: 0

Apache Tika stage-specific properties

Property Description, Type

Property	Description, Type
addFailedDocs Add failed documents	type: `boolean` default value: '`false`'
addOriginalContent Add original document content (raw bytes)	type: `boolean` default value: '`false`'
contentEncoding Content transport encoding of the content (per RFC1341)	type: `string` default value: '`binary`' enum: { binary base64 }
enabled Enable this Parser Stage	type: `boolean` default value: '`true`'
errorHandling Error Handling	type: `string` default value: '`mark`' enum: { ignore log fail mark }
excludeContentTypes Content types to exclude	List of content types to exclude from parsing type: `array` of `string`
extractHtmlLinks Extract XHTML links	Collect links explicitly declared in document structure (e.g. using HTML tags, bookmarks, etc) type: `boolean` default value: '`true`'
extractOtherLinks Extract other links	Use regex-based heuristic extractor to collect likely links from plain text content in all fields. type: `boolean` default value: '`false`'
flattenCompound Flatten compound documents	type: `boolean` default value: '`false`'
id Parser ID	type: `string` default value: '`ccf98da2-461d-4c75-a92b-864c6e8aaefc`'
includeImages Include images	type: `boolean` default value: '`false`'
inheritMediaTypes Match default media types in this Parser Stage	Each parser stage has a built-in list of media types it handles by default. If this setting is true, that list will be used along with any optional additional types provided in the mediaTypes list. If this setting is false, this stage will only be selected for media types in the mediaTypes list, and the mediaTypes list becomes a mandatory property which must have at least one valid media type. type: `boolean` default value: '`true`'
keepOriginalStructure Return original XML and HTML instead of Tika XML output (only applies if 'Return parsed content as XML is true')	type: `boolean` default value: '`false`'
mediaTypes Media Types to match	Documents with a media type on this list will be matched by this parser stage. See inheritMediaTypes / use default media types for more. type: `array` of `string`
outputFieldPrefix Prefix parsed fields with	Fields extracted by this parser will be prefixed with this string. The remainder of the field name will be as detected in the stream type: `string` maxLength: 20 pattern: ^$\|^[A-Za-z_][A-Za-z0-9_\-\.]+$
pathPatterns File names to parse	Specify a file name or pattern that must be matched for this parser stage to run. Forward slashes ("/") are used to join names of files inside archives with the archive name. type: `array` of `object` object attributes: { `pattern` : { display name: File name or pattern type: `string` description : e.g.: "z.txt" or ".md" or "/a//b/f.txt" for glob; "z.txt$" or ".\.txt$" or "^/a/[^\/]/b/f.txt$" for regex } `syntax` : { display name: Pattern type type: `string` default value: '`glob`' description : glob uses bash shell-style wildcards; regex uses Java (PCRE-style) regex enum: { glob regex } } }
returnXml Return parsed content as XML	type: `boolean` default value: '`false`'
type required	type: `string` default value: '`tika`' enum: { tika }
zipBombCompressionRatio Maximum input-to-output byte ratio	Maximum number of output bytes fusion will generate per input byte. If you are indexing highly compressed files, you may increase this value to avoid triggering 'Zip Bomb' detection type: `integer` default value: '`200`'
zipBombMaxDepth Maximum nesting depth	Returns the maximum XML element nesting level. If you are indexing highly nested files, you may increase this value to avoid triggering 'Zip Bomb' detection type: `integer` default value: '`200`'
zipBombMaxPackageEntryDepth Maximum package entry depth	Sets the maximum package entry nesting level. If you are indexing highly nested files, you may increase this value to avoid triggering 'Zip Bomb' detection type: `integer` default value: '`20`'

addFailedDocs

Add failed documents

type: boolean

default value: 'false'

addOriginalContent

Add original document content (raw bytes)

type: boolean

default value: 'false'

contentEncoding

Content transport encoding of the content (per RFC1341)

type: string

default value: 'binary'

enum: { binary base64 }

enabled

Enable this Parser Stage

type: boolean

default value: 'true'

errorHandling

Error Handling

type: string

default value: 'mark'

enum: { ignore log fail mark }

excludeContentTypes

Content types to exclude

List of content types to exclude from parsing

type: array of string

extractHtmlLinks

Extract XHTML links

Collect links explicitly declared in document structure (e.g. using HTML tags, bookmarks, etc)

type: boolean

default value: 'true'

extractOtherLinks

Extract other links

Use regex-based heuristic extractor to collect likely links from plain text content in all fields.

type: boolean

default value: 'false'

flattenCompound

Flatten compound documents

type: boolean

default value: 'false'

Parser ID

type: string

default value: 'ccf98da2-461d-4c75-a92b-864c6e8aaefc'

includeImages

Include images

type: boolean

default value: 'false'

inheritMediaTypes

Match default media types in this Parser Stage

Each parser stage has a built-in list of media types it handles by default. If this setting is true, that list will be used along with any optional additional types provided in the mediaTypes list. If this setting is false, this stage will only be selected for media types in the mediaTypes list, and the mediaTypes list becomes a mandatory property which must have at least one valid media type.

type: boolean

default value: 'true'

keepOriginalStructure

Return original XML and HTML instead of Tika XML output (only applies if 'Return parsed content as XML is true')

type: boolean

default value: 'false'

mediaTypes

Media Types to match

Documents with a media type on this list will be matched by this parser stage. See inheritMediaTypes / use default media types for more.

type: array of string

outputFieldPrefix

Prefix parsed fields with

Fields extracted by this parser will be prefixed with this string. The remainder of the field name will be as detected in the stream

type: string

maxLength: 20

pattern: ^$|^[A-Za-z_][A-Za-z0-9_\-\.]+$

pathPatterns

File names to parse

Specify a file name or pattern that must be matched for this parser stage to run. Forward slashes ("/") are used to join names of files inside archives with the archive name.

type: array of object

object attributes: {
  pattern : {
    display name: File name or pattern
    type: string
    description : e.g.: "z.txt" or "*.md" or "/a/*/b/f.txt" for glob; "z.txt$" or ".*\.txt$" or "^/a/[^\/]*/b/f.txt$" for regex
    }
  syntax : {
    display name: Pattern type
    type: string
    default value: 'glob'
    description : glob uses bash shell-style wildcards; regex uses Java (PCRE-style) regex
    enum: { glob regex     }

}
}

returnXml

Return parsed content as XML

type: boolean

default value: 'false'

type

required

type: string

default value: 'tika'

enum: { tika }

zipBombCompressionRatio

Maximum input-to-output byte ratio

Maximum number of output bytes fusion will generate per input byte. If you are indexing highly compressed files, you may increase this value to avoid triggering 'Zip Bomb' detection

type: integer

default value: '200'

zipBombMaxDepth

Maximum nesting depth

Returns the maximum XML element nesting level. If you are indexing highly nested files, you may increase this value to avoid triggering 'Zip Bomb' detection

type: integer

default value: '200'

zipBombMaxPackageEntryDepth

Maximum package entry depth

Sets the maximum package entry nesting level. If you are indexing highly nested files, you may increase this value to avoid triggering 'Zip Bomb' detection

type: integer

default value: '20'

Was this page helpful?