Text Parser Stage
The Plain Text parser can split a text file by lines or consume it into a single document.
Options for treatment of this filetype include:
|
|
When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.
|
Parses plain text content with optional trimming and splitting. Character encoding can be specified or automatically detected
id - string
Default: 1ea45951-7c72-4632-ac85-988a73d2c3b1
label - string
A label for this Parser Stage
<= 255 characters
enabled - boolean
Default: true
mediaTypes - array[string]
Documents with a media type on this list will be matched by this parser stage. See inheritMediaTypes / use default media types for more.
inheritMediaTypes - boolean
Each parser stage has a built-in list of media types it handles by default. If this setting is true, that list will be used along with any optional additional types provided in the mediaTypes list. If this setting is false, this stage will only be selected for media types in the mediaTypes list, and the mediaTypes list becomes a mandatory property which must have at least one valid media type.
Default: true
pathPatterns - array[object]
Specify a file name or pattern that must be matched for this parser stage to run. Forward slashes ("/") are used to join names of files inside archives with the archive name.
object attributes:{syntax : {
display name: Pattern type
type: string
}pattern : {
display name: File name or pattern
type: string
}}
errorHandling - string
Default: mark
Allowed values: ignorelogfailmark
outputFieldPrefix - string
Fields extracted by this parser will be prefixed with this string. The remainder of the field name will be as detected in the stream
<= 20 characters
Match pattern: ^$|^[A-Za-z_][A-Za-z0-9_\-\.]+$
charset - stringrequired
Example: "UTF-8"
Default: detect
ignoreBOM - booleanrequired
Ignore Byte-Order Mark (BOM) if present and always use the configured character set. When set to false a valid BOM character set overrides the configured default character set.
Default: false
splitLines - boolean
Split text into lines to create multiple records, default false
Default: false
skipHeaderLines - integer
Skip a number of header lines, default 0
Default: 0
trimWhitespace - boolean
Trim off leading and trailing whitespace from lines, default false
Default: false
skipEmptyLines - boolean
Skip any empty lines encountered, default false
Default: false
outputField - string
Name of the output field where text is stored, default 'body'
>= 1 characters
Default: body
maxLength - integer
Maximum number of characters to allow for the body, -1 for unlimited, default 1MB
<= 2147483647
exclusiveMinimum: false
exclusiveMaximum: false
Default: 1048576
maxLineLength - integer
Maximum number of characters to allow for any single line, default 1MB
<= 2147483647
exclusiveMinimum: false
exclusiveMaximum: false
Default: 1048576
commentField - string
Name of the output field where comment is stored, default 'comment'
>= 1 characters
Default: comment
comment - string
Characters at start of line to indicate a comment, default # (hash)
>= 1 characters
Default: #
commentHandling - string
How to handle comments: include as-is, ignore (and remove from text), add as field (and remove from text), default include
Default: include
Allowed values: ignoreincludeas_field
type - stringrequired
Default: text
Allowed values: text