Solr Update Parser Stage

The Solr Update Parser stage parses Solr’s "update" messages and transforms them into Fusion PipelineDocuments. This makes it possible to integrate with existing datasets that happen to be in Solr’s "update" format and send them to an index pipeline.

This parser stage supports XML, JSON, CSV and JavaBin.

Tip
 When parsing CSV files, consider using the CSV parser, which provides more options than the Solr Update Parser. All CSV files are treated the same in terms of structure and schema.

To learn more about how Fusion’s Solr core indexes "update" documents, see this tutorial. The fusion/5.2.x/apps/solr-dist/example/exampledocs/ directory contains many of these "update" format documents. Fusion also ships with a fusion/5.2.x/apps/solr-dist/bin/post utility which is used to index documents from the file system. However, the post utility has no way of integrating with Fusion; its output is sent directly to Solr.

Tip
 When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.

Solr Update stage-specific properties

Property Description, Type
enableCsv

Enable CSV

Enables the parser to recognize and parse, CSV based Solr update messages.

type: boolean

default value: 'true'
enableJson

Enable JSON

Enables the parser to recognize and parse, JSON based Solr update messages.

type: boolean

default value: 'true'
enableXml

Enable XML

Enables the parser to recognize and parse, XML based Solr update messages.

type: boolean

default value: 'true'
enabled

Enable this Parser Stage

type: boolean

default value: 'true'
errorHandling

Error Handling

type: string

default value: 'mark'

enum: { ignore log fail mark }
id

Parser ID

type: string

default value: '548a3222-a17c-43cc-94a2-8aa090da30f3'
ignoredMediaTypes

Media Types to ignore

Documents with a media type on this list will be not be processed by this parser stage.

type: array of string

inheritMediaTypes

Match default media types in this Parser Stage

Each parser stage has a built-in list of media types it handles by default. If this setting is true, that list will be used along with any optional additional types provided in the mediaTypes list. If this setting is false, this stage will only be selected for media types in the mediaTypes list, and the mediaTypes list becomes a mandatory property which must have at least one valid media type.

type: boolean

default value: 'true'
mediaTypes

Media Types to match

Documents with a media type on this list will be matched by this parser stage. See inheritMediaTypes / use default media types for more.

type: array of string

outputFieldPrefix

Prefix parsed fields with

Fields extracted by this parser will be prefixed with this string. The remainder of the field name will be as detected in the stream

type: string

maxLength: 20

pattern: ^$|^[A-Za-z_][A-Za-z0-9_\-\.]+$
pathPatterns

File names to parse

Specify a file name or pattern that must be matched for this parser stage to run. Forward slashes ("/") are used to join names of files inside archives with the archive name.

type: array of object

object attributes: {
  pattern : {
    display name: File name or pattern
    type: string
    description : e.g.: "z.txt" or "*.md" or "/a/*/b/f.txt" for glob; "z.txt$" or ".*\.txt$" or "^/a/[^\/]*/b/f.txt$" for regex
    }
  syntax : {
    display name: Pattern type
    type: string
    default value: 'glob'
    description : glob uses bash shell-style wildcards; regex uses Java (PCRE-style) regex
    enum: { glob regex     }

    }
  }
type

required

type: string

default value: 'solr-update'

enum: { solr-update }