HTML Parser Stage

This parser stage processes the following HTML elements:

  • <title>

  • <body> (with tags removed)

  • <meta>

  • <a> and <link>

Additionally, you can configure JSoup selectors to extract specific HTML and CSS elements from a document and map them to PipelineDocument fields. For example, you could use this to process navigational DIV elements one way, then process content-ful DIV elements another way.

Note
The HTML Transformation index pipeline stage is deprecated in favor of this parser stage.

HTML and CSS elements can be selected for extraction into new documents or fields:

  • To create new documents from selected elements, configure recordSelector.

  • To create new fields from selected elements, configure mappings.

Title, body, metadata, and links are only populated in the parent document. Both of these parameters support JSoup selectors, which provides a rich syntax for selecting HTML and CSS elements.

Tip
When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.