Product Selector

Fusion 5.11
    Fusion 5.11

    Use Tika Asynchronous Parsing

    This document describes how to setup your application to use Tika asynchronous parsing.

    Unlike synchronous Tika parsing, which uses a parser stage, asynchronous Tika parsing is configured in the datasource and index pipeline. For more information, see Asynchronous Tika Parsing.

    Field names change with asynchronous Tika parsing.

    In contrast to synchronous parsing, asynchronous Tika parsing prepends parser_ to fields added to a document. System fields, which start with _lw_, are not prepended with parser_.

    If you are migrating to asynchronous Tika parsing, and your search application configuration relies on specific field names, update your search application to use the new fields.

    Configure the connectors datasource

    1. Navigate to your datasource.

    2. Enable the Advanced view.

    3. Enable the Async Parsing option.

      Enable async option

      Your parser configuration is ignored when using asynchronous parsing.

      The asynchronous parsing service performs Tika parsing using Apache Tika Server. Other parsers, such as HTML and JSON, are not supported by the asynchronous parsing service. By enabling asynchronous parsing, the parser configuration linked to your datasource is ignored.

    4. Save the datasource configuration.

    Configure the index pipeline

    1. Go to the Index Pipeline screen.

    2. Add the Solr Partial Update Indexer stage.

    3. Turn off the Reject Update if Solr Document is not Present option and turn on the Process All Pipeline Doc Fields option:

      Tika config setup

    4. Include an extra update field in the stage configuration using any update type and field name. In this example, an incremental field docs_counter_i with an increment value of 1 is added:

      Tika config setup

    5. Enable the Allow reserved fields option:

      Tika config setup

    6. Click Save.

    7. Turn off or remove the Solr Indexer stage, and move the Solr Partial Update Indexer stage to be the last stage in the pipeline.

      Tika config setup

    Asynchronous Tika parsing setup is now complete. Run the datasource indexing job and monitor the results.