When you use the Tika Asynchronous Parser, the connector fetches and indexes a document containing only metadata. It does not index the Tika parsed metadata and Tika parsed body. A separate Tika Asynchronous parsing job will fetch the raw file content, parse it with Tika and do a partial update on the document with both the Tika parsed metadata and Tika parsed body.
See Fusion 4.x V1 Connector Downloads to access the latest versions of the connectors.

​
Configure the index pipelines

When you use asynchronous parsing, you will need to establish two index pipelines.

​
Pre-parse pipeline

The pre-parse pipeline will only fetch and index metadata since it does not have the Tika parsed body or Tika parsed metadata. Within this index pipeline, you will want to make sure you are buffering requests to Solr: Configure Solr index pipeline
This is the default setting. If this option is not selected, select the box. Doing so will increase the indexing speed.

​
Post-parse pipeline

The Tika Asynchronous job will download, parse, and send the parsed document to a second Fusion index pipeline. This pipeline is responsible for updating the Tika body and metadata to the document created in the pre-parse pipeline. The index pipeline MUST contain a Solr Partial Update stage with the following selected parameters: Tika Async post parse pipeline The Solr Partial Update Indexer pipeline will add the Tika fields into the existing document that was added by the connector.
If you do not select the Process All Pipeline Doc Fields option, you will receive results similar to the following:"body_t":"[{\"field\":\"I’m the content from JSON!\"}]"

​
Configure the Asynchronous Parser

Next, we will create the new parser. All settings can be left on the default selections. Apache Tika Async Parser

​
Configure the datasource

Configure the datasource to use the asynchronous parser. Configure Tika Async datasource When you run the datasource job, it will display the documents indexed and will have a special async parsing metadata.
  • _lw_async_parsing_id_i
  • _lw_async_parsing_fail_count_i
If the connectors have an async parser, they will not download or parse the documents when running the connector job. This will be performed asynchronously by the Tika Asynchronous Parsing job.

​
Configure the Tika Async Parsing job

  1. Select Jobs under the Collections header. Locate jobs
  2. Click Jobs, and select Add +. Search for Tika to select the option for Tika Async Job. Add Tika Async job
  3. Configure fields according to your desired specifications. Tika Async configure job
Field options for the Tika Async parsing job include:
FieldNotes
Tika EndpointsList of Tika server endpoints that are in the cluster. The endpoints will be automatically updated. You do not need to specify the endpoints.
Pipeline IDThe Pipeline ID is determined by the partial-update index pipeline you created earlier.
Find Filter QueriesThis is an optional filter to limit the documents fetched by the Tika Async parser.
Finder batch sizeThis is the Solr rows batch size of the requests.
Max bytes returnedThe maximum number of characters indexed.
Max embedded documentsThis determines the amount of embedded documents that should be parsed until additional embedded documents are ignored. Use -1 for unlimited documents.
Parse timeoutThis configures the length of time for parsing in milliseconds.
Max Parse FailureIf a document fails to parse, you can configure the reattempts up to your designated amount of times.
Insert Batch Max DocsThis controls the batch size in number of documents before submitting to the index pipeline.
Insert Batch Max BytesThis controls the batch size number of bytes contained in the documents before submitting to the index pipeline.
  1. Finally, configure a schedule for the job to run according to your designated specifications.
Configure a schedule for the Tika Async job to run
The logs for the job will be in $FUSION_HOME/var/log/tika-server/tika-async.log