Skip to main content
The ETL (Extract, Transform, Load) Indexer job enables bulk ingestion of structured data from Google Cloud Storage into Lucidworks Search. Unlike the Parallel Bulk Loader job, the ETL Indexer runs as a Kubernetes operator instead of a Spark job, so it works in Lucidworks Search environments where Spark isn’t available.
ETL indexing is available in Lucidworks Search 5.19 and later. It is turned off by default for existing clients in Lucidworks Search 5.19. Contact your Lucidworks representative to request ETL indexing.

​
Usage

Use this job to load CSV, JSON, JSONL, or Parquet files from a Google Cloud Storage bucket into one or more Lucidwokrs Search collections, either directly to Solr for higher throughput or through a Lucidworks Search indexing pipeline when you need to transform data at index time. For conceptual information about the ETL Indexer, see ETL indexing.

​
Index to multiple collections

A single ETL Indexer job can index data to more than one collection. Add a collection configuration for each destination collection.

​
Indexing groups

The Indexing Group field controls whether collections in the same job are indexed serially or in parallel:
  • Serially: Collections with the same indexing group are indexed one after the other.
  • In parallel: Collections with different indexing groups are indexed at the same time.
The default indexing group is 0, so collections are indexed serially unless you assign different indexing groups.
Parallel indexing only improves throughput when the collections are on separate hardware. If the collections share hardware, use the same indexing group.

​
Glob expressions

The Data Source field accepts either a URL to a specific file in cloud storage or a glob expression that matches multiple files within a directory structure. For example, this expression matches every CSV file two directory levels below any date-named directory from 2026 in etl-data:

​
Configuration properties