Indexing Your Data
Indexing stores your data in a format that is optimized for searching. How you index your data is critical for ensuring that your data is stored in a format that is ideal for your search application.
- Index pipelines determine the details of the conversion.
- Index pipeline stages are the building blocks of index pipelines.
- The Index Workbench is Fusion’s index pipeline development tool.
Collections
Collections are a way of grouping data sets so that related data sets can be managed together. Every data set that you ingest belongs to a collection. Any app can contain one or more collections. See Collection Management.Datasources
Datasources are configurations that determine how your data is handled during ingest by Fusion’s connectors, parsers, and index pipelines. When you run a fully-configured datasource, the result is an indexed data set that is optimized for search, depending on the shape of your data and how you want to search it. See Configure a New Datasource.Configure a New Datasource
Configure a New Datasource
- In your Fusion app, navigate to Indexing > Datasources.
- Click Add.
- Select your connector. The connector configuration panel appears. The specific configuration options vary depending on the connector.
- Configure your connector’s options:
- Enter a useful name for your datasource in the Datasource ID field.
- You can leave the Pipeline ID and Parser fields as-is to start with the defaults, or select different ones if you have them.
- Configure the connector-specific options. Select your specific release and connector detail from the configuration reference topic.
- Click Save.
- Test your datasource configuration:
- Navigate to the Index Workbench at Indexing > Index Workbench.
- Click Load….
- Select the datasource ID you specified when you created the datasource. Now you can see your datasource configuration and a simulation of the results you can expect when you run this datasource job to index your data.
- Adjust the configurations of your datasource, parsers, and index pipeline until the simulated results are satisfactory.
- Click Save.
- Index your data:
- In the Index Workbench, click Start Job.
- When the job status is Finished, navigate to the Query Workbench at Querying > Query Workbench to view the indexed documents and configure your query pipeline.
Connectors
Connectors are Fusion components that ingest and parse specific kinds of data. There is a Fusion connector for just about any data type.Additional resources
LucidAcademyLucidworks offers free training to help you get started.The Quick Learning for Indexing Menu Tour focuses on the Indexing Menu features and functionality along with a brief description of each screen available in the menu:
LucidAcademyLucidworks offers free training to help you get started.The Quick Learning for Anatomy of Indexing focuses on the Fusion objects that work together to index your data:
LucidAcademyLucidworks offers free training to help you get started.The Course for Indexing Data focuses on how to ingest and store your data in a format that’s optimized for search: