Data Models
Data models simplify the process of getting started with Fusion by providing pre-configured objects to reduce the effort spent on basic starting tasks. This helps keep documents consistent between datasources and intuitive to the object’s type.
Components
Data models consist of the following:- Object type - Describes the object type, or what a document in the index is. For example, an object may be a file, email, person, product, etc.
- Index pipeline - Specifies the index pipeline that is indexing documents relevant to the data model.
- Query pipeline - Specifies the query pipeline that is invoked when a query involves the data model.
- Fields configuration - Creates a schema map that describes how the object schema should be translated into the Solr index. For example, the field name
first_namecan map to Solr fieldsfirst_name_sandfirst_name_t.
Object type
The object type defines what a document is and what associated data is expected. Example When indexing employees for a company directory, certain fields are expected by a user. This includes the employees’s name, title, and contact information. An object type,person, is created for this purpose. The data model object is created to include this information in a clear, intuitive way:
| Data Model Field | Solr Fields |
|---|---|
first_name | firstName_tfirstName_s |
last_name | lastName_tlastName_s |
email | email_s |
job_title | jobTitle_tlastName_s |
Index pipeline
When you add a connector in Fusion, a default index pipeline is created that is used to map the documents’ fields to the data model fields. The pipeline contains two pipeline stages, the Data Model Mapping index stage and the Call Data Model Pipeline index stage. Once configured, this stage indexes the documents as various object types. Fields irrelevant to that object type are optionally discarded.Query pipeline
The query pipeline is applied by default whenever a user is searching documents of the object type. This enables query pipeline configurations specialized for the object type. For example, a query pipeline for the object typeperson could boost or filter results for current employees over previous employees.
The query pipeline is optional for data models.
Fields configuration
Data model fields allow you to map document fields to Solr fields in a way that respects the data model object type. The fields configuration allows this map to be intuitive to new users, offering a guided experience for configuring these options.UI fields configuration
| Field | Required | Description | Example |
|---|---|---|---|
| Name of the Data Model | ✅ | string A user-friendly name for the data model. This typically describes the object type. | person |
| Index Pipeline | ✅ | string The name of the index pipeline used for the data model. | companyDirectory-index-pipeline |
| Query Pipeline | ✅ | string The name of the query pipeline used for the data model. | companyDirectory-query-pipeline |
| Field Name | ✅ | string | first_name |
Required | ✘ | boolean | true |
| Solr Fields ↓ | |||
| Solr Field Name | ✅ | string | firstName_t |
| Is Query Field | ✘ | boolean | true |
| Enable Phrase Match | ✘ | boolean | false |
| Boost Value | ✘ | integer The amount of boost to give to the query. | 1 |
| Phrase Boost | ✘ | integer The amount of boost to give to the query, if it matches as an exact phrase. | 2 |
More information
- Configure a Data Model
- Data Models API reference
- Data Model Mapping Index Stage reference
- Call Data Model Pipeline Index Stage reference
Configure a Data Model
Configure a Data Model
Data models simplify the process of getting started with Fusion by providing pre-configured objects to reduce the effort spent on basic starting tasks. This helps keep documents consistent between datasources and intuitive to the object’s type.See the Data Models topic for more information.
Configure the datasource
This section references the Slack V2 Connector.
- Navigate to Indexing > Datasources.
-
Click the Add button, and choose the Slack V2 connector from the list.
Some connectors include built-in data models as a standard component. Others require you to manually create data models.
-
Complete the datasource configuration:
- Ideally, use a pipeline specifically created for this data model. For now, you can create the pipeline by navigating to Indexing > Index Pipelines and clicking the Add button. Configure it using the information in Configure the index pipeline.
-
Under Profiles Settings, select Index profiles:
-
Under Channels and Messages Settings, select Index channels and Index from public channels:
- Click Save.
- Click Run to run the indexing job.
Configure the data model
Some connectors include built-in data models with pre-configured object types. However, you can add new data models or customize existing ones to fit your particular needs.- Navigate to Indexing > Data Models.
- Click the Add button.
-
Create a new data model. For example,
person. -
Assign the index pipeline for the data model. In this example, create a new index pipeline named
companyDirectory-data-pipeline. This pipeline is ideally for operation with data models only. For the sake of this example, we will only be using the default stages. Applying additional logic/stages specific to the object would occur here. -
Assign the query pipeline you will use to query the indexed documents. In this example, create a new query pipeline named
companyDirectory-query-pipeline. - Click the New button under Fields Configuration.
-
Complete the required configurations, as detailed in Data Models API Reference. Create fields for the following:
Field Name Solr Fields first_namefirstName_t,firstName_slast_namelastName_t,lastName_semailemail_sjob_titlejobTitle_t,jobTitle_s
-
If needed, edit the JSON for the data model.
- Click the Save button to save the data model.
Configure the index pipeline
To begin, navigate to Indexing > Index Workbench. Alternatively, thecompanyDirectory-index-pipeline pipeline can be configured in Indexing > Index Pipelines, but you are not able to preview results.The raw data fields in your index coming from Slack may differ from the example data fields used in this article.
Data Model Mapping stage
- Click the Add a Stage button, and choose Data Model Mapping from the list.
-
Use the Data Model Type dropdown to select the
persondata model. -
(optional) Check the Match Trigger checkbox and assign the following values:
Field Value Field to match type_sValue to match userThe Value to match field supports RegEx. You can assign multiple values. Alternatively, you can create additional Data Model Mapping stages. -
Assign field mappings for the Slack datasource’s raw datasources:
Source Field Data Model Field first_namefirst_namelast_namelast_nameemailemail - (optional) Check the Keep unmapped fields checkbox to preserve fields that are unmatched to the data model.
- Click Apply.
Call Data Model Pipeline stage
This stage must be placed after the Data Model Mapping stage.
- Click the Add a Stage button, and choose Call Data Model Pipeline from the list.
- Assign the value
_lw_data_model_type_sto the Data Model Type Field field. This field is created when a document meets the criteria specified in the Data Model Mapping. - Click Apply.
Next steps
With the index pipeline configured, you are now ready to complete the indexing job by clicking the Start Job button.Once complete, the documents are ready for viewing in the Query Workbench:- Navigate to Querying > Query Workbench.
- Click the Load button.
- Choose the query pipeline you specified when creating the data model,
companyDirectory-query-pipeline.