Skip to main content
This topic explains what you’ll find in the GitHub repository used by ConfigSync. See also Set up a GitHub repo for ConfigSync.
ConfigSync uses a GitHub repository to store the configuration files. The repository should be created independently and shared with the ConfigSync service via the Helm chart values. This document describes the process of setting up a GitHub repository for Fusion config files.

​
Create a new repository

Create a new repository in the github organization where the ConfigSync should store the config files.

​
Create a deploy key

Create a deploy key for the new repository. The deploy key should have write access to the repository. See the GitHub documentation for instructions.

​
Add the deploy key and repository information to the ConfigSync values

​
Option 1: Add the deploy key in the Helm chart

fusion-config-sync:
  # other values
  sshKey: <base64-encoded-private-key>
  cfg:
    github:
      repo: <repository-url> # use the ssh url e.g. git@github.com:lucidworks-managed-fusion/maksim-config-sync.git
      branch: <customer-branch> # you can leave it as is
      mainBranch: <main-branch> # most probably it's `main`
To encode the private key to base64 you can use the following command:
cat ~/.ssh/id_rsa | base64

​
Option 2: Add the deploy key as an external secret

Alternatively, you can store the deploy key as an external secret and reference it in the ConfigSync values.

​
Create a secret with the deploy key:

apiVersion: v1
kind: Secret
metadata:
  name: config-sync-ssh
type: Opaque
data:
  config: SG9zdCAqCiAgU3RyaWN0SG9zdEtleUNoZWNraW5nIG5v
  id_rsa: <base64-encoded-private-key>
To encode the private key to base64 you can use the following command:
cat ~/.ssh/id_rsa | base64

​
Update the ConfigSync values to reference the secret with the deploy key:

fusion-config-sync:
  # other values
  sshKey: null
  additionalVolumeMounts:
    - name: config-sync-ssh-key
      mountPath: /root/.ssh
      readOnly: true
  additionalVolumes:
    - name: config-sync-ssh-key
      secret:
        secretName: config-sync-ssh
        defaultMode: 256

​
Additional configuration

github.* settings control the remote GitHub repo, branch, and path within the repo.
github connection properties, common for pub/sub modes

cfg:
  github:
    repo: https://github.com/lucidworks/cloud-config-sync.git # git repo, mandatory
    branch: dev # git branch, mandatory
    mainBranch: main # main branch, mandatory
    dir: "/tmp/config-sync-repo" # repo dir, default "/tmp/config-sync-repo"
    path: / # configs path in the repo dir, default "/"
    username: "fusion-config-sync" # git user name, default "fusion-config-sync"
    email: "fusion-config-sync@lucidworks.com" # git user email, mandatory, default "fusion-config-sync@lucidworks.com"

​
Update the ConfigSync deployment

Update the ConfigSync deployment with the new values.

​
Branch and Path

If you need to share the same GitHub repo for different clusters or environments, you can use the cfg.github.branch setting to specify the branch in the GitHub repo where the config files are stored. Additionally (or alternatively) ConfigSync supports a cfg.github.path (referred to as root in the future) setting to specify the path in the GitHub repo where the config files are stored. By default, ConfigSync stores the config files in the root of the GitHub repo.

​
Apps and Object Types

Under the root directory, there is a directory for each app, such as app1, app2. Under each app directory, there are directories for each object type, such as query-pipelines, blobs, etc.

​
System Objects

ConfigSync uses _lw_system as the app name for system objects. This directory is always present in the repo and contains system objects like users, roles, solr configs, etc.

​
Reserved Files

Additionally, ConfigSync has set of reserved files in the root directory:
  • .lock - A lock file to prevent concurrent updates
  • .gitattributes - Used to list LFS files
  • .gitignore - Used to ignore files that should not be stored in the repo
ImportantThese files should not be modified or deleted.

​
Example repo structure

|-- root
|   |-- _lw_system
|   |   |-- users
|   |   |   |-- user1.json
|   |   |   |-- user2.json
|   |   |-- roles
|   |   |   |-- role1.json
|   |   |   |-- role2.json
|   |   |-- solr-configs
|   |   |   |-- config1
|   |   |   |   |-- solrconfig.xml
|   |   |   |   |-- managed-schema.xml
|   |-- app1
|   |   |-- query-pipelines
|   |   |   |-- pipeline1.json
|   |   |   |-- pipeline2.json
|   |   |-- blobs
|   |   |   |-- blob1.json
|   |   |   |-- blob2.json
|   |-- app2
|   |   |-- query-pipelines
|   |   |   |-- pipeline3.json
|   |   |   |-- pipeline4.json
|   |   |-- blobs
|   |   |   |-- blob3.json
|   |   |   |-- blob4.json
|-- .lock
|-- .gitattributes
|-- .gitignore
ConfigSync uses the Fusion links API to determine app association for Fusion objects. However, it does not track links in GitHub as it uses the directory structure to maintain app association for an object.