Fusion 5 On-prem Kubernetes Deployment

The setup_f5_k8s.sh script in the fusion-cloud-native repository provides deployment support for any Kubernetes platform, including on-premise, private cloud, public cloud, and hybrid platforms. This script is used by the setup_f5_gke.sh, setup_f5_eks.sh, and setup_f5_aks.sh scripts, which provide additional platform-specific support for Google Kubernetes Engine (GKE), Amazon Elastic Kubernetes Service (EKS), and Azure Kubernetes Service (AKS). See Fusion 5 Upgrades for more information about deployment instructions for:

Fusion 5 Upgrades

This guide describes how to perform Fusion 5 upgrades.

Before upgrading, be aware of changes by checking for Deprecations and Removals between versions.

Lucidworks recommends upgrading to the next minor version only. For example, you should upgrade from Fusion 5.6.1 to Fusion 5.7.1 before upgrading to Fusion 5.8.0.The general upgrade process is described in this article. However, the specific upgrade procedures may vary depending on your upgrade path. For the most accurate instructions, please refer to the upgrade article specific to your upgrade.

General upgrade process

Fusion natively supports deployments on supported Kubernetes platforms, including AKS, EKS, and GKE.Fusion includes an upgrade script for AKS, EKS, and GKE. This script is not generated for other Kubernetes deployments.Upgrades differ from platform to platform. See below for more information about upgrading on your platform of choice.Whenever you upgrade Fusion, you must also update your remote connectors, if you are running any. You can download the latest files at V2 Connectors Downloads.

Natively supported deployment upgrades

Deployment type	Platform
Azure Kubernetes Service (AKS)	`aks`
Amazon Elastic Kubernetes Service (EKS)	`eks`
Google Kubernetes Engine (GKE)	`gke`

Fusion includes upgrade scripts for natively supported deployment types. To upgrade:

Open the <platform>_<cluster>_<release>_upgrade_fusion.sh upgrade script file for editing.
Update the CHART_VERSION to your target Fusion version, and save your changes.
Run the <platform>_<cluster>_<release>_upgrade_fusion.sh script. The <release> value is the same as your namespace, unless you overrode the default value using the -r option.

After running the upgrade, use kubectl get pods to see the changes applied to your cluster. It may take several minutes to perform the upgrade, as new Docker images are pulled from DockerHub. To see the versions of running pods, do:

kubectl get po -o jsonpath='{..image}'  | tr -s '[[:space:]]' '\n' | sort | uniq

Other Kubernetes deployment upgrades

To update an existing installation, do:

RELEASE=f5
NAMESPACE=default
helm repo update
helm upgrade ${RELEASE} "lucidworks/fusion" --namespace "${NAMESPACE}" --values "${MY_VALUES}"

Except for ZooKeeper, all K8s deployments and statefulsets use a RollingUpdate update policy:

  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate

ZooKeeper instances use OnDelete to avoid changing critical stateful pods in the Fusion deployment. To apply changes to Zookeeper after performing the upgrade (uncommon), you need to manually delete the pods. For example:

kubectl delete pod f5-zookeeper-0

Delete one pod at a time. Verify the new pod is healthy and serving traffic, before deleting the next healthy pod.

You can also set the updateStrategy under the zookeeper section in your "${MY_VALUES}" file:

solr:
  ...
  zookeeper:
    updateStrategy:
      type: "RollingUpdate"

Upgrades with Helm v3

One of the most powerful features provided by Kubernetes and a cloud-native microservices architecture is the ability to do a rolling update on a live cluster. For example, Fusion 5 allows customers to upgrade from Fusion 5.1.0 to a later 5.x.y version on a live cluster with zero downtime or disruption of service.When Kubernetes performs a rolling update to an individual microservice, there is a mix of old and new services in the cluster. Requests from other services route to both versions.

Lucidworks ensures all changes we make to our service do not break the API interface exposed to other services in the same minor release version (5.x). We also ensure that the stored configuration remains compatible in the same minor release version.

Lucidworks releases minor updates to individual services frequently. Pull in those upgrades using Helm at your discretion.How to upgrade Fusion

Clone the fusion-cloud-native repo, if you haven’t already.
Locate the setup_f5_<platform>.sh script that matches your Kubernetes platform.
Run the script with the --upgrade option.
To see what would be upgraded, pass the --dry-run option to the script.

The scripts in the fusion-cloud-native repo automatically pull in the latest chart updates from our Helm repository and deploy any updates needed by doing a diff of your current installation and the latest release from Lucidworks.

Helm upgrade script

Once you deploy a working cluster, use the upgrade script created by the customize_fusion_values.sh script. The upgrade script hard-codes the parameters and eases the need to remember which parameters to pass to the script. This is helpful when working with multiple K8s clusters. Make sure you check the script into version control alongside your custom values YAML files.Whenever you change the custom values YAML files for your cluster, you need to run the upgrade script to apply the changes. The script calls helm upgrade with the correct parameters and --values options.

If you run helm upgrade without passing the custom values YAML files, the deployment will revert to using chart defaults, which you never want to do.

The script assumes your kubeconfig is pointing to the correct cluster and you’re using Heml v3. If not, the upgrade fails. Select the correct kubeconfig before running the script.

Natively supported deployments
Other Kubernetes deployments

Considerations for on-prem deployments

Solr

Making changes to Solr’s memory settings after the initial installation, including changing the number of pods, is simple. Changing the size and type of disk, however, can be difficult. Before you begin, identify some basic sizing information for Solr:

Memory settings
Disk size
Disk type (e.g. SSD)

Docker images

Fusion’s public Docker images are hosted on DockerHub. If you do not allow your Kubernetes environment to pull images from DockerHub, you will need to copy the images to your internal registry. This can be a time-consuming process.

Helm charts

Helm is a package manager for Kubernetes that helps you install and manage applications on your Kubernetes cluster. We supply a Helm chart for the Fusion installation. If you are using Helm V2, the setup_f5_k8s.sh script installs Helm’s Tiller component into your Kubernetes cluster with the cluster admin role. If you want to prevent the installation of Tiller, upgrade to Helm V3.

Nodes

We recommend installing Fusion on at least 3 nodes, each starting in its own namespace. Only 1 Fusion instance per namespace is supported. Development and test environments only require 1 node with at least 12 GB of RAM.

It is possible to install other apps alongside Fusion in the same namespace, if desired.

Pod resources

We recommend not placing resource limits on pods until you have run production-like workloads with your data and fully understand your resource requirements.

Fusion gateway service

If you will be exposing the Fusion gateway service externally, you will need to consider your Kubernetes Ingress strategy:

How will you route to internal services, such as the gateway?
Will you do TLS termination at the Ingress?

Deploying with a read-only root file system

Fusion 5.9.9 and up supports a read-only root file system to safeguard against unauthorized modifications. Most services operate efficiently in this mode, and for components that need write access, a separate writable mount is available using specific helm configurations. See Enable Read-Only Root File System for examples of how to configure this feature.

Enable Read-Only Root File System

This topic describes how to configure a read-only root file system for a Fusion deployment. Read-only mode safeguards your file system against unauthorized changes, such as by malicious software or other attacks. Enabling this feature entails configuring the readOnlyRootFilesystem attribute in the Fusion Helm chart.

This feature is only available in Fusion 5.9.9 and later versions of Fusion 5.9.

Beginning with Fusion 5.9.10, all of Fusion’s services are designed to work with a read-only root file system, but some external services may require write access to the file system. In that case, you can configure a read-only root file system and mount a separate writable file system for these services.The examples below show how to configure a read-only root file system for various Fusion services.

Solr

fusion:
  solr:
    containerSecurityContext:
      readOnlyRootFilesystem: true

With optional services:

fusion:
  solr:
    exporter:
      enabled: true
      securityContext:
        readOnlyRootFilesystem: true
    containerSecurityContext:
      readOnlyRootFilesystem: true

Zookeeper

fusion:
  zookeeper:
    containerSecurityContext:
      readOnlyRootFilesystem: true

With optional services:

fusion:
  zookeeper:
    containerSecurityContext:
      readOnlyRootFilesystem: true
    exporters:
      jmx:
        enabled: true
      zookeeper:
        enabled: true
    jobs:
      chroots:
        enabled: true
        config:
          create:
            - /root-read-only

Kafka

fusion:
  kafka:
    containerSecurityContext:
      readOnlyRootFilesystem: true

With optional services:

fusion:
  kafka:
    containerSecurityContext:
      readOnlyRootFilesystem: true
    volumePermissions:
      enabled: true
      containerSecurityContext:
        readOnlyRootFilesystem: true
    metrics:
      kafka:
        containerSecurityContext:
          readOnlyRootFilesystem: true
          enabled: true
      jmx:
        enabled: true
        containerSecurityContext:
          enabled: true
          readOnlyRootFilesystem: true
    externalAccess:
      enabled: true
      autoDiscovery:
        enabled: true
        containerSecurityContext:
          enabled: true
          readOnlyRootFilesystem: true
    provisioning:
      enabled: true
      topics:
      - name: test
        partitions: 1
        replicationFactor: 1
        ## https://kafka.apache.org/documentation/#topicconfigs
        config:
          max.message.bytes: 64000
          flush.messages: 1
      containerSecurityContext:
        readOnlyRootFilesystem: true
        enabled: true

ML Model Service

fusion:
  ml-model-service:
    ambassador:
      containerSecurityContext:
        readOnlyRootFilesystem: true
    milvus:
      mysql:
        containerSecurityContext:
          readOnlyRootFilesystem: true
      securityContext:
        readOnlyRootFilesystem: true
With optional services:
fusion:
  ml-model-service:
    ambassador:
      containerSecurityContext:
        readOnlyRootFilesystem: true
      prometheusExporter:
        enabled: true
        securityContext:
          readOnlyRootFilesystem: true
    milvus:
      mysql:
        metrics:
          enabled: true
          securityContext:
            readOnlyRootFilesystem: true
        containerSecurityContext:
          readOnlyRootFilesystem: true
      cluster:
        enabled: false
      mishards:
        securityContext:
          readOnlyRootFilesystem: true
      admin:
        enabled: true
        securityContext:
          readOnlyRootFilesystem: true
      securityContext:
        readOnlyRootFilesystem: true

Argo

fusion:
  argo:
    minio:
      containerSecurityContext:
        readOnlyRootFilesystem: true
      extraVolumes:
        - name: config-dir
          emptyDir: {}
        - name: certs-dir
          emptyDir: {}
      extraVolumeMounts:
        - name: config-dir
          mountPath: /.minio
        - name: certs-dir
          mountPath: /etc/minio/certs
    server:
      securityContext:
        readOnlyRootFilesystem: true
    controller:
      securityContext:
        readOnlyRootFilesystem: true
    mainContainer:
      securityContext:
        readOnlyRootFilesystem: true
    executor:
      securityContext:
        readOnlyRootFilesystem: true

Seldon Core Operator

fusion:
  seldon-core-operator:
    containersSecurityContext:
      readOnlyRootFilesystem: true

Argo Common Workflows

fusion:
  argo-common-workflows:
    containerSecurityContext:
      readOnlyRootFilesystem: true

Question Answering

fusion:
  question-answering:
    containersSecurityContext:
      readOnlyRootFilesystem: true

Classification

fusion:
  classification:
    containersSecurityContext:
      readOnlyRootFilesystem: true

All-in-one values.yaml example

fusion:
  solr:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  solr-managed:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  zookeeper:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  kafka:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  ml-model-service:
    ambassador:
      containerSecurityContext:
        readOnlyRootFilesystem: true
    milvus:
      mysql:
        containerSecurityContext:
          readOnlyRootFilesystem: true
      securityContext:
        readOnlyRootFilesystem: true
  argo:
    minio:
      containerSecurityContext:
        readOnlyRootFilesystem: true
      extraVolumes:
        - name: config-dir
          emptyDir: {}
        - name: certs-dir
          emptyDir: {}
      extraVolumeMounts:
        - name: config-dir
          mountPath: /.minio
        - name: certs-dir
          mountPath: /etc/minio/certs
    server:
      securityContext:
        readOnlyRootFilesystem: true
    controller:
      securityContext:
        readOnlyRootFilesystem: true
    mainContainer:
      securityContext:
        readOnlyRootFilesystem: true
    executor:
      securityContext:
        readOnlyRootFilesystem: true
  seldon-core-operator:
    containersSecurityContext:
      readOnlyRootFilesystem: true
  argo-common-workflows:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  question-answering:
    containersSecurityContext:
      readOnlyRootFilesystem: true
  classification:
    containersSecurityContext:
      readOnlyRootFilesystem: true

All-in-one values.yaml with optional services example

fusion:
  argo:
    minio:
      persistence:
        enabled: false
      containerSecurityContext:
        readOnlyRootFilesystem: true
      extraVolumes:
        - name: config-dir
          emptyDir: {}
        - name: certs-dir
          emptyDir: {}
      extraVolumeMounts:
        - name: config-dir
          mountPath: /.minio
        - name: certs-dir
          mountPath: /etc/minio/certs
    server:
      securityContext:
        readOnlyRootFilesystem: true
    controller:
      securityContext:
        readOnlyRootFilesystem: true
    mainContainer:
      securityContext:
        readOnlyRootFilesystem: true
    executor:
      securityContext:
        readOnlyRootFilesystem: true
  solr:
    exporter:
      enabled: true
      securityContext:
        readOnlyRootFilesystem: true
    containerSecurityContext:
      readOnlyRootFilesystem: true
    # tls:
    #   enabled: true
  solr-managed:
    exporter:
      enabled: true
      securityContext:
        readOnlyRootFilesystem: true
    enableExternalFiles: true
    enabledStorage:
    - gcs
    processRaw:
      image:
        repository: fusion-dev-docker.ci-artifactory.lucidworks.com
      cloudRoot: gs://lw-managed-fusion-data/tmp
      gcs:
        secret: gcs-key
        secretFieldName: key.json
      securityContext:
        readOnlyRootFilesystem: true
    containerSecurityContext:
      readOnlyRootFilesystem: true
    # tls:
    #   enabled: true
  zookeeper:
    containerSecurityContext:
      readOnlyRootFilesystem: true
    exporters:
      jmx:
        enabled: true
      zookeeper:
        enabled: true
    jobs:
      chroots:
        # enabled: true
        config:
          create:
            - /root-read-only
  kafka:
    containerSecurityContext:
      readOnlyRootFilesystem: true
    volumePermissions:
      enabled: true
      image:
        repository: "fusion-dev-docker.ci-artifactory.lucidworks.com/os-shell"
      containerSecurityContext:
        readOnlyRootFilesystem: true
    metrics:
      kafka:
        containerSecurityContext:
          readOnlyRootFilesystem: true
          enabled: true
      jmx:
        enabled: true
        containerSecurityContext:
          enabled: true
          readOnlyRootFilesystem: true
    externalAccess:
      enabled: true
      autoDiscovery:
        enabled: true
        containerSecurityContext:
          enabled: true
          readOnlyRootFilesystem: true
    provisioning:
      enabled: true
      topics:
      - name: test
        partitions: 1
        replicationFactor: 1
        ## https://kafka.apache.org/documentation/#topicconfigs
        config:
          max.message.bytes: 64000
          flush.messages: 1
      containerSecurityContext:
        readOnlyRootFilesystem: true
        enabled: true
  ml-model-service:
    enabled: true
    ambassador:
      containerSecurityContext:
        readOnlyRootFilesystem: true
      prometheusExporter:
        enabled: true
        securityContext:
          readOnlyRootFilesystem: true
    milvus:
      mysql:
        metrics:
          enabled: true
          securityContext:
            readOnlyRootFilesystem: true
        containerSecurityContext:
          readOnlyRootFilesystem: true
      cluster:
        enabled: false
      mishards:
        securityContext:
          readOnlyRootFilesystem: true
      admin:
        enabled: true
        securityContext:
          readOnlyRootFilesystem: true
      securityContext:
        readOnlyRootFilesystem: true
  seldon-core-operator:
    containersSecurityContext:
      readOnlyRootFilesystem: true
  argo-common-workflows:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  question-answering:
    containersSecurityContext:
      readOnlyRootFilesystem: true
  classification:
    containersSecurityContext:
      readOnlyRootFilesystem: true

Supported services

The table below lists the services that support a read-only root file system, the ones that have it enabled by default, and the Fusion release in which support was added:


Chart Name	Pod Name	Container Name	Supported	Default enabled	Supported version
admin-ui	admin-ui	admin-ui	✅	✅	5.9.9+
api-gateway	api-gateway	init/api-gateway	✅	✅	5.9.9+
api-gateway	api-gateway	api-gateway	✅	✅	5.9.9+
api-gateway	api-gateway	generate-jks	✅	✅	5.9.9+
apps-manager	apps-manager	apps-manager	✅	✅	5.9.9+
argo	argo-server	argo-server	✅	❌	5.9.10+
argo	argo-executor	executor	✅	❌	5.9.10+
argo	argo-mainContainer	mainContainer	✅	❌	5.9.10+
argo	argo-controller	controller	✅	✅	5.9.10+
argo/minio	minio	minio	✅	❌	5.9.10+
argo/minio	minio	minio	✅	❌	5.9.10+
argo/minio	make-bucket-job	minio-mc	✅	❌	5.9.10+
argo-common-workflows	delete-model	init/main/wait	✅	❌	5.9.10+
argo-common-workflows	deploy-model	init/main/wait	✅	❌	5.9.10+
argo-common-workflows	milvus-maintenance	init/main/wait	✅	❌	5.9.10+
argo-common-workflows	upload-model-to-cloud	init/main/wait	✅	❌	5.9.10+
async-parsing	async-parsing	tika-server	✅	✅	5.9.9+
async-parsing	async-parsing	async-parsing	✅	✅	5.9.9+
auth-ui	auth-ui	auth-ui	✅	✅	5.9.9+
classic-rest-service	classic-rest-service	init/import-certs	✅	✅	5.9.10+
classic-rest-service	classic-rest-service	classic-rest-service	✅	✅	5.9.9+
classification	argo/classification	init/wait/main	✅	❌	5.9.10+
connector-plugin	connector-plugin	init/import-certs	✅	❌	5.9.10+
connector-plugin	connector-plugin	connector-plugin	✅	✅	5.9.9+
connectors	connectors	connectors	✅	✅	5.9.9+
connectors-backend	CRD		✅	✅	5.9.9+
connectors-backend	connectors-backend	connectors-backend	✅	✅	5.9.9+
fusion-admin	fusion-admin	admin	✅	✅	5.9.9+
fusion-commons		check-admin	✅	✅	5.9.9+
fusion-commons		check-api-gateway	✅	✅	5.9.9+
fusion-commons		check-indexing	✅	✅	5.9.9+
fusion-commons		check-kafka	✅	✅	5.9.9+
fusion-commons		check-logstash	✅	✅	5.9.9+
fusion-commons		check-pulsar	✅	✅	5.9.9+
fusion-commons		setup-keystore-and-properties	✅	✅	5.9.9+
fusion-commons		check-zk	✅	✅	5.9.9+
fusion-config-sync	fusion-config-sync	fusion-config-sync	✅	✅	5.9.9+
fusion-data-augmentation	argo/data-augmentation/volume-fix	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/init-workspace	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/write-job-configs	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/write-io-configs	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/add-zkhost	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/pull-data-training-and-metadata	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/pull-data-training-and-metadata-cloud	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/volume-fix2	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/synonym-list	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/download-synonym-dictionary	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/keystroke-list	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/download-keystroke-blob	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/augment	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/push-augmented-data	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/push-augmented-data-cloud	init/main/wait	✅	❌	5.9.10+
fusion-indexing	fusion-indexing	fusion-indexing	✅	✅	5.9.9+
fusion-resources	fusion-resources-secret-hook	setup-keystore-and-properties	✅	✅	5.9.10+
insights	insights	insights	✅	✅	5.9.9+
job-launcher	job-launcher	job-launcher	✅	✅	5.9.9+
job-launcher	job-launcher	kubectl-runner	✅	✅	5.9.9+
job-launcher	job-launcher-spark-cleanup	kubectl-runner	✅	✅	5.9.9+
job-launcher	spark-kubernetes-driver	spark-kubernetes-driver	✅	❌	5.9.10+
job-launcher	spark-kubernetes-executor	spark-kubernetes-executor	✅	❌	5.9.10+
job-rest-server	job-rest-server	job-rest-server	✅	✅	5.9.9+
kafka	kafka-metrics	kafka-exporter-archived	✅	❌	5.9.10+
kafka	kafka-provisioning	init/wait-for-available-kafka	✅	❌	5.9.10+
kafka	kafka-provisioning	kafka-provisioning	✅	❌	5.9.10+
kafka	kafka	kafka	✅	❌	5.9.10+
kafka	kafka	jmx-exporter	✅	❌	5.9.10+
kafka	kafka	init/check-zk	✅	✅	5.9.9+
kafka	kafka	init/auto-discovery	✅	❌	5.9.10+
kafka	kafka	init/volume-permissions	✅	❌	5.9.10+
lwai-gateway	lwai-gateway	lwai-gateway	✅	✅	5.9.9+
ml-model-service	ml-model-service	java-service	✅	✅	5.9.9+
ml-model-service	ml-model-service-namespace-hook	kubectl-runner	✅	✅	5.9.9+
ml-model-service/ambassador	ambassador	ambassador	✅	❌	5.9.10+
ml-model-service/ambassador	ambassador	prometheus-exporter	✅	❌	5.9.10+
ml-model-service/milvus	milvus-writable	milvus	✅	❌	5.9.10+
ml-model-service/milvus	milvus-writable	init/wait-for-mysql	✅	✅	5.9.10+
ml-model-service/milvus	milvus-writable	init/create-for-share-storage	✅	❌	5.9.10+
ml-model-service/milvus	milvus-admin	admin	✅	❌	5.9.10+
ml-model-service/milvus	milvus-admin	init/wait-for-milvus	✅	✅	5.9.10+
ml-model-service/milvus	milvus-mishards	init/wait-for-mysql	✅	✅	5.9.10+
ml-model-service/milvus	milvus-mishards	init/wait-for-mysql	✅	✅	5.9.10+
ml-model-service/milvus	milvus-mishards	mishards	✅	❌	5.9.10+
ml-model-service/milvus/mysql	mysql	init/remove-lost-found	✅	✅	5.9.10+
ml-model-service/milvus/mysql	mysql	mysql	✅	❌	5.9.10+
pm-ui	pm-ui	pm-ui	✅	✅	5.9.9+
query-pipeline	query-pipeline	query-pipeline	✅	✅	5.9.9+
question-answering	argo/qna-coldstart/init-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/write-job-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/write-io-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/write-io-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/add-zkHost	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/pull-data	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/pull-data-cloud	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/train	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/list-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/post-model	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/apply-seldon-deployment	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/init-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/write-job-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/write-io-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/add-zkhost	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/pull-qa-data-cloud	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/pull-qa-data	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/train	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/train-with-texts	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/list-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/post-model	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/apply-seldon-deployment	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/init-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/write-job-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/write-io-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/add-zkhost	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/list-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/pull-eval-data	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/pull-eval-data-cloud	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/evaluate	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/push-eval-results-cloud	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/push-eval-results	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/init-workspace	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/write-job-configs	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/write-io-configs	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/add-zkhost	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/pull-data-training-and-metadata	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/pull-data-training-and-metadata-cloud	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/train-with-metadata	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/train-without-metadata	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/push-recommendations	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/push-recommendations-cloud	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/add-default-exclude-query	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/init-workspace	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/copy-model	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/write-job-configs	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/write-io-configs	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/add-zkhost	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/pull-data	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/pull-data-cloud	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/train	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/push-content	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/push-content-cloud	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/add-default-exclude-query	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/delete-old-content-recommendations	init/wait/main	✅	❌	5.9.10+
reverse-search	reverse-search	init/set-reverse-search-zone	✅	✅	5.9.10+
reverse-search	reverse-search	init/check-zk	✅	✅	5.9.9+
reverse-search	reverse-search	init/enable-tls-in-reverse-search	✅	✅	5.9.10+
reverse-search	reverse-search	reverse-search	✅	❌	5.9.10+
rules-ui	rules-ui	rules-ui	✅	✅	5.9.9+
seldon-core-operator	seldon-controller-manager	manager	✅	❌	5.9.10+
seldon-core-operator	seldon-spartakus-volunteer	seldon-spartakus-volunteer	✅	❌	5.9.9+
seldon-core-operator	crd/SeldonDeployment		✅/❌	❌	5.9.9+
solr	solr	init/set-solr-zone	✅	✅	5.9.10+
solr	solr	init/enable-tls-in-solr	✅	✅	5.9.10+
solr	configset-bootstrap	configset-bootstrap	✅	✅	5.9.10+
solr	solr	solr	✅	❌	5.9.10+
solr	solr-exporter	exporter	✅	❌	5.9.10+
solr	solr-exporter	init/solr-init	✅	✅	5.9.10+
solr-managed	configset-bootstrap	configset-bootstrap	✅	✅	5.9.10+
solr-managed	exporter	exporter	✅	❌	5.9.10+
solr-managed	exporter	init/solr-init	✅	✅	5.9.10+
solr-managed	solr	init/set-solr-zone	✅	✅	5.9.10+
solr-managed	solr	init/enable-tls-in-solr	✅	✅	5.9.10+
solr-managed	solr	solr	✅	❌	5.9.10+
solr-managed	solr	process-raw	✅	❌	5.9.10+
solr-backup-runner	solr-backup-runner-backup	solr-backups	✅	❌	5.9.10+
solr-backup-runner	solr-backup-runner-prune	solr-prune	✅	❌	5.9.10+
templating	templating	templating	✅	✅	5.9.9+
webapps	webapps	webapps	✅	✅	5.9.9+
zookeeper	zookeeper	zookeeper	✅	❌	5.9.10+
zookeeper	zookeeper	jmx-exporter	✅	❌	5.9.10+
zookeeper	zookeeper	zookeeper-exporter	✅	❌	5.9.10+
zookeeper	zookeeper-chroots	main	✅	❌	5.9.10+

Introduction to Fusion

Getting Data In

Getting Data Out

Operations

Reference

Developer Docs

Neural Hybrid Search

Release Notes

General upgrade process

Natively supported deployment upgrades

Other Kubernetes deployment upgrades

Upgrades with Helm v3

Helm upgrade script

Considerations for on-prem deployments

Solr

Docker images

Helm charts

Nodes

Pod resources

Fusion gateway service

Deploying with a read-only root file system

Solr

Zookeeper

Kafka

ML Model Service

Argo

Seldon Core Operator

Argo Common Workflows

Question Answering

Classification

All-in-one values.yaml example

All-in-one values.yaml with optional services example

Supported services

Introduction to Fusion

Getting Data In

Getting Data Out

Operations

Reference

Developer Docs

Neural Hybrid Search

Release Notes

​Considerations for on-prem deployments

​Solr

​Docker images

​Helm charts

​Nodes

​Pod resources

​Fusion gateway service

​Deploying with a read-only root file system

Considerations for on-prem deployments

Solr

Docker images

Helm charts

Nodes

Pod resources

Fusion gateway service

Deploying with a read-only root file system