Product

Fusion 5.4

Fusion Microservices

Overview

The table below lists the Fusion microservices deployed by the Helm chart.

Fusion is a complex distributed application composed of many stateful and stateless services designed to support demanding search-oriented workloads at high scale.
Microservice Protocol Deployment or StatefulSet Node Pool Assignment Autoscaling Supported Description

admin

REST/HTTP

Deployment

system

Not required. Minimum of 1 but 2 pods are recommended for HA.

Exposes endpoints for admin tasks, such as creating applications and running jobs.

admin-ui

Web

Deployment

system

Not required. One pod is sufficient for most clusters.

Serves static Web assets for the admin UI.

argo

HTTP

Deployment

system

Yes (CPU or custom metric).

Orchestrates parallel jobs on Kubernetes.

argo-ui

Web

Deployment

system

Not required. One pod is sufficient for most clusters.

Stores logs and prior Argo workflow runs.

auth-ui

Web

Deployment

system

Not required. One pod is sufficient for most clusters.

Serves static Web assets for the login form.

classic-rest-service

REST/HTTP

StatefulSet

analytics or system

Yes (CPU or custom metric).

REST service for supporting non-RPC connector plugins.

connectors

REST/HTTP

Deployment

analytics or system

Not required. One pod is sufficient for most clusters.

Routes REST API requests to classic-rest-service and connectors-rpc.

connectors-backend

gRPC

Deployment

analytics or system

Yes (CPU or custom metric).

gRPC service for managing SDK-based connector plugins.

connector-plugin-<connector_plugin>

HTTP/TCP

Deployment

analytics or system

Yes (CPU or custom metric).

Deployment for each connector plugin type. Note: There is a base deployment, connector-plugin with 0 replicas. This is used as a deployment template for each connector plugin type. It should not be deleted or scaled.

devops-ui

Web

Deployment

system

Not required. One pod is sufficient for most clusters.

Serves static Web assets for the DevOps UI.

indexing

REST/HTTP

Deployment

search or analytics depending on write-volume

Yes (CPU or custom metric).

Processes indexing requests.

insights

Web

Deployment

system

Not required. One pod is sufficient for most clusters.

Serves the App Insights UI

job-launcher

REST/HTTP

Deployment

analytics

Not required. One pod is sufficient for most clusters.

Configures and launches the Spark driver pod for running Spark jobs

job-rest-server

REST/HTTP

Deployment

analytics

Not required. One pod is sufficient for most clusters.

Performs admin tasks for creating and running Spark jobs.

jupyter

HTTP

Deployment

analytics

Not required. One pod is sufficient for most clusters.

Jupyter notebook for ad hoc analytics and visualization.

logstash

HTTP

StatefulSet

system

Not required. Minimum of 1 but 2 pods are recommended for HA.

Collects logs from the other microservices and either indexes into system_logs or ships them to an external service like Elastic

milvus

REST/HTTP

Deployment

analytics or system

Not required. One pod is sufficient for most clusters.

Dense Vector Search Engine for ML models active.

milvus-mysql

REST/HTTP

Deployment

analytics or system

Not required. One pod is sufficient for most clusters.

Handles metadata for Milvus service active.

ml-model-service

REST/HTTP and gRPC

Deployment

search

Yes (CPU or custom metric).

Exposes gRPC endpoints for generating predictions from ML models.

pm-ui

Web

Deployment

system

Not required. One pod is sufficient for most clusters.

Serves static Web assets for the Predictive Merchandiser app.

proxy / api-gateway

HTTP

Deployment

search

Not required. Minimum of 1 but 2 pods are recommended for HA.

Performs authentication, authorization, and traffic routing.

pulsar-bookkeeper

HTTP

StatefulSet

search

At least 3 nodes in HA.. You need to run 3 or 5 to keep quorum.

Write Ahead Log (WAL) used for persistent message storage.

pulsar-broker

HTTP and TCP

Deployment

search

At least 3 nodes in HA.

Contains REST API for managing administration and dispatcher for handling all message transfers.

query

REST/HTTP

Deployment

search

Yes (CPU or custom metric).

Processes query requests.

rules-ui

Web

Deployment

system

Not required. One pod is sufficient for most clusters.

Serves static Web assets for the Rules UI.

seldon-ambassador

Web

Deployment

system

Not required. Minimum of 1 but 2 pods are recommended for HA.

Load balancing and proxy for Seldon Core deployments.

seldon-core

REST/GRPC

Deployment

system

Yes (CPU or custom metric).

Serves models built in any model building framework.

seldon-webhook-service

Web

Deployment

system

Not required. One pod is sufficient for most clusters.

Maintains Seldon Core deployments for ML model serving active.

solr

HTTP

StatefulSet

At least 3 nodes in search, 2 in analytics, and 2 in system

Yes (CPU or custom metric).

Search engine.

spark-driver

n/a

single pod per job

analytics or dedicated Node Pool for Spark jobs

One per job.

Launched by the job-launcher to run a Spark job

spark-executor

n/a

one or more pods launched by the Spark driver for executing job tasks

analytics or dedicated Node Pool for Spark jobs

Depends on job configuration. Controlled by the spark.executor.instances setting.

Executes tasks for a Spark job

sql-service

REST/HTTP and JDBC

Deployment

analytics

Not required. One pod is sufficient for most clusters.

Performs admin tasks for creating and managing SQL catalog assets.

Exposes a JDBC endpoint for the SQL service.

templating

Web

Deployment

system

Not required. One pod is sufficient for most clusters.

Retrieves and renders Predictive Merchandiser templates.

tikaserver

HTTP

Deployment

system

Not required. One pod is sufficient for most clusters.

Is responsible for text extraction using Tika Server Parser

webapps

REST/HTTP

Deployment

system

Not required. One pod is sufficient for most clusters.

Serves App Studio-based Web apps.

zookeeper

TCP

StatefulSet

system

No. You need to run 1, 3, or 5 ZooKeeper pods to keep quorum. Do not use HPA for scaling ZooKeeper.

Stores centralized configuration and performs distributed coordination tasks.

Ports used by Fusion

Below you will find the list of ports required to access Fusion services.

Service Port

admin-ui

8080

api-gateway

6764

auth-ui

8080

classic-rest-service

9000

connector-plugin-service

9020

devops-ui

8080

dnsAccess

53

fusion-admin

8765

fusion-indexing

8765

fusion-jupyter

8888

insights

8080

job-launcher

8083

job-rest-server

8081

kubernetesAccess

443

ml-model-service

8086

ml-model-service ambassador

8877

ml-model-service milvus

3000

ml-model-service mysql

3306

pm-ui

8080

pulsar

8080, 8443, 6650, 6651, 9527

query-pipeline

8787

rest-service

9010

rpc-service

8771, 8772

rules-ui

8080

solr

8983, 9983

solr-backup-runner

8983

sql-service

5250

templating

5250

webapps

8780

zookeeper

2181, 2888, 3888, 2281, 9141

Transport Layer Security (TLS)

When enabled, Fusion generates a TLS certificate for each pod when the pod starts. This allows Fusion to use the Kubernetes endpoints API to reach each pod by its IP address and perform load balancing, circuit breaking, and retries in the Fusion microservices.