Fusion Microservices
Fusion is comprised of microservices that drive features and functionality within a deployment. The services running in your deployment depend on the version of Fusion and the features you have enabled.
Get deployment details
You can view your deployment details using kubectl.
-
First, verify that you have access to your cluster, replacing the example values with your parameters. This example is for a Fusion instance deployed using GKE.
gcloud container clusters get-credentials EXAMPLE-CLUSTER --region EXAMPLE-REGION --project EXAMPLE-PROJECT -
Get the ports and services:
kubectl get svc -n EXAMPLE-NAME -
Get the StatefulSets:
kubectl get statefulsets -n EXAMPLE-NAME -
Get the deployments:
kubectl get deploy -n EXAMPLE-NAME
Overview
The table below lists the Fusion microservices deployed by the Helm chart. It also include Kubernetes services that manage traffic to microservices.
| Fusion is a complex distributed application composed of many stateful and stateless services designed to support demanding search-oriented workloads at high scale. |
For Docker image versions associated with microservices, see the list of Docker images and versions for each Fusion release.
| Microservice | Required for Fusion | Protocol | Deployment or StatefulSet | Node Pool Assignment | Autoscaling Supported | Description |
|---|---|---|---|---|---|---|
|
Yes |
REST/HTTP |
Deployment |
system |
Not required. Minimum of 1, but 2 pods are recommended for high availability. |
Exposes endpoints for admin tasks, such as creating applications and running jobs. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the admin UI. |
|
No |
HTTP |
Deployment |
system |
Yes (CPU or custom metric). |
Orchestrates parallel jobs on Kubernetes. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Stores logs and prior Argo workflow runs. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the login form. |
|
No |
REST/HTTP |
StatefulSet |
analytics or system |
Not supported. |
REST service for supporting non-RPC connector plugins. This microservice was previously named classic-rest-service. |
|
No |
REST/HTTP |
Deployment |
analytics or system |
Not required. One pod is enough for most clusters. |
Routes REST API requests to connectors-classic and connectors-rpc. |
|
No |
gRPC |
Deployment |
analytics or system |
Yes (CPU or custom metric). |
gRPC service for managing SDK-based connector plugins. |
|
No |
HTTP/TCP |
Deployment |
analytics or system |
Yes (CPU or custom metric). |
Deployment for each connector plugin type. Note: There is a base deployment, connector-plugin with 0 replicas. This is used as a deployment template for each connector plugin type. It should not be deleted or scaled. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the DevOps UI. Available in Fusion 5.0 through 5.5. |
|
Yes |
REST/HTTP |
Deployment |
search or analytics depending on write-volume |
Yes (CPU or custom metric). |
Processes indexing requests. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves the App Insights UI |
|
No |
REST/HTTP |
Deployment |
analytics |
Not required. One pod is enough for most clusters. |
Configures and launches the Spark driver pod for running Spark jobs |
|
No |
REST/HTTP |
Deployment |
analytics |
Not required. One pod is enough for most clusters. |
Performs admin tasks for creating and running Spark jobs. |
|
No |
HTTP |
Deployment |
analytics |
Not required. One pod is enough for most clusters. |
Jupyter notebook for ad hoc analytics and visualization. |
|
No |
HTTP |
StatefulSet |
system |
Not required. Minimum of 1, but 2 pods are recommended for high availability. |
Collects logs from the other microservices and either indexes into system_logs or ships them to an external service like Elastic |
|
No |
REST/HTTP |
Deployment |
analytics or system |
Not required. One pod is enough for most clusters. |
Dense Vector Search Engine for ML models active. |
|
No |
REST/HTTP |
Deployment |
analytics or system |
Not required. One pod is enough for most clusters. |
Handles metadata for Milvus service active. |
|
No |
REST/HTTP and gRPC |
Deployment |
search |
Yes (CPU or custom metric). |
Exposes gRPC endpoints for generating predictions from ML models. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the Predictive Merchandiser app. |
|
Yes |
HTTP |
Deployment |
search |
Not required. Minimum of 1, but 2 pods are recommended for high availability. |
Performs authentication, authorization, and traffic routing. |
|
Yes |
HTTP |
StatefulSet |
search |
At least 3 nodes in high availability. You need to run 3 or 5 to keep quorum. |
Write Ahead Log (WAL) used for persistent message storage. |
|
Yes |
HTTP and TCP |
Deployment |
search |
At least 3 nodes in high availability. |
Contains REST API for managing administration and dispatcher for handling all message transfers. |
|
Yes |
REST/HTTP |
Deployment |
search |
Yes (CPU or custom metric). |
Processes query requests. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the Rules UI. |
|
No |
Web |
Deployment |
system |
Not required. Minimum of 1, but 2 pods are recommended for high availability. |
Load balancing and proxy for Seldon Core deployments. |
|
No |
REST/GRPC |
Deployment |
system |
Yes (CPU or custom metric). |
Serves models built in any model building framework. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Maintains Seldon Core deployments for ML model serving active. |
|
Yes |
HTTP |
StatefulSet |
At least 3 nodes in search, 2 in analytics, and 2 in system |
Yes (CPU or custom metric). |
Search engine. |
|
No |
n/a |
single pod per job |
analytics or dedicated Node Pool for Spark jobs |
One per job. |
Launched by the job-launcher to run a Spark job |
|
No |
n/a |
one or more pods launched by the Spark driver for executing job tasks |
analytics or dedicated Node Pool for Spark jobs |
Depends on job configuration. Controlled by the spark.executor.instances setting. |
Executes tasks for a Spark job |
|
No |
REST/HTTP and JDBC |
Deployment |
analytics |
Not required. One pod is enough for most clusters. |
Performs admin tasks for creating and managing SQL catalog assets. Exposes a JDBC endpoint for the SQL service. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Retrieves and renders Predictive Merchandiser templates. |
|
No |
HTTP |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Is responsible for text extraction using Tika Server Parser |
|
No |
REST/HTTP |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves App Studio-based Web apps. |
|
Yes |
TCP |
StatefulSet |
system |
No. You need to run 1, 3, or 5 ZooKeeper pods to keep quorum. Do not use HPA for scaling ZooKeeper. |
Stores centralized configuration and performs distributed coordination tasks. |
Ports used by Fusion
Below you will find the list of pod ports for intra-cluster communications.
| Service | Port |
|---|---|
|
8765 |
|
8080 |
|
8080 |
|
9020, 5701 |
|
9010 |
|
8771 |
|
9000 |
|
8080 |
|
2746 |
|
8888 |
|
9092, 9093 |
|
9092, 9093 |
|
80, 443 |
|
3306 |
|
3181, 8000 |
|
8080, 6650 |
|
8989 |
|
8983 |
|
8983 |
|
2181, 2281 |
|
2181, 3888, 2888, 2281 |
|
8765 |
|
8080 |
|
8083 |
|
8081 |
|
19530, 19121 |
|
6565 |
|
8086 |
|
8080 |
|
8080 |
|
9091 |
|
6764 |
|
8080, 6650 |
|
8787 |
|
8080 |
|
443 |
|
8088 |
|
8085 |
|
8088 |
|
80 |
|
5250 |
|
9998 |
|
8780 |