Security - Lucidworks documentation

ImportantLucidworks recommends not virus scanning the fusion/data folder. Virus scanning can cause slow performance, and it can cause downtime if it quarantines an index file identified as a possible virus.

Fusion uses a number of security measures:

Authenticating UI users – Fusion authenticates users when they log in. Logging in creates a new Fusion session. Fusion also authenticates users when the Sessions REST API creates a session.
Authorizing UI users – Fusion authorizes users to use specific parts of the Fusion UI.
UI users must also be authorized to make API requests, because the UI makes API requests.
Authenticating and authorizing users who make API requests
Password Encryption- Fusion uses 128-bit AES keys to encrypt passwords and “AES/CBC/PKCS7Padding” for the cipher. The ciphertext is also signed.
Using session cookies
Using an external authentication provider (optional) - A security realm can specify use of an external authentication provider, such as LDAP, JWT, or SAML.
Using SSL/TLS to ensure that data in flight between your application and Fusion is not observable (optional) - See SSL Security (Unix) or SSL Security (Windows).
Constraining the documents that are indexed (optional)
Trimming the documents that are returned by queries based on authorization (optional)

SSL Security (Unix)

Fusion’s UI (which is accessed through the Fusion Proxy service) can run over SSL for secure communication with any HTTP client, using the Java Secure Socket Extension (JSSE) framework. You configure Fusion for SSL by configuring Fusion’s Proxy and UI services.

Fusion 4.0.2 or later is required to enable SSL security. Version 4.0.2 fixed a bug in the authentication proxy that incorrectly redirected some requests when SSL is enabled.

Required software

Configuring Fusion for SSL requires the following software:

Java Development Kit. To store certificates, you can use the Java keytool Key and Certificate Management utility which is a part of the JDK. The JDK is also a requirement for Fusion Server.
OpenSSL. You might need the openssl command line tool:
- If you have the certificate chain and private key as separate files, then you can use the openssl command line tool to create a PKCS #12 file.
- If you have an intermediate CA certificate, then you can use it and openssl to generate the certificate chain and private key files.

Load an SSL certificate into a Fusion keystore

The SSL protocol is based on public-key cryptography where encryption keys come in public key/private key pairs. An SSL certificate is used to verify the authenticity of a particular server. It contains the web site name, contact email address, company information and the public key used to encrypt the communication which is shared with the entities that communicate with the owner of the public/private key pair.The server has a locally-protected private key that is accessible via a JSEE keystore.The keystore maintains both the server certificate and the private key, so that when a server authenticates itself to the client, it uses the private key from its keystore for the initial SSL handshake.Load the certificate into a Fusion keystore. Perform the tasks in the appropriate section:

Self-signed certificate. If Fusion is behind a firewall, you can use a self-signed certificate for SSL communication with other hosts in your internal network. Create a keystore for the Fusion Proxy service and load the keystore with the self-signed PKCS #12 certificate.
Certificate signed by a certificate authority. In a production environment, SSL certificates typically originate with certificate signing requests (CSRs) and are signed by a trusted third-party Certificate Authority (CA). Create a keystore for the Fusion Proxy service and load the keystore with the PKCS #12 certificate from a CA.

Alternative 1: Self-signed certificate

If you are using a CSR-originated certificate from a trusted certificate authority, proceed to Alternative 2: CA-signed certificate.

If Fusion is behind a firewall, you can use a self-signed certificate for SSL communication with other hosts in your internal network. Create a keystore for the Fusion Proxy service and load the keystore with the self-signed PKCS #12 certificate.To store certificates, you can use the Java keytool Key and Certificate Management utility which is a part of the JDK.How to create a keystore and load a self-signed certificate

Set environment variables:

export JAVA_HOME=JavaHomeDirectory
export FUSION_HOME=FusionHomeDirectory

For example, in Fusion 4.1:

export JAVA_HOME=/usr/java/jdk1.8.0_171
export FUSION_HOME=/opt/lucidworks/fusion/4.1.2

In Fusion 4.0.x:

export JAVA_HOME=/usr/java/jdk1.8.0_171
export FUSION_HOME=/opt/lucidworks/fusion/4.0.2

Create the Fusion Proxy service keystore, generate the key pair and self-signed certificate, and load them into the keystore:

"$JAVA_HOME/bin/keytool" -genkeypair -keystore "$FUSION_HOME/apps/jetty/proxy/etc/keystore" -dname "CN=CommonName, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown" -keypass KeyPassword -storepass KeystorePassword -keyalg RSA -alias selfsigned -deststoretype pkcs12 -ext SAN=dns:ServerFqdn,ip:ServerIpAddress

You must include the qualified domain name and/or the IP address of the Fusion server in the -ext SAN part of the command. Failure to do so results in SSL validation errors.

Example command:

"$JAVA_HOME/bin/keytool" -genkeypair -keystore "$FUSION_HOME/apps/jetty/proxy/etc/keystore" -dname "CN=search.mycorp, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown" -keypass 59Winter.Is.Long45 -storepass 46I.Prefer.Vanilla24 -keyalg RSA -alias selfsigned -deststoretype pkcs12 -ext SAN=dns:search.mycorp,ip:192.168.1.40,dns:localhost

The resulting certificate enables validated SSL transport to these hosts:

https://search.mycorp:ProxyPort
https://192.168.1.40:ProxyPort
https://localhost:ProxyPort

Where ProxyPort is the Fusion Proxy port.

Alternative 2: CA-signed certificate

If Fusion is behind a firewall and you are using a self-signed certificate, skip this section. Perform the tasks in Alternative 1: Self-signed certificate.

In a production environment, SSL certificates typically originate with certificate signing requests (CSRs) and are signed by a trusted third-party Certificate Authority (CA).The steps here assume that you are the person who will be obtaining the SSL certificate chain and private key files. If you are not that person, contact your system administrator.

Preliminary steps

Obtain a domain from a domain registrar.
Change the A record of your domain to the public IP address of your web server instance.

Generate SSL certificate files

Use an SSL certificate provider to generate the certificate chain and private key files, or a PKCS #12 certificate, from a trusted CA:

Certificate chain and private key files. In this case, you will need to convert these files into a single certificate file in PKCS #12 format.
A PKCS #12 certificate. This must contain both the certificate chain and private key. In this case, no conversion is necessary.

How to obtain certificate files

In most cases, you will need to temporarily open ports 80 and 443 in your firewall configuration. The SSL certificate provider must be able to make successful HTTP and HTTPS requests to your server through the Domain Name System (DNS).
Use an SSL certificate provider to generate the certificate chain (fullchain.pem) and private key (privkey.pem) files, or the PKCS #12 certificate, from a trusted CA. Steps will vary based on the certificate provider. Contact your certificate provider for details.
Close ports 80 and 443 in your firewall configuration.
Change the A record of your domain to the public domain-name address of your web server instance.

Proceed to the next sections as follows:

If you have certificate chain and private key files, perform the steps in Convert the certificate chain and private key files to a PKCS #12 certificate and Import the PKCS #12 certificate into the Fusion Proxy service keystore.
If you have a PKCS #12 certificate, perform the steps in Import the PKCS #12 certificate into the Fusion Proxy service keystore.

Convert the certificate chain and private key files to a PKCS #12 certificate

If you have a PKCS #12 certificate, skip this section and proceed to the section Import the PKCS #12 certificate into the Fusion Proxy service keystore.

With the certificate chain and private key as separate files, use the openssl command line tool in OpenSSL to create a PKCS #12 certificate.

openssl pkcs12 -export -out /path/to/keystore.p12 -in /path/to/fullchain.pem -inkey /path/to/privkey.pem

Do not enter a blank password.

Create the Fusion Proxy service keystore and import the PKCS #12 certificate

Use the Java keytool Key and Certificate Management utility to create a keystore for the Fusion Proxy service ($FUSION_HOME/apps/jetty/proxy/etc/keystore) and import the PKCS #12 certificate file. Fusion uses this certificate to perform SSL.

To create the keystore and import the PKCS #12 certificate:

Use the keytool import command to create a JSSE keystore.

keytool -importkeystore -srckeystore /path/to/keystore.p12 -srcstoretype PKCS12 -destkeystore "$FUSION_HOME/apps/jetty/proxy/etc/keystore" -deststoretype PKCS12

(Optional) If desired, delete the PKCS #12 certificate file that resides outside of the Fusion Proxy service keystore (the one you created from the certificate chain and private key files, or obtained from a trusted CA.
```
rm /path/to/keystore.p12
```

Enable HTTPS in the Fusion Proxy service

Before beginning these steps, load an SSL certificate into a Fusion keystore.How to enable HTTPS in the Fusion Proxy service:

(Only for Fusion Server 4.0.x and 4.1.0) Prevent the start.jar program from downloading a default keystore file, which is not needed. Edit $FUSION_HOME/apps/jetty/home/modules/ssl.mod. Comment out the indicated line using #. Change:

[files]
https://raw.githubusercontent.com/eclipse/jetty.project/master/jetty-server/src/test/config/etc/keystore?id=${jetty.tag.version}|etc/keystore

To:

[files]
# https://raw.githubusercontent.com/eclipse/jetty.project/master/jetty-server/src/test/config/etc/keystore?id=${jetty.tag.version}|etc/keystore

Set environment variables:

export JAVA_HOME=JavaHomeDirectory
export FUSION_HOME=FusionHomeDirectory

For example, in Fusion 4.1:

export JAVA_HOME=/usr/java/jdk1.8.0_171
export FUSION_HOME=/opt/lucidworks/fusion/4.1.2

In Fusion 4.0:

export JAVA_HOME=/usr/java/jdk1.8.0_171
export FUSION_HOME=/opt/lucidworks/fusion/4.0.2

Add HTTPS protocol support to the Jetty TLS (SSL) connector:

cd "$FUSION_HOME/apps/jetty/proxy/"
java -jar "$FUSION_HOME/apps/jetty/home/start.jar" --add-to-start=https

Example output:

INFO: ssl             initialized (transitively) in ${jetty.base}/start.ini
INFO: https           initialized in ${jetty.base}/start.ini
INFO: Base directory was modified

Get the obfuscated version of your keystore password:

java -cp "$FUSION_HOME/apps/jetty/home/lib/*" org.eclipse.jetty.util.security.Password PASSWORD

Replace PASSWORD with the password you used for the keystore. If the password contains special characters, URL encode them. Example output:

2018-05-15 12:32:48.988:INFO::main: Logging initialized @133ms
password345XYZ
OBF:1vn21ugu1saj1v9i1v941sar1ugw1vo0
MD5:b91cd1a54781790beaa2baf741fa6789

Edit the file $FUSION_HOME/apps/jetty/proxy/start.ini:

Include obfuscated passwords by adding these properties to the end of the file:

* `jetty.sslContext.keyStorePassword`
* `jetty.sslContext.keyManagerPassword`
* `jetty.sslContext.trustStorePassword`

Use the OBF-encrypted password from step 4 (including the OBF: string) as the value for all three of the properties. For example:

## Keystore password
jetty.sslContext.keyStorePassword=OBF:2uha1vgt1jg01a4b1a4j1jda1vg11ugg

...

## KeyManager password
jetty.sslContext.keyManagerPassword=OBF:2uha1vgt1jg01a4b1a4j1jda1vg11ugg

## Truststore password
jetty.sslContext.trustStorePassword=OBF:2uha1vgt1jg01a4b1a4j1jda1vg11ugg

Set the local SSL port by adding the jetty.ssl.port property to the end of the file, and providing the port number. For example:
```
## Connector port to listen on
jetty.ssl.port=8443
```
Save the file $FUSION_HOME/apps/jetty/proxy/start.ini.

Restart Fusion and test access through HTTPS

Restart all Fusion services:
```
./bin/fusion restart
```
HTTPS should now be enabled in the Fusion Proxy service.
Sign in to the Fusion UI. Specify the HTTPS URL scheme and SSL port, for example, https://search.mycorp:8443.

Disable HTTP access to the Fusion Proxy service

Disable HTTP access. You have a choice. Perform the tasks in the appropriate section:

Disable HTTP access on the firewall or load balancer - This is the preferred approach.
Disable listening for HTTP requests in the Fusion Proxy service

Alternative 1: Disable HTTP access on the firewall or load balancer

Disable HTTP access to the Fusion Proxy service on the firewall or load balancer:

Disallow all requests for port 8764 from the outside world. Only localhost should be able to communicate with Fusion on the non-SSL port 8764. Block all other requestors.
If you are using a firewall or load balancer in front of Fusion, use it to redirect all HTTP requests to use HTTPS instead. For example, Apache would redirect all incoming HTTP traffic to HTTPS.

Alternative 2: Disable listening for HTTP requests in the Fusion Proxy service

Ideally, you should disable HTTP access using the firewall or load balancer. Follow the steps in this section only if disabling HTTP access on the firewall or load balancer is not feasible.

You can only use this alternative if your SSL certificate covers a hostname that can be accessed from the local host. For example, if your certificate only covers https://fusion.com, then your local machine must be able to access Fusion from that exact host. If necessary, change the hosts file so that this can work.How to disable HTTP

Edit /opt/lucidworks/fusion/latest.x/apps/jetty/proxy/start.d/http.ini.
1. Change this line:
  --module=http
  To:
  #--module=http
2. Save the file.
Edit the Fusion configuration file, /opt/lucidworks/fusion/latest.x/conf/fusion.cors (fusion.properties in Fusion 4.x).
1. Ensure that the Agent JVM uses the Fusion Proxy service’s keystore by adding this to the end of the file:
  agent.jvmOptions=-Djavax.net.ssl.trustStore="${FUSION_HOME}/apps/jetty/proxy/etc/keystore" -Djavax.net.ssl.trustStorePassword=PASSWORD -Djavax.net.ssl.keyStore="${FUSION_HOME}/apps/jetty/proxy/etc/keystore" -Djavax.net.ssl.keyStorePassword=PASSWORD
  Replace PASSWORD with your Fusion keystore password.
2. Uncomment the default.address and change it to the hostname of the server that is validated by your SSL certificate. If the hostname saved in default.address is not validated by your SSL certificate, then the Fusion Proxy service will not start, because the agent’s liveness detector will not be able to access the HTTPS port to determine whether Fusion is running.
  If you self-signed the certificate, then the default.address must match the hostname you specified while signing the certificate. Failure to do this will result in the Fusion Proxy service not starting after you have disabled HTTP.
  For example, if your SSL certificate’s validated hostname is search.mycorp, then change:
  #default.address = 127.0.0.1
  To:
  default.address = search.mycorp
3. Change the proxy.port to the SSL port you chose.
4. Uncomment proxy.ssl and change its value to true. Change:
  # proxy.ssl=false
  To:
  proxy.ssl=true

Stopping and starting Fusion is needed for these changes to take effect. But there is a bit more to do concerning the Fusion Agent before restarting Fusion.

References and tutorials

Transport Layer Security (Wikipedia)
Public Key Certificate (Wikipedia)
OpenSSL Cookbook (free ebook)
OpenSSL Command Line Utilities (OpenSSL wiki)
Java Tutorials: Generating and Verifying Certificates
IBM developerWorks: What is the JSSE all about?

SSL Security (Windows)

Fusion 4.0.2 or later is required to enable SSL security. Version 4.0.2 fixed a bug in the authentication proxy that incorrectly redirected some requests when SSL is enabled.

Required software

Configuring Fusion for SSL requires the following software:

Java Development Kit. To store certificates, you can use the Java keytool Key and Certificate Management utility which is a part of the JDK. The JDK is also a requirement for Fusion Server.
OpenSSL. You might need the openssl command line tool:
- If you have the certificate chain and private key as separate files, then you can use the openssl command line tool to create a PKCS #12 file.
- If you have an intermediate CA certificate, then you can use it and openssl to generate the certificate chain and private key files.

Overview of procedure

How to configure Fusion for SSL:

Load an SSL certificate into a Fusion keystore.
Enable SSL in the Fusion Proxy service.
Restart Fusion and test access through HTTPS.
Disable HTTP access to the Fusion Proxy service.
Ensure that the Fusion Agent can do heartbeats against the Proxy service.
The heartbeat service is deprecated in Fusion 4.2+.

Load an SSL certificate into a Fusion keystore

Self-signed certificate. If Fusion is behind a firewall, you can use a self-signed certificate for SSL communication with other hosts in your internal network. Create a keystore for the Fusion Proxy service and load the keystore with the self-signed PKCS #12 certificate.
Certificate signed by a certificate authority. In a production environment, SSL certificates typically originate with certificate signing requests (CSRs) and are signed by a trusted third-party Certificate Authority (CA). Create a keystore for the Fusion Proxy service and load the keystore with the PKCS #12 certificate from a CA.

Alternative 1: Self-signed certificate

If you are using a CSR-originated certificate from a trusted certificate authority, proceed to Alternative 2: CA-signed certificate.

Set environment variables:

set JAVA_HOME=JavaHomeDirectory
set FUSION_HOME=FusionHomeDirectory

Create the Fusion Proxy service keystore, generate the key pair and self-signed certificate, and load them into the keystore:

"%JAVA_HOME%\bin\keytool.exe" -genkeypair -keystore "%FUSION_HOME%\apps\jetty\proxy\etc\keystore" -dname "CN=CommonName, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown" -keypass KeyPassword -storepass KeystorePassword -keyalg RSA -alias selfsigned -deststoretype pkcs12 -ext SAN=dns:ServerFqdn,ip:ServerIpAddress

You must include the qualified domain name and/or the IP address of the Fusion server in the -ext SAN part of the command. Failure to do so results in SSL validation errors.

Example command:

"%JAVA_HOME%\bin\keytool.exe" -genkeypair -keystore "%FUSION_HOME%\apps\jetty\proxy\etc\keystore" -dname "CN=CommonName, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown" -keypass 59Winter.Is.Long45 -storepass 46I.Prefer.Vanilla24 -keyalg RSA -alias selfsigned -deststoretype pkcs12 -ext SAN=dns:search.mycorp,ip:192.168.1.40,dns:localhost

The resulting certificate enables validated SSL transport to these hosts:

https://search.mycorp:ProxyPort
https://192.168.1.40:ProxyPort
https://localhost:ProxyPort

Where ProxyPort is the Fusion Proxy port.

Alternative 2: CA-signed certificate

If Fusion is behind a firewall and you are using a self-signed certificate, skip this section. Perform the tasks in Alternative 1: Self-signed certificate.

Preliminary steps

Obtain a domain from a domain registrar.
Change the A record of your domain to the public IP address of your web server instance.

Generate SSL certificate files

Use an SSL certificate provider to generate the certificate chain and private key files, or a PKCS #12 certificate, from a trusted CA:

Certificate chain and private key files. In this case, you will need to convert these files into a single certificate file in PKCS #12 format.
A PKCS #12 certificate that contains both the certificate chain and private key. In this case, no conversion is necessary.

How to obtain certificate files

In most cases, you will need to temporarily open ports 80 and 443 in your firewall configuration. The SSL certificate provider must be able to make successful HTTP and HTTPS requests to your server through the Domain Name System (DNS).
Use an SSL certificate provider to generate the certificate chain (fullchain.pem) and private key (privkey.pem) files, or the PKCS #12 certificate, from a trusted CA. Steps will vary based on the certificate provider. Contact your certificate provider for details.
Close ports 80 and 443 in your firewall configuration.
Change the A record of your domain to the public domain-name address of your web server instance.

Proceed to the next sections as follows:

If you have certificate chain and private key files, perform the steps in Convert the certificate chain and private key files to a PKCS #12 certificate and Import the PKCS #12 certificate into the Fusion Proxy service keystore.
If you have a PKCS #12 certificate, perform the steps in Import the PKCS #12 certificate into the Fusion Proxy service keystore.

Convert the certificate chain and private key files to a PKCS #12 certificate

If you have a PKCS #12 certificate, skip this section and proceed to the section Import the PKCS #12 certificate into the Fusion Proxy service keystore.

How to create a PKCS #12 certificateWith the certificate chain and private key as separate files, use the openssl command line tool in OpenSSL to create a PKCS #12 certificate.

openssl pkcs12 -export -out \path\to\keystore.p12 -in \path\to\fullchain.pem -inkey \path\to\privkey.pem

Do not enter a blank password.

Create the Fusion Proxy service keystore and import the PKCS #12 certificate

Use the Java keytool Key and Certificate Management utility to create a keystore for the Fusion Proxy service (%FUSION_HOME%\apps\jetty\proxy\etc\keystore) and import the PKCS #12 certificate file. Fusion uses this certificate to perform SSL.

If you are starting with a certificate file in PFX format (file extension .pfx), that format is now identical to PKCS #12 format.

To create the keystore and import the PKCS #12 certificate:

"%JAVA_HOME%\bin\keytool.exe" -genkeypair -alias mykeystore -keyalg RSA -keysize 2048 -keystore keystore.jks

"%JAVA_HOME%\bin\keytool.exe" -importkeystore -srckeystore mycertificate.p12 -srcstoretype PKCS12 -destkeystore keystore.jks

Use the keytool import command to create a JSSE keystore.

keytool -importkeystore -srckeystore \path\to\keystore.p12 -srcstoretype PKCS12 -destkeystore "%FUSION_HOME%\apps\jetty\proxy\etc\keystore" -deststoretype PKCS12

(Optional) If desired, delete the PKCS #12 certificate file that resides outside of the Fusion Proxy service keystore (the one you created from the certificate chain and private key files, or obtained from a trusted CA.
```
del \path\to\keystore.p12
```

Enable HTTPS in the Fusion Proxy service

Before beginning these steps, load an SSL certificate into a Fusion keystore.How to enable HTTPS in the Fusion Proxy service:

Set environment variables:

set JAVA_HOME=JavaHomeDirectory
set FUSION_HOME=FusionHomeDirectory

Add HTTPS protocol support to the Jetty TLS (SSL) connector:

cd "%FUSION_HOME%\apps\jetty\proxy\"
java -jar "%FUSION_HOME%\apps\jetty\home\start.jar" --add-to-start=https

Example output:

INFO: ssl             initialized (transitively) in ${jetty.base}/start.ini
INFO: https           initialized in ${jetty.base}/start.ini
INFO: Base directory was modified

Get the obfuscated version of your keystore password:

java -cp "$FUSION_HOME/apps/jetty/home/lib/*" org.eclipse.jetty.util.security.Password PASSWORD

Replace PASSWORD with the password you used for the keystore. If the password contains special characters, URL encode them. Example output:

2018-05-15 12:32:48.988:INFO::main: Logging initialized @133ms
password345XYZ
OBF:1vn21ugu1saj1v9i1v941sar1ugw1vo0
MD5:b91cd1a54781790beaa2baf741fa6789

Edit the file %FUSION_HOME%\apps\jetty\proxy\start.ini:
1. Include obfuscated passwords by adding these properties to the end of the file:
  - jetty.sslContext.keyStorePassword
  - jetty.sslContext.keyManagerPassword
  - jetty.sslContext.trustStorePassword
2. Use the OBF-encrypted password from step 4 (including the OBF: string) as the value for all three of the properties. For example:
  ## Keystore password jetty.sslContext.keyStorePassword=OBF:2uha1vgt1jg01a4b1a4j1jda1vg11ugg ... ## KeyManager password jetty.sslContext.keyManagerPassword=OBF:2uha1vgt1jg01a4b1a4j1jda1vg11ugg ## Truststore password jetty.sslContext.trustStorePassword=OBF:2uha1vgt1jg01a4b1a4j1jda1vg11ugg
3. Set the local SSL port by adding the jetty.ssl.port property to the end of the file, and providing the port number. For example:
  ## Connector port to listen on jetty.ssl.port=8443
4. Save the file %FUSION_HOME%\apps\jetty\proxy\start.ini.

Restart Fusion and test access through HTTPS

Restart all Fusion services:
```
bin\fusion.cmd restart
```
HTTPS should now be enabled in the Fusion Proxy service.
Sign in to the Fusion UI. Specify the HTTPS URL scheme and SSL port, for example, https://search.mycorp:8443.

Disable HTTP access to the Fusion Proxy service

Disable HTTP access. You have a choice. Perform the tasks in the appropriate section:

Disable HTTP access on the firewall or load balancer - This is the preferred approach.
Disable listening for HTTP requests in the Fusion Proxy service

Alternative 1: Disable HTTP access on the firewall or load balancer

Disable HTTP access to the Fusion Proxy service on the firewall or load balancer:

Disallow all requests for port 8764 from the outside world. Only localhost should be able to communicate with Fusion on the non-SSL port 8764. Block all other requestors.
If you are using a firewall or load balancer in front of Fusion, use it to redirect all HTTP requests to use HTTPS instead. For example, Apache would redirect all incoming HTTP traffic to HTTPS.

Alternative 2: Disable listening for HTTP requests in the Fusion Proxy service

Ideally, you should disable HTTP access using the firewall or load balancer. Follow the steps in this section only if disabling HTTP access on the firewall or load balancer is not feasible.

Edit \lucidworks\fusion{backslash}latest.x\apps\jetty\proxy\start.d\http.ini.
1. Change this line:
  --module=http
  To:
  #--module=http
2. Save the file.
Edit the Fusion configuration file, \lucidworks\fusion{backslash}latest.x\conf\fusion.cors (fusion.properties in Fusion 4.x).
1. Ensure that the Agent JVM uses the Fusion Proxy service’s keystore by adding this to the end of the file:
  agent.jvmOptions=-Djavax.net.ssl.trustStore="%{FUSION_HOME}%\\apps\\jetty\\proxy\\etc\\keystore" -Djavax.net.ssl.trustStorePassword=PASSWORD -Djavax.net.ssl.keyStore="%{FUSION_HOME}%\\apps\\jetty\\proxy\\etc\\keystore" -Djavax.net.ssl.keyStorePassword=PASSWORD
  Replace PASSWORD with your Fusion keystore password.
2. Uncomment the default.address and change it to the hostname of the server that is validated by your SSL certificate. If the hostname saved in default.address is not validated by your SSL certificate, then the Fusion Proxy service will not start, because the agent’s liveness detector will not be able to access the HTTPS port to determine whether Fusion is running.
  If you self-signed the certificate, then the default.address must match the hostname you specified while signing the certificate. Failure to do this will result in the Fusion Proxy service not starting after you have disabled HTTP.
  For example, if your SSL certificate’s validated hostname is search.mycorp, then change:
  #default.address = 127.0.0.1
  To:
  default.address = search.mycorp
3. Change the proxy.port to the SSL port you chose. For example, change:
  proxy.port = 8764
  To:
  proxy.port = 8443
4. Uncomment proxy.ssl and change its value to true. Change:
  # proxy.ssl=false
  To:
  proxy.ssl=true

Stop and start Fusion for these changes to take effect.

References and tutorials

Transport Layer Security (Wikipedia)
Public Key Certificate (Wikipedia)
OpenSSL Cookbook (free ebook)
OpenSSL Command Line Utilities (OpenSSL wiki)
Java Tutorials: Generating and Verifying Certificates
IBM developerWorks: What is the JSSE all about?

When logging into the Fusion UI, a user provides a username and password, as well as their assigned security realm. An administrator must specify these in Fusion (using the native security realm) or configure Fusion to use an external authentication provider (for example, LDAP or SAML). See Access control. Fusion uses roles defined by permissions to authorize Fusion UI access and perform tasks in Fusion, including searching. The recommended method to delegate permissions is as follows:

Assign each user to a role and create custom roles as needed.
Assign permissions on a per-app basis.

Manage users with security realms

Fusion uses security realms to authenticate users of the Fusion UI. Each user has an assigned security realm, which the user must select when logging in. If the user selects a different realm, authentication fails. A security realm also provides a list of roles as follows:

The list always includes the role(s) that are specified in the security realm.
(Optional) If an external directory service (such as LDAP) is used for authentication, the list can also contain roles that are mapped from the names of the directory-service groups. That is, you can configure a security realm to return group information for users from that same directory service.
(Optional) The security realm can reference one or more Fusion roles or, when using an external directory service provider, use group membership information from the provider to determine roles for users. Fusion maps the group names to role names and adds these roles to the user’s list of roles.

Fusion does not use permissions from LDAP to authorize UI access or API requests. It only obtains group names (optionally), which are used as role names or are mapped to role names. If an Active Directory Security Query Trimming Stage is used, then directory-service permissions are used for trimming. If a connector supports security trimming, then connector permissions are used for trimming.

Per-request authentication

Requests to the Fusion REST API must specify a security realm for per-request authentication, unless a session cookie is used (which contains information about the security realm). Fusion authorizes requested operations based on API permissions specified for the user and for the user’s role(s). Fusion considers the role(s) specified in the user definition and in the security realm. Fusion creates a list of roles when a session is created, that is, when a user logs in or when the Sessions REST API creates a session. Authorization based on permissions and its layering is at request time. You can define multiple security realms for a Fusion instance. A Fusion instance can manage multiple security realms, which allows users from different domains to have (different levels of) access to specific Fusion collections.

Read-only root file system

Fusion 5.9.9 and up supports a read-only root file system to safeguard against unauthorized modifications, protecting your deployment against malicious software and other attacks. Most services operate efficiently in this mode, and for components that need write access, a separate writable mount is available using specific helm configurations. Read-only mode should only be enabled for services that do not require CRUD access. See Enable Read-Only Root File System for examples of how to configure this feature.

Enable Read-Only Root File System

This topic describes how to configure a read-only root file system for a Fusion deployment. Read-only mode safeguards your file system against unauthorized changes, such as by malicious software or other attacks. Enabling this feature entails configuring the readOnlyRootFilesystem attribute in the Fusion Helm chart.

This feature is only available in Fusion 5.9.9 and later versions of Fusion 5.9.

Beginning with Fusion 5.9.10, all of Fusion’s services are designed to work with a read-only root file system, but some external services may require write access to the file system. In that case, you can configure a read-only root file system and mount a separate writable file system for these services.The examples below show how to configure a read-only root file system for various Fusion services.

Solr

fusion:
  solr:
    containerSecurityContext:
      readOnlyRootFilesystem: true

With optional services:

fusion:
  solr:
    exporter:
      enabled: true
      securityContext:
        readOnlyRootFilesystem: true
    containerSecurityContext:
      readOnlyRootFilesystem: true

Zookeeper

fusion:
  zookeeper:
    containerSecurityContext:
      readOnlyRootFilesystem: true

With optional services:

fusion:
  zookeeper:
    containerSecurityContext:
      readOnlyRootFilesystem: true
    exporters:
      jmx:
        enabled: true
      zookeeper:
        enabled: true
    jobs:
      chroots:
        enabled: true
        config:
          create:
            - /root-read-only

Kafka

fusion:
  kafka:
    containerSecurityContext:
      readOnlyRootFilesystem: true

With optional services:

fusion:
  kafka:
    containerSecurityContext:
      readOnlyRootFilesystem: true
    volumePermissions:
      enabled: true
      containerSecurityContext:
        readOnlyRootFilesystem: true
    metrics:
      kafka:
        containerSecurityContext:
          readOnlyRootFilesystem: true
          enabled: true
      jmx:
        enabled: true
        containerSecurityContext:
          enabled: true
          readOnlyRootFilesystem: true
    externalAccess:
      enabled: true
      autoDiscovery:
        enabled: true
        containerSecurityContext:
          enabled: true
          readOnlyRootFilesystem: true
    provisioning:
      enabled: true
      topics:
      - name: test
        partitions: 1
        replicationFactor: 1
        ## https://kafka.apache.org/documentation/#topicconfigs
        config:
          max.message.bytes: 64000
          flush.messages: 1
      containerSecurityContext:
        readOnlyRootFilesystem: true
        enabled: true

ML Model Service

fusion:
  ml-model-service:
    ambassador:
      containerSecurityContext:
        readOnlyRootFilesystem: true
    milvus:
      mysql:
        containerSecurityContext:
          readOnlyRootFilesystem: true
      securityContext:
        readOnlyRootFilesystem: true
With optional services:
fusion:
  ml-model-service:
    ambassador:
      containerSecurityContext:
        readOnlyRootFilesystem: true
      prometheusExporter:
        enabled: true
        securityContext:
          readOnlyRootFilesystem: true
    milvus:
      mysql:
        metrics:
          enabled: true
          securityContext:
            readOnlyRootFilesystem: true
        containerSecurityContext:
          readOnlyRootFilesystem: true
      cluster:
        enabled: false
      mishards:
        securityContext:
          readOnlyRootFilesystem: true
      admin:
        enabled: true
        securityContext:
          readOnlyRootFilesystem: true
      securityContext:
        readOnlyRootFilesystem: true

Argo

fusion:
  argo:
    minio:
      containerSecurityContext:
        readOnlyRootFilesystem: true
      extraVolumes:
        - name: config-dir
          emptyDir: {}
        - name: certs-dir
          emptyDir: {}
      extraVolumeMounts:
        - name: config-dir
          mountPath: /.minio
        - name: certs-dir
          mountPath: /etc/minio/certs
    server:
      securityContext:
        readOnlyRootFilesystem: true
    controller:
      securityContext:
        readOnlyRootFilesystem: true
    mainContainer:
      securityContext:
        readOnlyRootFilesystem: true
    executor:
      securityContext:
        readOnlyRootFilesystem: true

Seldon Core Operator

fusion:
  seldon-core-operator:
    containersSecurityContext:
      readOnlyRootFilesystem: true

Argo Common Workflows

fusion:
  argo-common-workflows:
    containerSecurityContext:
      readOnlyRootFilesystem: true

Question Answering

fusion:
  question-answering:
    containersSecurityContext:
      readOnlyRootFilesystem: true

Classification

fusion:
  classification:
    containersSecurityContext:
      readOnlyRootFilesystem: true

All-in-one values.yaml example

fusion:
  solr:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  solr-managed:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  zookeeper:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  kafka:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  ml-model-service:
    ambassador:
      containerSecurityContext:
        readOnlyRootFilesystem: true
    milvus:
      mysql:
        containerSecurityContext:
          readOnlyRootFilesystem: true
      securityContext:
        readOnlyRootFilesystem: true
  argo:
    minio:
      containerSecurityContext:
        readOnlyRootFilesystem: true
      extraVolumes:
        - name: config-dir
          emptyDir: {}
        - name: certs-dir
          emptyDir: {}
      extraVolumeMounts:
        - name: config-dir
          mountPath: /.minio
        - name: certs-dir
          mountPath: /etc/minio/certs
    server:
      securityContext:
        readOnlyRootFilesystem: true
    controller:
      securityContext:
        readOnlyRootFilesystem: true
    mainContainer:
      securityContext:
        readOnlyRootFilesystem: true
    executor:
      securityContext:
        readOnlyRootFilesystem: true
  seldon-core-operator:
    containersSecurityContext:
      readOnlyRootFilesystem: true
  argo-common-workflows:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  question-answering:
    containersSecurityContext:
      readOnlyRootFilesystem: true
  classification:
    containersSecurityContext:
      readOnlyRootFilesystem: true

All-in-one values.yaml with optional services example

fusion:
  argo:
    minio:
      persistence:
        enabled: false
      containerSecurityContext:
        readOnlyRootFilesystem: true
      extraVolumes:
        - name: config-dir
          emptyDir: {}
        - name: certs-dir
          emptyDir: {}
      extraVolumeMounts:
        - name: config-dir
          mountPath: /.minio
        - name: certs-dir
          mountPath: /etc/minio/certs
    server:
      securityContext:
        readOnlyRootFilesystem: true
    controller:
      securityContext:
        readOnlyRootFilesystem: true
    mainContainer:
      securityContext:
        readOnlyRootFilesystem: true
    executor:
      securityContext:
        readOnlyRootFilesystem: true
  solr:
    exporter:
      enabled: true
      securityContext:
        readOnlyRootFilesystem: true
    containerSecurityContext:
      readOnlyRootFilesystem: true
    # tls:
    #   enabled: true
  solr-managed:
    exporter:
      enabled: true
      securityContext:
        readOnlyRootFilesystem: true
    enableExternalFiles: true
    enabledStorage:
    - gcs
    processRaw:
      image:
        repository: fusion-dev-docker.ci-artifactory.lucidworks.com
      cloudRoot: gs://lw-managed-fusion-data/tmp
      gcs:
        secret: gcs-key
        secretFieldName: key.json
      securityContext:
        readOnlyRootFilesystem: true
    containerSecurityContext:
      readOnlyRootFilesystem: true
    # tls:
    #   enabled: true
  zookeeper:
    containerSecurityContext:
      readOnlyRootFilesystem: true
    exporters:
      jmx:
        enabled: true
      zookeeper:
        enabled: true
    jobs:
      chroots:
        # enabled: true
        config:
          create:
            - /root-read-only
  kafka:
    containerSecurityContext:
      readOnlyRootFilesystem: true
    volumePermissions:
      enabled: true
      image:
        repository: "fusion-dev-docker.ci-artifactory.lucidworks.com/os-shell"
      containerSecurityContext:
        readOnlyRootFilesystem: true
    metrics:
      kafka:
        containerSecurityContext:
          readOnlyRootFilesystem: true
          enabled: true
      jmx:
        enabled: true
        containerSecurityContext:
          enabled: true
          readOnlyRootFilesystem: true
    externalAccess:
      enabled: true
      autoDiscovery:
        enabled: true
        containerSecurityContext:
          enabled: true
          readOnlyRootFilesystem: true
    provisioning:
      enabled: true
      topics:
      - name: test
        partitions: 1
        replicationFactor: 1
        ## https://kafka.apache.org/documentation/#topicconfigs
        config:
          max.message.bytes: 64000
          flush.messages: 1
      containerSecurityContext:
        readOnlyRootFilesystem: true
        enabled: true
  ml-model-service:
    enabled: true
    ambassador:
      containerSecurityContext:
        readOnlyRootFilesystem: true
      prometheusExporter:
        enabled: true
        securityContext:
          readOnlyRootFilesystem: true
    milvus:
      mysql:
        metrics:
          enabled: true
          securityContext:
            readOnlyRootFilesystem: true
        containerSecurityContext:
          readOnlyRootFilesystem: true
      cluster:
        enabled: false
      mishards:
        securityContext:
          readOnlyRootFilesystem: true
      admin:
        enabled: true
        securityContext:
          readOnlyRootFilesystem: true
      securityContext:
        readOnlyRootFilesystem: true
  seldon-core-operator:
    containersSecurityContext:
      readOnlyRootFilesystem: true
  argo-common-workflows:
    containerSecurityContext:
      readOnlyRootFilesystem: true
  question-answering:
    containersSecurityContext:
      readOnlyRootFilesystem: true
  classification:
    containersSecurityContext:
      readOnlyRootFilesystem: true

Supported services

The table below lists the services that support a read-only root file system, the ones that have it enabled by default, and the Fusion release in which support was added:


Chart Name	Pod Name	Container Name	Supported	Default enabled	Supported version
admin-ui	admin-ui	admin-ui	✅	✅	5.9.9+
api-gateway	api-gateway	init/api-gateway	✅	✅	5.9.9+
api-gateway	api-gateway	api-gateway	✅	✅	5.9.9+
api-gateway	api-gateway	generate-jks	✅	✅	5.9.9+
apps-manager	apps-manager	apps-manager	✅	✅	5.9.9+
argo	argo-server	argo-server	✅	❌	5.9.10+
argo	argo-executor	executor	✅	❌	5.9.10+
argo	argo-mainContainer	mainContainer	✅	❌	5.9.10+
argo	argo-controller	controller	✅	✅	5.9.10+
argo/minio	minio	minio	✅	❌	5.9.10+
argo/minio	minio	minio	✅	❌	5.9.10+
argo/minio	make-bucket-job	minio-mc	✅	❌	5.9.10+
argo-common-workflows	delete-model	init/main/wait	✅	❌	5.9.10+
argo-common-workflows	deploy-model	init/main/wait	✅	❌	5.9.10+
argo-common-workflows	milvus-maintenance	init/main/wait	✅	❌	5.9.10+
argo-common-workflows	upload-model-to-cloud	init/main/wait	✅	❌	5.9.10+
async-parsing	async-parsing	tika-server	✅	✅	5.9.9+
async-parsing	async-parsing	async-parsing	✅	✅	5.9.9+
auth-ui	auth-ui	auth-ui	✅	✅	5.9.9+
classic-rest-service	classic-rest-service	init/import-certs	✅	✅	5.9.10+
classic-rest-service	classic-rest-service	classic-rest-service	✅	✅	5.9.9+
classification	argo/classification	init/wait/main	✅	❌	5.9.10+
connector-plugin	connector-plugin	init/import-certs	✅	❌	5.9.10+
connector-plugin	connector-plugin	connector-plugin	✅	✅	5.9.9+
connectors	connectors	connectors	✅	✅	5.9.9+
connectors-backend	CRD		✅	✅	5.9.9+
connectors-backend	connectors-backend	connectors-backend	✅	✅	5.9.9+
fusion-admin	fusion-admin	admin	✅	✅	5.9.9+
fusion-commons		check-admin	✅	✅	5.9.9+
fusion-commons		check-api-gateway	✅	✅	5.9.9+
fusion-commons		check-indexing	✅	✅	5.9.9+
fusion-commons		check-kafka	✅	✅	5.9.9+
fusion-commons		check-logstash	✅	✅	5.9.9+
fusion-commons		check-pulsar	✅	✅	5.9.9+
fusion-commons		setup-keystore-and-properties	✅	✅	5.9.9+
fusion-commons		check-zk	✅	✅	5.9.9+
fusion-config-sync	fusion-config-sync	fusion-config-sync	✅	✅	5.9.9+
fusion-data-augmentation	argo/data-augmentation/volume-fix	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/init-workspace	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/write-job-configs	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/write-io-configs	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/add-zkhost	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/pull-data-training-and-metadata	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/pull-data-training-and-metadata-cloud	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/volume-fix2	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/synonym-list	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/download-synonym-dictionary	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/keystroke-list	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/download-keystroke-blob	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/augment	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/push-augmented-data	init/main/wait	✅	❌	5.9.10+
fusion-data-augmentation	argo/data-augmentation/push-augmented-data-cloud	init/main/wait	✅	❌	5.9.10+
fusion-indexing	fusion-indexing	fusion-indexing	✅	✅	5.9.9+
fusion-resources	fusion-resources-secret-hook	setup-keystore-and-properties	✅	✅	5.9.10+
insights	insights	insights	✅	✅	5.9.9+
job-launcher	job-launcher	job-launcher	✅	✅	5.9.9+
job-launcher	job-launcher	kubectl-runner	✅	✅	5.9.9+
job-launcher	job-launcher-spark-cleanup	kubectl-runner	✅	✅	5.9.9+
job-launcher	spark-kubernetes-driver	spark-kubernetes-driver	✅	❌	5.9.10+
job-launcher	spark-kubernetes-executor	spark-kubernetes-executor	✅	❌	5.9.10+
job-rest-server	job-rest-server	job-rest-server	✅	✅	5.9.9+
kafka	kafka-metrics	kafka-exporter-archived	✅	❌	5.9.10+
kafka	kafka-provisioning	init/wait-for-available-kafka	✅	❌	5.9.10+
kafka	kafka-provisioning	kafka-provisioning	✅	❌	5.9.10+
kafka	kafka	kafka	✅	❌	5.9.10+
kafka	kafka	jmx-exporter	✅	❌	5.9.10+
kafka	kafka	init/check-zk	✅	✅	5.9.9+
kafka	kafka	init/auto-discovery	✅	❌	5.9.10+
kafka	kafka	init/volume-permissions	✅	❌	5.9.10+
lwai-gateway	lwai-gateway	lwai-gateway	✅	✅	5.9.9+
ml-model-service	ml-model-service	java-service	✅	✅	5.9.9+
ml-model-service	ml-model-service-namespace-hook	kubectl-runner	✅	✅	5.9.9+
ml-model-service/ambassador	ambassador	ambassador	✅	❌	5.9.10+
ml-model-service/ambassador	ambassador	prometheus-exporter	✅	❌	5.9.10+
ml-model-service/milvus	milvus-writable	milvus	✅	❌	5.9.10+
ml-model-service/milvus	milvus-writable	init/wait-for-mysql	✅	✅	5.9.10+
ml-model-service/milvus	milvus-writable	init/create-for-share-storage	✅	❌	5.9.10+
ml-model-service/milvus	milvus-admin	admin	✅	❌	5.9.10+
ml-model-service/milvus	milvus-admin	init/wait-for-milvus	✅	✅	5.9.10+
ml-model-service/milvus	milvus-mishards	init/wait-for-mysql	✅	✅	5.9.10+
ml-model-service/milvus	milvus-mishards	init/wait-for-mysql	✅	✅	5.9.10+
ml-model-service/milvus	milvus-mishards	mishards	✅	❌	5.9.10+
ml-model-service/milvus/mysql	mysql	init/remove-lost-found	✅	✅	5.9.10+
ml-model-service/milvus/mysql	mysql	mysql	✅	❌	5.9.10+
pm-ui	pm-ui	pm-ui	✅	✅	5.9.9+
query-pipeline	query-pipeline	query-pipeline	✅	✅	5.9.9+
question-answering	argo/qna-coldstart/init-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/write-job-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/write-io-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/write-io-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/add-zkHost	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/pull-data	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/pull-data-cloud	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/train	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/list-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/post-model	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-coldstart/apply-seldon-deployment	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/init-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/write-job-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/write-io-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/add-zkhost	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/pull-qa-data-cloud	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/pull-qa-data	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/train	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/train-with-texts	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/list-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/post-model	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-supervised/apply-seldon-deployment	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/init-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/write-job-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/write-io-configs	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/add-zkhost	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/list-workspace	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/pull-eval-data	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/pull-eval-data-cloud	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/evaluate	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/push-eval-results-cloud	init/wait/main	✅	❌	5.9.10+
question-answering	argo/qna-evaluation/push-eval-results	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/init-workspace	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/write-job-configs	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/write-io-configs	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/add-zkhost	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/pull-data-training-and-metadata	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/pull-data-training-and-metadata-cloud	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/train-with-metadata	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/train-without-metadata	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/push-recommendations	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/push-recommendations-cloud	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/add-default-exclude-query	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-user/init-workspace	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/copy-model	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/write-job-configs	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/write-io-configs	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/add-zkhost	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/pull-data	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/pull-data-cloud	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/train	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/push-content	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/push-content-cloud	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/add-default-exclude-query	init/wait/main	✅	❌	5.9.10+
recommender	argo/item-recommender-content/delete-old-content-recommendations	init/wait/main	✅	❌	5.9.10+
reverse-search	reverse-search	init/set-reverse-search-zone	✅	✅	5.9.10+
reverse-search	reverse-search	init/check-zk	✅	✅	5.9.9+
reverse-search	reverse-search	init/enable-tls-in-reverse-search	✅	✅	5.9.10+
reverse-search	reverse-search	reverse-search	✅	❌	5.9.10+
rules-ui	rules-ui	rules-ui	✅	✅	5.9.9+
seldon-core-operator	seldon-controller-manager	manager	✅	❌	5.9.10+
seldon-core-operator	seldon-spartakus-volunteer	seldon-spartakus-volunteer	✅	❌	5.9.9+
seldon-core-operator	crd/SeldonDeployment		✅/❌	❌	5.9.9+
solr	solr	init/set-solr-zone	✅	✅	5.9.10+
solr	solr	init/enable-tls-in-solr	✅	✅	5.9.10+
solr	configset-bootstrap	configset-bootstrap	✅	✅	5.9.10+
solr	solr	solr	✅	❌	5.9.10+
solr	solr-exporter	exporter	✅	❌	5.9.10+
solr	solr-exporter	init/solr-init	✅	✅	5.9.10+
solr-managed	configset-bootstrap	configset-bootstrap	✅	✅	5.9.10+
solr-managed	exporter	exporter	✅	❌	5.9.10+
solr-managed	exporter	init/solr-init	✅	✅	5.9.10+
solr-managed	solr	init/set-solr-zone	✅	✅	5.9.10+
solr-managed	solr	init/enable-tls-in-solr	✅	✅	5.9.10+
solr-managed	solr	solr	✅	❌	5.9.10+
solr-managed	solr	process-raw	✅	❌	5.9.10+
solr-backup-runner	solr-backup-runner-backup	solr-backups	✅	❌	5.9.10+
solr-backup-runner	solr-backup-runner-prune	solr-prune	✅	❌	5.9.10+
templating	templating	templating	✅	✅	5.9.9+
webapps	webapps	webapps	✅	✅	5.9.9+
zookeeper	zookeeper	zookeeper	✅	❌	5.9.10+
zookeeper	zookeeper	jmx-exporter	✅	❌	5.9.10+
zookeeper	zookeeper	zookeeper-exporter	✅	❌	5.9.10+
zookeeper	zookeeper-chroots	main	✅	❌	5.9.10+

The table below lists the services that support a read-only root file system, the ones that have it enabled by default, and the Fusion release in which support was added.

Chart Name	Pod Name	Container Name	Supported	Default enabled	Supported version
`admin-ui`	`admin-ui`	`admin-ui`	✅	✅	5.9.9+
`api-gateway`	`api-gateway`	`init/api-gateway`	✅	✅	5.9.9+
`api-gateway`	`api-gateway`	`api-gateway`	✅	✅	5.9.9+
`api-gateway`	`api-gateway`	`generate-jks`	✅	✅	5.9.9+
`apps-manager`	`apps-manager`	`apps-manager`	✅	✅	5.9.9+
`argo`	`argo-server`	`argo-server`	✅	❌	5.9.10+
`argo`	`argo-executor`	`executor`	✅	❌	5.9.10+
`argo`	`argo-mainContainer`	`mainContainer`	✅	❌	5.9.10+
`argo`	`argo-controller`	`controller`	✅	✅	5.9.10+
`argo/minio`	`minio`	`minio`	✅	❌	5.9.10+
`argo/minio`	`minio`	`minio`	✅	❌	5.9.10+
`argo/minio`	`make-bucket-job`	`minio-mc`	✅	❌	5.9.10+
`argo-common-workflows`	`delete-model`	`init/main/wait`	✅	❌	5.9.10+
`argo-common-workflows`	`deploy-model`	`init/main/wait`	✅	❌	5.9.10+
`argo-common-workflows`	`milvus-maintenance`	`init/main/wait`	✅	❌	5.9.10+
`argo-common-workflows`	`upload-model-to-cloud`	`init/main/wait`	✅	❌	5.9.10+
`async-parsing`	`async-parsing`	`tika-server`	✅	✅	5.9.9+
`async-parsing`	`async-parsing`	`async-parsing`	✅	✅	5.9.9+
`auth-ui`	`auth-ui`	`auth-ui`	✅	✅	5.9.9+
`classic-rest-service`	`classic-rest-service`	`init/import-certs`	✅	✅	5.9.10+
`classic-rest-service`	`classic-rest-service`	`classic-rest-service`	✅	✅	5.9.9+
`classification`	`argo/classification`	`init/wait/main`	✅	❌	5.9.10+
`connector-plugin`	`connector-plugin`	`init/import-certs`	✅	❌	5.9.10+
`connector-plugin`	`connector-plugin`	`connector-plugin`	✅	✅	5.9.9+
`connectors`	`connectors`	`connectors`	✅	✅	5.9.9+
`connectors-backend`	`CRD`		✅	✅	5.9.9+
`connectors-backend`	`connectors-backend`	`connectors-backend`	✅	✅	5.9.9+
`fusion-admin`	`fusion-admin`	`admin`	✅	✅	5.9.9+
`fusion-commons`		`check-admin`	✅	✅	5.9.9+
`fusion-commons`		`check-api-gateway`	✅	✅	5.9.9+
`fusion-commons`		`check-indexing`	✅	✅	5.9.9+
`fusion-commons`		`check-kafka`	✅	✅	5.9.9+
`fusion-commons`		`check-logstash`	✅	✅	5.9.9+
`fusion-commons`		`check-pulsar`	✅	✅	5.9.9+
`fusion-commons`		`setup-keystore-and-properties`	✅	✅	5.9.9+
`fusion-commons`		`check-zk`	✅	✅	5.9.9+
`fusion-config-sync`	`fusion-config-sync`	`fusion-config-sync`	✅	✅	5.9.9+
`fusion-data-augmentation`	`argo/data-augmentation/volume-fix`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/init-workspace`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/write-job-configs`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/write-io-configs`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/add-zkhost`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/pull-data-training-and-metadata`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/pull-data-training-and-metadata-cloud`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/volume-fix2`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/synonym-list`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/download-synonym-dictionary`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/keystroke-list`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/download-keystroke-blob`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/augment`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/push-augmented-data`	`init/main/wait`	✅	❌	5.9.10+
`fusion-data-augmentation`	`argo/data-augmentation/push-augmented-data-cloud`	`init/main/wait`	✅	❌	5.9.10+
`fusion-indexing`	`fusion-indexing`	`fusion-indexing`	✅	✅	5.9.9+
`fusion-resources`	`fusion-resources-secret-hook`	`setup-keystore-and-properties`	✅	✅	5.9.10+
`insights`	`insights`	`insights`	✅	✅	5.9.9+
`job-launcher`	`job-launcher`	`job-launcher`	✅	✅	5.9.9+
`job-launcher`	`job-launcher`	`kubectl-runner`	✅	✅	5.9.9+
`job-launcher`	`job-launcher-spark-cleanup`	`kubectl-runner`	✅	✅	5.9.9+
`job-launcher`	`spark-kubernetes-driver`	`spark-kubernetes-driver`	✅	❌	5.9.10+
`job-launcher`	`spark-kubernetes-executor`	`spark-kubernetes-executor`	✅	❌	5.9.10+
`job-rest-server`	`job-rest-server`	`job-rest-server`	✅	✅	5.9.9+
`kafka`	`kafka-metrics`	`kafka-exporter-archived`	✅	❌	5.9.10+
`kafka`	`kafka-provisioning`	`init/wait-for-available-kafka`	✅	❌	5.9.10+
`kafka`	`kafka-provisioning`	`kafka-provisioning`	✅	❌	5.9.10+
`kafka`	`kafka`	`kafka`	✅	❌	5.9.10+
`kafka`	`kafka`	`jmx-exporter`	✅	❌	5.9.10+
`kafka`	`kafka`	`init/check-zk`	✅	✅	5.9.9+
`kafka`	`kafka`	`init/auto-discovery`	✅	❌	5.9.10+
`kafka`	`kafka`	`init/volume-permissions`	✅	❌	5.9.10+
`lwai-gateway`	`lwai-gateway`	`lwai-gateway`	✅	✅	5.9.9+
`ml-model-service`	`ml-model-service`	`java-service`	✅	✅	5.9.9+
`ml-model-service`	`ml-model-service-namespace-hook`	`kubectl-runner`	✅	✅	5.9.9+
`ml-model-service/ambassador`	`ambassador`	`ambassador`	✅	❌	5.9.10+
`ml-model-service/ambassador`	`ambassador`	`prometheus-exporter`	✅	❌	5.9.10+
`ml-model-service/milvus`	`milvus-writable`	`milvus`	✅	❌	5.9.10+
`ml-model-service/milvus`	`milvus-writable`	`init/wait-for-mysql`	✅	✅	5.9.10+
`ml-model-service/milvus`	`milvus-writable`	`init/create-for-share-storage`	✅	❌	5.9.10+
`ml-model-service/milvus`	`milvus-admin`	`admin`	✅	❌	5.9.10+
`ml-model-service/milvus`	`milvus-admin`	`init/wait-for-milvus`	✅	✅	5.9.10+
`ml-model-service/milvus`	`milvus-mishards`	`init/wait-for-mysql`	✅	✅	5.9.10+
`ml-model-service/milvus`	`milvus-mishards`	`init/wait-for-mysql`	✅	✅	5.9.10+
`ml-model-service/milvus`	`milvus-mishards`	`mishards`	✅	❌	5.9.10+
`ml-model-service/milvus/mysql`	`mysql`	`init/remove-lost-found`	✅	✅	5.9.10+
`ml-model-service/milvus/mysql`	`mysql`	`mysql`	✅	❌	5.9.10+
`pm-ui`	`pm-ui`	`pm-ui`	✅	✅	5.9.9+
`query-pipeline`	`query-pipeline`	`query-pipeline`	✅	✅	5.9.9+
`question-answering`	`argo/qna-coldstart/init-workspace`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/write-job-configs`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/write-io-configs`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/write-io-configs`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/add-zkHost`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/pull-data`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/pull-data-cloud`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/train`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/list-workspace`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/post-model`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-coldstart/apply-seldon-deployment`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/init-workspace`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/write-job-configs`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/write-io-configs`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/add-zkhost`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/pull-qa-data-cloud`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/pull-qa-data`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/train`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/train-with-texts`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/list-workspace`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/post-model`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-supervised/apply-seldon-deployment`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/init-workspace`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/write-job-configs`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/write-io-configs`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/add-zkhost`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/list-workspace`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/pull-eval-data`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/pull-eval-data-cloud`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/evaluate`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/push-eval-results-cloud`	`init/wait/main`	✅	❌	5.9.10+
`question-answering`	`argo/qna-evaluation/push-eval-results`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/init-workspace`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/write-job-configs`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/write-io-configs`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/add-zkhost`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/pull-data-training-and-metadata`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/pull-data-training-and-metadata-cloud`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/train-with-metadata`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/train-without-metadata`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/push-recommendations`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/push-recommendations-cloud`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/add-default-exclude-query`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-user/init-workspace`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/copy-model`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/write-job-configs`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/write-io-configs`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/add-zkhost`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/pull-data`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/pull-data-cloud`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/train`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/push-content`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/push-content-cloud`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/add-default-exclude-query`	`init/wait/main`	✅	❌	5.9.10+
`recommender`	`argo/item-recommender-content/delete-old-content-recommendations`	`init/wait/main`	✅	❌	5.9.10+
`reverse-search`	`reverse-search`	`init/set-reverse-search-zone`	✅	✅	5.9.10+
`reverse-search`	`reverse-search`	`init/check-zk`	✅	✅	5.9.9+
`reverse-search`	`reverse-search`	`init/enable-tls-in-reverse-search`	✅	✅	5.9.10+
`reverse-search`	`reverse-search`	`reverse-search`	✅	❌	5.9.10+
`rules-ui`	`rules-ui`	`rules-ui`	✅	✅	5.9.9+
`seldon-core-operator`	`seldon-controller-manager`	`manager`	✅	❌	5.9.10+
`seldon-core-operator`	`seldon-spartakus-volunteer`	`seldon-spartakus-volunteer`	✅	❌	5.9.9+
`seldon-core-operator`	`crd/SeldonDeployment`		✅/❌	❌	5.9.9+
`solr`	`solr`	`init/set-solr-zone`	✅	✅	5.9.10+
`solr`	`solr`	`init/enable-tls-in-solr`	✅	✅	5.9.10+
`solr`	`configset-bootstrap`	`configset-bootstrap`	✅	✅	5.9.10+
`solr`	`solr`	`solr`	✅	❌	5.9.10+
`solr`	`solr-exporter`	`exporter`	✅	❌	5.9.10+
`solr`	`solr-exporter`	`init/solr-init`	✅	✅	5.9.10+
`solr-managed`	`configset-bootstrap`	`configset-bootstrap`	✅	✅	5.9.10+
`solr-managed`	`exporter`	`exporter`	✅	❌	5.9.10+
`solr-managed`	`exporter`	`init/solr-init`	✅	✅	5.9.10+
`solr-managed`	`solr`	`init/set-solr-zone`	✅	✅	5.9.10+
`solr-managed`	`solr`	`init/enable-tls-in-solr`	✅	✅	5.9.10+
`solr-managed`	`solr`	`solr`	✅	❌	5.9.10+
`solr-managed`	`solr`	`process-raw`	✅	❌	5.9.10+
`solr-backup-runner`	`solr-backup-runner-backup`	`solr-backups`	✅	❌	5.9.10+
`solr-backup-runner`	`solr-backup-runner-prune`	`solr-prune`	✅	❌	5.9.10+
`templating`	`templating`	`templating`	✅	✅	5.9.9+
`webapps`	`webapps`	`webapps`	✅	✅	5.9.9+
`zookeeper`	`zookeeper`	`zookeeper`	✅	❌	5.9.10+
`zookeeper`	`zookeeper`	`jmx-exporter`	✅	❌	5.9.10+
`zookeeper`	`zookeeper`	`zookeeper-exporter`	✅	❌	5.9.10+
`zookeeper`	`zookeeper-chroots`	`main`	✅	❌	5.9.10+

Introduction to Fusion

Getting Data In

Getting Data Out

Operations

Reference

Developer Docs

Neural Hybrid Search

Release Notes

​Fusion user login

​Manage users with security realms

​Per-request authentication

​Read-only root file system

Fusion user login

Manage users with security realms

Per-request authentication

Read-only root file system