core-mw

App Suite Middleware Core Helm Chart

Maintainers

Source Code

• https://github.com/open-xchange/appsuite-middleware

Requirements

Additional informations

4.0.0

• This version introduces new configuration.redis and configuration.sessiond section which adds support for Redis. Please refer to the documentation in values.yaml.
• Changed the default service type from NodePort to ClusterIP for http-api, sync and admin service.
• Removed all ingress configuration settings.
• Removed services.documentconverterHost, services.imageconverterHost and services.spellcheckHost. Not necessary anymore but it's possible to override them e.g. via .Values.global.dc.serviceName
• Renamed environment variables
  • OX_IMAGECONVERTER_URL → IC_SERVER_URL
  • OX_SPELLCHECK_URL → SPELLCHECK_SERVER_URL
  • OX_DOCUMENTCONVERTER_URL → DC_SERVER_URL
• Partially or fully override ox-common.names.fullname via nameOverride or fullnameOverride.

5.10.1

• This version introduces new configuration.redis.cache section which adds support for a separate cache for volatile data. Please refer to the documentation in values.yaml.

6.0.1

• Hazelcast is now required for OX Documents only.
• Disabled packages open-xchange-documents-backend and open-xchange-hazelcast per default.
• Introduced a new role documents that enables packages open-xchange-documents-backend and open-xchange-hazelcast and adds necessary HZ_* env variables to containers. The role has been added to the defaultScaling.
• Removed packages.minimalWhitelist as it was not used anymore
• Removed role hazelcast-data-holding and hazelcast-lite-member

6.4.0

• Whether to use TLS to connect to the Redis endpoint can now be configured using redis.tls.enabled and redis.cache.tls.enabled.

6.19.5

• Added support for PodDisruptionBudget (PDB) via the new pdb configuration section.
• This version adds a new middleware.open-xchange.com/type label to pods. Upgrading to this version will trigger a rolling restart of all pods due to the label change.

6.21.0

Third-party Java agent support

The chart supports injecting third-party Java agents (e.g. Elastic APM) into the middleware pod via extraInitContainers. A typical setup uses an init container to copy the agent JAR into a shared volume, which is then mounted into the middleware container and activated via javaOpts.other.

Support scope: OX supports the Helm injection mechanism (extraInitContainers, extraVolumes, extraMounts, javaOpts.other). The internal behavior of third-party agents is outside OX support scope.

Example: Elastic APM agent

extraInitContainers:
  - name: elastic-apm-init
    image: docker.elastic.co/apm/apm-agent-java:1.52.0
    command: ["cp", "/usr/agent/elastic-apm-agent.jar", "/apm-agent/"]
    volumeMounts:
      - name: apm-agent
        mountPath: /apm-agent

extraVolumes:
  - name: apm-agent
    emptyDir: {}

extraMounts:
  - name: apm-agent
    mountPath: /opt/apm-agent
    readOnly: true

javaOpts:
  other: "-javaagent:/opt/apm-agent/elastic-apm-agent.jar"

Upgrading

To 6.0.1

If you are using custom node definitions (scaling.nodes) in your Helm values, please make sure to remove roles hazelcast-data-holding and hazelcast-lite-member. Beside of that, a new role called documents has been introduced. The role needs to be added to every node definition that contains the http-api role and should run OX Documents. Furthermore, it ensures that a Hazelcast headless service is still deployed for OX Documents.

Please consider the following scaling.nodes definition:

core-mw:
  scaling:
    nodes:
      groupware:
        replicas: 2
        roles:
          - admin
          - http-api
          - hazelcast-data-holding
      sync:
        replicas: 1
        roles:
          - sync
          - businessmobility
          - hazelcast-lite-member

A migrated version could look like this:

core-mw:
  scaling:
    nodes:
      groupware:
        replicas: 2
        roles:
          - admin
          - http-api
          - documents
      sync:
        replicas: 1
        roles:
          - sync
          - businessmobility

Warning

Nodes that do not have the role documents will not be deployed as a StatefulSet anymore. Instead they will be deployed as a Deployment. Upgrading a StatefulSet to a Deployment is not easily possible without some preparation. Simply upgrading via Helm would remove all pods of the StatefulSet before starting the first pod of the new Deployment. This would result in a short downtime for endusers.

Example

Take a look at the following migrated example:

core-mw:
  scaling:
    nodes:
      groupware-without-docs:
        replicas: 2
        roles:
          - admin
          - http-api
      sync:
        replicas: 1
        roles:
          - sync
          - businessmobility

Nodes groupware-without-docs will not run OX Documents and roles hazelcast-data-holding and hazelcast-lite-member have been removed, too. Prior 6.0.1, those nodes were deployed as a StatefulSet. After the upgrade, they will be deployed as Deployment. If downtime is not feasible, you need to do the following before the upgrade:

1. Remove the StatefulSet without removing the pods with:

kubectl delete statefulset appsuite-core-mw-groupware-without-docs --cascade=orphan

2. Upgrade via Helm

If you're low on resources, you should scale replicas to 0 in your values.yaml. Otherwise, you will end up with the two pods from the StatefulSet and another two pods for the Deployment:

core-mw:
  scaling:
    nodes:
      groupware-without-docs:
        replicas: 0
        [...]

You can now upgrade the deployment via Helm:

helm upgrade [...]

After the upgrade, you can manually delete the StatefulSet pods and scale up the Deployment:

kubectl delete pod appsuite-core-mw-groupware-without-docs-1
kubectl scale deployment appsuite-core-mw-groupware-without-docs --replicas=1
kubectl delete pod appsuite-core-mw-groupware-without-docs-0
kubectl scale deployment appsuite-core-mw-groupware-without-docs --replicas=2

This process needs to be followed for all node definitions that previously had the hazelcast-data-holding role without having the documents role after the migration.

Don't forget to scale up the replicas in your values.yaml again.

Using existing*Secret values

The existing*Secret values let you supply configuration from Secrets you create and manage yourself (for example via Vault, sealed-secrets or external-secrets) instead of placing sensitive data directly in values.yaml. The chart only references these Secrets, so each one must already exist in the release namespace before you install or upgrade.

There are two behaviors. Know which one applies to the value you are setting:

• Additive – your Secret is mounted alongside the chart-rendered configuration; both apply.
• Replace – your Secret replaces the chart-rendered equivalent, and the corresponding plain values.yaml settings are then ignored.

What each Secret must contain

Every Secret below is one you create and manage yourself, in the release namespace, with the name you put into the corresponding value. The chart never generates them and never validates them: a missing or misspelled key is not a template error, it surfaces later as a failed database connection, a missing configuration file or a wrong password.

Two rules apply throughout:

• Use stringData for text and data for anything binary. Keys under stringData are stored verbatim; keys under data must be base64-encoded and are decoded before they reach the container.
• A Secret key becomes a file name. Kubernetes forbids / in Secret keys, so none of these Secrets can place a file into a sub-directory. Where a chart value accepts a nested path (yamlFiles, secretYAMLFiles), the existing*Secret equivalent does not.

Database credentials — mysql.existingSecret

Replaces the whole configdb Secret the chart would otherwise render, and is consumed as environment variables. Because it is a full replacement, supply all sixteen keys the chart's own Secret would define — a key you leave out is not a template error, it is an unset environment variable that surfaces much later. On a single-server setup, point the _WRITE_ and _READ_ keys at the same host as the plain ones.

apiVersion: v1
kind: Secret
metadata:
  name: my-db-creds
type: Opaque
stringData:
  MYSQL_HOST: "mysql-master"
  MYSQL_PORT: "3306"
  MYSQL_DATABASE: "configdb"
  MYSQL_USER: "openexchange"
  MYSQL_PASSWORD: "secret"
  MYSQL_ROOT_PASSWORD: "supersecret"
  MYSQL_WRITE_HOST: "mysql-master"
  MYSQL_WRITE_PORT: "3306"
  MYSQL_WRITE_DATABASE: "configdb"
  MYSQL_WRITE_USER: "openexchange"
  MYSQL_WRITE_PASSWORD: "writePassword"
  MYSQL_READ_HOST: "mysql-replica"
  MYSQL_READ_PORT: "3306"
  MYSQL_READ_DATABASE: "configdb"
  MYSQL_READ_USER: "openexchange"
  MYSQL_READ_PASSWORD: "readPassword"

One Secret serves every node type, referenced under exactly the name you give. The exception is a node type that overrides mysql in its own scaling.nodes.<type>.values block: the chart then looks for <name>-<typeName> instead, and since it is not generating the Secret either, those pods fail to start with CreateContainerConfigError. Do not combine mysql.existingSecret with a per-type mysql override.

global.mysql.existingSecret does the same thing across sub-charts and takes precedence over mysql.existingSecret when both are set. Note that this Secret is not covered by the checksums.existingSecrets annotation, so editing it does not roll the pods — restart the deployment yourself.

Redis credentials — redis.existingSecret

This one is easy to get wrong: it is not a redis-password key. It replaces the chart's generated Redis properties file, so its value must be a middleware configuration document — a YAML file whose top-level key is a configuration scope (anywhere unless you know you need a narrower one) holding com.openexchange.redis.* properties. Give it the same 1000_ prefix the chart's own file uses: it lands in the same directory as the property files and is applied in file-name order, so an unprefixed name would sort after — and silently outrank — an existingPropertiesSecret file you prefixed 1001_.

apiVersion: v1
kind: Secret
metadata:
  name: my-redis-props
type: Opaque
stringData:
  1000_redis-properties.yaml: |
    anywhere:
      com.openexchange.redis.mode: "sentinel"
      com.openexchange.redis.hosts: "redis-sentinel.example.svc.cluster.local:26379"
      com.openexchange.redis.sentinel.masterId: "mymaster"
      com.openexchange.redis.username: ""
      com.openexchange.redis.password: "redisPassword"
      com.openexchange.redis.ssl: "false"
      com.openexchange.redis.cache.enabled: "true"
      com.openexchange.redis.cache.mode: "cluster"
      com.openexchange.redis.cache.hosts: "redis-cache.example.svc.cluster.local:6379"
      com.openexchange.redis.cache.username: ""
      com.openexchange.redis.cache.password: "cachePassword"
      com.openexchange.redis.cache.ssl: "false"

For standalone or cluster mode drop the sentinel.masterId property and list the Redis nodes in hosts. Quote every value — the middleware expects strings, and an unquoted false or 6379 is parsed as a boolean or integer. Setting this value makes the chart ignore redis.auth and redis.mode, so anything you leave out of the Secret is simply absent rather than defaulted.

Keep populating redis.hosts regardless. That value has a second consumer the Secret does not replace: when the list is empty the chart deploys its own single-node Redis alongside the middleware, and that decision does not look at redis.existingSecret. Leaving it at the default gets you an in-cluster Redis that nothing connects to.

Environment variables — existingEnvSecret

Keys are environment-variable names, added to the containers via envFrom. Include only the variables you actually want to override; unlike the two above, this Secret is additive, so anything you omit keeps the value the chart generates. These are the credential-bearing variables worth putting here:

apiVersion: v1
kind: Secret
metadata:
  name: my-env
type: Opaque
stringData:
  MASTER_ADMIN_USER: "oxadminmaster"
  MASTER_ADMIN_PW: "masterPassword"
  OX_BASIC_AUTH_LOGIN: "basicAuthUser"
  OX_BASIC_AUTH_PASSWORD: "basicAuthPassword"
  JOLOKIA_LOGIN: "jolokiaUser"
  JOLOKIA_PASSWORD: "jolokiaPassword"
  CREDSTORAGE_PASSCRYPT: "credStoragePassphrase"

Mind the caveat described under Override precedence: this changes the container environment only, and the chart's own templating still reads the plain masterAdmin / masterPassword values.

Provisioning gateway certificate — provisioningGateway.tls.existingSecret

Required as soon as provisioningGateway.tls.enabled is true: the chart generates no certificate, and rendering fails with a required error if the value is empty. It is an ordinary kubernetes.io/tls Secret, so the two keys are fixed and cannot be renamed — the gateway reads them from tls.crt and tls.key under provisioningGateway.tls.mountPath.

apiVersion: v1
kind: Secret
metadata:
  name: my-gateway-tls
type: kubernetes.io/tls
data:
  tls.crt: <base64-encoded PEM certificate>
  tls.key: <base64-encoded PEM private key>

Like mysql.existingSecret, this one is not folded into the checksums.existingSecrets annotation, so rotating the certificate does not restart the gateway on its own.

Configuration files

The remaining existing*Secret values all work the same way — each key becomes a file, and where that file lands is what differs. The .yaml/.yml/.txt names below are examples except where called out.

existingPropertiesSecret and existingUISettingsSecret share the single /configuration/ directory, so give their keys distinct names — the same key in both means one silently overwrites the other.

Several Secrets per value

Every additive existing*Secret value has a plural twin that takes a list of further Secret names: existingPropertiesSecrets, existingUISettingsSecrets, existingMetaSecrets, existingContextSetsSecrets, existingETCFilesSecrets, existingETCBinariesSecrets, existingYAMLFilesSecrets and existingEnvSecrets. Use them when the sensitive data comes from several sources, for example one ExternalSecret per credential. The singular value stays supported and comes first; a name given twice is used once.

For the file-based values all listed Secrets are projected into the same directory as the singular one, so the rule about distinct keys applies across the whole set. Which value wins on a duplicate property is still decided by the numeric file-name prefix, not by the order of the list.

For existingEnvSecrets each entry becomes one more envFrom source after existingEnvSecret, so on a duplicate key a later entry wins over an earlier one, and extraEnv still wins over all of them.

The Replace values (existingASConfigSecret, redis.existingSecret, mysql.existingSecret) and the gateway TLS Secret have no plural form: each stands in for exactly one chart-generated resource.

existingPropertiesSecret: my-extra-props    # still supported, mounted first
existingPropertiesSecrets:
  - my-ldap-credentials                     # key 1002_ldap.yaml
  - my-smtp-credentials                     # key 1003_smtp.yaml
existingEnvSecrets:
  - my-s3-env
  - my-oauth-env                            # wins over my-s3-env on a duplicate key

Override precedence

For the additive property/UI-settings secrets, configuration is applied in numeric file-name order. To make your file win over the chart-generated configuration, prefix it with a number higher than the chart's:

• properties: use a prefix >= 1001 (the chart uses <= 1000)
• UI settings: use a prefix >= 2001 (the chart uses <= 2000)

# stringData of the secret named by existingPropertiesSecret
1001_existing.yaml: |
  anywhere:
    com.openexchange.foobar: "true"

existingEnvSecret is added after the common-env and chart-generated env secrets and before any existingEnvSecrets entry, so on a duplicate key it overrides those two but neither a later existingEnvSecrets entry nor extraEnv. For example, a MASTER_ADMIN_USER key in your secret wins over the MASTER_ADMIN_USER the chart generates from masterAdmin.

This override happens only at the container-environment level. The chart's own templating still reads the plain masterAdmin / masterPassword values (it does not look at existingEnvSecret) when it generates other resources. So to change the master admin everywhere, set masterAdmin / masterPassword rather than only overriding the env var through existingEnvSecret.

Rolling restarts on content change

With checksums.existingSecrets: true (the default), the checksum of each referenced Secret's contents is folded into a pod annotation, so editing a Secret triggers a rolling restart. mysql.existingSecret is intentionally excluded (connection details rarely change).

Example

existingPropertiesSecret: my-extra-props   # additive; file prefixed 1001_ to override chart props
existingPropertiesSecrets: [my-ldap, my-smtp] # additive; every existing*Secret above has such a plural list
existingASConfigSecret: my-as-config       # replaces asConfig entirely
existingEnvSecret: my-env                  # extra environment variables, last-wins on duplicate keys

Configuration

The following table lists the configurable parameters of the App Suite Middleware Core chart and their default values.