App Suite 8 Operations Guide

Configuration

This file should explain the various App Suite 8 configuration types and how they are represented in the values.yaml files. It should help with transferring legacy App Suite 7 settings into App Suite 8 as well.

Note: the entire configuration mechanism is to be understood as a temporary vehicle in our initial kubernetes implementation for App Suite. We are working on a re-implementation of the entire configuration mechanism.

Overview

The entire configuration happens, as usual, via values.yaml files, given as paramter to the helm install call on the App Suite Helm Chart.

helm install as8 oci://registry.open-xchange.com/appsuite/charts/appsuite --values values.yaml

Single vs multiple values.yaml files

As usual with Helm Charts, it is possible to specify multiple values.yaml files on one helm install command line. helm will then merge the contents on helm install.

While this can be helpful in special situations, in general we do not recommend to split up the values functionally; it reduces readability and is a potential source of errors.

There is one execption, though: It is considered best practices to manage secret settings in a separate file, while keeping the other settings in the main values.yaml file. The values.secret.yaml file can then be managed by appropriate means of version control and deployment automation.

helm install as8 oci://registry.open-xchange.com/appsuite/charts/appsuite --values values.yaml --values values.secret.yaml

YAML High Level Structure

The Core App Suite Stack Chart oci://registry.open-xchange.com/appsuite/charts/appsuite and similar charts expect a values.yaml structure which looks, from high level, as follows:

global:
  # global settings here

core-mw:
  # core-mw settings here
core-ui:
  # core-ui settings here
core-ui-middleware:
  # core-ui-middleware settings here
istio:
  # istio settings here

There are charts which contain the Stack Chart and deploy it alongside other chars. This includes on the one hand the App Suite Pro chart, Charts for customized versions of App Suite, and also the (open source) Public Sector chart oci://registry.open-xchange.com/appsuite-public-sector/charts/appsuite-public-sector.

Following the structure of the chart itself, these umbrella or meta charts expect a values.yaml structure as follows:

global:
  # global settings here

appsuite:
  core-mw:
    # core-mw settings here
  core-ui:
    # core-ui settings here
  core-ui-middleware:
    # core-ui-middleware settings here
  istio:
    # istio settings here

a-custom-ui:
  # settings for a-custom-ui component here

another-custom-ui:
  # settings for another-custom-ui component here

Note how the stack chart settings become attached to a top level appsuite: entry, and additional components get configured alongside of it. The global: section remains a top-level section.

Depending on which kind of chart you employ, you need to adjust your values.yaml structure accordingly. If in doubt, consult the helm show values command.

$ helm show values helm show values oci://registry.open-xchange.com/appsuite/charts/appsuite --version ...
# match the output against the structures shown above

Middleware Properties

Middleware properties like com.openexchange.some.property can be set in the properties object:

core-mw: 
  properties:
    com.openexchange.mail.loginSource: "mail"
    com.openexchange.mail.mailServerSource: "global"

Secret properties get their own object. These are stored in kubernetes secrets as opposed to configmaps for regular properties. This works the same way for any of the other secret variants later in this document.

core-mw: 
  secretProperties:
    com.openexchange.push.credstorage.passcrypt: "8297f949a39d49618ed957e6cfead030"

The system tries to find the properties file that contains the setting and overrides it. If it can't find such a file, it adds the setting to a generic new additional.properties file, which is home for all such settings.

Most of the settings work regardless of the file they are specified in. Some (legacy) settings however require to be defined in exactly the right file. (Most of these can be identified by looking like SHELL_ENVIRONMENT_VARIABLE variables.)

If you need to force a property to wind up in a certain .properties file, you can use the propertiesFiles or secretPropertiesFiles objects:

core-mw: 
  propertiesFiles:
    /opt/open-xchange/etc/server.properties:
      MAX_UPLOAD_SIZE: "92233720368000"
      com.openexchange.tools.images.transformations.maxResolution: "30087962"
  secretPropertiesFiles:
    /opt/open-xchange/etc/mail.properties:
      com.openexchange.mail.someSecret: "some secret value"

The property is then overwritten or added to the file referred to in the parent object.

UI Settings

UI settings that are delivered via the JSLob interface live in the .properties files in /opt/open-xchange/etc/settings. These properties can be set using the uiSettings or secretUISettings objects:

core-mw: 
  uiSettings:
    io.ox/mail//dumpster/folder: "default0/DUMPSTER"
    io.ox/core//apps/quickLaunchCount: "5"
  secretUISettings:
    io.ox/core//mySecretName: "Rumpelstilskin"

This again tries to guess the proper .properties file or adds a new one. Similarly to above, you can also force a specific file to be used:

core-mw: 
  uiSettingFiles:
    /opt/open-xchange/etc/settings/dumpster.properties:
      io.ox/mail//dumpster/folder: "default0/DUMPSTER"
  secretUISettingsFiles:
    /opt/open-xchange/etc/settings/hushush.properties:
      io.ox/core//mySecretName: "Rumpelstilskin"

Note though, that these MUST still be placed in /opt/open-xchange/etc/settings/ for the middleware to pick them up.

Meta

The settings that usually live under /opt/open-xchange/etc/meta, like protecting and unprotecting UI properties or aliasing a middleware property to a UI property can be set in a single meta object:

core-mw: 
  meta:
    io.ox/core//design:
      protected: true
    io.ox/core//someProp:
      protected: false
    io.ox/core//apps/quickLaunchCount:
      protected: false

This winds up as a sinlge .yaml file in the meta directory. If you need to structure these settings, do so with comments in the yaml file.

ContextSets

ContextSets that selectively override settings based on a label on a context are configured in files under /opt/open-xchange/etc/contextSets. To do so in helm, use the contextSets object. This also works as a secretContextSets object.

core-mw: 
  contextSets:
    brand.com:
      withTags: brand.com
      com.openexchange.sessiond.maxSession: 10
  secretContextSets:
    brand.com:
      withTags: brand.com
      com.openexchange.secret: 123abc

as-config.yaml

Certain Properties are handled by /opt/open-xchange/etc/as-config.yml. Just add the yaml you need to the asConfig object:

core-mw: 
  asConfig:
    default:
      host: all
    myhost:
      host: myexchange.myhost.mytld
      someConfig: some overriding value

Other YAML Files

You can also specify any other YAML files as-is in either the regular or secret variants. These are added to /opt/open-xchange/etc/:

core-mw: 
  yamlFiles:
    client-onboarding-scenarios.yml:
      mailmanual:
        # Disbled by default
        enabled: true
        type: manual
        providers: [mail]
        alternatives: []
        # The comma-separated Font Awesome names
        icon: fa-envelope-o
        # The translatable display name
        displayName_t10e: "eMail"
        # The translatable description
        description_t10e: "To manually setup your Mail account, please use the following information"
  secretYAMLFiles:
    globaldb.yml:
      default:
        groups: [default]
        id: 4

Other Files

Text Files

You can also place arbitrary files in /opt/open-xchange/etc/:

core-mw: 
  etcFiles:
    importerExporter.xml: |-
      <beans>
        <bean id="importerExporter" class="com.openexchange.groupware.importexport.ImporterExporter">

              <property name="importers">
                <list>
                  <ref bean="iCalImporter" />
                  <ref bean="vCardImporter" />
                </list>
              </property>

              <property name="exporters">
                <list>
                  <ref bean="iCalExporter" />
                  <ref bean="vCardExporter" />
                  <ref bean="csvContactExporter" />
                </list>
              </property>

          </bean>

        <bean id="iCalImporter" 		class="com.openexchange.groupware.importexport.importers.ICalImporter" />
        <bean id="vCardImporter" 		class="com.openexchange.groupware.importexport.importers.VCardImporter" />
        <bean id="iCalExporter" 		class="com.openexchange.groupware.importexport.exporters.ICalExporter" />
        <bean id="vCardExporter" 		class="com.openexchange.groupware.importexport.exporters.VCardExporter" />
        <bean id="csvContactExporter" 	class="com.openexchange.groupware.importexport.exporters.CSVContactExporter" />
      </beans>
  secretETCFiles:
    tokenlogin-secrets: |-
      public-ox-redeem-secret-1337

Binary Files

To add binary files to /opt/open-xchange/etc you need to base64encode their content and put them in the etcBinaries or secretETCBinaries data structure like so:

core-mw: 
  etcBinaries:
    - name: binary1
      filename: something.bin
      b64Content: <base 64 encoded file content>
  secretETCBinaries:
    - name: binary2
      filename: something-secret.bin
      b64Content: <base 64 encoded file content>

These are each put into individual secrets and configmaps, whose respective sizes cannot exceed 1MiB.

Bundle Activation

In the AppSuite 7 days some features would be toggled on or off depending on whether a certain package was installed on the system. Nowadays the appsuite middleware image contains all bundles, but which ones are actually active can be controlled in a variety of ways. We organize bundles into packages and packages into features. Multiple bundles make up a package, multiple packages make up a feature.

To enable a feature, do the following:

core-mw:
  features:
    status:
      plugins: enabled
      cloudPlugins: enabled
      reseller: enabled

To know which packages are part of the mentioned features, take a look at the values file of the appsuite middleware chartopen in new window. Here you can see the features.definitionssection that defines a feature:

features:
  definitions:
    reseller:
      - open-xchange-admin-reseller
      - open-xchange-admin-soap-reseller

Which means, that the feature reseller consists of the packages open-xchange-admin-reseller and open-xchange-admin-soap-reseller. In that values file you can also see the default activation state for all the packages that we have assembled into features. You can also enable individual packages like so:

core-mw:
  packages:
    status:
      open-xchange-sso: enabled
      open-xchange-hostname-config-cascade: enabled
      open-xchange-hostname-bla: disabled

By default a package will be enabled unless disabled in either the appsuite middleware's values file as a package or as a member or a disabled feature. You can also see the list of packages disabled by default in the values file of the appsuite middleware chartopen in new window.

To find out which bundle is part of a package is a little more involved. You don't need this information but in very special cases. It is possible in principle by one of the following means:

  • investigating in a running middleware container, via kubectl exec, as long as it is still possible, or kubectl debug
  • spinning up a container locally (hint: podman run -it ... --entrypoint bash).
  • pulling the middleware image and extracting locally

Regardless the means you use to access the filesystem: The metadata used to determine which bundles to switch on can be found at /opt/open-xchange/osgi/bundle.d

cd /opt/open-xchange/osgi/bundle.d

ls

The directories here correspond to the packages the files inside the directories correspond to individual bundles:

cd open-xchange-admin-reseller/

ls -l

-rw-r--r--   1 open-xchange open-xchange    68 May  5 07:55 com.openexchange.admin.reseller.ini
-rw-r--r--   1 open-xchange open-xchange    72 May  5 07:55 com.openexchange.admin.reseller.rmi.ini
-rw-r--r--   1 open-xchange open-xchange    77 May  5 07:55 com.openexchange.config.cascade.reseller.ini
-rw-r--r--   1 open-xchange open-xchange    67 May  5 07:58 com.openexchange.reseller.impl.ini

So the open-xchange-admin-reseller packages consists of the bundles com.openexchange.admin.reseller, com.openexchange.admin.reseller.rmi, com.openexchange.config.cascade.reseller and com.openexchange.reseller.impl. If, conversely, you'd like to know which package contains a bundle, use e.g. find:

cd /opt/open-xchange/osgi/bundle.d

find . -name "com.openexchange.reseller.impl.ini"

./open-xchange-admin-reseller/com.openexchange.reseller.impl.ini

which tells you the bundle will be part of open-xchange-admin-reseller.

Deep Dive: How are settings applied to the containers?

All the settings mentioned above find their way into ConfigMaps or Secrets.

Properties

Properties are set using the ox_props command line tool. This tool takes as input a yaml file that looks something like this:

anywhere:
  com.openexchange.some.property: Some Value

/opt/open-xchange/etc/settings/theme.properties:
  io.ox/core/theme: light

and applies it using either a heuristic to find the file that contains a property (everything below anywhere) or adds the property to a newly created file or uses the file specified. These YAML files are created with input from either the properties or propertiesFiles settings (and created as a ConfigMap) or the secretProperties and secretPropertiesFiles and rendered as a Secret with the appsuite-properties respectively. UI Settings wind up in a ConfigMap or Secret called appsuite-ui-settings. The AppSuite Pods mount these files at /injections/configuration/properties/ and /injections/configuration/ui-settings/ and the entrypoint.sh startup script copies them to /configuartion where the ox_props tool expects to find them. ox_props also uses the go templating engine (similar to how Helm renders its files) prior to parsing the resulting YAML.

Text and binary files below /opt/open-xchange/etc/

All the other settings are likewise stored in either ConfigMaps or Secrets:

  • meta: becomes appsuite-meta ConfigMap and is mounted at /injections/etc/meta/meta
  • asConfig: becomes appsuite-as-config ConfigMap and is mounted at /injections/etc/etc-properties
  • contextSets: becomes appsuite-context-sets ConfigMap and is mounted at /injections/etc/context-sets/contextSets
  • yamlFiles:, secretYAMLFiles: become appsuite-yaml-files or appsuite-yaml-secrets ConfigMap and Secret and is mounted at /injections/etc/yaml and /injections/etc/secretYaml
  • etcFiles:, secretETCFiles: become etc-files or etc-secrets ConfigMap and Secret and is mounted at /injections/etc/etc and /injections/etc/secretEtc
  • etcBinaries:, secretETCBinaries: become individual ConfigMaps or Secrets called appsuite-[NAME]-etc-binary which are also mounted at /injections/etc/etc and /injections/etc/secretEtc/ like text files above.

These are all copied to /opt/open-xchange/etc by the entrypoint script.

Entrypoint Startup Script

If you want to see the details of what happens before appsuite starts up, see the startup script here: https://gitlab.open-xchange.com/oss/appsuite/core/middleware/-/blob/stable-8.19/docker/entrypoint.sh

Advanced Options

AppSuite Container Hooks

In case the above system misses a feature, we can also configure bash scripts to run at certain points during startup. We can run either at the start, right before the ox_props tool applies the settings stored at /configuration/ in the YAML files, or right before the appsuite process is started:

core-mw: 
  hooks:
    start: 
      welcome_to_the_machine.sh: |
        #! /bin/bash
        echo "WELCOME TO THE MACHINE"
    beforeApply: {}
    beforeAppsuiteStart: {}

And we can also call the ox_props tool in these scripts to generate or set specific properties:

core-mw: 
  hooks:
    start: {}
    beforeApply: {}
    beforeAppsuiteStart:
      welcome_to_the_machine.sh: |
        #! /bin/bash
        for ((i=20;i<=100;i++)); 
        do 
          ox_props set --file /opt/open-xchange/etc/scality-filestores.properties --property com.openexchange.filestore.sproxyd.${filestoreId}.baseUrl=http://scality-store/${filestoreId}
        done

Extra Mounts

We can also mount our own ConfigMaps with as extra mounts into the container by using extraMounts and extraVolumes

core-mw: 
  extraMounts:
    - mountPath: /filestore
      name: filestore
  extraVolumes:
    - name: filestore
      nfs:
        server: nfs-server
        path: /export/filestore
        readOnly: false

Extra Mounts with go templated configuration yaml files

TODO: Example