Introduction

The concept of pre-assembled contexts consists of the asynchronous pre-creation of deactivated contexts that will be reused (and adapted to the provided settings) during the usual createcontext call. Because the contexts already exist in the configdb and user database and only some parameters needs to be adjusted based on the input of the operation, it is possible to significantly improve the performance of context creation. Additionally, it is ensured that new contexts are assigned to their local database segments within multi-site / active/active environments. Pre-assembled contexts are characterized by being deactivated with reason_id equals 666 and their name beginning with the preassembled- prefix followed by an UUID.

Creating pre-assembled contexts

In order to pick up pre-assembled contexts during regular provisioning operations, context skeletons need to be inserted into the database first. This can be achieved in two ways, by calling a REST API or via a background job. Both concepts can be used in combination, e.g. to create an initial amount of pre-assembled contexts via REST API before enabling the usage and to use the background job to make sure (with reasonable settings) to always have pre-assembled contexts available for context creation.

Preconditions

Pre-assembly of contexts includes an automatic assignment of context identifiers. Therefore, the functionality is contained in package open-xchange-admin-autocontextid which needs to be actively enabled in your chart's values.yaml:

core-mw:
  packages:
    status:
      open-xchange-admin-autocontextid: enabled

REST API

The REST API to create pre-assembled contexts will always be accessible independently from the value of com.openexchange.admin.context.preassembly.enabled. This allows to create a dedicated amount of pre-assembled contexts prior actually requiring them for a regular createcontext call, without waiting for the background job (see below) having been executed before. There is a limit of a maximum of 500 pre-assembled contexts per request when using the REST API.

The mentioned REST API is located at http://oxhost:8009/admin/v1/contexts/pre-assemble. Pre-assembled contexts can be generated by a POST request using master authentication with a body containing a JSON object describing the schema name and how many context skeletons should be created. Optionally you can add the id of the filestore that should be used for the pre-assembled context by defining filestore_id parameter.

Example:

POST /admin/v1/contexts/pre-assemble HTTP/1.1
Content-Type: application/json
Authorization: Basic amFuOmphbg==
User-Agent: PostmanRuntime/7.36.3
Accept: */*
Cache-Control: no-cache
Postman-Token: 3992936c-9d95-43c6-beb7-8b110e0088e2
Host: devenv.oxmw.io:8009
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 41

{
    "number":4,"schema": "oxuserdb_5"
}


HTTP/1.1 200 OK
Server: grizzly/2.4.4
X-Robots-Tag: none
Content-Type: application/json; charset=UTF-8
Content-Length: 52

{
    "contextIds": [
        11832,
        11833,
        11834,
        11835
    ],
    "errors": []
}

The response contains a JSON array of identifiers of the created pre-assembled contexts at field contextIds. Any errors that might have occurred when inserting the data are supplied in the errors array as well. In case of fatal errors, a response status different from HTTP 200 will be returned depending on the error that occurred. Please have a look at the full REST API description of the endpoint for more details.

Background job

The background job ensures that an adequate amount of pre-assembled contexts does always exist on the to create new contexts. The job itself checks how many pre-assembled contexts should exist per database schema and creates new pre-assembled context skeletons up to the configured limit. By default this job runs once a day in presumably low-load times. Please mind that the times in the configured schedule need to be specified in the JVM processes timezone on the pod.

The filestore used for the contexts will be determined by the server based on the current usage and free space. The background job has to be enabled explicitly by setting the (non-reloadable) property com.openexchange.admin.context.preassembly.job.enabled=true. This can be done independently of the setting to actually make use of pre-assembled data for the regular provisioning calls (via com.openexchange.admin.context.preassembly.enabled), i.e. it is possible to generate a sufficient amount of pre-assembled contexts through the background job first before making use of them.

Other (non-reloadable) configuration options are:

• com.openexchange.admin.context.preassembly.job.schedule, defaults to Mon-Sun 0-4. The pattern specifying when to execute context the pre-assemble job. The default time zone of the Java virtual machine is assumed, which is typically the system's default time zone; unless the user.timezone property is set otherwise.
  
• com.openexchange.admin.context.preassembly.job.contextsPerSchema, defaults to 100. Configures the number of pre-assembled contexts that should exist per schema after the job was executed. This number will never be exceeded by using the background job. Of course it is possible to exceed this limit by using the REST API.
• com.openexchange.admin.context.preassembly.job.contextLimitFactor, defaults to 0.9. Configures the maximum filling level of schemas when adding pre-assembled contexts via periodic background job, as a factor of CONTEXTS_PER_SCHEMA. I. e. pre-assembly is only performed until the total number of contexts exceeds <factor> * <contexts_per_schema>.
• com.openexchange.admin.context.preassembly.job.frequency, defaults to 3600000 (1 hour). The frequency in milliseconds when to check for new job executions within configured schedule.
• com.openexchange.admin.context.preassembly.job.executionDelay, defaults to 86400000 (1 day). The (minimum) delay between repeated executions of the pre-assembly job in milliseconds.

Please refer to the property documentation for further details.

Use pre-assembled contexts

To use pre-assembled contexts the feature must be enabled via property com.openexchange.admin.context.preassembly.enabled=true. Once enabled, the regular createcontext operations (SOAP or command line tool) will select and use data of a pre-assembled context, and modify it according to the supplied properties like context name, quota, access combination etc.

However, as those pre-assembled contexts do already exist in the database (and account to sharding aspects like filestore- and db schema usage weights), it is no longer possible (for the administrator) to provide a context identifier, the filestore identifier or the database schema. In multi-site or active/active environments with segmented database schemas, the latter will automatically be selected so that it is currently served by the 'local' site where the provisioning operation was routed to.

Besides that, data provided (by the admin) for the new context will be applied to the pre-assembled context and it will get activated and usable. The context will be alike a context created "from scratch" with no additional users except the context administrator.

In order to prevent situations where the usage of pre-assembled data is crucial (e.g. as within multi-site or active/active environments), a createcontext request will fail if no pre-assembled context skeleton is available anymore.

Monitoring

As mentioned above, as soon as property com.openexchange.admin.context.preassembly.enabled becomes true, contexts can only be created using pre-assembled data. Therefore it is crucial to ensure that always enough contexts are available on the system, e.g. by configuring the background job parameters accordingly. To monitor the actual number of pre-assembled contexts that are available, the following metric is exposed at /metrics:

# HELP appsuite_provisioning_preassembled_available_total The number of available pre-assembled contexts for the deployment.
# TYPE appsuite_provisioning_preassembled_available_total gauge
appsuite_provisioning_preassembled_available_total 1234.0