Data export deprecated

The data export feature allows users to submit a data export of his/her personal data packed into one or more ZIP archives. The actual export is a background task that completes at any time in future. Once ready, the user is notified via E-Mail that he/she is now able to download the packages.

As mentioned before, a data export is designed as a background task. When triggered by the user, it collects user-related data, which can be a lengthy as well as an I/O- and compute-intensive operation. For this reason, the data export feature provides several options to distribute and schedule this operation as best as possible. The most important options for this, which are also mentioned and presented below:

Only certain nodes handle the processing of triggered data exports via com.openexchange.gdpr.dataexport.active. These could be special nodes in a cluster not serving regular calls.
Processing of data exports takes place only at certain times via com.openexchange.gdpr.dataexport.schedule
The number of data exports processed in parallel is controlled by com.openexchange.gdpr.dataexport.numberOfConcurrentTasks

Since the artefacts and result files are typically quite big files, the configuration for the data export feature requires to specify a dedicated file storage, which is used to manage those big files. Thus common file storage associated with a context or user is not burdened and quota is unaffected as well.

Furthermore, scheduling of data export tasks within an Open-Xchange installation is a global task. Therefore, the common payload databases are not suitable to store data export information since determination execution of the next adequate data export task requires to look-up every payload database on every registered database host. Therefore, the data export feature requires setup of the global database to have a central database for fast and easy look-up.

Currently, the Open-Xchange Middleware ships with 5 standard providers for the data export:

E-Mails
Calendar
Contacts
Tasks
Drive

Prerequisites

A special transport must be configured for data export notifications. This transport is configured in noreply.properties. All properties therein are config-cascade capable, so their values can be sensitive to the current user or context.
Choose the appropriate file storage to use for storing export artefacts and result files (both not accounting to quota)
- Either register a dedicated file storage via registerfilestore or
- Select an existing file storage having enough capacities since artefacts/result files are typically quite big files
Enable global database as explained here

Configuration

As a first step, enable the open-xchange-gdpr-dataexport package in your chart's values.yaml:

core-mw:
  packages:
    status:
      open-xchange-gdpr-dataexporte: enabled

Furthermore, please set the following configuration properties:

com.openexchange.gdpr.dataexport.fileStorageId Specifies the identifier of the file storage to use. This configuration property is config-cascade aware.
com.openexchange.gdpr.dataexport.enabled Whether the feature is available for a user. This configuration property is config-cascade aware.
com.openexchange.gdpr.dataexport.active Whether processing of data export tasks is enabled in this node.
com.openexchange.gdpr.dataexport.schedule The pattern specifying when processing of data export tasks is allowed. A comma-separated listing of tokens, while a token consists of a weekday and an optional time range A weekday is one of: Mon, Tue, Wed, Thu, Fri, Sat and Sun. It also accepts ranges e.g. Mon-Fri A time range is a hour of day range of the 24h clock. E.g. 0-6 for 0 to 6 in the morning or 22:30-24 for 22:30h until 24h in the evening. Also accepts comma-separated pairs; e.g. 0-6,22:30-24 A more complex example: Mon 0:12-6:45; Tue-Thu 0-7:15; Fri 0-6,22:30-24; Sat,Sun 0-8
com.openexchange.gdpr.dataexport.numberOfConcurrentTasks How many task are allowed being processed concurrently on a node

Please find further configuration properties in the configuration documentation section

Command-line tools

The data export feature ships with a set of command-line tools to

List the status of submitted data export tasks. Either all, by context or by user
Request cancelation for certain data export tasks by user or context
Check and repair consistency of the associated file storage
Export the data of a user on behalf of an authority, without the user logging in

Listing and canceling the tasks of a user or a context is also available over HTTP and JSON, through the provisioning gateway under /prov/v1/contexts/{context_id}/data-export-tasks and /prov/v1/contexts/{context_id}/users/{user_id}/data-export-task.

Export the data of a user on behalf of an authority via `submitdataexport`

An administrator can export the data of a user without the user logging in, e.g. when an authority requests it. Such an export is kept apart from the user's own: the user may have one of each, and an administrative export is not listed among the user's exports. The context administrator's login and a mandatory case reference are kept with the task and written to the audit trail, logger com.openexchange.gdpr.dataexport.auditTrail. A user has at most one administrative export at a time. The modules mail, calendar, contacts, tasks and infostore can be exported this way.

An administrator has no password of the user, so mail is exported through the administrative mail access: with a credential the administrator supplies, or else through master authentication (see com.openexchange.mail.passwordSource=global and com.openexchange.mail.masterPassword) where com.openexchange.mail.admin.strategies lists master. A supplied credential is a password, sent through PLAIN, or an OAuth access token, sent through XOAUTH2 (default) or OAUTHBEARER - whatever mail authentication is configured, master authentication included - with the user's mail login. Either may be a master user's instead (--mail-master-user): the middleware then logs in as <user's mail login><separator><master user> with com.openexchange.mail.admin.masterUserSeparator, e.g. Dovecot's auth_master_user_separator. The administrator names the master user only, never the mailbox, so the export always reads the mailbox of the user it is for. With an OAuth master user, the master passdb has to meet what is described for shared accounts under Acting User Login, in particular refuse anyone who is not a master user with USER_DISABLED; otherwise Dovecot logs the token's owner in to their own mailbox. The credential is kept encrypted in the mail credential vault for the lifetime of the export only, never logged, and checked against the mail server on submit. An access token is refreshed only with a refresh token and the OpenID Connect back-end that issued it; otherwise it has to outlive the mail export. Should the mail server refuse the credential while the export runs, mail ends there and the diagnostics report says so.

Without a usable credential, and for modules the user has no permission for, the module is skipped instead of failing the export. The skipped modules and the reason are shown by listdataexports ... -a and downloaddataexport, and noted in the diagnostics report inside the archive, which an administrative export always carries. If no requested module can be exported, the request is refused.

Usage

submitdataexport -c <contextId> -i <userId> -m <modules> -r <caseReference> -A <contextAdmin> -P <contextAdminPassword>
                 [-p <RMI-Port>] [-s <RMI-Server] [--responsetimeout <responseTimeout>] | [-h]
 -c,--context <contextId>             The context identifier
 -i,--userid <userId>                 The identifier of the user whose data is exported
 -m,--modules <modules>               The modules to export, comma-separated: mail, calendar, contacts, tasks, infostore
 -r,--case-reference <caseReference>  The reference of the case the export is requested for; at most 255 characters
    --mail-password-file <file>       Reads a password for the user's mailbox from this file, '-' for standard input
    --mail-master-user <user>         The master user whose password or access token is supplied
    --mail-access-token-file <file>   Reads an OAuth access token for the user's mailbox from this file, '-' for standard input
    --mail-access-token-expires <time> When the access token expires, ISO 8601 such as 2026-10-21T12:00:00Z
    --mail-refresh-token-file <file>  Reads an OAuth refresh token from this file
    --mail-oidc-backend <path>        The path of the OpenID Connect back-end that issued the tokens; empty for the default one
    --mail-oauth-mechanism <mech>     The SASL mechanism for the access token: XOAUTH2 (default) or OAUTHBEARER

Secrets are read from files rather than given on the command line, so that they show up neither in the process list nor in the shell history.

Examples

$ submitdataexport -c 1234 -i 3 -m calendar,contacts -r "AZ 4711/26" -A oxadmin -P secret

$ submitdataexport -c 1234 -i 3 -m mail,calendar -r "AZ 4711/26" --mail-master-user exporter --mail-password-file - -A oxadmin -P secret

$ submitdataexport -c 1234 -i 3 -m mail -r "AZ 4711/26" --mail-access-token-file token.txt --mail-access-token-expires 2026-10-21T12:00:00Z -A oxadmin -P secret

Over the provisioning gateway, the same request is a POST to /prov/v1/contexts/{context_id}/users/{user_id}/administrative-data-export-task.

The user is not notified about an administrative export, neither when it completes nor when it fails or is canceled. Canceling all exports of a context with canceldataexports -c leaves administrative exports alone. Their results are kept for com.openexchange.gdpr.dataexport.administrative.maxTimeToLive, which defaults to the time of the user's own exports.

Retrieve, verify and delete an administrative export

Every package of an export is kept with the SHA-256 checksum computed while it was written. downloaddataexport downloads all packages of the administrative export of a user, verifies each against its checksum and, with --delete, deletes the export on the server once all of them were verified. Downloads, cancelations and deletions are written to the audit trail.

downloaddataexport -c <contextId> -i <userId> -o <directory> [-d] -A <contextAdmin> -P <contextAdminPassword>
 -c,--context <contextId>   The context identifier
 -i,--userid <userId>       The identifier of the user whose administrative data export is downloaded
 -o,--output <directory>    The existing directory to write the packages to; existing files are not overwritten
 -d,--delete                Deletes the export on the server once every package has been downloaded and verified

listdataexports -c <contextId> -i <userId> -a shows the administrative export of a user with the checksums of its packages, canceldataexports -c <contextId> -i <userId> -a cancels it.

Examples

$ downloaddataexport -c 1234 -i 3 -o /var/tmp/case-4711 --delete -A oxadmin -P secret

$ canceldataexports -c 1234 -i 3 -a -A oxadmin -P secret

List data export tasks via `listdataexports`

Usage

listdataexports [-c <contextId>] [-i <userId>] -A <masterAdmin | contextAdmin> -P <masterAdminPassword |
                       contextAdminPassword> [-p <RMI-Port>] [-s <RMI-Server] [--responsetimeout <responseTimeout>] |
                       [-h]
 -A,--adminuser <adminUser>       Admin username
 -c,--context <contextId>         The context identifier. If context identifier is given, only data export tasks
                                  associated with denoted context are listed; otherwise all data export tasks are
                                  listed.
 -h,--help                        Prints this help text
 -i,--userid <userId>             The user identifier. If user identifier is given, only the data export task associated
                                  with denoted user is listed. This user-sensitive output also includes the
                                  task-associated work items and result files.
 -p,--port <rmiPort>              The optional RMI port (default:1099)
 -P,--adminpass <adminPassword>   Admin password
    --responsetimeout <timeout>   The optional response timeout in seconds when reading data from server (default: 0s;
                                  infinite)
 -s,--server <rmiHost>            The optional RMI server (default: localhost)

Examples

List all data exports:

$ listdataexports -A oxadminmaster -P secret

List all data exports of a certain context:

$ listdataexports -c 1234 -A oxadminmaster -P secret

List the data export of a certain user. This outputs includes task-associated work items and result files (if any):

$ listdataexports -c 1234 -i 3 -A oxadminmaster -P secret

Request cancelation for data export tasks via `canceldataexports`

Usage

canceldataexports -c <contextId> [-i <userId>] -A <masterAdmin | contextAdmin> -P <masterAdminPassword |
                         contextAdminPassword> [-p <RMI-Port>] [-s <RMI-Server] [--responsetimeout <responseTimeout>] |
                         [-h]
 -A,--adminuser <adminUser>       Admin username
 -c,--context <contextId>         The context identifier. If only context identifier is given, all data export tasks
                                  associated with denoted context are requested for being canceled
 -h,--help                        Prints this help text
 -i,--userid <userId>             The user identifier. If this option is set, only the data export task associated with
                                  that user is canceled
 -p,--port <rmiPort>              The optional RMI port (default:1099)
 -P,--adminpass <adminPassword>   Admin password
    --responsetimeout <timeout>   The optional response timeout in seconds when reading data from server (default: 0s;
                                  infinite)
 -s,--server <rmiHost>            The optional RMI server (default: localhost)

Examples

Request to cancel all data exports for a certain context:

$ canceldataexports -c 1234 -A oxadminmaster -P secret

Request to cancel the data export for a certain user

$ canceldataexports -c 1234 -i 3 -A oxadminmaster -P secret

List (& repair) orphaned file storage entities or references to non-existent file storage entities via `dataexportconsistency`

Usage

dataexportconsistency -f <filestoreId_1> ... [-r] -A <masterAdmin> -P <masterAdminPassword> [-p <RMI-Port>] [-s
                             <RMI-Server] [--responsetimeout <responseTimeout>] | [-h]
 -A,--adminuser <adminUser>       Admin username
 -f,--filestores <filestoreIds>   Accepts one or more file storage identifiers
 -h,--help                        Prints this help text
 -p,--port <rmiPort>              The optional RMI port (default:1099)
 -P,--adminpass <adminPassword>   Admin password
 -r,--repair                      Repairs orphaned files and export task items.
    --responsetimeout <timeout>   The optional response timeout in seconds when reading data from server (default: 0s;
                                  infinite)
 -s,--server <rmiHost>            The optional RMI server (default: localhost)

Examples

List orphaned file storage entities or references to non-existent file storage entities:

$ dataexportconsistency -f 34 -A oxadminmaster -P secret

Repair orphaned file storage entities or references to non-existent file storage entities:

$ dataexportconsistency -f 34 --repair -A oxadminmaster -P secret

Data export deprecated

Prerequisites

Configuration

Command-line tools

Export the data of a user on behalf of an authority via submitdataexport

Examples

Retrieve, verify and delete an administrative export

Examples

List data export tasks via listdataexports

Examples

Request cancelation for data export tasks via canceldataexports

Examples

List (& repair) orphaned file storage entities or references to non-existent file storage entities via dataexportconsistency

Examples

Export the data of a user on behalf of an authority via `submitdataexport`

List data export tasks via `listdataexports`

Request cancelation for data export tasks via `canceldataexports`

List (& repair) orphaned file storage entities or references to non-existent file storage entities via `dataexportconsistency`