Default File Storage deprecated

Default File Storage

A user of the Drive module holds any number of file storages: the built-in infostore, whose files are held in the filestore of the context, and the accounts the user or the operator added for OpenCloud, Nextcloud, WebDAV and other services. One of them is the user's default file storage: the storage that Drive Mail stores its attachments in, that mail attachments are saved to, whose folders are the default folders for documents, pictures, music, videos and templates, and whose home, public and shared folders a client shows first. By default that is the infostore. Since 8.55 another file storage can be the default, so that new files of that user go to OpenCloud or Nextcloud while the infostore and the other storages stay available alongside.

This document describes how the default storage is configured, what it changes for a user, how a client reaches the storages of a user, what a file storage has to provide to be the default, and which limits apply.

Access to the Drive module

The Drive module is open to a user with the infostore permission, as before, and to a user who holds the capability drive: the capability entitles a user without the infostore to the module, who may use it with other storages, or with none yet, adding storages later. By default the capability is granted to the users with the infostore permission only, through contextSets/drive.yml; grant it per context or user through the config cascade to a user who uses other storages only, see com.openexchange.capability.drive. Denying it to a user with the infostore keeps the OX Drive clients away, as before, and leaves the module open.

Configuration

The default storage is configured per user through the config cascade:

com.openexchange.file.storage.default=opencloud://10

The value is a qualified account identifier of the form <service-id>://<account-id>. The default is com.openexchange.infostore://infostore, the built-in infostore. The account has to exist for the user, and the storage has to be enabled for the user, through com.openexchange.capability.filestorage_opencloud=true for OpenCloud and com.openexchange.capability.filestorage_nextcloud=true for Nextcloud, as the account of a user without the capability cannot be accessed. A user without an account in the configured storage is served by the infostore, which the log notes at debug level.

Only file storage services that are prepared for the role can be the default. Today those are the built-in infostore, OpenCloud and Nextcloud. A value that denotes another file storage service, or one that is not deployed, is logged as a warning and the infostore is the default storage instead, so that a mistaken value never locks a user out of the groupware. A value that cannot be parsed is treated the same way. The warning is logged once per user and value in a while, not with every request.

What changes for the user

Folder tree

The folder tree is the same for every user, whatever the default storage: the system folders My files, Shared files and Public files (9, 10 and 15) serve the built-in infostore, and the root folders of the other file storage accounts are listed below My files. Nothing is mapped onto those folders, and no data is migrated when the default storage changes.

The account listing (fileaccount?action=all) marks the default storage: exactly one account has isDefaultAccount set, the infostore unless another storage is configured.

A client that wants to show the storages of a user side by side lists the virtual folder drive://0/ in tree 1. Its subfolders are the root folders of every account: the default storage first, then the infostore as folder 9, then the other accounts in the order of their names. Below a root the storage lists its own folders, e.g. Personal, Shared, Spaces and Trash for OpenCloud, or, with altNames=true, My files, Shared files, Public files and Trash for the infostore. The folder is read-only and holds no items.

Home, public and shared folders

The folders actions home, public and sharedWithMe answer the folders of the default storage: its home folder, its public folders and the folders other users of the storage shared with the user. Where the user has no account in an external default storage, the infostore answers with the user's default folder and the folders below Public files and Shared files, federated shares included; a user without the infostore gets an empty listing. A storage without a notion of a home folder, such as plain WebDAV, answers home with its root folder.

Sharing

Sharing with users and groups of the installation works as before, as far as the storage can map them to its own users and groups. For the external recipients of a share, i.e. the people invited by their mail address, the storage decides:

  • The infostore creates a guest user of the installation for the recipient and notifies the recipient by mail, as it always did.
  • A storage that grants external recipients access itself, such as OpenCloud, gets the recipient handed down with its mail address. No guest user is created, the storage resolves the address among its own users, grants them access and notifies them on its own. A grant to such a user is stated as a guest permission of the item, with the recipient's mail address and display name, so that clients see and revoke it like any other permission. An address the storage does not know is rejected with the error FILE_STORAGE-0092.
  • A storage that can do neither rejects the request with the error FILE_STORAGE-0091 before a guest user exists.

A folder of a storage that resolves external recipients itself states external_recipients among its supported_capabilities, so that a client can offer the invitation.

Sharing links are created by the storage where it manages its own links, in which case the field com.openexchange.folder.storage.share.external of a folder, and com.openexchange.file.storage.share.external of a file, lists the links the storage holds for the item. Only links are listed there; the shares with users and groups are the item's permissions.

Drive Mail

Drive Mail stores the attachments of a mail in the default storage and shares them through a sharing link of the storage where the storage manages its own links. The expiration date of such a mail expires the link. The option to delete the attachments once the link has expired is only offered and applied where the storage deletes expired files on its own, which is the infostore only: the middleware does not delete data held by an external storage. Clients learn that through the setting io.ox/mail//compose/shareAttachments/autoDeleteSupported; see com.openexchange.mail.compose.share.forceAutoDelete in the property documentation.

Data export

The GDPR data export of the Drive module always exports the built-in infostore of the user. An external default storage is exported in addition, below a directory named after the account, unless the property com.openexchange.gdpr.dataexport.provider.infostore.includeExternalDefaultFileStorage leaves it out, in which case the diagnostics report of the export says so. As the files of an external storage are read over the network at export time, the export of such a user takes longer. What the archive holds depends on what the storage serves; OpenCloud, for example, provides no contents for deleted files, so the trash folders are exported with their structure only. The diagnostics report, which notes such cases, is only written with com.openexchange.gdpr.dataexport.addDiagnosticsReport=true.

Saving mail attachments

Saving a mail attachment to Drive, and attaching a file from Drive to a mail, work with the default storage like with the infostore.

Provisioning the account

The default storage is configured by the account identifier, so the account has to exist for the user. There are two ways to get there.

An account of the user

Any account the user created in the storage, with credentials or linked to an OAuth account of a provider the operator configured, can be made the default by its identifier, e.g. opencloud://7. The operator reads the identifier from the user's accounts (fileaccount?action=all). The account's credentials stay with the account: for an OAuth account the user granted access once, and the server renews the tokens itself.

The identity the user logged in with

A deployment whose identity provider grants access to the groupware and to the storage alike, e.g. keycloak in front of OX App Suite and OpenCloud, needs no account at all. Once the storage is configured for a user, the user has an account in it under the reserved identifier idp:

com.openexchange.capability.filestorage_opencloud=true
com.openexchange.file.storage.opencloud.identityProvider.url=https://cloud.example.com
com.openexchange.file.storage.opencloud.identityProvider.audience=opencloud
com.openexchange.file.storage.opencloud.identityProvider.scope=openid opencloud-audience
com.openexchange.file.storage.default=opencloud://idp

The same properties exist with the prefix com.openexchange.file.storage.nextcloud. for Nextcloud. The server exchanges the session's token at the identity provider for one meant for the storage (RFC 8693) and accesses OpenCloud with it, so no credentials of OpenCloud are kept anywhere. The exchanged token is kept with the session until it is about to expire and exchanged anew once the storage rejects it, at most once every 30 seconds. The account is derived from the configuration and the session rather than stored, so it is listed next to the accounts the user created, and cannot be created, changed or deleted. It only exists for a session that was logged in through the identity provider; a user that logged in with a password has no account in the storage, which the log notes at debug level.

Nextcloud is accessed with an app password instead, which the server obtains once with the exchanged token. Nextcloud logs the user in anew for every request that authenticates with a token and lists each of these logins among the user's devices and sessions, so authenticating every request with the token would flood that list. The app password is kept per user and instance, encrypted with the secret of the user, and shows up among the user's devices and sessions under the name com.openexchange.file.storage.nextcloud.identityProvider.deviceName states, by default OX App Suite (required, do not delete). If the user removes it, the server obtains a new one with the next request, so the user notices nothing. The secret source (com.openexchange.secret.secretSource) must not depend on the password, which a session logged in through the identity provider lacks; without a usable secret, the server accesses Nextcloud with the token itself and logs a warning once per session. Nextcloud has to allow app passwords, which auth_can_create_app_token does by default.

The identity provider has to allow the exchange. For keycloak, with OX App Suite registered as client appsuite-client and the storage as client opencloud:

  • Enable Standard token exchange on appsuite-client.
  • Create a client scope per storage, e.g. opencloud-audience, with an Audience mapper that includes the client opencloud, and the mappers for the claims the storage maps and updates its users by, for OpenCloud the predefined mappers username, email and full name, with Add to userinfo on. Add the scope to appsuite-client as an optional scope, and name it in the scope property together with openid.

The scope property names that scope, and has to name openid as well: the storages read the user's claims from the userinfo endpoint of the identity provider, which keycloak only serves to a token that carries that scope. A default scope of the client is rejected there. Binding the audience scope as a default scope would work as well, but it would put the storage's audience into every token of the client, the session's own included, and an exchange would succeed without naming a scope, so a missing or wrong scope would go unnoticed. With an optional scope, an exchange without the right scope fails with Requested audience not available, which the log shows. A failed exchange, a missing audience included, is logged as a warning with the identity provider's answer, and the user has no account in the storage for the next five minutes before the exchange is tried, and if need be logged, again. On the storage side nothing is provisioned by the server: OpenCloud accepts the tokens of its identity provider as they are; Nextcloud does so through its OpenID Connect app, whose provider has to be configured to check bearer tokens on API and WebDAV requests. Both provision the user on first use. The OpenCloud and Nextcloud articles have the specifics.

Not supported: tokens provided by the operator

An operator cannot configure an app password or an app token of the storage for a user. Such a token belongs to the user, who may revoke it at any time, which the server could neither notice nor repair, and an operator should not hold tokens issued to someone else. The app password the server obtains from Nextcloud itself is different: the server holds the exchanged token to obtain a new one with whenever the user revokes it.

What a file storage has to provide

A file storage service that is to be the default storage has to prepare for the role, as the Drive module and its clients rely on more than the file storage API demands of an ordinary storage. The storage declares its readiness by implementing the marker interface FileStoragePrimaryService; a service without the marker is never taken as the default, see above. Beyond that:

  • Typed folders. The folders a client shows first carry their type through TypeAware: the home folder is a HOME_DIRECTORY, the public folders are PUBLIC_FOLDERs and the trash folder is a TRASH_FOLDER. The folder access answers getPersonalFolder(), getPublicFolders() and getTrashFolder() accordingly.
  • Shared items. The folder access implements FileStorageSharedWithUserFolderAccess and the file access FileStorageSharedWithUserFileAccess, which list the folders and files other users share with the user. Those are what sharedWithMe answers.
  • Folder name search. The folder access implements SearchableFolderNameFolderAccess, so that a search finds the storage's folders.
  • External recipients. The file access implements GuestPermissionAware if the storage can grant the guest users of the installation access, which the infostore does, or ExternalRecipientAware if the storage grants external recipients access itself by their mail address. A storage that implements neither cannot share with external recipients.
  • Sharing links. The folder access implements FileStorageExternalShareFolderAccess and the file access FileStorageExternalShareFileAccess if the storage manages sharing links itself; Drive Mail then shares its attachments through such a link.
  • Stable identifiers. The identifier of a folder or a file survives a rename and a move. Clients keep identifiers in favorites, in mails and in their synchronization state, and Drive Mail references its folders by identifier across the renames it performs.
  • Version count. The metadata of a single file states the real number of its versions, as the clients only offer the versions of a file if that number is above one; a value of -1, which means "unknown" for an ordinary storage, is taken as "none" there. A listing may state -1, as counting the versions of every listed file would cost a request per file.

Implicit contracts of the clients

The clients of the Drive module grew up with the infostore, and some of their behavior relies on traits of the infostore that the file storage API never spelled out. The ones found so far:

  • The versions panel asks whether the storage counts versions, not whether it supports them, see above.
  • A permission of a folder or file is expected to name an entity of the installation. A permission that names a recipient instead, as a storage that resolves external recipients states it, is rendered with type: guest and the recipient's address, and clients send it back the same way.
  • Clients render the entries of the external share field as links, so that field must hold links only.

Limits

  • A storage that was never prepared for the role must not be configured as the default. The middleware falls back to the infostore for such a value, so the user keeps a working groupware.
  • The middleware does not delete data on an external storage on its own: neither expired Drive Mail attachments nor anything else. A storage that offers retention itself can be used for that; the infostore remains the only storage with automatic cleanup.
  • The OX Drive synchronization clients synchronize the file storage services listed in com.openexchange.drive.enabledServices only, which defaults to the infostore. Synchronizing an external default storage with them is not part of this release.
  • Whether deleted files can be read, how versions are listed and whether sharing with external recipients is possible depends on the storage. The OpenCloud and Nextcloud articles list the specifics of those storages.