Provisioning over HTTP

The provisioning API is available over HTTP and JSON, as an alternative to SOAP. A gateway translates the JSON requests into the gRPC calls the middleware serves internally, so the same operations are reachable without a SOAP client.

SOAP is not deprecated. Both interfaces stay available and act on the same data; a system can use either, or both at once.

Where an identity provider is to drive the users, groups, resources and shared accounts of a context, SCIM provisioning covers that lifecycle with the connector the provider already ships. It builds on the same provisioning services and authenticates with the tokens issued here.

Enabling it

The gateway runs as a sidecar next to the middleware pods that serve provisioning. It is off by default:

provisioningGateway:
  enabled: true

That is enough for a first look. Everything below has a working default:

Reaching it

The chart creates a Service named <release>-provisioning-gateway in the release namespace, of type ClusterIP by default. It creates no Ingress — this chart defines none for any of its services, including the middleware itself. Routing traffic to the gateway from outside the cluster is therefore the same step, and uses the same mechanism, as exposing the middleware: an Ingress, an Istio VirtualService or whatever the installation already uses, pointed at that Service.

Do not expose it the way the end-user HTTP API is exposed. This endpoint is administrative and belongs behind whatever restriction the SOAP provisioning endpoint sits behind today.

Securing it

The endpoint carries administrative credentials. Treat it like the SOAP endpoint: do not publish it, and terminate TLS in front of it.

provisioningGateway:
  tls:
    enabled: true
    existingSecret: provisioning-gateway-tls
  service:
    port: 443

The Secret is an ordinary kubernetes.io/tls Secret with tls.crt and tls.key. The chart does not generate a certificate — rendering fails if existingSecret is empty while TLS is enabled. Without TLS the credentials travel in clear text, which is only defensible on a trusted, non-routable network.

Which operations are exposed

Only the services listed under services are reachable. That list is the endpoint's attack surface, so it is worth a deliberate decision rather than a copy of everything available.

The default is the set SOAP already offers, so a caller can move off SOAP without losing an operation it had: contexts, users, groups, resources, shared and secondary accounts, reseller, deputy permissions, the server/database/filestore registry (SOAP's OXUtilService), job management (OXTaskMgmtService), user copy, session management (OXSessionService) and provisioning tokens (ProvisioningTokenService, SOAP's OXProvisioningTokenService, see below).

SessiondService carries one operation SOAP does not, ClearSessionStorage, which drops the whole session storage rather than the sessions of one user or context. Envoy filters by service, not by method, so exposing the service exposes that one too; leave it out of services where that is not wanted.

ChronosService is on the list for the same reason. Its single operation reassigns the organizer of an event after the previous organizer was deleted, and it is part of the documented provisioning API - without it an HTTP-only caller would have no way to reach that operation at all.

Everything beyond that stays opt-in — in particular the cluster maintenance services (ExtendedUpdateTaskService, DBMigrationService, SchemaService, ContextRestoreService, ConsistencyService). These are not what a provisioning client needs, and they include operations such as starting an update run.

MailFilterAdminService is opt-in as well. It reads and changes the mail filter (Sieve) rules of any user in a context the caller administers, without that user's password. Opt-in refers to the gateway only: over gRPC and RMI the service is available to administrators like every provisioning service. It works only where the middleware has administrative access to the mailbox: a token, DoveAdm or master authentication, in the order of com.openexchange.mail.admin.strategies. A rule travels as the JSON object of the mail filter HTTP API v2:

# state, strategy and the hash of the active script
curl -u oxadmin:secret https://<host>/prov/v1/contexts/1/users/3/mailfilter

# add a rule, but only if nobody changed the script since the hash was read
curl -u oxadmin:secret \
     -H 'Content-Type: application/json' \
     -d '{"rulename": "Spam", "active": true, "test": {"id": "header", "comparison": "contains", "headers": ["X-Spam-Flag"], "values": ["YES"]}, "actioncmds": [{"id": "move", "into": "default0/INBOX/Spam"}]}' \
     'https://<host>/prov/v1/contexts/1/users/3/mailfilter/rules?if_match=<script_hash>'

if_match is optional; an empty value counts as not set. Without it, a change made in the meantime - by the user, another node or another client - is overwritten, and repeating a POST .../rules, e.g. after a 504, adds the rule twice. With it, a repeated call answers 409 if the first one went through: read the rules again rather than retrying. A script_hash is only comparable while the access strategy stays the same. if_match is best effort: the check and the write are not atomic across nodes or towards other ManageSieve clients, so a write landing in between is still overwritten.

Without flag, GET .../rules leaves out the rules of mail categories (flags category and syscategory); POST .../mailfilter:purge deletes those too, so take a backup with GET .../mailfilter/script first. POST .../rules:reorder only takes rules the list shows. GET .../mailfilter answers "available": false if no strategy applies or the mail filter is switched off for the user. JSON fields are camel case (scriptHash); the first rule of a script has the id 0.

Where the mail server accepts OAuth tokens, a call can bring one along for the user it addresses, in the header mail-authorization: Bearer <token>. The gateway passes it on as gRPC metadata. With the strategy token in com.openexchange.mail.admin.strategies (the default order is token,doveadm,master), that token logs in to the user's mailbox and Sieve scripts, over SASL OAUTHBEARER; without the header the next strategy applies. The token serves only the user the call names by context_id and user_id: other users the call touches, a user named by login name, and calls about several users fall to the next strategy. It is used for this one call only, never on a connection opened for the user before, is passed on to another site with the call, and never logged. If the mail server rejects it, the call fails with 403; if the mail server offers no SASL mechanism for tokens, it fails with 400 without any login attempt. The same header works for DeputyPermissionService, e.g. to grant deputy permissions on mail.

curl -s -u oxadmin:secret -H 'mail-authorization: Bearer <token>' \
     'https://<host>/prov/v1/contexts/1/users/3/mailfilter'

Every call counts against the provisioning rate limit of the calling administrator (com.openexchange.admin.rmi.rate.limit.default, off by default, and com.openexchange.admin.rmi.rate.limit.<admin>). A client that changes many filters in parallel is worth a limit.

To expose one, add its full gRPC service name:

provisioningGateway:
  services:
    - com.openexchange.grpc.provisioning.ContextService
    - com.openexchange.grpc.provisioning.UserService
    # … the rest of the default list …
    - com.openexchange.grpc.provisioning.SchemaService

A service name that does not exist is rejected when the gateway starts, so a typo shows up as a failing container rather than as a silently missing endpoint.

Calling it

Paths start with /prov/v1/, and resources are nested the way they belong together:

GET    /prov/v1/contexts
POST   /prov/v1/contexts
PATCH  /prov/v1/contexts/{context_id}
DELETE /prov/v1/contexts/{context_id}
POST   /prov/v1/contexts/{context_id}:enable
GET    /prov/v1/contexts/{context_id}/users
GET    /prov/v1/contexts/{context_id}/groups/{group_id}
POST   /prov/v1/contexts/{context_id}/users/{user_id}/deputies:grantMultiple
POST   /prov/v1/contexts/{context_id}/tokens
DELETE /prov/v1/contexts/{context_id}/tokens/{token_id}

Operations that are not a plain create/read/update/delete carry a verb suffix after a colon, as in :enable or :grantMultiple.

Two things about these paths are worth knowing before writing a client:

• Path parameters carry plain identifiers and are named after the request field in snake case: {context_id}, {user_id}, {group_id}. Where an object may be addressed by name instead, the path offers a second parameter such as {resource_name}.
• There is no endpoint for reading a single context. One is read through the collection, by passing its identifier as a filter to GET /prov/v1/contexts. The same holds for users and resources: the collection endpoint takes identifiers, a pattern and paging as parameters. Groups are the exception and can be read one at a time.

Credentials go into the Authorization header, as HTTP basic — never into the request body. The login is the admin's own; which context it acts on comes from the path, not from the login. This is the point where a SOAP client will trip: SOAP carries them inside the message, and the same habit produces a request the gateway rejects.

curl -u oxadminmaster:secret https://<host>/prov/v1/contexts

curl -u oxadmin:secret \
     -H 'Content-Type: application/json' \
     -d '{"user": {"name": "jdoe", "displayName": "John Doe"}}' \
     https://<host>/prov/v1/contexts/1/users

Contexts are addressed numerically. SOAP accepts a context by name in places where the HTTP path takes the identifier.

An OpenAPI document describing every path, parameter and response is generated from the API definition itself and published alongside the other API documentation:

https://documentation.open-xchange.com/components/middleware/provisioning/<version>/

That page renders the description in a browser. The document itself sits next to it as provisioning.swagger.json and is the file to point a client generator at.

Provisioning tokens

Automated clients that cannot hold an administrator's password, an identity provider driving the SCIM endpoint or a script driving this gateway for instance, authenticate with a provisioning token instead. A token is bound to one scope and opens that one interface and nothing else: scope SCIM opens the SCIM endpoint, scope PROVISIONING opens /prov itself, presented as Authorization: Bearer <secret>. A token acts as an administrator of the contexts it is bound to, never as the master administrator, and it cannot manage tokens.

Tokens are managed over the gateway by the context administrator, a reseller administrator owning the context or, where MASTER_ACCOUNT_OVERRIDE permits it, the master administrator - the same rule every other operation inside a context follows. The same operations exist without a gateway, as the command line tools createprovisioningtoken, listprovisioningtokens, revokeprovisioningtoken and detachprovisioningtoken and as the SOAP service OXProvisioningTokenService (create, list, revoke, listCrossContextReachingInto, detachCrossContext); over the gateway they are:

# create; the secret is part of this response only
curl -u oxadmin:secret \
     -H 'Content-Type: application/json' \
     -d '{"label": "Entra ID", "scope": "SCIM", "expires": 0}' \
     https://<host>/prov/v1/contexts/1/tokens

# list (metadata only)
curl -u oxadmin:secret https://<host>/prov/v1/contexts/1/tokens

# revoke
curl -u oxadmin:secret -X DELETE https://<host>/prov/v1/contexts/1/tokens/<token_id>

# use a token of scope PROVISIONING on the gateway
curl -H 'Authorization: Bearer ox_1_...' https://<host>/prov/v1/contexts/1/users

The secret has the form ox_<context-id>_<64 hex characters>. Only its SHA-256 hash is stored, so a lost secret cannot be recovered; revoke the token and create a new one. expires is a point in time in milliseconds since the epoch, 0 means the token does not expire. Every successful use records its time, visible as lastUsed in the listing, which is how a forgotten token can be spotted.

Cross-context tokens

A client working across contexts - one granting shared account permissions between them, say - needs a credential that stands above every context involved, which no context administrator's password does. A cross-context token opens several contexts at once: it is created for an explicit list of at least two contexts, and only by the master administrator or a reseller administrator owning every one of them. It is stored installation-wide rather than in a context, its secret has the form ox_x_<64 hex characters>, and it is accepted on /prov for exactly the contexts it names - a context it does not name is answered with 401, as is any attempt to act as the master administrator, such as creating a context. When one of its contexts is deleted, the token drops that context; when the last one is deleted, the token is deleted with it.

Creating one requires MASTER_ACCOUNT_OVERRIDE, the switch that decides whether an administrator may reach into a context at all. With the override off nobody issues a token that acts inside a context - otherwise the master administrator could hand itself the access the override denies it.

Listing and revoking are not tied to the switch for the master administrator: whatever was issued before it was turned off can still be seen and taken back. A reseller administrator, like everywhere else, reaches its contexts only while the override permits it, so with the override off its own tokens are out of its reach - the master administrator revokes them. A token stays valid until it is revoked; turning the override off does not stop the tokens already handed out.

What the context reached into can do

A cross-context token is issued above the contexts, so the context it opens has no part in it. It is not left without a say either: its administrator can see which cross-context tokens open the context and end that one context's exposure, the way both sides of a shared account permission can end it.

The listing names the asked-about context and no other - which further contexts a token opens is not disclosed to it. Detaching is not a revocation: the token keeps working for every other context it opens, and is deleted only if this was the last one. It is not a lasting veto either, since an administrator above the contexts may issue a new token covering the context again; to end a token everywhere at once, revoke it.

# what reaches into context 1, as its administrator sees it
curl -u contextAdmin:secret https://<host>/prov/v1/contexts/1/cross-context-tokens

# end this context's exposure; the token keeps opening the others
curl -u contextAdmin:secret -X DELETE \
     https://<host>/prov/v1/contexts/1/cross-context-tokens/<token_id>

# create; needs the master administrator or a reseller owning contexts 1 and 2
curl -u oxadminmaster:secret \
     -H 'Content-Type: application/json' \
     -d '{"contextIds": [1, 2], "label": "Shared account automation", "scope": "PROVISIONING"}' \
     https://<host>/prov/v1/tokens

# list the cross-context tokens the caller stands above (metadata only)
curl -u oxadminmaster:secret https://<host>/prov/v1/tokens

# revoke
curl -u oxadminmaster:secret -X DELETE https://<host>/prov/v1/tokens/<token_id>

# use it: one header, two contexts
curl -H 'Authorization: Bearer ox_x_...' \
     -H 'Content-Type: application/json' \
     -d '{...}' \
     https://<host>/prov/v1/contexts/2/sharedaccounts/permissions

The command line tools take --contexts 1,2 instead of -c, and list or revoke cross-context tokens when no context is given; the SOAP service offers createCrossContext, listCrossContext and revokeCrossContext. A listing shows the contexts a token opens in contextIds; a cross-context token has no contextId of its own (0). A reseller administrator sees, and can revoke, only the tokens whose contexts it all owns.

Trying it without a deployment

The gateway can be run against a middleware on a developer machine with ./gradlew :grpc-proto:provisioningGateway, which makes the same paths available at http://localhost:8090/prov/v1/.... Useful for reproducing a report without a cluster; see grpc-api/readme.md in the middleware sources for the details.

Errors

The gateway maps the internal status onto an HTTP status and returns a JSON body with a message:

MailFilterAdminService adds these:

One wrinkle worth knowing: a missing context is reported as 404 only where the operation acts on the context from the outside — creating, listing, capabilities and the like, which authenticate the master admin and then look it up.

Operations that act inside a context — users, groups, resources, deputies — authenticate against that context, and that step fails before anything is looked up. They answer 401 for a context that does not exist, the same as for a wrong password. A caller cannot tell those two apart by status code, and giving a context id it has no rights on looks identical to giving one that is not there.

When the gateway gives up

A call that takes longer than timeout is answered with 504 and a body of upstream request timeout. The operation is not canceled: the middleware completes it, and only the outcome never reaches the caller. A 504 therefore means "unknown", not "failed" — the context may well be deleted or the user changed.

Read the object back before repeating the call. The middleware logs every such call once it completes, with the method and its outcome:

WARN  [grpc-default-executor-0] com.openexchange.provisioning.grpc.servlet.utils.AbandonedCallInterceptor.log(AbandonedCallInterceptor.java:100)
com.openexchange.grpc.provisioning.ContextService/UpdateContext finished with OK after 8095 ms, but its caller had given up on it before, e.g. the HTTP gateway on its timeout. The caller did not receive this outcome.

The same applies to any timeout between the caller and the gateway: an Ingress or load balancer that gives up first leaves the caller in the same position, so it needs at least the gateway's timeout.

Differences from SOAP

The operations are the same, but they are named and grouped differently. SOAP spells out combinations as separate operations; the HTTP API folds them into one call with parameters. Listing contexts is a single endpoint where SOAP has eight operations — pattern, paging, restriction to a database or filestore, and lookup by identifier are all parameters of GET /prov/v1/contexts.

Three behaviors differ deliberately:

• Fetching a group without specifying one returns the context's default group.
• Blocking and unblocking a deputy permission is one endpoint with a boolean, not two operations.
• The reseller data of a context — its custom identifier, its owner and its restrictions — is a resource of its own at /prov/v1/contexts/{context_id}/reseller, where SOAP carries those fields in the context itself. Sending restrictions replaces them wholesale, so an empty list removes all of them; leaving the field out keeps the ones the context has. The owner is reported by identifier and name only — read the full record with GET /prov/v1/resellers/{reseller_id}.

Granting several deputy permissions at once rolls back on error: if one grant fails, those already granted in that call are revoked before the error is reported, so a failed call leaves nothing behind.