Implementation Details deprecated

This section provides some deeper insights for certain topics of the new calendar implementation. Whenever appropriate, semantical differences to the previous calendar stack or the standards are highlighted.

Date/Time handling

There are some different modes for the start- and end-date of events. This affects all areas where dates are used and exchanged (HTTP API, iCal, Java, database). Most basically, there are dates (without time fraction) and datetimes (a date with time fraction). For the latter one, three different forms are possible.

Date

A calendar date with no time component, i.e. only year, month and date are set. It is not possible to assign a concrete timezone to a date, so that dates do always appear in the timezone of the actual viewer. All-day events start- and end on a date. This characteristic is called floating, and needs to be handled appropriately whenever concrete periods of such all-day events need to be considered, e.g. during free/busy lookups or conflict checks.

Date

  • Used for all-day events
  • Are floating, implicitly
    • Example in iCal: DTSTART:19980118
    • In the database, the unix timestamp is stored, and the timezone is set to NULL, additionally the allDay column is true
    • In the HTTP-API, only the value property is set, without time-fraction, e.g.: { "value":"19980118" }

Date with Time

A precise calendar date and time consisting of day, year, month, date, hour, minute, second. A datetime may be defined in three different forms, which are local time (no specific timezone), in UTC, or in local time with timezone reference.

  1. Local time - no specific timezone

    • Used for floating events
    • Represent the same year, month, day, hour, minute, second no matter in what time zone they are observed
    • Example in iCal: DTSTART:19980118T230000
    • In the database, the unix timestamp is stored, and the timezone is set to NULL
    • In the HTTP-API, only the value property is set, e.g.: { "value":"19980118T230000" }

  2. UTC - UTC timezone

    • Easiest form, but not much useful for events
    • Example in iCal: DTSTART:19980118T230000Z
    • In the database, the unix timestamp is stored, and the timezone is set to UTC
    • In the HTTP-API, the value property is set with a trailing Z as in RFC 5545, e.g.: { "value":"19980118T230000Z" }

  3. Local time with Timezone reference - specific timezone

    • Most common form for events
    • Example in iCal: DTSTART;TZID=America/New_York:19980119T020000
    • In the database, the unix timestamp ans the timezone are stored
    • In the HTTP-API, an additional tzid property is included, e.g.: { "value":"19980118T230000", "tzid":"America/New_York" }

Usage

Internally, throughout the new Chronos stack, a DateTime object from the 3rd party library rfc5545-datetime is used, where dedicated support for the different modes are available. In the database, we have separate columns for the actual value of the start- and end-date of an event, the associated timezones and the all-day flag.

In the HTTP-API, the start- and end-date properties of events are serialized within a JSON object, which is based on the definitions in RFC 5545, as follows:

DateTimeData {
  value (string, optional): A date-time value without timezone information as specified in rfc 5545 chapter 3.3.5. E.g. "20170708T220000" ,
  tzid (string, optional): A timezone identifier. E.g. "America/New_York"
}

References / further reading:

Timezones

Timezones play an important role in calendaring and scheduling, as they allow to define an instant of time in a certain geopolitical region, relative to the Coordinated Universal Time (UTC). Especially when scheduling events with attendees that are located in different timezones, it is important to derive the same point of time of the event in each timezone, based on the underlying timezone definitions.

Internal Handling

Whenever events whose start- and enddate are decorated with a specific timezone identifier, the concrete instant of time is evaluated dynamically based on the corresponding timezone information from the Java runtime environment. This instant of time is then used for all sorts of operations with the start- and endtime, e.g. scheduling, recurrence calculations, free/busy lookups, sorting and so on.

The matching timezone is looked up based on the timezone identifier, which is usually a string in the form "continent/city" (for example "America/New_York"), as defined in the in the Olson Timezone Database, which is the most common internationally agreed standard for timezones. Therefore, it is required that all used timezone identifiers are available in the server, which is normally the case if the Java runtime environment is updated regularly.

Parsing

Not all clients are using the same set of timezone definitions, and especially clients that do not use timezones from the common Olson database may be problematic as an internally known timezone needs to be selected so that it can be processed by the server. The most prominent example are clients from Windows, which often rely on a Windows-internal set of timezone definitions that use different identifiers.

In order to also accept non-Olson timezones, such unknown timezones are attempted to be mapped in the following, best-effort way:

  • If an unknown timezone is parsed during an update operation, and the parsed timezone has the same rules as the timezone of the originally set timezone, fall back to the original timezone
  • If an unknown timezone is parsed, and the parsed timezone has the same rules as the timezone of the calendar user, fall back to the timezone of the calendar user
  • If an unknown timezone is parsed, and the parsed timezone has the same rules as the timezone of the current session's user, fall back to the timezone of the session user
  • If an unknown timezone is parsed, and a known mapping from the parsed Windows timezone identifier to Olson exists, use the mapped Olson timezone
  • If an unknown timezone is parsed, and at least one known timezone with the same rules exists, use the timezone whose identifier is most similar (Levenshtein distance) to the parsed one
  • Use the calendar user timezone, otherwise

References / further reading:

Relation of Organizer / Principal / Folder-Owner / Creator

Terminology

Folder-Owner ~ The owner of a personal calendar folder

  • In OX, this is the user with the identifier of the folder's createdBy property, if the folder type is private
  • In CalDAV, this is expressed in the calendar collection's calendar-owner property:
    RFC 4791, 12.1: This property is used for browsing clients to find out the user, group or resource for which the calendar events are scheduled. Sometimes the calendar is a user's calendar, in which case the value SHOULD be the user's principal URL from WebDAV ACL. (In this case the DAV:owner property probably has the same principal URL value.)
  • In Outlook, this owner is usually called the principal, which is the principal name for the mailbox user.

Organizer ~ The organizer of an event

  • In OX (legacy), this is the user who created the event
  • In iCalendar, this is the ORGANIZER property and points to the organizer of the event
  • In CalDAV, the organizer determines the type of scheduling object resources:
    RFC 6638, 3.1: A calendar object resource is considered to be a valid organizer scheduling object resource if the "ORGANIZER" iCalendar property is present and set in all the calendar components to a value that matches one of the calendar user addresses of the owner of the calendar collection. and A calendar object resource is considered to be a valid attendee scheduling object resource if the "ORGANIZER" iCalendar property is present and set in all the calendar components to the same value and doesn't match one of the calendar user addresses of the owner of the calendar collection, and if at least one of the "ATTENDEE" iCalendar property values matches one of the calendar user addresses of the owner of the calendar collection.
    So, for newly created events, the ORGANIZER property of the iCal data must match the owner of the parent calendar collection.
  • In Outlook, this is the meeting organizer

Difficulties

In case an event is created within the personal calendar of a specific user, this user is automatically also the organizer of the event. This is the most common case and is easy to handle and understand. Difficulties arise as soon as an event is created in the personal calendar folder of another user. Then, the user is acting on behalf of the calendar owner. This is often referred-to as the secretary functionality in the calendar, where the manager grants full permissions to his secretary to manage his appointments. In this case, some special handling applies towards the assignment and interpretation of the organizer attribute.

Legacy: Organizer and Principal

The legacy calendar implementation used the properties "principal" and "organizer" for storing an event's organizer and a possibly different owner. The organizer (and organizerId) always holds the user who creates the appointment, regardless of the parent folder. The principal (and principalId, respectively) properties are set to the parent calendar folder's owner, in case it is a personal calendar that is shared by another user. The properties are stored as-is in the database, and are accessible in the same way via the HTTP API. The createdBy property is set to the identifier of the folder owner as well.

Example:

  • Eva (id 5) has granted read/write permissions for his personal calendar folder to Tobias (id 4)
  • Tobias creates a new appointment in Eva's calendar
  • Database: created_from=5,principalId=5,principal='eva@local.ox',organizerId=4,organizer='tobias@local.ox'
  • HTTP API: "created_by":5,"principal":"eva@local.ox","organizerId":4,"organizer":"tobias@local.ox"

Chronos: Organizer and sent-by

In the Chronos stack, there's no dedicated "principal". Instead, it is ensured that the organizer is always the actual calendar owner for newly created events (An exception to this rule are imported scheduling object resources from external organizers, as described at RFC 6638, section 3.2.2.2). In case the event is created on behalf of the folder owner by another calendar user (e.g. the secretary), this is expressed via the SENT-BY attribute within the organizer. This parameter is only added during the creation of scheduling messages to external attendees.

Changing the organizer

Since v.7.10.2 the organizer of an event can hand the event over to another internal CU. Doing so, the other CU becomes the new organizer of the event thus is able to change the event. The feature is deactivated by default and can be activated by setting the property com.openexchange.calendar.allowChangeOfOrganizer to true. When changing the organizer the server will generate a notification to all attendees, optional giving the old organizer the possibility to announce why the event was transmitted to another CU.

The change can only be performed if all attendees, the acting and the new organizer are internal users. The reason for this restriction is that even though the iTIP standard supports such changes most implementations do not. Changing the organizer on events with external CUs will lead to inconsistent events across the different systems and is therefore prohibited.

Conversion

In order to convert between the legacy properties for organizer/principal and the organizer as it is used within the Chronos stack, the following conversions are performed:

HTTP API, Appointment->Event and Database, prg_dates->Event

  • principal/principalId not set:
    Organizer is taken from organizer/organizerId Organizer's "sent-by" is empty
  • principal/principalId are set:
    Organizer is taken from principal/principalId
    Organizer's sentBy is constructed from organizer/organizerId

HTTP API, Event->Appointment and Database, Event->prg_dates

  • Organizer's "sent-by" is set:
    organizer/organizerId is taken from Organizer's sent-by
    principal/principalId is taken from Organizer
  • Organizer's "sent-by" not set:
    organizer/organizerId is taken from Organizer
    principal/principalId empty

References / further reading:

Per-Attendee delete exceptions

As invited attendees may delete a meeting from their personal calendar if they do not want to attend ("decline, and remove the event from my calendar"), they may also do so for specific occurrences of a recurring event series. From the organizer's and the other attendee's point of view, this leads to a new change exception event, where the attendee's participation status is set to declined. However, for the attendee who has deleted a specific occurrence of the series from his calendar view, this rather means the creation of a new delete exception in the event series. Similarly, an attendee can be removed or uninvited from a specific occurrence of an event series by the organizer, so that the attendee list no longer includes him.

According to the RFC 6638, in such a scenario the attendee effectively gets a different set of delete exception dates (EXDATE property in iCal), while the organizer and the other attendees see this exception date as overridden instance (change exception):

As another example, an "Attendee" could be excluded from one instance of a recurring event. In that case, the organizer scheduling object resource will include an overridden instance with an "ATTENDEE" list that does not include the "Attendee" being excluded. Any scheduling messages delivered to the "Attendee" will not specify the overridden instance but rather will include an "EXDATE" property in the "master" component that defines the recurrence set.

While appropriate handling has originally been in place as incoming/outgoing "patches" within the CalDAV implementation, this is now considered directly within the Chronos service itself, so that the change- and delete-exception arrays in series events may be different based on the actual calendar user.

References / further reading:

Classification / Private flag

The legacy private flag (pflag in database) is used to hide sensitive details from appointments to other users. Participants of the appointment may always see all details of such appointments, while other users who are able to access such appointments based to their permissions (e.g. in shared folders) will only have a restricted view on them. This basically includes the start- and end-time, identifying properties such as the UID, and the appointment's shown-as value. Instead of the appointment title, usually "Private" is shown instead.

In iCalendar, this relates to the classification property of an event, with the possible values PUBLIC (default), PRIVATE and CONFIDENTIAL. While documentation about the exact meanings of PRIVATE and CONFIDENTIAL are quite rare, the legacy private flag best matches the semantics of CONFIDENTIAL, i.e. only start- and end-times of the events are visible when being read by non-participating users. So, the legacy private flag will be converted to the classification CONFIDENTIAL; vice-versa, both PRIVATE and CONFIDENTIAL will make the private flag true. Consequently, the parameter showPrivate of the legacy HTTP API is applied to CONFIDENTIALly marked events.

Internally, the following semantics apply for the classifications:

  • PUBLIC (default) No special handling, i.e. all event properties are exposed to non-attending users in shared folders.
  • CONFIDENTIAL Only certain non-classified event properties are exposed to non-attending users in shared folders (basically all date- and time-related properties as well as administrative fields like identifiers), so that effectively such events appear as anonymous 'blocks' when being viewed in a calendar. The event's summary is replaced with the static text "Private". Additionally, such events are still considered in conflict- and free/busy-queries.
  • PRIVATE Events are not exposed to non-attending users in shared folders at all. Additionally, such events are only considered in conflict- and free/busy-queries in case of an attending resource attendee.

When setting the classification property to a value different from PUBLIC, this can only be done for events that are located in one of the acting user's personal calendar folders. This is done to ensure that a non-owner cannot lock himself out of access to a calendar object resource, with no way to undo the operation. Additionally, it is not allowed to set the classification to PRIVATE for events with resource attendees, to ensure they're always considered during conflict- and free/busy-lookups.

For synchronization via CalDAV, the event classification is passed and read as-is, with the exception of the Apple calendar clients who require some special treatment (in form of a proprietary spec) in this topic. When exporting calendar object resources whose classification is different from PUBLIC for an Apple client, the parent VCALENDAR component is decorated with an additional property X-CALENDARSERVER-ACCESS, set to the value of the (series master) event's classification. In the same way during import, the passed value of X-CALENDARSERVER-ACCESS is considered and transferred from the parent VCALENDAR component to each contained VEVENT.

References / further reading:

Transparency / Busy Status

The time transparency is used in an event to specify whether it actually consumes time on a calendar, hence is detected as such in free/busy time searches. In iCalendar, this is represented as OPAQUE in the TRANSP property. Events that should not consume the calendar user's time should be recorded as TRANSPARENT, so that they're not taken into account in free/busy time searches.

In the legacy calendar implementation, there were two additional states available, aligned to the corresponding property in Microsoft Exchange/Outlook PidLidBusyStatus: absent (olOutOfOffice) and temporary (olTentative). However, for increased compatibility with the iCalendar standard, and to avoid ambiguities with the event's overall STATUS, they're no longer used and both mapped to a transparency of OPAQUE when using the new APIs.

The TRANSP property is managed separately for each attendee. It can either explicitly be set via the updateAttendee API call or implicitly via CalDAV. CalDAV clients tend to allow changes on the event transparency, even though the user might not be the organizer. This (illegal) change on the event will be interpreted as an per-attendee transparency change on the server. If an attendee hasn't set an explicit value, the transparency of the event property TRANSP is considered instead. The per-attendee TRANSP property is implicitly delivered through the userized event that is send back through the API or via CalDAV. The participation status of DECLINED will automatically make an event TRANSPARENT in free/busy queries for the attendee. Whenever the transparency of the event is changed, all per-attendee transparencies are removed. Thus, the updated transparency is effective for all attendees.

References / further reading:

Move between Folders

Whenever events are moved between different folders, some special handling applies, especially for move operations between different types of folders. We basically differentiate between two types of folders, which are on the one hand personal calendar folders of the internal users, and public folders on the other hand that are not directly associated explicitly with a user. The following gives an overview about the possible move actions and their outcome.

General restrictions

The user performing the move must be equipped with appropriate permissions for the source- and target folder. For the source folder, these are at least Read folder, Read own/all objects, Write own/all objects and Delete own/all objects. All or own depends on the event being created by a different user or not. In the target folder, at least the following permissions must be granted: Create objects in folder.

In group-scheduled events with multiple attendees, the calendar user's role in the event is checked as well when moving between one folder type to another (see also chapter "Permissions" below). In particular, it's required to be or impersonate as the organizer when moving such a group scheduled event from a personal into a public folder and vice-versa. Additionally, recurring event series (or change exception events) cannot be moved. And finally, there's a limitation towards moving events with a classification of PRIVATE or CONFIDENTIAL (which correlates to the legacy private flag). In this case, events must not be moved to a public folder, or moved between personal folders of different users (~ shared to private and vice versa).

Move Scenarios

After the general restrictions have been checked and are fulfilled, the following happens depending on the source- and target folder type. Implicitly, this also includes triggering of further updates, like updating the folder informations for stored alarm triggers or inserting tombstone objects in the database so that the deletion from the source folder can be successfully tracked by differential synchronization protocols.

Public calendar folder 1 -> Public calendar folder 2

  • Update the common public folder identifier of the event

Personal calendar folder 1 of User A -> Personal calendar folder 2 of same User A

  • Update attendee A's parent folder identifier

Personal calendar folder of User A -> Personal calendar folder of other User B (non group-scheduled event)

  • Update the common folder identifier of the event
  • Update the calendar user of the event to user B

Personal calendar folder of User A -> Personal calendar folder of other User B ("pseudo" group-scheduled event with one attendee/organizer)

  • Update the original calendar user attendee A to user attendee B and take over his target folder identifier

Personal calendar folder of User A -> Personal calendar folder of other User B (group-scheduled event with multiple attendees)

  • Not allowed to avoid ambiguities

Public calendar folder -> Personal calendar folder of User A

  • Update attendee A's parent folder identifier accordingly
  • Assign default parent folder identifier for all other user attendees

Personal calendar folder of User A -> Public calendar folder

  • Take over common public folder identifier for all user attendees

References / further reading:

Free/busy and conflict checks

As mentioned earlier, the server is capable of performing free/busy and conflict checks. So what are those in detail?

First, the mentioned checks are a help for the end user creating or updating an event. Lets' start with the free/busy check. The free/busy check will generate a list of all attendees that shall participate in the event to create or update. Furthermore, (and more importantly), also a list of events an attendee is already participating in will be generated, per attendee. The data allows a client to visually display time slots in which attendees don't have time or rather are too "busy" to participate in the new or updated event. This allows the acting user to find a "free" time slot, so most or even all attendees can participate in the new or updated event. This free/busy check however is only a snapshot. There is always the possibility while the acting user finds a "free" slot, that one or more attendees will be invited to yet another event, efficiently generating a conflicting event. Therefore, whenever an event is created or updated there is the possibility to perform a conflict check. This conflict check will again generate a list of conflicting events, with an additional list of attendees who have the conflict. In addition to the free/busy check, a conflict check is enhanced with the knowledge if the conflict is a so called hard or soft conflict. A hard conflict must be resolved before creating or updating the event. This can e.g. happen if a ROOM attendee is participating that has already an event at the desired time slot, so rooms can't be overbooked. A soft conflict in contrast tells you that an attendee has an event at the desired time slot, however, the event can still be created or updated by ignoring this circumstance.

For further details on how the checks are performed and influences, please read the other sections about Transparency / Busy Status and Classification / Private flag, too.

Cross-context

Normally, to one Open Xchange server belong many different contexts in which different users live. To enable a more user-friendly experience, it is possible to perform free/busy and conflict checks for users from different contexts on the same server, too. The cross-context checks become handy if e.g. each context is a different department in a larger organization. Thus, an end user can perform free/busy and conflict checks with her colleagues without the need of a shared calendar or an iTIP/iMIP roundtrip.

Because not everyone using the Open Xchange server will find this feature useful, it is configurable. Please have a look at the property com.openexchange.calendar.enableCrossContextFreeBusy to enable the free/busy check and the property com.openexchange.calendar.enableCrossContextConflicts to enable the conflict check.

Please also note that the properties are configurable down to context level via the config-cascade and have to be enabled for at least two contexts to work, the context of the user who is requesting the free/busy or conflict check and for the context of the user for whom the free/busy or conflict check is performed. To resolve the calendar users in foreign contexts by their email address, the mail resolver service is utilized, see here for further details.

Please mind that the mechanism to enable free/busy lookups through a subscribed calendar share in a foreign context (that was available in v7.10.5) has been superseded by this general approach which, which works without the indirection of established cross-context folder shares.

Free/Busy Visibility

By default the availability of a user can be inspected by others, e.g. prior scheduling a meeting. As the exposure of this potentially sensitive data may not be desired for one or the other user, the freeBusyVisibility setting has been introduced. It's default value can be configured by the admin via property com.openexchange.calendar.freeBusyVisibility.default. Also, it is possible to mark the setting as "protected" (via com.openexchange.calendar.freeBusyVisibility.protected), so that it cannot be changed by the end user.

On a per-appointment level, visibility can also be controlled by assigning a "secret" classification, which also hides them from free/busy lookups initiated by other users (see chapter "Classification / Private flag" above).

Free/busy servlet

Because this topic is complex enough on its own, it is described in its own article. Please have a look at the article Internet Free Busy Servlet

Configuration

Free/busy and conflict checks are limited to return a certain amount of response entries and to search only in defined time frames. This has performance reasons, since those queries can grow rapidly and consuming many resources. For a detailed list of configuration options that are taken into account please have a look here

Conversion of Recurrence Rules

Within the Chronos stack, the recurrence information is transferred and treated as plain RRULE string, like it is defined in RFC 5545. Any series calculations are performed ad-hoc, under the constraints of some further influencing properties (start-date, start timezone, all-day flag) of the series master event. In the legacy implementation, the recurrence information was split over different properties in the class CalendarDataObject, and provided as such to clients accessing the HTTP API. In particular, the properties recurrence_type, days, day_in_month, month, interval and until. In the database, this recurrence information is stored in a combined field as so-called series pattern.

For interoperability with the old API and database format, distinct conversion routines are in place, which were built upon already existing (de-)serialization routines to and from iCalendar. This also means that consequently still only those recurrence rules are supported that can also be expressed with the legacy series pattern, at least as long as the data is stored in this format. However, any recurrence calculations in the Chronos stack do already support all RRULE parts as described in RFC 5545.

Supported RRULE parts

The following list gives an overview about the supported RRULE parts, based on the limitations described before.

  • FREQ:DAILY: INTERVAL, UNTIL, COUNT
  • FREQ:WEEKLY: INTERVAL, UNTIL, COUNT, BYDAY
  • FREQ:MONTHLY: INTERVAL, UNTIL, COUNT, BYDAY, BYSETPOS, BYMONTHDAY
  • FREQ:YEARLY: INTERVAL, UNTIL, COUNT, BYDAY, BYMONTH, BYSETPOS, BYMONTHDAY

UNTIL handling

If an event series is decorated with an end date, this used to be stored as the date of the last occurrence without time fraction in milliseconds since the Unix epoch. However, in RFC 5545, the corresponding UNTIL part of a RRULE is just defined as

The UNTIL rule part defines a DATE or DATE-TIME value that bounds the recurrence rule in an inclusive manner.

Therefore, some information might currently get lost when converting a recurrence rule to a legacy series pattern, since the time fraction of the date-time value from the RRULE has to be truncated prior saving. Also, some special checks need to be in place during conversion to prevent an additional occurrence if the client-supplied UNTIL lies behind the last occurrence (and possibly right before the start date of a next occurrence).

References / further reading:

Reset of Participation Status

Whenever an existing event with attendees is rescheduled, each attendee's participation status is reset to NEEDS-ACTION. The optional comment, however, is persisted if present. A reschedule occurs, when any DTSTART, DTEND, DURATION, RRULE, RDATE, or EXDATE property changes such that existing recurrence instances are impacted by the changes (RFC 6638, 3.2.8) - i.e., whenever the period of an event changes.

Internal Handling

Internally, these kind of checks are performed along with each event update is checked. Optionally, the standard allows to behave similarly in case of other changes, e.g. a changed event location, however, currently only time changes are considered.

In contrast to the rather strict definition in the standard, the following, slightly adjusted rules are applied when determining if the attendee's participation status will be resetted or not:

  • For series events, reset if there are 'new' occurrences (caused by a modified or extended rule)
  • For series events, reset if there are 'new' occurrences (caused by the reinstatement of previous delete exceptions)
  • Reset if updated start is before the original start
  • Reset if updated end is after the original end
  • Don't reset otherwise

Doing so, the parstats are not resetted whenever the event's effective timeframe is reduced, e.g. the event is re-scheduled to start half an hour later, or end a couple of minutes earlier.

Besides these rules that apply for most attendees, there are two special cases in which a reset of the participant status for certain attendees still won't be performed. One special case applies for the calendar owner. Whenever the calendar owner, or someone on behalf of the calendar owner, changes an event in a way it is rescheduled, it is assumed that the calendar owner wants to preserve its current participant status. Therefore, the participant status is not resetted. The second special case emerges when the attendee the status shall be resetted for is a resource or room. If the acting user that provokes the rescheduling has at least the permission to book the resource directly, and thus manages it in some way, it is assumed that change does not influence the booking status of the resource. Thus, the participant status of the resource is preserved.

References / further reading:

Reset of Change Exceptions

When an event series is updated, there are situations where all previously created change exceptions need to be reset, i.e. they get removed so that there are only regular occurrences again. In the legacy implementation, this has always been the case whenever the series pattern is changed (in any way), or the recurring master's start-/endtime are updated. Implicitly, this would also reset any participation status of the attendees, see above.

In the Chronos stack, a slightly enhanced approach is used so that not necessarily all previous change- and delete-exceptions are lost along with an updated recurrence rule. Now, whenever the recurrence rule is changed, the implementation checks if there are existing exceptions whose RECURRENCE-ID would still be matched by the recurrence set of the updated rule. If so, the change- or delete-exception is kept, otherwise it is still removed. In other words, if the original start time of an existing exception would still be a occurrence of the new recurrence rule, then it can be preserved. For example, any exceptions prior an updated UNTIL or COUNT part survive the update. Or, when changing a previously DAILY rule to only occur on working days, any previous exceptions on those days are kept.

Additionally, it is now possible to apply a changed recurrence rule to a specific and all future occurrences of the series, which is handled by splitting the event series into two parts - see below for more details.

References / further reading:

  • com.openexchange.calendar.api.CalendarCollection.detectTimeChange
  • com.openexchange.calendar.CalendarOperation.checkPatternChange
  • com.openexchange.calendar.CalendarMySQL.deleteAllRecurringExceptions

Smart update of Event Series

There are cases where changes to an event series should be performed for one recurrence instance on to the future. For example, when a new member is added for a regular team meeting, or, if the location of an event changes. In order to support these use cases, the server supports splitting event series into two parts, as well as propagating series changes to existing change exceptions.

Series Split

Splitting an existing event series into two parts is the basis to support use cases where changes to an event series should be applied from a specific occurrence on to the future. The logic to split event series is based on a preliminary specification of the Apple Calendar Server (see links below). Such a split is performed at a specific split point which is typically the recurrence identifier of the occurrence that is considered for the update.

The split is actually performed by creating a new, detached event series for the part prior the split point, and shifting the date of the first occurrence of the existing series to the split point. Existing overridden instances and any per-attendee data is preserved in both resulting event series, additionally, the series are linked via the RELATED-TO property.

Technically, the following steps are performed when an event series is splitted:

  1. A new series event representing the 'detached' part prior to the split time is created, based on the original series master
  2. The recurrence rule of the detached series is adjusted to have a fixed UNTIL one second or day prior the split point
  3. The recurrence rule, start- and end-date for the existing event series are modified to begin on or after the split point
  4. Existing delete and change exception dates are distributed between both series (prior / on or after the split time)
  5. Existing overridden occurrences are assigned to the new detached event series, if prior split time
  6. Both series are decorated with the same, newly generated RELATED-TO property

Afterwards, typically the new event series is adjusted in a second step, so that a client effectively can perform an update to an event series that should be applied to a specific and all future occurrences.

Propagate changes to Exceptions

As described above, when dealing with recurring events, it is often necessary to apply a change from one occurrence on into the future (e.g., add a new attendee part way through the series of meetings).

Previously, existing change exceptions are not touched at all when the series master event is modified. In the calendar implementation, we integrated some smart detection if a change to the recurring master event could also be applied to existing change exceptions, e.g. after a new attendee is added, or if the event location changes. As the web client does not have all change exceptions handy, the logic to 'propagate' the changes is implemented in the middleware.

To avoid possible ambiguities, only certain changes considered, where the change can also be applied in some or all change exceptions intuitively. Also, for group-scheduled events, change propagation only happens when updating the event as organizer. In particular, the following cases are taken into account:

  • For all simple changed event fields, it is checked if the modified property is equal in the original series master event and in the change exception. Simple event fields are (preliminary): CLASSIFICATION, TRANSP, SUMMARY, LOCATION, DESCRIPTION, CATEGORIES, COLOR, URL, GEO, TRANSP, STATUS. If not, leave the property in the change exception as-is (i.e. do not propagate this change). If yes, also apply the change in the change exception.
  • Newly added attendees are also added in existing change exception events, unless they're not already attending there.
  • Removed attendees are also removed from change exceptions, in case they previously attended there, too.
  • Changes within the series event's conferences are also reflected in overridden instances, unless they were modified compared to the master event
  • For changes to an event's start- and/or enddate, the same change is only propagated if both properties are equal to the original value in the change exception, i.e. the change exception's timeslot is still matching the recurrence.

Propagate Participation Status to Exceptions

Especially for event series with many attendees, chances are quite high that there are many change exceptions in the series after a while. Main reason is here that users change their participation status for individual occurrences to indicate that they cannot attend an event due to vacation or conflicting appointments at the same time. Or, they apply an individual reminder to an occurrence.

While these are all change exception events from a technical point of view, users do not see them as such, which previously caused some quirks when they want to change their participation status of the whole series, or get hints in the notification area that there are multiple events of the same series they need to reply to.

So, similarly as updates of the event data may get propagated, also an updated participation status of an attendee in the series master event may get applied to existing change exceptions under certain circumstances, which are:

  • The original participation status of a change exception must be equal to the original participation status of the series master event
  • The change exception must not be a real exception from the series, i.e. it must not be considered as re-scheduled compared to the original occurrence. In particular, the following event fields are considered to distinguish between real and synthetic change exceptions: SUMMARY, LOCATION, DESCRIPTION, ATTACHMENTS, GEO, ORGANIZER, START_DATE, END_DATE, TRANSP, RECURRENCE_RULE. On top, any change in the attendee line-up (in terms of added or removed participants) qualifies an overridden instance as real exception.

If these preconditions are met, the updated participation status in the series master event is taken over in overridden instances, too. In the participation status update comes along with a certain comment, this attendee comment is also updated in overridden instances, if its value has been equal to the series master event before, too.

Due to the fact that the participation status is also propagated to overriden instances, the notification area will only show those events the user really has to reply to and leave out those technical exceptions matching the circumstances mentioned above.

References / further reading:

External Calendar Users

At many places, one needs to differentiate between internal and external calendar users. Internal calendar users are all provisioned groupware entities within the same context, i.e. all resources, users and groups. All other calendar users are treated as external entities. This is especially also true for entities from other contexts of the same installation.

Internal calendar user without calendar access

Like explained above, normally, all non-guest users of one specific context belong to the group classified of internal users. However, there is one more restriction to that. An internal user of the same context will be handled as an external user if the user has no calendar access within the groupware. This can be the case when different access combinations are applied for different users of the same context. Like any other external calendar user, the internal user without calendar access will get scheduling mails via iMIP, instead of internal notification mails like the other internal users. This will enable the user to use a third party calendar agent, like Thunderbird. With the third party calendar agent, the user without calendar access can still manage calendar events and interact like any other external calendar user. This covers both cases in which the user is either the organizer of an event or is an attendee participating in an event.

Restrictions and Behavior

Once an internal user without calendar access is handled as an external calendar user in an event, this will be the case throughout the whole lifetime of the event. I.e. even if the user gains calendar access in the groupware later on, the user will still be handled as an external calendar user. The easiest case, which shows why there is this restriction, is the case in which the user is an external organizer. The organizer holds the master copy of an event. It has all information and is always the most current copy of the event. This copy however is within the third party calendar agent. The groupware has no access to this data. Now, when the user without calendar access gets the access, there simply is no data in the groupware of the event. Thus, incoming scheduling mails from attendees can't be processed within the groupware.

Also beware that the automatic processing of iMIP mails is still active. This means two things:

  • All incoming scheduling mails from the user without calendar access are automatically applied. The user is still within the global address book and therefore a known contact. For more details on the processing, please have a look here
  • Once the user gains back calendar access, iMIP mails in the INBOX will be processed. This can lead to scenarios in which events from iMIP mails are automatically inserted into the calendar. Please note that the user still is an external user then, even tough the event might be organized from someone within the same context

Scheduling

Currently, all external calendar users require a valid e-mail address that can be used for scheduling via iMIP. This is enforced by validating the URIs of external attendees prior saving. Attempting to save or update an event containing an external calendar user w/o valid e-mail address will fail; the check may be skipped when setting com.openexchange.chronos.skipExternalAttendeeURIChecks to false.

URI Encoding / International Domain Names (IDN)

E-Mail addresses of calendar users may contain non-ASCII characters, e.g. from international domain names like müller.com. When being used in a calendar user's URI along with the mailto: scheme, they are usually transferred as percent-encoded UTF-8. For example, for an attendee with e-mail address "horst@müller.com", the value of the corresponding ATTENDEE property would look like the following (while the CN parameter would still just use the UTF-8 string as-is):

ATTENDEE;CN="Horst Müller";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:horst@m%C3%BCller.com

Internally within the Chronos stack, all URIs are kept as (escaped) URI string, which would match the representation of the corresponding value in an iCalendar ATTENDEE or ORGANIZER property.

When being converted back to a plain e-mail address string (as used for external participants in the legacy stack), such URIs are decoded implicitly; additionally, ASCII-encoded (punycode) addresses with international domain names (IDN) are converted back to their unicode representation, too. This is the case when converting to an external participant in the legacy HTTP API, as well as when storing such an attendee in the dateExternal table.

References / further reading:

Provider/Account Framework

Besides the default, internal calendar, there may be further calendar sources that should be integrated into a user's calendar module. For example, calendar feeds from external sources, or a virtual calendar containing the upcoming birthdays found in the user's address book. Therefore, a new layer is introduced that provides access to all available calendar accounts from different providers of a user using the same API.

Calendar Providers and Accounts

A calendar provider implements the functionality to access the calendar data of a specific calendar source. Calendar access is always bound to a specific account of a user within the calendar provider, i.e. each calendar provider provides a number of accounts for different users. A user may have a fixed number of accounts (e.g. exactly one) within a concrete provider, or there can be multiple accounts for the user.

Have a look at the chapter 'Default implementations' to get an overview over providers shipped by Open-Xchange.

Discovery and Addressing

Due to the newly introduced calendar access layer, the discovery and addressing of calendar folders and their contents was extended accordingly.

HTTP API

For the folders module, a new content type has been introduced in order to provide backwards-compatibility on the one hand, and support for the newly introduced provider/account architecture on the other hand. Doing so, any existing requests to the API will return the same responses as before (i.e. list the same folders with the same identifiers). All "new" folders will be returned when requesting folders of the content type event, which includes both the previously used calendar groupware folders, as well as calendar folders representing external accounts. Differentiation is performed based on composite identifiers, which carry the information about the underlying calendar account.

Those calendar folders are now driven by a dedicated folder storage, and there's no folder hierarchy anymore, i.e. there are no subfolders. The calendar folder storage makes use of the the newly introduced calendar access layer, where the requests are routed to the actual calendar accounts. The account information is encoded within the composite folder identifiers, which are of the format cal://<account_id>/<relative_folder_id>?. The default groupware calendar provider has the reserved identifier 0, all further calendar accounts are generated with an unique identifier.

CalDAV

For calendar clients accessing the CalDAV interface, the composite folder identifiers will appear in the respective collection URIs in a base 64 representation. The formerly exposed numerical collection names are no longer advertised. Upon the first synchronization after the server has been upgraded, a typical CalDAV client will remove all previously synchronized collections, and add the new ones automatically. This usually works seamlessly, since clients typically start each synchronization cycle by querying all available calendars in the user's calendar home (by issuing a PROPFIND request with depth 1). The response includes all calendars visible for the user and their paths (and implicitly those that are no longer available), i.e. the client basically replaces the local list of collections with the ones of the response, then continues to synchronize the contents of each collection.

Attention: Depending on the usage and the configured interval for CalDAV synchronization, this may cause some additional load after an upgrade since existing clients will effectively re-synchronize the contents of those newly advertised calendar collections.

As one exception, the calendar client from Thunderbird/Lightning does not synchronize all calendars in the user's calendar home. Instead, it directly accesses a single calendar using the fixed path to that collection. As there is no automatic bootstrapping process for this client that directly synchronizes a single collection, there's a fallback handling in place so that the client can still access previously configured collections under the "old" path, i.e. using the non-qualified identifiers.

External Calendar Account Caching

The default calendar providers implemented by Open-Xchange (like ICal, ...) use a generic caching approach to avoid expensive calls to external resources as much as possible. This means that events (and all related information like attendees and description) from the external resource gets persisted like events from the internal calendar after their first retrieval. All upcoming requests will be answered by the cache until a defined refresh interval is exceeded. Please be aware that alarms and attachments are not synched.

The caching layer itself is able to handle information on a per-folder base. Cached folders that have been deleted on the remote site will be removed immediately from the cache while new folders on remote will be cached instantly.

In general the cache should be a blackbox for those implementations using it. Of course there are some utilities and hooks for the provider to be able to invalidate the folder cache as the cache doesn't know all possible cases.

Refreshing cache

Each Calendar Provider implementation can use its own refresh interval, again on a per-folder base. These value defines how long the cache should be used to answer requests for a dedicated folder of the underlying calendar. After the interval for the folder is exceeded the cache state will be updated by contacting the external calendar. New events will be added, removed ones will be deleted and changed events will be updated.

The refresh interval is defined in minutes. If the calendar provider implementation does provide a value bigger than 0 this will be taken into account. Otherwise the default of one day will be used.

In cases where new folders have been added or previously existing folders have been removed on the external calendar site the caching layer will directly update its internal state which means changes on the folder structure will be instantly visible without considering the refresh interval!

Error handling

The caching allows to handle data that will normally fail to be executed. Among other things the following error cases will be handled by the generic caching layer:

  • data truncation: external data that is too big to be stored will be truncated so that no import will fail.
  • corrupt data: data that won't met the requirements to be stored will be handled as smooth as possible. For instance if start or timestamp of an event is missing the complete event will be ignored.
  • missing UID: if an event lacks on appropriate UID definitions it is not possible to generate a reproducible diff for already persisted and externally available events. Due to this fact the caching will delete all persisted events for the folder and re-import those from the calendar provider.

As always the errors will be handled per folder and the underlying implementation has the opportunity to deal with it. In addition it can define an interval in which the external resource should not be requested after an error occurred. This prevents a continuous requests loop to the external provider which permanently will fail because of the same error. After the given interval time is exceeded the cache connects to the external provider again and hopefully the error is gone ...

Default implementations

As already mentioned Open-Xchange provides some default implementations for external accounts.

ICal Calendar Provider

Using the ICal Calendar Provider allows to subscribe to various remote calendar feeds that meet the requirements defined by the administrator (size, schemes, ...). The user just has to give an URI and optional credentials to subscribe the feed. To optimize requests to external resources the ICal Calendar Provider implementation is based on the generic caching layer described above. This means the calendar provider has to define its own refresh interval.

Refresh interval

In general the administrator is able to configure a refresh interval (by using the well known ConfigCascade) that will be used for each subscribed feed. As feeds might also define a refresh interval (by having 'REFRESH-INTERVAL' set) this value will be used under the assumption that the feed does even better know which interval makes sense.

Cache invalidation

As the caching layer isn't aware of changes that should result in a cache invalidation the implementation has to call back in such cases. For the ICal implementation some user actions have to trigger such an invalidation. These actions can be summarized as account reconfiguration which contains endpoint (URI or auth) changes that might result in a complete new ICal subscription with other events.

Configuration

The following configuration can be made by the administrator:

  • a maximal file size that is allowed for a feed: This ensures smooth processing of the feed by the Open-Xchange server and prevents undesired states. Each feed exceeding the limit will be denied. Per default the limit is set to 5MB.
  • a host blacklist: Due to security reasons the blacklist avoids connections to nodes of the entire network. Per default the blacklist contains 127.0.0.1-127.255.255.255 and localhost.
  • supported schemes: Due to security reasons (for instance avoid 'file') only a defined set of schemes should be supported. Per default these are http, https and webcal
  • various connection parameters: This contains timeouts and a maximum number of connections.

Have a look at the property documentation page on http://documentation.open-xchange.com for more details.

Google Calendar

The google calendar provider provides the possibility to subscribe to the primary calendar of a google account. The only requirement is a working google oauth account with the 'calendar_ro' scope. Please note that the google provider is a read only calendar provider and therefore doesn't work when the 'calendar' scope is used instead of 'calendar_ro'. To reduce the amount of data transported from google to the the middleware the google provider uses the incremental update feature of the google calendar. After the initial synchronization the google provider only requests the changes from the google server. This allows the use of a shorter refresh interval and therefore leads to more up-to-date data.

Please also note that old google subscription are migrated and removed afterwards. So they are lost in case of a downgrade.

Birthdays Calendar

The Birthdays Calendar is a calendar that automatically populates with the known birthdays of everyone in a user's address books. A corresponding calendar account is added by default for each user once that has the contacts module permission. However, this can be disabled beforehand or afterwards by removing the calendar_birthdays capability via config-cascade.

The calendar works by providing a yearly event series for each contact that has a set birthday property. The calendar data is always created on the fly dynamically by querying the contact storage, i.e. no additional event data is actually stored. However, it is still possible to add alarms for birthday events, which are then stored within the usual calendar storage.

Since the underlying calendar account is automatically provisioned, it cannot be deleted by the user - though the folder may still be unsubscribed to hide it from the user interface.

Group-scheduled Events

In iCalendar, there is a strict separation between simple events without further attendees in a user's calendar, and so-called group-scheduled events. Group-scheduled events are meetings with a defined organizer and one or more attendees, while not group-scheduled ones are published events, or events in a single user's calendar only.

Legacy: Implicit Participant

In the legacy implementation, appointments were always stored with at least the calendar user being a participant. This has the side effect of the actual parent folder information always being stored along with the participants, and not within the appointment itself (except for the fixed parent folder identifier for appointments in public folders). Doing so, lookups based on the parent folder could be performed by matching against the attendee's folders for private and shared folders.

Furthermore, since also simple, not group-scheduled appointments were stored with the calendar user as participant, the list of "all appointments of a certain user" could be built up in a very effective way. This also aided free/busy lookups and conflict checks, respectively.

However, based on RFC 5545, this is handling was wrong (the CalDAV layer already tried to work around this difference with some patches to remove this implicit attendee during export, and re-apply it during import again).

Chronos: No implicit Participant (possible)

While not necessarily needed, for now events are still stored using the calendar user as implicit participant, so that compatibility can be guaranteed for the legacy data structure and for existing clients. For CalDAV, the same workarounds as described above are still in place.

Eventually, once no backwards compatibility is needed anymore, this quirk will be removed, i.e. we'll no longer add the current calendar user as attendee and organizer implicitly in case no further attendees are defined.

Organizer and Attendee Copies

Group-scheduled meetings appear in the calendars of all attendees and the organizer. So, figuratively speaking, this means that multiple copies of the event are stored individually in the corresponding folders of the participants. The copies are updated based on certain scheduling rules that model the message flow between organizer and attendees - the iCalendar Transport-Independent Interoperability Protocol (iTIP). The concrete handling of organizer and attendee copies depends on if the organizer copy resides on the server, i.e. the event has been organized by an internal calendar user, or if an internal calendar user received a meeting request from an external calendar user.

In the first case, where an internal calendar user is the organizer, a single event instance - the master copy - is used and shared with all other internal attendees. Updates performed on this master copy are directly applied and visible for all other internal calendar users. If configured, notification messages about the changes are also sent to internal users. Externally invited attendees receive the updates via iTIP, usually via email (iMIP).

Group-scheduled meetings with an external organizer usually arrive via iMIP at the inbox of an internal recipient. When replying, the event data is taken over into their personal calendar folder, representing the so-called attendee copy of the event - the organizer's master copy resides on a foreign calendaring system in this scenario. The same handling also applies for events that arrive via iMIP, but contain an organizer whose email address refers to another internal user (e.g. when a calendar client is used by the organizer that is not connected to the server). In contrast to an internally organized event where the server has control over the master copy, the attendee copy is stored for each invited user individually in that case. So, even if multiple internal users from the same context attend in an externally organized event, each of them will have an own attendee copy, whose lifecycle is completely decoupled from other attendee copies of the same event. This is necessary, as meeting requests from external scheduling systems may be different for different recipients, so that a single copy cannot be maintained. For example, the organizer's calendar system might hide the attendees in the guest list from each other, or, when attendees are removed, the cancel message may only be sent to the deleted attendees, while no updated meeting request is forwarded to the other ones.

Technically, in such events where the organizer copy is not located on the server, only the attendee representing the actual calendar user gets resolved into an internal entity, while all other attendees as well as the organizer are still treated as external calendar users. So, even if one attendee copy changes locally (due to one user changing its participation status), this is not persisted in the other one as long as an update by the organizer is received. However, when the event data is read from one copy, the server will still try to lookup and inject data for other internal attendees from the copies found on their calendars if possible. Doing so, a scheduling reply performed by one internal attendee is immediately visible for peer attendees, too, without the need of receiving an updated iMIP request from the organizer first.

References / further reading:

Event Flags

For displaying events in the user interface, or to enable/disable appropriate actions for events, clients typically need to query multiple properties of events from the server. This includes the obvious properties like summary, start- or enddate, but also further properties to determine if the calendar user is the organizer of the event or an attendee, or information about the user's own participation status.

Previously, a client needed to fetch virtually all available properties when loading events from the server, and then performed multiple implicit checks dynamically, e.g. to decide whether the background is drawn "striped" or not, or if events can be "dragged" around in the grid. However, this caused some significant amount of data to be transferred between server and client, and also required some evaluations in the client which are rather in the calendar server's domain, especially towards the handling of different roles within group-scheduled events.

Therefore, a new, virtual read-only property for events was introduced: event flags.

Via this property, events will get decorated with different aspects that are relevant for the client, e.g. "has attachments", "is recurring", "has alarm(s)", "is organizer", and so on. In particular, the following event flags are supported:

  • attachment: The event contains at least one attachment.
  • conferences: The event contains at least one conference.
  • alarms: The calendar user has at least one alarm associated with the event.
  • scheduled: Event is a group-scheduled meeting with an organizer.
  • organizer: The calendar user is the organizer of the meeting.
  • organizer_on_behalf: The calendar user is the organizer of the meeting, and the current user acts on behalf of him.
  • attendee: The calendar user is attendee of the meeting.
  • attendee_on_behalf: The calendar user is attendee of the meeting, and the current user acts on behalf of him.
  • private: Event is classified private, so is invisible for others.
  • confidential: Event is classified as confidential, so only start and end time are visible for others.
  • transparent: Event is transparent for the calendar user, i.e. invisible to free/busy time searches.
  • event_tentative: Indicates that the event's overall status is tentative.
  • event_confirmed: Indicates that the event's overall status is definite.
  • event_cancelled: Indicates that the event's overall status is canceled.
  • needs_action: The calendar user's participation status is needs action.
  • accepted: The calendar user's participation status is accepted.
  • declined: The calendar user's participation status is declined.
  • tentative: The calendar user's participation status is tentative.
  • delegated: The calendar user's participation status is delegated.
  • series: The event represents the master of a recurring event series, or an expanded (regular) occurrence of a series.
  • overridden: The event represents an exception / overridden instance of a recurring event series.
  • first_occurrence: The event represents the first occurrence of a recurring event series.
  • last_occurrence: The event represents the last occurrence of a recurring event series.
  • all_others_declined: All other individual attendees in the event have a participations status of declined.

References / further reading:

  • com.openexchange.chronos.EventFlag
  • com.openexchange.chronos.common.CalendarUtils#getFlags

Last-Modified / Created

In the legacy stack, the properties that store the creation- and last-modification time of an event were handled by the server implicitly each time the entry was written in the database, so that they're actually read-only for clients. Similarly, the entity identifiers of the acting user were stored along as created- and modified-by. Additionally, the last-modified property was also used to prevent so-called lost updates, in a way that clients were only allowed to update or delete an event where the last-modified timestamp matches the one of the version last read by this client. Also, the last-modified property was used to compute resource ETags and to drive the sync-collection report in CalDAV.

While letting the server assign those timestamps based on the current time automatically is feasible for events being initially created on that system, it is problematic whenever data is imported from an external source - e.g. a subscribed external calendar feed, or a manually imported iCalendar file. Also, the created- and modified-by properties would be rather misleading in such scenarios. Therefore, the new calendar implementation will handle those properties in the following way:

  • The properties created-by and modified-by are no longer mandatory (i.e. may be NULLed if not applicable)
  • If set, the properties created-by and modified-by still refer to internal calendar users
  • The properties last-modified and created are no longer mandatory (i.e. may be NULLed if not applicable)
  • If set, the properties created and last-modified do not necessarily store the server timestamp the event was (last) written, and may contain values from an external source
  • For synchronization purposes, an additional synthetic property timestamp is introduced, which is set to the current server timestamp whenever the event is written

For backwards compatibility, the new timestamp property is used to drive the last-modified property in the legacy HTTP API. Additionally, due to the NOT NULL column definitions, the respective values in the legacy storage are derived from the acting calendar user and the timestamp property as needed.

References / further reading:

Allowed Attendee Changes

Previously, only the user's permissions in the underlying folder were considered when checking if an event can be updated or not. For meetings with multiple attendees that usually appear in each of the attendee's default calendar folders, this meant that every internal user that attends a meeting was able to edit or delete the event.

However, iCalendar standards require to consider different roles here - mainly depending on the calendar user being the organizer of an event or not. While the organizer can perform any changes to the event, the other attendees are quite limited regarding the allowed changes (see RFC 6638, section 3.2.2.1):

"Attendees" are allowed to make some changes to a scheduling object resource, though key properties such as start time, end time, location, and summary are typically under the control of the "Organizer".

In order to comply with the standards, the new calendaring stack introduces appropriate restrictions (per default) in case the user is not the organizer, or is not acting on behalf of him. Effectively, the permitted changes then boil down to modifications of the user's personal alarms and his own participation status. Additionally, the attendee is still allowed to remove himself from an event (beyond declining it). Those changes can also be performed on a single instance of a recurring event series (which may indirectly cause new change and/or delete exceptions for the series).

When acting on behalf of another user in a shared calendar folder, always this shared folder's owner is considered when determining if the event is updated as organizer or attendee. In public folders, the original creator is stored as the organizer implicitly. All consecutive changes by other internal users can then be performed on behalf of this organizer, provided that the underlying permissions in the folder are sufficient. See also chapter Permissions below for further details.

Attendee Privileges

As stated above, attendees are usually quite limited regarding the allowed changes. To still allow modifications by other calendar users for specific events, the organizer of an event can assign elevated attendee privileges on a per-event basis. Doing so, he allows other attendees to modify event properties that would normally require the organizer role. Those modifications are then performed on behalf of the organizer implicitly. This on behalf-relationship will be used throughout the stack, e.g. also when sending subsequent scheduling messages like iMIP. However, regardless of the granted attendee privileges, the restrictions as per RFC 6638, section 3.2.2.1 still apply when accessing the server via CalDAV.

Delete as Attendee

Previously, when an internal user attendee performed a delete operation on an event he's been invited to, he was removed from the attendee list in the storage. While this is the way the legacy calendar implementation handled it (and the way our users are used to), it is de-facto a not allowed attendee change, as the collection of attendees can only be altered by the event's organizer. As such, it would also be wrong to propagate such a change to an external organizer via iMIP, as it would rather be a COUNTER and no DECLINE. Same goes for possibly internal notification messages for an updated event that would be sent out superfluously.

To work around this, the delete operation performed as attendee in a group-scheduled event needs to be treated pretty much as setting the attendee's participation status to "declined". Additionally, in case an attendee "delete-declines" by EXDATE'ing an instance, this is also handled by setting the participation status accordingly.

Internally, whenever the request to delete an event from an attendee's point of view is processed, the attendee will effectively not be removed from the attendee list, but his participation status is set to "declined". From the iTIP perspective, this will also be indicated as CANCEL to the organizer. Whenever the event is re-scheduled (e.g. the start time is adjusted), the attendee's participation status is reset to "needs-action" again, even if the attendee "deleted" the event previously from his point of view.

Therefore, on the storage layer, a new hidden column is introduced for attendees in group scheduled events, that indicates that such an event appears deleted from this attendee's point of view. The API remains unchanged, i.e. all existing delete operations will be adjusted to set this hidden flag instead of removing the attendee. To aid synchronization, we will still insert a tombstone record for the attendee whenever the "hidden" flag is set, so that such an event is also indicated with status 404 on sync-collection responses in CalDAV.

HTTP API

In case a client attempts to modify a group-scheduled event in a not allowed way, an appropriate exception is thrown (code CAL-4038). To aid the differentiation between attendee- and organizer-scheduling object resources, the event's flags contain additional hints (flags scheduled, attendee and organizer) which can be evaluated appropriately, in addition to the permissions of the folder the events is located in.

Additionally, to aid the typical reply of an attendee to a meeting request, a new, dedicated action updateAttendee has been introduced that allows to adjust the calendar user's own attendee property, as well as to apply his set of alarms for the event.

CalDAV

In case a client attempts to modify a group-scheduled event using the CalDAV interface in a not allowed way, the request is answered with the CALDAV:allowed-attendee-scheduling-object-change precondition error. Besides the per-user properties defined in RFC 6638, there are no further exceptions. However, if a client attempts to store a non-standard X-property in the iCalendar resource, no error is thrown and the extended property is dropped silently (as clients actually do it, e.g. a custom X-APPLE-TRAVEL-ADVISORY-BEHAVIOR or X-LIC-ERROR).

References / further reading:

Permissions

When interacting with the calendar and scheduling subsystem, three different permission concepts need to be considered. On the one hand, the user's effective permissions in the folder (that represents the actual view on the event data) is taken into account, just like one is used to from other groupware modules. On the other hand, the user's role within a certain group-scheduled event affects the possible actions, i.e. if a user is also an attendee or the organizer of the event. The latter aspect basically introduces an additional layer of object permissions for events. On top of that, the classification of an event may be used to restrict access to certain properties of an event for other users.

Permissions by Role

As stated above, within group-scheduled events with multiple attendees, a calendar user can have certain roles - i.e. the user is organizer of the event, or he is an attendee. For events that are located in personal calendar folders (shared or private), these are actually the only possibilities. In public calendar folders, the user might also be neither organizer, nor attendee.

Considering the role, a user that is attendee or organizer of a group-scheduled event is always able to read event data. Also, an attendee is always allowed to modify his own attendee property (especially his participation status), as well as he has control over his own alarms. Additionally, he may still delete himself from the attendee list. However, whenever event data should be manipulated in way beyond the allowed attendee changes (see above), the calendar user needs to be the organizer of the event. Note that the role is always interpreted for the actual calendar user based on the folder that represents the current view on an event, which means that this calendar user may be different from the current session user in case of a shared calendar folder, while the calendar user equals the current session user in private folders. So, when interacting with an event under the perspective of a shared folder, the current user always acts on behalf of the folder owner - even if he also attends or is the organizer of the event.

The access rights that are implicitly given by the user's role in an event are always valid, even if the event is statically located in a (public) calendar folder that is not visible for the user. For example, the API allows to create an event in a public calendar folder and invite attendees that do not have access to this folder. Interaction with such events would still be possible (e.g. by using the corresponding links from the invitation mail, by looking up the events via search, or by querying virtual event collections for the user ("all my events"). Note that the role is always interpreted for the actual calendar user based on the folder that represents the current view on an event, which means that this calendar user may be different from the current session user in case of a shared calendar folder, while the calendar user equals the current session user in private and public folders. See chapter "Relation of Organizer / Principal / Folder-Owner / Creator" for further details.

In contrast to personal calendar folders, public folders are not associated with a dedicated calendar owner. So, whenever a new event is created within a public folder, the creator automatically becomes the organizer of this event. Each consecutive modification of the event data by other users beyond the allowed attendee changes is then implicitly performed on behalf of the organizer. That means that all other users can edit events in public folders by implicitly impersonating as organizer of the event, given that the underlying folder access rights are sufficient. However, it is not possible to act on behalf of another attendee in public folders, so that the current session user is always interacting with his "own" attendee property here, i.e. he cannot change the participation status of another attendee.

Folder Permissions

For calendar folders, the same set of permissions can be applied as for folders in other groupware modules, i.e. users and groups can be added to the permission lists; each entity with an individual set of folder-/read-/write- and delete-permission. The effective folder permission of a specific user is the calculated maximum permission (based on the users own and/or any group entity permissions that are defined).

In the calendar module, the folder permissions are primarily considered for create operations, since modifications and deletions of group-scheduled events are rather restricted based on the calendar user's particular role in the event once it has been created, see above. Additionally, in shared folders, where a user acts on behalf the calendar owner, the folder permissions determine which actions are allowed for the proxy user (besides of the restrictions based on the calendar user's role).

Permissions by Classification

The classification may be used to restrict access to the event as such, or to specific properties of an event, to other users that do not participate in the event as attendee. So, even if an event would be visible within a shared folder of the calendar user for other users, depending on the event's classification, it would be either invisible (for a PRIVATE classification), or only the event's period would appear (for a CONFIDENTIAL classification).

More details about the possible event classifications are described in chapter "Classification / Private flag" above.

HTTP API

Clients that need to be aware of which actions are actually possible by a certain calendar user for a group-scheduled event in a specific folder should consider the appropriate event flags scheduled, organizer and attendee, along with the effective permissions of the current session user in the underlying folder. Then, the following rules apply:

  • New events can be created in case the folder permissions include at least create own objects.
  • Whenever data should be manipulated in a way beyond the allowed attendee changes (see above), the calendar user needs to be the organizer of the event, hence the organizer flag would have to be present. Additionally, the folder permissions need to allow write object permissions for the event in question (i.e. either write all or write own objects).
  • Whenever attendee-related data such as the participation status or the user's personal alarms should be modified, the calendar user needs to be an attendee of the event, hence the attendee flag would have to be present. No additional folder permissions are required.

UTF-8 Support

The new calendaring stack uses new database tables to store event data. Data from existing events is migrated during the upgrade, see chapter "Migration of legacy data" above. The same database tables are also used for caching data of external calendar subscriptions.

The newly introduced database tables come with full support for storing all symbols from the Unicode character set, whose code points range from U+000000 to U+10FFFF. However, this may also require to adjust some MySQL configuration.

UTF-8 in MySQL

UTF-8 is a variable-width encoding that encodes each symbol using one to four bytes, i.e. commonly used symbols with a lower code point (ASCII) are encoded using fewer bytes, while it is still possible to store BMP (U+000000 to U+00FFFF) and astral symbols (U+010000 to U+10FFFF).

The default utf8 character set of MySQL uses a maximum of three bytes per character, so it only allows unicode characters in the BMP range, while any supplementary astral symbols cannot be stored at all. Attempting to do so results in an incorrect string warning, which is treated as error by the middleware. Whenever 4-byte characters have to be stored, the extended character set utf8mb4 is required.

JDBC Connection Character Set

All strings sent from the Java middleware using the JDBC driver to the database are converted automatically from native Java Unicode to the client encoding. The encoding between client (Connector/J) and MySQL server is automatically detected upon connection. So, once the server advertises support for utf8mb4, this will be negotiated automatically, hence there is no need to modify the driver settings (as defined in configdb.properties).

So, in order to finally use 4-byte UTF-8 character sets, the MySQL server should be configured with character_set_server=utf8mb4. Otherwise, the allowed characters may still be restricted to the BMP range (utf8 with three bytes per character), and

Replaying Storage

As noted above in "Migration of legacy data", the database will work temporarily in a mode where all write operations are 'replayed' to the legacy database tables after the migration is finished, to prevent data loss in case a downgrade should ever be required. As the previously used database tables do still not support astral symbols, those problematic characters are removed implicitly when storing them there.

References / further reading:

Import and Export

This chapter will describe the impact on calendar data during an import or an export of an calendar.

Import

During the import all information about attendees and the organizer will be ignored. Instead the acting user will be set as organizer and only attendee of the imported events. The following case lead to this decision.

Lets consider the case that the event is organized by another entity than the acting user importing the event. Furthermore lets assume the user importing isn't attending the event (normally this is handled via iTIP or other internal mechanisms). Based on our current concept of "personal" calendar folders of internal user (see Group-scheduled Events) we aren't able to save the event without adding the current user as attendee. This means that the user would always become a 'party-crasher'. To avoid this situation the only feasible way is to replace the organizer. Further to ensure data consistency and avoid struggles like scheduling mails for other attendees in the imported event, all existing attendees are removed as well. This is done to void the impression that the event was taken over by a new organizer.

Import of orphaned event instances

Orphaned event instances are event exceptions belonging to a series, we don't have further information about. The master event is unknown and only data for the exception event is available. There can be multiple orphaned instances of the same series within a single iCAL file. Due to the decision to replace the organizer on event imports, orphaned instances can't be imported as-are. In the moment we change the organizer of the event to import, it is claim that the event is organized and managed by the calendar system doing the import. This means, the calendar system must known everything about the event, efficiently requiring the knowledge about the series master of an orphaned event instances. Since we don't have this knowledge, we can't import the orphaned instances as a part of an series. Therefore, recurrence information are removed from the orphaned event instances, transforming them into independent single event instances. Thus, for more than one imported event instance ,new unique identifier are required and will be reassigned as needed.

Export

The export will touch so called 'pseudo group scheduled' events. Events that are only attended by the organizer herself are considered 'pseudo group scheduled'. Those events will be exported without the organizer and the attendee information as those are only persisted to legacy reasons (see Group-scheduled Events). When exporting events the event flags as per Event Flags will be ignored, as they represent internal state. Besides this, alarms will be removed on exported events. The motivation behind this is that alarms are bound to a user and not to the exported event. Additionally some alarms that will trigger mails which might be imported as-is on other systems, generating unwanted mails.

Resources

In the groupware, resources are used to represent rooms or other inventory like conference equipment, cars, or similar. For appointments, resources can be booked / reserved as attendees. Resource attendees are also considered in conflict and free/busy checks, effectively preventing an "over-booking" of resources.

Resource Calendars

In order to check if a specific resource is available or not, users can use the "scheduling view" (which uses the freeBusy action to determine occupied slots). Sometimes however, it is also helpful to have an overview about all resource bookings in a separate calendar, which can be enabled within the currently shown calendar views.

Therefore, all events of a resource are exposed as distinct, virtual calendar, which can be used through the API just like other calendars. These identifiers of the virtual resource calendars follow the pattern cal://0/resource<resource_id>, and don't get advertised through the folders module, or for external clients via CalDAV.

With its virtual folder identifier, they can be addressed in the typical all requests to retrieve the appointments where the associated resource attendee is contained in, as well as an inclusive folder within the search requests. The returned event data will replay the virtual resource folder ids, via which they can then be accessed in get and list requests by clients.

Depending on the requesting user's access rights to other folders and his participation in events, either full details of the appointments in resource calendars is visible, or only an anonymized version where only the time periods the resource is booked are exposed, similar as in the free/busy view. In case a user acts as booking delegate for a resource, i.e. he is privileged to accept or decline incoming booking requests, event details are always available (including events with a classification of CONFIDENTIAL), since appointment data is also included in the scheduling notification mails.

Scheduling Privileges

For each resource, different scheduling privileges can be defined for users and groups. Doing so, it is possible to restrict if and how a resource can be booked, i.e. how it can be used as attendee in an appointment. In particular, the following privileges can be granted:

  • none - Not allowed to book the resource
  • ask_to_book - May submit a request to book the resource if it is available
  • book_directly - May book the resource directly if it is available
  • delegate - Act as delegate of the resource and manage bookings

These privileges can be assigned to users or groups in the context for each resource using the resource management interface in App Suite, which becomes available when the editresource module permission is set. If no permissions are defined, an implicit fallback to book_directly applies for all users of the context, which matches the legacy behavior of App Suite ("unmanaged resources"). Each privilege implies the "smaller" privileges implicitly - especially booking delegates are always able to book a resource directly.

As soon as an ask_to_book privilege is granted to an entity, at lease one delegate privilege must be assigned to a user or a group of users, so that booking incoming requests that require a confirmation can be processed.

Please also note that scheduling privileges and calendar permissions are different and independent concepts. This becomes relevant for the case that a calendar user is working on behalf of another calendar user, the so called secretary functionality. In that case, the scheduling privileges of the acting user (the secretary) will be considered and not the ones of the calendar owner, which is the user the secretary is acting on behalf of. E.g., User A (the secretary) has full access to the calendar "meetings" of user B. User A has scheduling privileges of none for resource C, user B has scheduling privileges book_directly for resource C. When user A creates an event with resource C in the calendar "meetings", the operation will fail with an error. User A hasn't enough permissions to book the resource. If now user B creates an event in "meetings" with resource C, user B will succeed. User B has enough permission to perform the operation with resource C.

Scheduling Workflows

By default, all resources in the groupware can be used in meetings by all users directly. If the resource is not yet booked for another appointment at the same time, it can be added and the resource attendee's participation status is directly set to ACCEPTED. Otherwise, a hard conflict is raised, preventing an over-booking of the resource and the appointment cannot be saved.

With the introduction of booking delegates, i.e. users which are managing resources and confirm or reject booking requests, an indirect scheduling workflow is introduced. Whenever a user who has the ask_to_book scheduling privilege adds a resource attendee to an appointment, a corresponding booking request is implicitly sent to the booking delegates of the resource. They receive the request via email and can either accept or decline the request. Until the request is processed by a delegate, the participation status of the resource remains at NEEDS-ACTION.

The switch between both resource scheduling modes happens implicitly, that means that it is always possible to go back to the legacy / direct booking workflow by just removing all previously assigned scheduling privileges. Then, a default of book_directly is assumed again for all users in the context.