Plans for AI Usage

As AI usage relies on paid APIs, providers may want to charge their users for AI usage. The service supports a flexible way to apply different commercial models. This allows to:

  • set individual request and/or token limits per model
  • limits can defined be per month, per week, per day
  • optionally define a time-based limit to support trial phases
  • define different plans with different limits for user segments based on capabilities/permissions
  • trigger upsell processes to higher plans
  • in future this will also support defining different feature sets per plan

Limits are implemented by counting API requests per user. When this is enabled, the accounting feature ensures that all requests done by a user are counted in a database. This actual number is checked against the defined limits for each request. If the usage limit is reached, no additional requests can be made for the current time range.

For instance, a "free" plan might allow 10 requests per day, while a paid plan allows 1000 requests per month.

Defining plans

A plan is an object with the following properties:

  • id: Defaults to capability
  • capability: The user capability/permision to assign a plan to a user segment. Must be unique in the list of plans.
  • name (optional): The name of the plan (just for readability)
  • limit: Optional shorthand to define one request limit per day for all models.
  • limits: Allows defining limits per modelId (* can be used as a wildcard Id for all (other) models in a plan)
    • interval: month, week, day.
    • requests: The maximum requests per user per interval. -1 means no limit.
    • tokens: The maximum tokens per user per interval. -1 means no limit.
    • days: An optional time-based limit in days since first usage. -1 means no limit.
    • account (optional): The usage of different models (of same interval) can be grouped. Just set the same string. Defaults to modelId (to keep things separate as default).
  • selectableModels (Optional): Array of strings to define which models can be used for users of this plan. Defaults to selectableModels in models configuration.
  • upgrade (optional): This plan has an upgrade path via upsell (true / false; default: false)

Understanding Plan Limits: days, interval, and requests

There are two types of limits that can be used to control access: request-based and time-based.

Interval & requests (request-based; always used together)

These two define how many AI-related requests a user can make within a certain time window.

  • For example: requests: 25 and interval: day means a user can make 25 requests per day.
  • Likewise, requests: 140 and interval: week means a user can make 140 requests per week. This does not set any limit per day. Requests will be counted over the entire week.
  • The counter resets based on the interval:
    • Day → resets every midnight
    • Week → resets every Monday
    • Month → resets on the 1st of each month

Days (optional time-based limit)

  • This sets a one-time countdown that starts from the user’s first AI usage.
  • After the configured number of days has passed, access to AI features is blocked — even if they haven’t hit their request limit.
  • Think of it like a trial period.

Summary

  • Use interval & requests to control ongoing usage.
  • Use days to enforce time-limited access, like a trial.
  • These limits are independent and can be used together or separately, depending on your needs.

No limits required: No need to define plans at all.

One limit for everything

# it is possible to adjust the internal fallback plan
# (applied if no other plans match the current user)
plans:
  - id: fallback
    limit: 25

A premium plan with upsell

plans:
  - id: fallback
    limit: 10
    upgrade: true
  - capability: premium_users
    limits:
      openai:
        interval: 'day'
        limit: 25
      localLLM:
        interval: 'day'
        limit: 50

Further examples

plans:
  - id: premium
    capability: ug_premium
    name: AI Premium
    interval: month
    # shorthand to set limit for all models
    limit: 500
  - id: trial
    capability: TRIAL
    name: AI Trial
    limits:
      # wildcard for all models
      '*':
        days: 30
        interval: day
        requests: 20
  - id: specialTrial
    capability: SPECIAL_TRIAL
    name: AI Sneak Peek Week
    limits:
      default:
        days: 7
        interval: day
        requests: 15
      private:
        days: 7
        interval: day
        requests: 30
        tokens: 60000
    selectableModels:
      - default
      - private
    upgrade: true

Implementation details

Activation

To activate the Accounting feature in a Kubernetes environment, modify the configuration file by setting the accounting.enabled value to true in your values file.

When enabled, it monitors user activity and applies limits according to the user's capabilities and associated plan.

Functionality

Once the Accounting feature is activated, the service will inspect each user's JSON Web Token (JWT) for specific "capabilities" that match a configured "plan."

How It Works

  • JWT Inspection: When a user makes a request, the service inspects the user's JWT for any "capabilities" that correspond to a configured plan.
  • Plan Matching: If a capability matches a predefined plan, the service begins tracking the user's usage according to the rules of that plan.
  • Usage Tracking: The service tracks the number of requests made by the user as specified by the corresponding plan. For example, a plan may allow 100 requests per month.
  • Usage Limits: If the user reaches the usage limit (e.g., 100 requests per month), the service responds with a status code 402 Payment Required to indicate that the limit has been achieved.
  • Default Plan: If no capability is found that matches a configured plan, the user is automatically assigned a default plan. The default plan allows up to 10,000 requests per month.

Response Codes

  • 200 OK: Request was successful, and usage is within its limits. The responses body will contain the property remainingRequests with the number of remaining requests.
  • 402 Payment Required: The user has reached the usage limit for their plan.

Routes

User Routes

The service provides a route for users to check their usage:

  • GET /api/usage: Get the usage data for the current user for all models. The responses body will contain the array remainingRequests of objects containing the servicename as key and the number of remaning requests as value.
  • GET /api/usage/:model: Get the usage data for the current user for the specific :model. The responses body will contain the property remainingRequests with the number of remaining requests.

Admin routes

There are several routes available for administrators to inspect and reset usage data:

  • /api/admin/usage/history?[brand={brand}][&month={month}]: Get usage statistics over time. brand is the brand name, month the month, formatted as YYYY-MM. Both paramters are optional. Example /api/admin/usage/history?brand=brand1&2025-04

  • GET /api/admin/usage/:cid/:plan: Get the usage for a specific user and plan. :cid is the composite user ID which is a combination of the context ID and the numeric user ID. For example, :cid could be 1-123. :plan is the id of the plan (e.g., default).

  • DELETE /api/admin/usage/:cid/:plan/:model: Reset the usage data for a specific user, plan, and model. :plan is the id of the plan, :cid is the composite user ID, and :model is the model id.

To secure these routes, there are 3 options available for authentication:

  1. Basic Auth: Use basic authentication with the basicAuth.user and basicAuth.password values in the configuration file. If you want to use the charts secret file you have to set basicAuth.enabled to true. Otherwise, you can provide your own secret with basicAuth.enabled set to false and overrides.basicAuth set to the secret name. The provided secret should contain the keys basic_auth_user and basic_auth_password. In this case you don't need to set basicAuth.user and basicAuth.password. The request header should contain the Authorization header with the value Basic base64(username:password).

  2. Bearer Auth: Use bearer authentication with the bearerAuth.keys value in the configuration file. The bearerAuth.keys value should be a list of keys that are allowed to access the admin routes seperated by a comma. If you want to use the charts secret file you have to set bearerAuth.enabled to true. Otherwise, you can provide your own secret with bearerAuth.enabled set to false and overrides.bearerAuth set to the secret name. The provided secret should contain the key bearer_auth_keys. In this case you don't need to set bearerAuth.keys. The request header should contain the Authorization header with the value Bearer <key>.

  3. OAuth2: Given that you have an OAuth2 server running and this server supports openid connect and JWKS, you can use the oauth2.domain configuration to secure the admin routes. Turn this on via oauth2.enabled set to true. The service will then discover the JWKS from the given domain and validate the JWTs with it. The request header should contain the Authorization header with the value Bearer <JWT>.

Info

Take note that you can configure multiple authentication methods at the same time. The service will try to authenticate the request with the first method and if it fails, it will try the next one. If no auth is enabled (default), no admin route is exposed at all.

Capabilities and plans

How capabilities are matched to plans

The service will look for the capabilities property in the JWT. If capabilities are found, the service will look for a capability that matches the ID of a plan (premium, trial, or specialTrial, as in the example above). If a plan is found, the service will use the plan to track the user's usage. If no plan is found, the service will use the default plan (10,000 requests per month). If the user has more than one capability that matches a plan, the service will use the first one found and log a warning.

How to set the capabilities in the JWT

The capabilities are set in the JWT by the Switchboard. The Switchboard will check for all App Suite capabilities of a user, then apply a filter so that only relevant capabilities are added to the JWT. This means you need to set the desired capability, for example, specialTrial, on the App Suite side for your user. This will most likely be handled by your upsell code, your administrators via the SOAP API, or the config cascade.

Once the user has the correct capability, the Switchboard needs to be configured to add the capability to the JWT. This is done by adding your capability to the Switchboard’s jwt.serviceCapabilities value, which by default is set to "switchboard,openai,zoom,jitsi" (be sure to add your capability rather than overwriting the defaults, e.g., "switchboard,openai,zoom,jitsi,specialTrial"). See the Switchboard documentation. This list acts like an allowlist for the user's capabilities. If the user has the capability, it will be added to the JWT.

Steps to Configure the AI Service

  1. Fill out Database Configuration

    Add the following configuration to enable the database for the AI service:

     database:
   host: <your mysql host>
   name: <your db name>
   user: <your mysql user omit if you provide your own secret>
   password: <password of your mysql user omit if you provide your own secret>

  2. Optional: Custom Database Secret

    If you want to use your own secret for the database, set the mysqlSecret.enabled to false and provide the secret name in overrides.mysqlSecret.

  3. Prepare the Database

    The service expects an existing database to connect to and run initial migrations. Follow the steps below to set up the database correctly.

Database Schema Setup

To ensure the AI service operates correctly, you need to create the necessary tables and procedures in your database. Although the actual migration script will be executed automatically, it is essential to ensure that the database user has the appropriate permissions.

Minimal Required Grants

The minimal grants needed for the database user are:

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, TRIGGER
ON `your_database_name`.*
TO 'your_db_user'@'%';

Replace your_database_name with the name of your database and your_db_user with the name of your database user.

Example Configuration

Here's an example of how to grant the necessary permissions:

CREATE USER 'ai_service_user'@'%' IDENTIFIED BY 'secure_password';

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, TRIGGER
ON `ai_service_db`.*
TO 'ai_service_user'@'%';

FLUSH PRIVILEGES;

In this example:

  • ai_service_user is the database user for the AI service.
  • secure_password is the password for the database user.
  • ai_service_db is the database name for the AI service.
Last Updated::