AI-Service documentation
The OX AI-Service integrates 3rd party AIs in App Suite UI, currently ChatGPT from OpenAI.
Architecture
The service offers the API for different AI actions and a micro frontend, actually the App Suite UI integration in form of an UI plugin. The UI Middleware loads the plugin code from the service for App Suite UI. There is no dedicated UI source container needed.
Configuration
As the AI Service delivers the App Suite UI plugin, it needs to be added to the baseUrls
on the UI Middleware.
Add the AI Service url (K8s url) to the list of baseUrls
of the UI Middleware configuration. This is needed to ensure that the AI integration's code is send to the user's browser.
Example config map
baseUrls:
- http://core-ui-source-pod/
- http://ai-service-pod/
- http://some-other-ui-pod/
Further reading: UI middleware Config map
Capability
Important: Starting with the AI-Service 2.0 and Core UI 8.28 the needed capability was renamed.
Capability: ai-service
(was open-ai
)
Feature toggle
Important: Starting with the AI-Service 2.0 and Core UI 8.28 the feature toggle was renamed.
Feature toggle: ai
(was open-ai
)
Requirements
A running Switchboard is needed to run the AI Service, otherwise the plugin won't even start
How to enable for users
By default, AI integration is disabled for all users. To enable the feature for users, add the following:
- Add capability
ai-service
which indicates a running AI-Service in your deployment - Enable the feature toggle for users with core ui setting
io.ox/core//features/ai
totrue
Upsell configuration
AI integration supports the App Suite UI upsell functionality. To enable upsell for the AI features, set the following frontend configuration:
io.ox/core//upsell/activated = true
io.ox/core//upsell/enabled/ai-service = true
io.ox/core//features/ai = true
Remove the capability ai-service
for all users that should see the upsell triggers. When enabled, all AI buttons and dropdowns are shown in the UI but will trigger an upsell event on the ox
object.
Models
The models
parameter defines a list of AI models that are supported by the deployment. Each model has specific attributes, such as name
, maxTokens
, and an optional default
flag.
name
: The name of the AI model.maxTokens
(optional): The maximum number of tokens that the model can handle.default
(optional): A boolean flag that indicates whether this model is the default model. Only one model can be marked as the default.
example for models:
models:
- name: "gpt-4o"
maxTokens: 128000
- name: "gpt-4"
maxTokens: 8192
default: true
- name: "gpt-3.5-turbo"
maxTokens: 16385
enforceDefaultModel
The enforceDefaultModel
value determines if the default model is used for all requests.
If set to false
, the client configuration will specify which model to use for each request.
To set the model for the client use the following core ui configuration:
io.ox/ai/openai/model = "<model-name>"
Accounting Feature
As AI usage relies on paid APIs, providers may want to charge their users for AI usage. The service supports a flexible way to apply different commercial models. This is done by counting API requests per user. When enabled, the Accounting feature ensures that all requests done by a user are counted in a database. This number is checked against a so called "plan" which is assigned to the user. If the AI quota is reached, no additional request can be made for the current time range.
For instance, a "free" plan might allow 10 requests per day or month, while a paid plan allows 1000 requests per month. The user and brand limits can be configured via config map on the Helm chart. A plan is an object with its key as the id
for the plan and has the following properties:
name
: The name of the planplan
: The type of the plan (free, paid)strategy
: The strategy for the plan (duration, monthly)duration
: The duration if strategy is "duration" (format is30d
for 30 days)limit
: The limit of requests per user and brandenabled
: The plan is enabled or not
Activation
To activate the Accounting feature in a Kubernetes environment, modify the configuration file by setting the accounting.enabled
value to true
in your values file.
When enabled, it monitors user activity and enforces quotas according to the user's capabilities and associated plan.
Functionality
Once the Accounting feature is activated, the service will inspect each user's JSON Web Token (JWT) for specific "capabilities" that match a configured "plan."
How It Works
JWT Inspection: When a user makes a request, the service inspects the user's JWT for any "capabilities" that correspond to a configured plan.
Plan Matching: If a capability matches a predefined plan, the service begins tracking the user's usage according to the rules of that plan.
Usage Tracking: The service monitors the number of requests made by the user as specified by the plan's quota. For example, a plan may allow 100 requests per month.
Quota Enforcement: If the user reaches the quota limit (e.g., 100 requests per month), the service responds with a status code 402 Payment Required to indicate that the quota has been exhausted.
Default Plan: If no capability is found that matches a configured plan, the user is automatically assigned a default plan. The default plan allows up to 10,000 requests per month.
Response Codes
200 OK: Request was successful, and usage is within quota limits.
402 Payment Required: The user has reached the quota limit for their plan.
Example plan configuration:
plans:
- premium:
name: "AI Premium"
plan: "paid"
strategy: "monthly"
limit: 100
enabled: true
- trial:
name: "AI Trial"
plan: "free"
strategy: "duration"
duration: "30d"
limit: 25
enabled: true
- specialTrial:
name: "AI Sneak Peak Week"
plan: "free"
strategy: "duration"
limit: 7
limit: 15
enabled: true
Steps to Configure the AI Service
Enable Database Configuration:
Add the following configuration to enable the database for the AI service:
database: enabled: true host: <your mysql host> name: <your db name> user: <your mysql user omit if you provide your own secret> password: <password of your mysql user omit if you provide your own secret>
Enable Database Secret:
To create a Kubernetes secret for the database, set the
mysqlSecret.enabled
totrue
.If you want to use your own secret for the database, set the
mysqlSecret.enabled
tofalse
and provide the secret name inoverrides.mysqlSecret
.Prepare the Database:
The service expects an existing database to connect to and run initial migrations. Follow the steps below to set up the database correctly.
Database Schema Setup
To ensure the AI service operates correctly, you need to create the necessary tables and procedures in your database. Although the actual migration script will be executed automatically, it is essential to ensure that the database user has the appropriate permissions.
Minimal Required Grants
The minimal grants needed for the database user are:
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, TRIGGER
ON `your_database_name`.*
TO 'your_db_user'@'%';
Replace your_database_name
with the name of your database and your_db_user
with the name of your database user.
Example Configuration
Here's an example of how to grant the necessary permissions:
CREATE USER 'ai_service_user'@'%' IDENTIFIED BY 'secure_password';
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, TRIGGER
ON `ai_service_db`.*
TO 'ai_service_user'@'%';
FLUSH PRIVILEGES;
In this example:
ai_service_user
is the database user for the AI service.secure_password
is the password for the database user.ai_service_db
is the database name for the AI service.
Complete configuration reference
Parameter | Description | Default |
---|---|---|
image.repository | The image to be used for the deployment | registry.open-xchange.com/core/ai-service |
image.pullPolicy | The imagePullPolicy for the deployment | IfNotPresent |
image.tag | The image tag, defaults to app version | "" |
hostname | hostname for the ai-service deployment | "" |
origins | Allowed origins for CORS | * |
logLevel | specify log level for service | "info" |
logJson | Logs in JSON format | false |
exposeApiDocs | Expose API documentation via Swagger UI at /api-docs | false |
ingress.enabled | Generate ingress resource | false |
ingress.annotations | Map of key-value pairs that will be added as annotations to the ingress resource | {} |
overrides.name | Name of the chart | "ai-service" |
overrides.fullname | Full name of the chart installation | "RELEASE-NAME-ai-service" |
jwtSecret.enabled | Enable the secret for JWT | true |
jwt.sharedSecret | Shared secret for JWT verification. This must match the secret configured for switchboard | "" |
jwks.domain | Domain of JWKS issuer like example.com leave empty if you want to use sharedSecret | "" |
openaiSecret.enabled | Enable the secret for openai | true |
openaiAPIKey | OpenAI API Key | "" |
azureSecret.enabled | Enable the secret for Azure | false |
azureAPIUrl | OpenAI Azure API Url (Internal use only) | "" |
azureAPIKey | OpenAI Azure API Key (Internal use only) | "" |
mysqlSecret.enabled | Create the kubernetes secret for mysql (enable if you want to use the DB or provide own secret) | false |
overrides.mysqlSecret | If you provide your own secret for mysql put the secret name here | "" |
database.enabled | Use Database (mandatory for usage tracking and rate limiting) | false |
database.host | SQL server hostname | RELEASE-NAME-ai-service-sql |
database.name | Database name | RELEASE-NAME-ai-service |
database.connections | Number of concurrent connections to the DB server | "10" |
database.user | DB User with access rights to sqlDB | "" |
database.password | DB Password of swDBUser | "" |
database.rootPassword | Database root password to perform admin tasks | "" |
database.rollback | WARNING: This will roll back the migrations this version has rolled out | false |
cron.cleanupDb | Database cleanup interval (Cron notation) | 0 0 * * * * |
azureAPIVersion | OpenAI Azure API Key (Internal use only) | "" |
openaiBaseUrl | Url of the OpenAI service (internal use only) | "" |
accounting.enabled | Enable the accounting feature | false |
plans | List of plans with limits for users and brands | see example above |
models | List of models that the service supports | see example above |
enforceDefaultModel | Always use the configured default model regardless of the clients request | false |