LLM Gateway Setup
To use the LLM Gateway, administrators and developers each perform preparation on separate screens (SaaS only).
Overview
| Role | Screen | Tasks |
|---|---|---|
| Administrator | Admin panel (Admin) | Model definition (registering available models), IP restriction, and viewing usage logs |
| Developer | Console | Creating an LLM Gateway client (obtaining authentication information) |
- Administrators can log in to the admin panel (Admin), and developers can log in to Console (developer role or higher)
Administrator Preparation (Admin Panel)
Defining Models
Administrators register the available models in the "LLM Gateway Management" menu of the admin panel (Admin). The registered public model name is the name that developers pass in the request model field, and it is returned by GET /llm/v1/models. The form is organized into "Basic Settings / Routing & Budget / Connection / Model Info".
Basic Settings
| Item | Required | Description |
|---|---|---|
| Public Model Name | ✓ | The public name clients send (the lookup key). Developers pass it in model, and it is returned by GET /v1/models. |
| Deployment | An identifier that groups multiple deployments under the same public model name. | |
| Active | Whether this model definition is enabled or disabled. |
Routing & Budget
| Item | Required | Description |
|---|---|---|
| Tier | Routing tier (high / middle / low / unset). Registering multiple tiers under the same public model name enables "Auto Routing" (below). | |
| Weight | Weight for load balancing within the same tier (default 1). | |
| RPM / TPM limit | Per-minute request / token limits (leave blank for unlimited). | |
| Priority | Routing priority (a smaller number is higher priority). | |
| Daily Budget ($) | Daily cost cap (leave blank for unlimited). |
Exceeding RPM / TPM returns 429 (with Retry-After); exceeding the daily budget returns 429 (with x-should-retry: false). See Errors in the Reference for details.
Connection
Connection settings follow the LiteLLM format. You can enter them via a simple form or by editing the parameters directly as JSON (switchable via tabs). The simple-form fields are:
| Item | Required | Description |
|---|---|---|
| Backend Model | ✓ | Specify it in LiteLLM's provider/model format (e.g. azure/gpt-5.5). |
| API Key | The provider authentication key (not required for proxy-based or local models). | |
| API Base URL | The provider endpoint URL (not needed for official OpenAI; specify it for Azure / self-hosted). | |
| API Version | API version (required for Azure OpenAI, etc.). | |
| Additional Parameters | Specify other LiteLLM parameters as JSON. |
Model Info (Optional)
You can specify model attributes that are not used for connection or routing—such as mode or max_tokens—as JSON.
Available models are defined per tenant by administrators, so they vary by environment. In your application, retrieve them dynamically with GET /llm/v1/models (Reference) or check with your administrator.
Auto Routing (Optional)
When two or more tiers (high / middle / low) are registered for the same model name, an administrator can enable tenant-wide Auto Routing from the "LLM Gateway Management" menu in the admin panel (Admin). When enabled, prompts are automatically routed to a model of the appropriate tier based on their difficulty. Developers simply specify a model name as usual; the tier is selected automatically.
IP Restriction (Optional)
In the "LLM Gateway Allowed IPs" screen of the admin panel (Admin), you can restrict the networks that can access the LLM Gateway.
- Register allowed IPs — Register an IP address (IPv4 / IPv6 / CIDR formats are supported; e.g.
192.168.1.0/24) and a description. - Enable IP restriction — When enabled, only registered IP addresses can access the LLM Gateway. You cannot enable it when no allowed IPs are registered.
When enabled, requests from networks that are not allowed are rejected with 403.
Usage Logs
Usage such as model, provider, token count, cost, and latency can be viewed in the admin panel (Admin) and via the llm-gateway-logs endpoint in the Admin API Reference.
Developer Preparation (Console)
From the "LLM Gateway" menu in Console, register a new client, then use the detail screen to review credentials and manage the client lifecycle (enable / disable / edit / delete).
1. Enter Basic Information
In the "LLM Gateway" menu, click "Register" and enter the following to create the client.
| Item | Required | Description |
|---|---|---|
| LLM Client Name | ✓ | Display name of the client (max 256 characters). |
| Description | Description of the purpose or use. |
2. Review Authentication Information
After creation, you can review and manage credentials under "Developer Info" on the detail screen. Depending on your use case, use either an API key (set directly in the SDK) or the Client Credentials flow (obtain an access token with the Client ID / Client Secret; see the Authentication API Reference (SaaS) for the procedure).
| Item | Description |
|---|---|
| Client ID | Identifier of the client. Used in the Client Credentials flow. |
| Client Secret | Secret for the Client Credentials flow. Shown masked; it can be copied and regenerated ("Regenerate Secret"). |
| API Key | A fixed key (agtstr_...) that can be set directly in the OpenAI / Anthropic SDK. Sent via Authorization: Bearer or x-api-key. |
| Token URL | Endpoint for obtaining an access token in the Client Credentials flow. |
Issuing and Managing the API Key
The API key is not issued immediately after creation. Manage it on the detail screen with the following actions.
| Action | Description |
|---|---|
| Issue API Key | Issues a fixed API key (agtstr_...). |
| Regenerate API Key | Issues a new key. The previous key becomes invalid. |
| Delete API Key | Deletes the key. It can no longer be used for authentication. |
API keys and Client Secrets are confidential information equivalent to passwords. Do not hard-code them in source code; manage them with environment variables or a Secret Manager. Regenerating a key or secret causes all calls using the previous key / secret to fail, so review the impact beforehand.
3. Enable / Disable
From the header of the detail screen, you can enable / disable the client. A disabled client cannot call the LLM Gateway. You can delete clients that are no longer needed.
Only enabled clients can call the LLM Gateway. The number of enabled clients is limited by the number of users (seats) in your contract; once the limit is reached, you cannot create or enable additional clients ("The contracted seat limit has been reached, so LLM access cannot be granted."). Disabling a client frees one seat, allowing you to enable another.
Next Steps
- LLM Gateway Quickstart — Call from an SDK
- LLM Gateway Reference — Endpoint specifications