LLM Gateway
OpenAI / Anthropic compatible LLM calls (SaaS edition only). Requests and responses follow each vendor's native format, so all you change is the base URL in your existing OpenAI or Anthropic SDK.
Once you have swapped it, model selection, cost caps, access control and usage visibility are handled by the gateway. Your application code stays the same whether you change models or add more of them.
What the gateway takes care of
| What you get | |
|---|---|
| Keep your SDK | Native OpenAI / Anthropic formats. The only code change is the base URL and the API key |
| One key | Your application holds a single agtstr_... key. Each provider's credentials stay in the gateway's connection settings |
| Switch models without touching the app | Your application calls the public model name defined by the administrator. The actual target (OpenAI, Azure, self-hosted and so on) is decided by the model definition in Admin, so switching providers does not change your code |
| Group several targets under one name | Register multiple targets for the same public model name and distribute across them by weight and priority |
| Automatic routing by difficulty | Register high / medium / low tiers under the same model name and requests are routed automatically by prompt difficulty. Developers still just pass the model name |
| Cap the cost | Set a daily budget per model. Requests beyond it stop with 429 |
| Restrict where calls come from | Register allowed IPs and enable the restriction; other networks receive 403 |
| Usage in one place | Model, provider, token counts, cost and latency are visible in Admin and through the System Integration API |
Every setting is covered in the Setup Guide.
Available models are defined per tenant by the administrator, so they differ between environments. Fetch them dynamically with GET /llm/v1/models.
Direct calls from a browser (cross-origin) are not allowed by default. See the API Reference for details.
When you need agent capabilities such as tool execution, artifacts or conversation history, use the API instead.
Getting started
Follow Call from the SDK to swap the base URL and send your first request.
Guides / References
- Setup Guide — preparation for administrators (model definitions, automatic routing, daily budgets, IP restrictions, usage logs) and developers (client creation, API key issuance)
- API Reference — every endpoint (OpenAI / Anthropic compatible), authentication, errors and limits