Skip to main content

LLM Gateway Quickstart

Call OpenAI / Anthropic-compatible LLMs by simply swapping the base URL of your SDK (SaaS only).

Introduction​

Here are the quickest steps to use the LLM Gateway. You can call an LLM in just three steps.

The LLM Gateway and the Agent API are different

The LLM Gateway (/llm/v1/...) is a raw LLM call (a governance proxy) compatible with OpenAI / Anthropic. It is a separate service from the Agent API (/api/v1/chat/completions), which runs autonomous agents. If you need tool execution or deliverables, use the Agent API.

Prerequisites
  • Setup is complete
    • An administrator has defined the available models in the admin panel (Admin)
    • An LLM Gateway client has been created in Console and an API key (agtstr_...) has been obtained

Step 1: Check Available Models​

Available models are defined by an administrator. Retrieve the list with GET /v1/models.

curl
curl https://api.fd.agenticstar.tm.softbank.jp/llm/v1/models \
  -H "Authorization: Bearer agtstr_..."

Step 2: Call the LLM​

Specify the model ID you retrieved and make the call.

Chat completion
1from openai import OpenAI
2
3client = OpenAI(
4  api_key="agtstr_...",
5  base_url="https://api.fd.agenticstar.tm.softbank.jp/llm/v1",
6)
7
8resp = client.chat.completions.create(
9  model="<model ID from GET /v1/models>",
10  messages=[{"role": "user", "content": "Hello"}],
11)
12print(resp.choices[0].message.content)
Authentication Methods

In addition to Authorization: Bearer agtstr_..., the Anthropic SDK also accepts x-api-key: agtstr_.... You can also authenticate with the Client Credentials flow using a Client ID / Client Secret.

Step 3: Streaming​

Enable stream to receive tokens incrementally over SSE.

Python
stream = client.chat.completions.create(
    model="<model ID>",
    messages=[{"role": "user", "content": "Introduce yourself"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

Next Steps​

For more detailed specifications and setup steps, see the following documentation.

📖LLM Gateway Reference
SaaS

Specifications for all endpoints, errors, and limits

⚙️Setup
SaaS

Preparation steps for administrators and developers