Passthrough API

Use provider-native APIs through GoModel, including Anthropic SDK requests routed through the passthrough endpoint.

Overview

The passthrough API is in beta. Use it when you need a provider-native API shape, and prefer the OpenAI-compatible /v1 routes when you need the most stable GoModel interface.

The passthrough API proxies provider-native requests through GoModel without translating the request or response body.

Use it when a client library expects a provider’s native API instead of the OpenAI-compatible API. For example, the Anthropic SDK sends requests to /v1/messages, so you can point it at GoModel’s Anthropic passthrough base URL:

http://localhost:8080/p/anthropic

The SDK sends:

POST /p/anthropic/v1/messages

GoModel normalizes the optional v1 segment and forwards the request upstream as Anthropic’s native:

POST /messages

How it works

Passthrough routes use this shape:

/p/{provider}/{endpoint}

For Anthropic, these two paths map to the same upstream endpoint when ALLOW_PASSTHROUGH_V1_ALIAS=true, which is the default:

/p/anthropic/messages
/p/anthropic/v1/messages

GoModel handles gateway authentication first. If GOMODEL_MASTER_KEY or managed auth keys are enabled, the client must send a GoModel bearer token:

Authorization: Bearer change-me

GoModel strips client Authorization and X-Api-Key headers before forwarding the request, then applies the upstream provider credential configured on the server. For Anthropic, GoModel uses its configured ANTHROPIC_API_KEY.

Because passthrough is provider-native, the response is also provider-native. For Anthropic messages, the response uses Anthropic’s message schema, not an OpenAI chat completion schema.

When usage tracking is enabled, successful passthrough inference responses are recorded: token counts are read from SSE usage events on streaming responses and from the usage member of JSON responses, so costs and budgets account for passthrough traffic like any other route.

Guardrails do not run on passthrough

Passthrough forwards the caller’s provider-native body and relays the provider’s answer untouched, so no guardrail chain can read or edit either one. Rate limits, budgets, model access, and usage accounting still apply; guardrails do not.

To keep that from silently dropping a policy, GoModel refuses a passthrough request whenever a guardrail workflow applies to the caller:

{
  "error": {
    "type": "permission_error",
    "code": "passthrough_guardrails_unsupported",
    "message": "provider passthrough cannot run guardrails, and a guardrail workflow applies to this request; ..."
  }
}

The request is answered with 403. A passthrough workflow is matched on the model read from the provider-native body, and that read is best effort, so an inference call whose model GoModel cannot read is refused whenever any guardrail is configured — an unreadable body must not be a way around a policy.

Only text-generating routes are refused: chat completions, responses, Anthropic messages, the legacy completion endpoints, and any provider-native path of the same shape. Model lists, token counts, embeddings, files, batches, images, and audio run no guardrail chain on /v1 either, so passthrough serves them unchanged.

Route that traffic through the OpenAI-compatible endpoints (/v1/chat/completions, /v1/responses, /v1/messages), which do run the chains, or choose one of:

# Accept unguarded passthrough for every caller (default: false)
ALLOW_UNGUARDED_PASSTHROUGH=true

server:
  allow_unguarded_passthrough: true

Alternatively, scope the guardrail workflow so it does not match the callers that use passthrough. With no guardrail workflow matching a request, passthrough serves it as before.

Anthropic SDK example

Set the Anthropic SDK base URL to GoModel’s Anthropic passthrough route. Use the GoModel token as Anthropic SDK bearer auth.

Python

import os
from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:8080/p/anthropic",
    auth_token=os.environ["GOMODEL_MASTER_KEY"],
)

message = client.messages.create(
    model="claude-3-haiku-20240307",
    max_tokens=16,
    messages=[
        {
            "role": "user",
            "content": "Reply with exactly ok",
        }
    ],
)

print(message.content[0].text)

If your gateway does not require authentication, use a dummy SDK API key such as not-needed instead of auth_token or authToken. GoModel strips X-Api-Key from passthrough requests before forwarding them upstream.

Current limitations

Passthrough is intentionally narrow while the API is in beta.

• openai, anthropic, openrouter, kilo, zai, sglang, vllm, llamacpp, llmd, and deepseek are enabled by default.
• Chutes supports passthrough but requires explicit operator opt-in because passthrough can forward provider-native routes that do not identify a model. Add chutes to ENABLED_PASSTHROUGH_PROVIDERS only when you intend to expose that surface.
• GoModel does not translate passthrough request bodies or response bodies, so guardrails cannot run on them (see above).
• Provider-native error bodies and status codes are proxied instead of converted into OpenAI-compatible responses.
• Features that depend on OpenAI-compatible request or response shapes may not apply to passthrough traffic in the same way as /v1 traffic.

Related environment variables

Passthrough routes are enabled by default:

ENABLE_PASSTHROUGH_ROUTES=true
ALLOW_PASSTHROUGH_V1_ALIAS=true
ENABLED_PASSTHROUGH_PROVIDERS=openai,anthropic,openrouter,kilo,zai,sglang,vllm,llamacpp,llmd,deepseek,hetzner
ALLOW_UNGUARDED_PASSTHROUGH=false

Set ENABLED_PASSTHROUGH_PROVIDERS to the provider types you want to expose.