Routes & Policies
Configure provider selection, fallback, and request profiles for API keys.
Routes (routing policies) define how LatentKit executes requests for an API key. Application code sends task payloads; routes decide provider/model attempts.
What you configure
- Ordered provider/model attempts and failover
- Capability eligibility (chat, embeddings, image, etc.)
- BYOK vs Platform Access connections
- Optional request overrides such as
response_profile - Budgets and guardrails (workspace-dependent)
What developers send
Developers send endpoint-specific bodies only — for example messages for chat or input for embeddings. Do not send:
modelproviderroutepolicy
Routing strategies
Each published policy uses one strategy to decide which eligible route entry executes first:
| Strategy | Behavior |
|---|---|
priority | Try entries in the published order, top to bottom (default) |
cost | Sort eligible entries by published catalog price, cheapest first, then rank |
availability | Prefer entries whose providers are currently healthy, then published rank |
balanced | Weighted primary-route selection using each entry's weight (default 50) — useful for traffic splitting and gradual model migrations |
The developer integration is identical for every strategy: one API key and the standard /v1/* request body. Changing the strategy in the console changes execution behavior without any application redeploy.
Failover
When the selected entry fails with a retryable provider error (timeout, 5xx, invalid payload), LatentKit moves to the next eligible entry in strategy order. Your application receives one response either way — response.provider and response.model report the attempt that won. If every entry fails, the request returns 503 NO_HEALTHY_PROVIDER with diagnostic detail.
Publishing
Draft routes can be edited in the console. Published routes are what runtime requests use. Assign a published route to each API key.
The publish flow:
- Draft — add or reorder entries (each entry = connection + model), set weights and profiles. Drafts never affect live traffic.
- Save draft — persists the entry table on the server.
- Publish — makes the saved route version live for assigned API keys. A route must contain at least one valid entry before it can be published.
Every response includes policy_version_id, so you can correlate behavior changes with specific publishes.
Response profiles
When enabled on a route, applications may pass:
{ "response_profile": "balanced" }Allowed values: fast, balanced, deep. If overrides are disabled, the route default applies even when a profile is sent.
Deterministic routing hint
Eval and test clients may send prefer_deterministic: true or X-LK-Prefer-Deterministic: true with a seeded request. LatentKit keeps the published policy strategy, then ranks seed-supporting providers ahead of non-seed providers within that strategy.
Pin a preferred model in the policy when exact repeatability matters. A seed does not make two different providers produce identical output.
Testing routes
Use AI Router in the console to verify route health, run browser test requests, and copy snippets against the selected key.
If the route test fails in the console, fix provider connection, model eligibility, credits, or route publish state before changing application code.