Models & Providers
Which providers and model capabilities LatentKit supports, and how model selection actually works.
LatentKit maintains a provider and model catalog. Your workspace connects providers and chooses models per route in the console; application code never sends model names. This page explains the available capabilities and where to find the live list for your workspace.
How model selection works
- The catalog defines providers and their models, each tagged with capabilities and prices.
- Your workspace enables models on a connection (BYOK or Platform Access).
- A route (routing policy) lists ordered connection-model entries.
- At request time, LatentKit filters route entries by the endpoint's required capability and health, then executes in policy order.
The model that answered is reported back in response.model and response.provider.
Supported providers
| Provider | Type | Notes |
|---|---|---|
| OpenAI | Direct | Chat, vision, embeddings, images, TTS, and transcription models |
| Anthropic | Direct | Claude chat and vision models |
| Google Gemini | Direct | Chat, vision, and multimodal models with large context windows |
| Mistral AI | Direct | Chat and code models |
| DeepSeek | Direct | Chat and reasoning models |
| Moonshot AI (Kimi) | Direct | Current Kimi K3, K2.7 Code, and K2.6 text/vision/video-input models |
| xAI (Grok) | Direct | Grok chat/reasoning models plus image and video generation |
| OpenRouter | Aggregator | OpenAI-compatible access to many third-party models (Llama, and more) through one connection |
| AWS Bedrock | Cloud platform | OpenAI-compatible Bedrock Mantle access; BYOK with an optional regional endpoint override |
| Microsoft Foundry / Azure OpenAI | Cloud platform | OpenAI-compatible v1 endpoint; BYOK endpoint and deployment names are workspace-specific |
| Google Vertex AI | Cloud platform | OpenAI-compatible endpoint for Gemini and partner models; BYOK project/location endpoint required |
| Fireworks AI | Inference platform | OpenAI-compatible hosted models including Kimi, DeepSeek, and MiniMax |
| DeepInfra | Inference platform | OpenAI-compatible hosted text, vision, and reasoning models |
| fal.ai | Media platform | Direct image and video generation model endpoints |
| Replicate | Media platform | Official image and video models with synchronous wait and bounded polling |
The broader active catalog also includes Alibaba/Qwen, ByteDance/Volcano Ark, Cerebras, Cohere, Groq, Jina AI, MiniMax, Perplexity, Scaleway, Together AI, Z.ai, and the dedicated speech providers listed below. In total, the shipped catalog contains 30 active provider definitions; individual model visibility still depends on the applied database migrations and workspace connection.
For speech-to-text, the audio capability matrix additionally covers OpenAI transcribe models, AssemblyAI, Deepgram, ElevenLabs Scribe, Google Cloud Speech-to-Text, and Gemini audio — see Audio and STT for the current status of each.
For text-to-speech, OpenAI and ElevenLabs (Flash, Turbo, Multilingual v2, and v3) carry the audio_output capability — see Speech. ElevenLabs voice models route through BYOK connections only.
The catalog evolves continuously and model availability differs per workspace plan and connection. The Providers section in the console is the live source of truth for the models your workspace can route to right now.
Custom providers
Workspaces can also register custom OpenAI-compatible providers (self-hosted or third-party gateways that speak the OpenAI API shape) from the console, then route to them like any other connection.
Capabilities
Every catalog model is tagged with capabilities. Endpoints require capabilities, and only matching route models are eligible:
| Capability | Endpoints that require it |
|---|---|
text | /v1/chat, /v1/complete |
vision | /v1/vision; added automatically when chat messages contain image input |
video_input | Added automatically when supported chat messages contain video input |
embeddings | /v1/embeddings, /v1/embed |
image_generation | /v1/image |
audio_input | /v1/transcription, /v1/translation |
audio_output | /v1/speech, /v1/audio; added when chat requests "modalities": ["audio"] |
video_generation | /v1/video |
function_calling | Added automatically when a request includes tools, tool_choice, or tool messages |
If no route model satisfies the required capabilities, the request fails with NO_HEALTHY_PROVIDER — see Error reference.
Pricing and cost reporting
Per-token prices come from the same catalog the console displays. Each successful response includes a cost_usd field computed from provider-reported token usage and catalog prices, so your logs and the console usage dashboards agree.
BYOK vs Platform Access models
- BYOK connections can use any catalog model the provider account has access to; LatentKit does not bill model usage. The provider bills your own account at its own rate — LatentKit adds no margin to BYOK usage. The
cost_usdon a BYOK response is an estimate of that provider charge (the response is taggedbilling_mode: "byok_estimate"), not a workspace charge. - Platform Access (managed) connections bill usage against workspace credits. Some modalities — notably audio — may be BYOK-only until managed billing supports their pricing dimensions.
The first provider-breadth wave is BYOK-only. Azure OpenAI and Vertex AI need a
connection-specific endpoint, while AWS Bedrock can use the seeded US East
Mantle endpoint or a regional override. fal.ai and Replicate currently use
curated model rows rather than the generic Sync Models flow.
Picking models for a use case
| Goal | Approach |
|---|---|
| Cheapest acceptable answers | Use the cost routing strategy or order low-cost models first |
| Maximum stability for tests/evals | Pin one model in the policy and pass seed — see Determinism |
| High availability | Order equivalent models from different providers as fallbacks |
| Speed vs depth per request | Use route response_profile assignments (fast / balanced / thinking) |