Audio and STT

LatentKit supports audio input and speech-to-text through routed /v1 endpoints. Use /v1/transcription for transcription and /v1/translation for audio translation. For long audio, use the durable async path: POST /v1/transcription/jobs with polling and an optional completion webhook.

Choosing an input path:

For the reverse direction — generating spoken audio from text — see Speech (Text-to-Speech).

Your application authenticates with one LatentKit API key. The routing that key resolves to must include an audio_input model — OpenAI gpt-4o-transcribe, gpt-4o-mini-transcribe, gpt-4o-transcribe-diarize, whisper-1, or a dedicated speech provider such as ElevenLabs Scribe, AssemblyAI, or Deepgram — through the app's assigned route, an audio_input capability override, or a purpose bound to an audio route.

Batch Transcription

POST /v1/transcription accepts JSON or multipart form data.

Multipart upload

Use multipart form data when your backend has a local audio file.

curl https://ai.latentkit.com/v1/transcription \
  -H "Authorization: Bearer $LATENTKIT_API_KEY" \
  -F [email protected] \
  -F language=en \
  -F response_format=json

JSON input

Use JSON when your backend already has base64 audio or an approved audio URL.

curl https://ai.latentkit.com/v1/transcription \
  -H "Authorization: Bearer $LATENTKIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "audio": {
      "base64": "<base64-audio>",
      "media_type": "audio/mpeg",
      "filename": "meeting.mp3"
    },
    "language": "en",
    "response_format": "json"
  }'

Required audio input can be one of:

• multipart file
• audio.base64
• audio.data
• audio.url
• base64 input

Audio files are capped at 25 MiB by default (discover the live value from the max_audio_bytes field in GET /v1/me → limits). Over-budget audio returns a structured JSON 413 with code AUDIO_TOO_LARGE before any provider is called — you are never billed for a rejected oversized payload. Supported OpenAI-compatible upload formats include mp3, mp4, mpeg, mpga, m4a, wav, and webm.

Idempotent transcription (Idempotency-Key)

Send an Idempotency-Key header on POST /v1/transcription to make retries safe. If a response is lost (timeout, dropped connection) and you retry with the same key and the same payload, the gateway returns the stored transcript — no second provider call and no second charge. Replayed responses carry the header X-LK-Idempotent-Replay: true.

• Same key + same payload → stored result is returned; billing settles exactly once.
• Same key + different payload → structured 409 with code IDEMPOTENCY_KEY_REUSED.
• Same key while the original request is still running → 409 with code IDEMPOTENCY_REPLAY_IN_PROGRESS (retryable: true; retry after a few seconds).
• If the original attempt ended ambiguously (a gateway timeout or connection loss where the provider may or may not have completed), a retry with the same key returns 409 IDEMPOTENT_RESULT_UNAVAILABLE instead of silently re-running — the key is intentionally burned rather than risking a second charge. Use a new Idempotency-Key when you have decided a re-run is what you want.
• Slots expire after 24 hours (sync_idempotency_ttl_seconds in GET /v1/me → limits).
• Streaming requests bypass idempotency (an SSE stream cannot be replayed).

curl https://ai.latentkit.com/v1/transcription \
  -H "Authorization: Bearer $LATENTKIT_API_KEY" \
  -H "Idempotency-Key: segment-2026-08-16-0042" \
  -F [email protected]

Large audio: use audio.url

For big files, skip request-body limits entirely: upload the audio to your own storage (S3, GCS, R2, …), generate a short-lived signed URL, and pass it as audio.url. The gateway fetches up to 25 MB (max_audio_url_bytes) server-side.

{
  "audio": { "url": "https://uploads.example.com/meetings/2026-08-16.mp3?sig=…" },
  "language": "en"
}

Requirements (all enforced server-side, none configurable per request):

• HTTPS only; the host must be on your workspace's allowed domains list or the platform's global allowlist. Workspace admins manage the list via the workspace API (GET/PUT /workspace/{tenant_id}/audio-url-domains); a console settings card is planned.
• The URL must resolve to a public IP — private, loopback, link-local, and cloud metadata ranges are always rejected, regardless of allowlist entries.
• The response must declare Content-Length and a supported audio content type; redirects are capped and cross-host redirects are re-validated.

The per-workspace allowlist only chooses which domains may be named; the SSRF protections above are non-negotiable platform invariants.

Common STT fields:

• mode: batch, async, or realtime
• language
• diarization
• speaker_count_hint
• timestamps: word, segment, or both
• custom_vocabulary / keyterms
• smart_formatting
• detect_language
• profanity_filter
• redaction
• multichannel

Required features are checked before provider selection. If a fallback route would drop a required feature such as diarization, timestamps, redaction, realtime mode, or keyterms, routing fails closed instead of silently degrading the transcript.

Async Transcription Jobs

For long audio (a 30-minute meeting, a 90-minute recording), use the durable job API instead of holding a synchronous request open:

# 1. Submit — returns immediately with a job id
curl -X POST https://ai.latentkit.com/v1/transcription/jobs \
  -H "Authorization: Bearer $LATENTKIT_API_KEY" \
  -H "Idempotency-Key: meeting-2026-08-16" \
  -H "Content-Type: application/json" \
  -d '{"audio": {"url": "https://uploads.example.com/meeting.mp3?sig=…"}}'
# → {"job_id": "…", "status": "queued", "poll_url": "/v1/transcription/jobs/…"}

# 2. Poll
curl https://ai.latentkit.com/v1/transcription/jobs/$JOB_ID \
  -H "Authorization: Bearer $LATENTKIT_API_KEY"

# 3. Cancel (optional)
curl -X DELETE https://ai.latentkit.com/v1/transcription/jobs/$JOB_ID \
  -H "Authorization: Bearer $LATENTKIT_API_KEY"

Lifecycle

Jobs move through queued → processing → succeeded | failed | cancelled. Terminal states are immutable.

• Durable results. Results are persisted server-side and survive gateway restarts. Small transcripts are returned inline in result; large payloads are returned as a short-lived signed result_url instead.
• Retention. Results are kept for 72 hours by default (stt_job_result_ttl_seconds in GET /v1/me → limits), reported per job as expires_at. After that, the result and any signed-URL source are purged.
• Scoping. Jobs are visible only to the workspace + app that created them. A job id from another workspace behaves exactly like a nonexistent id: 404.
• Idempotent submission. Idempotency-Key on POST /jobs makes resubmission safe: a replayed key returns the same job_id.
• Cancellation. DELETE on a queued job cancels it. On a processing job it sets a cancel flag the worker honors between stages. Cancelling an already-terminal job is an idempotent no-op that returns the current state — a cancel that loses the race to completion returns succeeded.

Completion webhook (optional)

Register a webhook (workspace API: PUT /workspace/{tenant_id}/stt-webhook, admin role; a console settings card is planned) to be notified when jobs reach a terminal state, instead of polling:

• The payload contains only type, job_id, status, and occurred_at — never the transcript and never any secret. Fetch the result via the job API.
• Every delivery is signed: X-LK-Webhook-Signature is hex(hmac_sha256(secret, "{X-LK-Webhook-Ts}.{raw_body}")). Verify the signature and reject stale timestamps to prevent replays.
• Deliveries retry on failure with backoff, a bounded number of times.
• The signing secret is shown once at configuration time; rotating it re-generates it.

Transcription models

OpenAI-compatible transcription routes are first-class audio_input routes in LatentKit.

Dedicated speech providers

These providers ship with their own adapters and are available on your own API key (BYOK) connections:

Required STT features — diarization, realtime or async mode, word timestamps, redaction, custom vocabulary — are checked against the model's capability matrix before a provider is selected, and fallback fails closed rather than silently dropping a feature you asked for.

Audio transcription is metered in seconds of submitted audio, so a transcription response reports usage.audio_seconds with total_tokens: 0.

LatentKit decides the exact provider/model from the route assigned to the API key. Do not send provider or model in application requests; transcription route-control fields are rejected, including nested audio.provider and audio.model.

SDK Examples

import { LatentKit } from '@latentkit/sdk';

const client = new LatentKit({
  apiKey: process.env.LATENTKIT_API_KEY!,
});

const transcript = await client.transcription.create({
  audio: {
    base64: '<base64-audio>',
    media_type: 'audio/mpeg',
    filename: 'meeting.mp3',
  },
  language: 'en',
  prompt: 'Meeting about product roadmap and LatentKit routing.',
  response_format: 'json',
});

console.log(transcript.content);

import os
from latentkit import LatentKit

with LatentKit(api_key=os.environ["LATENTKIT_API_KEY"]) as client:
    transcript = client.transcription.create(
        audio={
            "base64": "<base64-audio>",
            "media_type": "audio/mpeg",
            "filename": "meeting.mp3",
        },
        language="en",
        prompt="Meeting about product roadmap and LatentKit routing.",
        response_format="json",
    )

print(transcript["content"])

Do you need a purpose for transcription?

Usually not. Capability routing already handles this. POST /v1/transcription requires the audio_input capability, so a route holding both chat and transcription models picks the transcription ones automatically — no label, no configuration, nothing to send. If your models differ by capability, the endpoint has already chosen for you.

A purpose earns its place when several models on the route can all serve the same endpoint and you want a particular one. Two Whisper-class models where one is tuned for your audio, say. Reach for it then, not before.

If your app uses an audio_input capability override to send audio to a different route, purposes on that route are the ones that apply — GET /v1/me reports them under the matching entry in routing_contexts.

Set it up in this order

The order matters: step 1 is the safety net for every step after it.

1. Give the app a working audio baseline. Either the app's assigned route contains an audio_input model, or you configure an audio_input capability override pointing at a route that does. Do this first.
2. Publish the route that should handle this transcription workload.
3. Bind a purpose to that route on the app's Purposes section.
4. Discover the label from GET /v1/me rather than hardcoding it.
5. Send purpose on transcription requests.
6. Log lk_applied_purpose and alert when it does not match what you sent.

A purpose selects which route; it never satisfies a capability requirement. The endpoint you call still decides that /v1/transcription needs an audio_input model.

Sending a purpose

curl https://ai.latentkit.com/v1/transcription   -H "Authorization: Bearer $LATENTKIT_API_KEY"   -H "Content-Type: application/json"   -d '{
    "audio": { "base64": "<base64-audio>", "media_type": "audio/mpeg" },
    "language": "en",
    "purpose": "meeting-transcription",
    "response_profile": "balanced"
  }'

purpose picks the route; response_profile picks how much effort that route spends. They compose — neither replaces the other.

Multipart uploads accept the same fields as form fields:

curl https://ai.latentkit.com/v1/transcription   -H "Authorization: Bearer $LATENTKIT_API_KEY"   -F [email protected]   -F language=en   -F purpose=meeting-transcription   -F response_profile=balanced

The X-LK-Purpose header works on both shapes and is a good fit when a client library controls the body. The body field wins if you send both.

import { LatentKit } from '@latentkit/sdk';

const client = new LatentKit({ apiKey: process.env.LATENTKIT_API_KEY! });

// Discover the labels this key may send instead of hardcoding one.
const { purposes } = await client.me.retrieve();
console.log(purposes); // [{ purpose: 'meeting-transcription', description: '…', is_enabled: true }]

const transcript = await client.transcription.create({
  audio: { base64: '<base64-audio>', media_type: 'audio/mpeg' },
  language: 'en',
  purpose: 'meeting-transcription',
  response_profile: 'balanced',
});

// null means the request ran on the app's default routing, not the bound route.
if (transcript.lk_applied_purpose !== 'meeting-transcription') {
  console.warn('transcription purpose degraded:', transcript.lk_purpose_source);
}

import os
from latentkit import LatentKit

with LatentKit(api_key=os.environ["LATENTKIT_API_KEY"]) as client:
    # Discover the labels this key may send instead of hardcoding one.
    for row in client.me.retrieve()["purposes"]:
        print(row["purpose"], "-", row["description"])

    transcript = client.transcription.create(
        audio={"base64": "<base64-audio>", "media_type": "audio/mpeg"},
        language="en",
        purpose="meeting-transcription",
        response_profile="balanced",
    )

    # None means the request ran on the app's default routing, not the bound route.
    if transcript.get("lk_applied_purpose") != "meeting-transcription":
        print("transcription purpose degraded:", transcript.get("lk_purpose_source"))

translation takes the same purpose and response_profile arguments.

One key is a routing decision, not a security decision

Purposes let one API key reach several routes. That removes a reason to hold multiple keys, but it is not a reason to collapse keys you hold for other purposes.

Keep separate keys when you need separate:

• revocation boundaries — rotating one integration without disrupting the rest
• app or environment boundaries — staging must not share production's key
• scopes or permissions
• quotas or spend controls — budgets and limits attach to the app, not the label
• audit identity — request logs attribute to the key that made the call

A purpose is a routing selector visible to anyone who can call your API. It is not an access-control boundary. Choose the number of keys on isolation grounds, then use purposes to route within each key.

Response Shape

STT responses use the same routed response envelope as chat responses and may include:

• text in content
• status
• language
• language_confidence
• duration_seconds
• segments
• words
• warnings
• usage.audio_seconds
• usage.billable_audio_seconds
• lk_requested_purpose, lk_applied_purpose, lk_purpose_source — see Purposes

Available providers

The live provider and model list depends on your workspace connections and plan. Open Connections and Routes in the console to see which audio_input models can be assigned to your key.

Audio URL Safety

Remote audio.url input requires at least one allowed domain: either your workspace's own allowlist (up to 20 domains, managed by workspace admins via PUT /workspace/{tenant_id}/audio-url-domains) or the platform's global allowlist. See Large audio: use audio.url for the full requirements. The platform SSRF protections (HTTPS, public-IP-only resolution, redirect caps, declared Content-Length) always apply and cannot be relaxed by allowlist entries.

Managed Billing

Platform Access audio availability depends on the selected model, plan, and supported usage metering. BYOK uses your provider account. Check the route and estimated cost in the console before sending production audio workloads.