Queue (Async Requests)

POST /v1/queue — enqueue background jobs for any runtime endpoint with idempotency support.

POST /v1/queue enqueues a request for background execution instead of waiting for a synchronous response. Use it for batch work, long-running generations, and fire-and-forget tasks that should not block your request path.

The public API does not currently provide a per-job result endpoint. Use a synchronous endpoint when your application needs the generated response.

Request

Wrap any runtime endpoint's normal body in a queue envelope:

curl https://ai.latentkit.com/v1/queue \
  -H "Authorization: Bearer $LATENTKIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "chat",
    "mode": "priority",
    "payload": {
      "messages": [{ "role": "user", "content": "Summarize this document..." }],
      "max_tokens": 500
    },
    "idempotency_key": "report-2026-06-12-tenant-42"
  }'

Fields

Field	Description
`endpoint`	Which runtime surface processes the job: `chat`, `complete`, `vision`, `embed`, `embeddings`, `image`, `speech`, `transcription`, `translation`, `audio`, or `video` (required)
`payload`	The same JSON body you would send to that endpoint directly (required)
`mode`	Queue mode; defaults to `priority`
`idempotency_key`	Optional deduplication key (max 256 characters) — see below

You can also send the idempotency key as an Idempotency-Key request header instead of a body field.

Response

{
  "queued": true,
  "job": {
    "id": "job_abc123",
    "endpoint": "chat",
    "mode": "priority",
    "status": "queued"
  }
}

The job executes server-side through the same routing, billing, and logging path as synchronous requests. Job activity, success, and failures appear in the console under Usage / request logs, and queue backlog state is visible on the workspace dashboard.

Delivery and retries

Queued work uses at-least-once delivery and may run again after an interruption. Keep any downstream side effects idempotent. Oversized queue requests are rejected before enqueueing.

Idempotency

When an idempotency_key is provided, LatentKit deduplicates identical enqueues during the service retention window. A duplicate enqueue returns the original job ID instead of creating another job.

This makes retry loops safe:

await client.queue.create({
  endpoint: 'chat',
  payload: { messages: [{ role: 'user', content: 'Nightly summary' }] },
  idempotencyKey: 'nightly-summary-2026-07-15',
});

client.queue.create(
    endpoint="chat",
    payload={"messages": [{"role": "user", "content": "Nightly summary"}]},
    idempotency_key="nightly-summary-2026-07-15",
)

A changed endpoint or request body is treated as different work. Use one stable key for each logical enqueue operation.

Async transcription

Audio transcription has a dedicated async surface: POST /v1/transcription/jobs. See Audio and STT for details and current rollout status.

Video — the most common long-running candidate
Audio and STT
Rate limits