Tool Calling

Define tools on /v1/chat, receive normalized tool calls, and return tool results.

LatentKit supports OpenAI-style tool (function) calling on POST /v1/chat. You define tools in the request; the routed model decides when to call them; LatentKit normalizes tool calls to one shape regardless of which provider executed — so the same handling code works whether the route lands on OpenAI, Anthropic, Gemini, or another tool-capable model.

Capability requirement

Any request that includes tools, a non-none tool_choice, or tool/function role messages requires the function_calling capability. Only route models with that capability are eligible, and routing fails with NO_HEALTHY_PROVIDER if none exist.

1. Define tools

curl https://ai.latentkit.com/v1/chat \
  -H "Authorization: Bearer $LATENTKIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{ "role": "user", "content": "What is the weather in Dubai?" }],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a city",
          "parameters": {
            "type": "object",
            "properties": {
              "city": { "type": "string" }
            },
            "required": ["city"]
          }
        }
      }
    ],
    "tool_choice": "auto"
  }'

Field	Description
`tools`	Array of OpenAI-style function definitions with JSON Schema parameters
`tool_choice`	`auto` (model decides), `none`, `required`, or a specific function selector
`parallel_tool_calls`	Allow or disallow multiple tool calls in one turn, when the provider supports it

2. Read normalized tool calls

When the model calls a tool, the response contains tool_calls in OpenAI function-call shape:

{
  "content": "",
  "tool_calls": [
    {
      "id": "call_abc123",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\": \"Dubai\"}"
      }
    }
  ]
}

function.arguments is a JSON string. Parse it, validate it against your schema, allowlist the function name, and apply your normal user authorization before executing anything. Treat all model-produced arguments as untrusted.

3. Return tool results

Execute the tool in your backend, then continue the conversation with a tool role message referencing the call ID:

{
  "messages": [
    { "role": "user", "content": "What is the weather in Dubai?" },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        { "id": "call_abc123", "type": "function",
          "function": { "name": "get_weather", "arguments": "{\"city\": \"Dubai\"}" } }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_abc123",
      "content": "{\"temp_c\": 41, \"condition\": \"sunny\"}"
    }
  ],
  "tools": [ ... ]
}

Repeat until the model returns a plain text answer instead of more tool calls.

SDK example

const response = await client.chat.create({
  messages,
  tools,
  tool_choice: 'auto',
});

for (const call of response.tool_calls ?? []) {
  const args = JSON.parse(call.function.arguments);
  // run your tool, append the result as a `tool` role message, call again
}

The Python SDK accepts the same tools and tool_choice keyword arguments on client.chat.create(...).

When the model refuses to call a tool

If your application requires a tool call (for example tool_choice: "required") but the model returns plain text, LatentKit returns the typed error TOOL_CALL_NOT_PRODUCED (category: model_output, retryable). Recommended handling:

Retry once — possibly with a stricter tool_choice or larger max_tokens.
If retries keep failing, revise the prompt or tool descriptions; repeated failures usually mean the model cannot map the request to your schema.

See Error reference for the full model_output category guidance.

Routing note

Tool-heavy workloads may require credits or BYOK even when plain chat is within a Free managed allowance, and route changes can shift which provider executes. If tool behavior must stay stable, pin the preferred model in the routing policy.

On this page