Tag: ai

The incident report blamed the model. “The agent called the delete endpoint twice and wiped a partition it shouldn’t have touched.” Everyone nodded, someone filed a ticket to “upgrade to a smarter model,” and the actual cause sat there in plain sight: the delete tool had no idempotency key, no confirmation gate, and a description that said only “deletes records.” The model did exactly what the interface let it do. A better model would have done the same thing faster.

This is the pattern almost nobody names correctly. Most agent failures look like model mistakes — wrong tool, bad arguments, mishandled errors — but the model is only ever reasoning from the interface you gave it: the tool name, its description, the parameter schema, and the parameter descriptions. When that interface is vague, loosely typed, or missing its guardrails, failures stop being accidents and become predictable. You can throw a stronger model at a bad tool surface and it will still fail, just with more confidence. This is the field guide to designing the tool surface itself — five patterns that work, five that break under real workloads, each paired with its opposite so you can see why it fails, not just what to replace it with.

If you’re still deciding whether a given capability should even be a tool versus a full subagent, start with our companion piece on tools vs subagents — this article assumes you’ve decided it’s a tool and focuses on designing that tool well.

TL;DR

→ Most agent failures are tool-design failures, not model failures. The model reasons only from the interface: name, description, schema, parameter docs. Fix the interface and the “model mistakes” largely disappear.

→ One tool, one responsibility. A tool that switches behavior on an action parameter forces the model to pick a mode before it can solve the task. Split it into single-purpose tools with unambiguous names.

→ Tight schemas make invalid states impossible. Enums, validators, and typed fields encode constraints so the model doesn’t guess. Validation fails at the tool boundary instead of as a cryptic downstream error.

→ Descriptions define scope, not just purpose — they say when to use the tool and when not to. Without the “do NOT use this for…” boundary, the model infers scope from the name and picks wrong at scale.

→ Structured error returns (error_code, recoverable, suggested_action) give the model something to branch on. A raw stack trace gives it noise to hallucinate against.

→ The failure modes that pass demos and break in production: thin wrappers over raw APIs, loading every tool into every context, silent partial success, overlapping tool names, and single-call destructive actions.

Why tool design — not model capability — is the root cause

A model can only reason from what the tool interface exposes. That’s the entire premise, and it’s worth sitting with because it inverts how most teams debug. When an agent picks the wrong tool, the instinct is to blame the model’s judgment. But the model’s judgment is a function of the tool name, the description, the parameter schema, and the parameter descriptions — nothing else. If two tools have near-identical descriptions, the model isn’t being dumb when it confuses them; it’s being given no basis to tell them apart.

Anthropic’s engineering team makes this point directly in their guide on writing effective tools for agents: the tool interface is model-facing documentation, and its clarity determines the agent’s reliability more than raw model horsepower does. Stronger models reduce some mistakes, but they cannot reliably compensate for a flawed interface. That framing matters for data teams especially, because the tools we hand agents — warehouse queries, pipeline triggers, catalog lookups — often started life as internal APIs never designed for a reasoning model to consume.

What works, pattern by pattern

1. One tool, one responsibility

A tool that multiplexes behavior through an action parameter makes the model choose a mode before it can act. Single-purpose tools remove that whole layer of ambiguity.

A tool should represent a single, clear operation. When one tool handles create, get, update, delete, and suspend through an action parameter, the model has to figure out which mode to invoke before it can reason about the actual task. That’s a second decision you’ve forced into every call.

# Avoid: action-based multi-behavior tool
@tool
def manage_customer(action: str, customer_id: str | None = None,
                    data: dict | None = None):
    """action: create | get | update | delete | suspend"""
    ...

# Prefer: single-responsibility tools
@tool
def create_customer(data: CustomerInput) -> Customer:
    """Create a new customer record."""
    ...

@tool
def suspend_customer(customer_id: str, reason: str) -> SuspensionResult:
    """Suspend a customer account."""
    ...

Single-responsibility tools give the model an unambiguous function and give you cleaner error handling and easier observability — the same reasoning behind the audit-per-operation approach in our ETL audit logger guide, where one clear operation per unit makes debugging tractable. One caveat: this is a strong default, not a universal law. Some domains — shell, filesystem, browser, calendar — legitimately benefit from a constrained multi-action interface because the action space is part of the abstraction itself.

2. Schemas that make invalid states impossible

The model constructs tool-call arguments by reasoning from your schema. A loose schema means it guesses at constraints; a tight schema encodes them so no guessing is required. This is where Pydantic models with enums and field validators earn their keep:

from pydantic import BaseModel, Field
from enum import Enum

class Priority(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"

class CreateTaskInput(BaseModel):
    title: str = Field(
        description="Short, actionable title. Imperative: 'Review PR', not 'PR Review'.",
        min_length=5, max_length=100)
    priority: Priority = Field(
        description="Use HIGH only for blockers affecting other work.",
        default=Priority.MEDIUM)
    due_date: str = Field(
        description="ISO 8601 date: YYYY-MM-DD. Must be a future date.",
        pattern=r"^\d{4}-\d{2}-\d{2}$")

Enums are especially valuable for small sets of valid values — they eliminate an entire class of plausible-but-invalid outputs. And validation failures surface right at the tool boundary rather than as a confusing error three steps downstream in your pipeline.

3. Descriptions that define scope, not just purpose

Tool descriptions are model-facing documentation, and they need to do two things: explain when to use the tool and when not to. Most descriptions only do the first, which leaves the model inferring scope from the tool name — a reliable source of selection errors at scale.

# Weak: says what it does, not when NOT to use it
"""Search for documents in the knowledge base."""

# Strong: purpose, scope, and boundaries
"""
Search the internal knowledge base for policies and reference material.
Use when the user asks about company procedures, product specs, or documented workflows.
Do NOT use for real-time data (prices, availability, current status) — use get_live_data().
Returns up to 5 results ranked by relevance. No results means it's not in the knowledge base.

The “do NOT use this for…” line is the one most teams omit and the one that most improves selection accuracy. A good tool definition draws its boundaries relative to the outcome, not relative to other tools — which is also the cleanest way to avoid the overlap problem covered below.

4. Structured, actionable error returns

When a tool fails, the model reads the error and decides what to do next. An unhandled exception produces noise-driven behavior; a structured error gives the model something concrete to branch on:

class ToolError(BaseModel):
    error_code: str        # machine-readable, for the model to branch on
    message: str           # human-readable description
    recoverable: bool      # can the agent retry?
    suggested_action: str  # what the agent should do next

return ToolError(
    error_code="RECORD_NOT_FOUND",
    message="No user record found with ID 'usr_123'.",
    recoverable=True,
    suggested_action="Call list_users() to get valid IDs before retrying.")

The recoverable flag and suggested_action field are what actually change agent behavior. Without them, models retry non-retryable errors — burning tokens and warehouse credits — or abandon recoverable ones. This matters doubly when the tool touches sensitive systems; see our guide on giving agents access to pipeline metadata safely for how structured returns keep a compromised agent’s blast radius small.

5. Idempotent state-changing operations

Every tool that mutates state — creates a record, sends a message, triggers a pipeline run — must be safe to call twice, because agents retry, networks fail, and the reasoning loop may fire a second call when confirmation of the first never arrived. The simplest defense is an idempotency key on every write:

@tool
def send_email(to: str, subject: str, body: str,
    idempotency_key: str = Field(
        description="Unique key for this send. Hash of recipient+subject+timestamp. "
                    "Same key on retry returns the original result without re-sending.")
) -> dict:
    """Send an email. Idempotent: same key will not trigger a second send."""
    existing = idempotency_store.get(idempotency_key)
    if existing:
        return existing
    result = email_service.send(to=to, subject=subject, body=body)
    idempotency_store.set(idempotency_key, result, ttl=86400)
    return result

Without idempotency, a transient failure quietly becomes a duplicate action — a doubled Slack alert, a re-run backfill, a second funds transfer.

What doesn’t work

1. Thin wrappers around unfiltered APIs

Pointing an agent at a REST API and exposing it raw is the most common shortcut and the most common source of production failures. As Anthropic’s tool-writing guide notes, APIs built for developers expose far more than an agent needs: responses packed with hundreds of fields, pagination, opaque internal IDs, and error codes that require domain knowledge to interpret. A purpose-built wrapper handles pagination internally, projects only the fields the agent needs, and maps API errors to the structured ToolError format above. The flip side: over-wrapping into dozens of hyper-narrow tools fragments the surface. The goal is a consistent, agent-friendly abstraction — not maximal abstraction.

2. Loading all tools into every context

Agent accuracy drops as the tool catalog grows. Loading only the tools relevant to the current step keeps the decision space small and the token budget lean.

Accuracy degrades as the tool catalog grows. LongFuncEval, a 2025 study on tool-calling across long contexts, found performance drops substantially as the tool catalog grows — even in models with 128K context windows. Loading every tool into every system prompt compounds it by eating token budget before any task content is processed. The fix is dynamic loading: determine which tools are relevant to the current step and include only those.

STEP_TOOL_MAP = {
    "research": ["search_documents", "search_web", "get_url_content"],
    "write":    ["create_document", "update_document", "format_text"],
    "send":     ["send_email", "post_to_slack", "create_calendar_event"],
}

def get_tools_for_step(step_type: str, available_tools: list) -> list:
    relevant = STEP_TOOL_MAP.get(step_type, [])
    return [t for t in available_tools if t.name in relevant]

This is the tool-level analogue of scoping a subagent’s tools, one of the justifications for reaching for a subagent at all in our tools vs subagents guide.

3. Silent partial success

Partial success becomes a bug when a tool completes only part of the work but returns something that looks fully successful, so the agent proceeds with a misleading view of system state. It usually happens when a tool swallows internal failures:

# Silently misleads the agent
@tool
def bulk_create_tasks(tasks: list) -> dict:
    created = []
    for task in tasks:
        try:
            created.append(task_api.create(task).id)
        except Exception:
            pass  # silent failure: this is the bug
    return {"created": created}

# Makes partial success explicit
@tool
def bulk_create_tasks(tasks: list) -> BulkCreateResult:
    created, failed = [], []
    for task in tasks:
        try:
            created.append(task_api.create(task).id)
        except TaskCreationError as e:
            failed.append({"input": task.title, "reason": str(e)})
    return BulkCreateResult(
        created_ids=created, failed_items=failed,
        success=len(failed) == 0,
        partial_success=len(created) > 0 and len(failed) > 0)

The partial_success flag gives the model a branch: retry the failed items, surface the partial result, or halt. Silent swallowing gives it a false green light.

4. Overlapping tool names and descriptions

When two tools do similar things, the model reasons about which to use on every single call — burning tokens and introducing errors. Classic offenders: search_documents and find_documents with identical purpose; get_user and fetch_user_profile with unclear difference; create_task, add_task, and new_task for one operation. Renaming alone isn’t the fix. Every tool needs a purpose describable without reference to the others — if a description needs “unlike X, this one…” to make sense, that’s a design problem. This is the same governance discipline we apply in governing agentic workflows: a tool surface audited before deployment, not after an incident.

5. Destructive actions without a confirmation gate

An irreversible action should never be completable in one reasoning step. Staging plus a short-lived confirmation token forces a deliberate second call.

Any tool that takes an irreversible action — deleting records, messaging real users, executing transactions — needs a structural two-step confirmation, not an in-prompt “are you sure?” Separate staging from execution and require a short-lived token between them:

@tool
def stage_deletion(record_ids: list[str], reason: str) -> StagedDeletion:
    """Stage records for deletion. Does NOT delete anything.
    Returns a confirmation token that expires in 60 seconds."""
    token = generate_deletion_token(record_ids)
    staged_deletions[token] = {"ids": record_ids, "expires": now() + 60}
    return StagedDeletion(token=token, records_to_delete=len(record_ids),
                          expires_in_seconds=60)

@tool
def confirm_deletion(token: str) -> DeletionResult:
    """Execute a staged deletion. IRREVERSIBLE. Confirm only after user approval."""
    staged = staged_deletions.get(token)
    if not staged or staged["expires"] < now():
        raise ValueError("Token invalid or expired. Stage the deletion again.")
    # proceed

Two distinct calls mean the model can’t complete a destructive operation in a single reasoning step — that’s the point. One caution: two-step flows aren’t sufficient on their own. Production systems also need single-use tokens, strict session binding, and replay protection so a token can’t be reused or executed across sessions.

The decisions at a glance

Every one of these is a design decision you make explicitly or make by accident: tool scope (single responsibility, not an action parameter); schema (tight enums and validators, not free strings); descriptions (scope boundaries and when-not-to-use, not happy path only); write operations (idempotent with keys, not fire-and-forget); error returns (structured with error_code/recoverable/suggested_action, not raw exceptions); tool count (dynamic per-step loading, not all tools in every context); API wrapping (purpose-built agent-facing schema, not unfiltered exposure); partial success (an explicit flag, not silent swallowing); destructive actions (two-step staging, not single-call); and tool overlap (semantically distinct and audited, not similar names competing).

The one principle

The agent reasons only from the interface you expose, so most “model failures” are design failures you can fix at the tool boundary. Give each tool one responsibility, a schema tight enough to make invalid calls impossible, a description that says when not to use it, structured errors it can branch on, and a confirmation gate on anything irreversible. Do that and a mid-tier model behaves reliably. Skip it and the smartest model available will still delete the wrong partition — confidently, and on the first try.

Related reading: Anthropic: Writing effective tools for agents · LongFuncEval: tool-calling in long contexts · Pydantic documentation · Tools vs Subagents: When to Use Each · Giving Agents Safe Access to Pipeline Metadata · Governing the AI Agent: CoCo + MCP Security