Anthropic CCDV-F: Building Claude Tools Safely
Tool use changes an AI application from a system that proposes actions into one that can cause them. A model might query a database, create a ticket, send a message, modify a repository, or trigger an operational workflow. That power is useful only when the application treats every tool call as a request crossing a security boundary rather than as trusted code emitted by the model.
Within Claude Development, safe tool design starts with a simple contract: the model selects a tool and supplies structured input, while your application or an Anthropic-hosted tool runtime performs the operation. The tool implementation remains responsible for authorization, validation, side effects, logging, and recovery.
Design a narrow contract before a clever prompt
A tool should do one understandable job with an input schema that expresses the real constraints. `delete_customer_data(customer_id)` is safer to reason about than a generic `run_admin_action(action, payload)` interface. Narrow tools reduce ambiguity for the model and make authorization, testing, and auditing easier for the application team.
Descriptions should explain when to use the tool, what it returns, and what it must not be used for. If two tools overlap, the descriptions should make the boundary explicit. Anthropic’s current tool-use guidance emphasizes that the model chooses among the tools you expose; unclear descriptions therefore become an application-design problem, not merely a prompting problem.
Use strict schema validation where the supported interface allows it, but do not confuse schema validity with business validity. A syntactically correct account ID can still identify an account the user is not allowed to access. JSON Schema protects shape; your application must still protect meaning and authority.
Keep authorization outside the model
The model can help decide what action appears useful, but it should not decide whether the caller is authorized. Bind tool execution to the authenticated user, workspace, tenant, or service identity in trusted application code. Resolve permissions at execution time so a prompt cannot grant itself access by merely claiming a role.
Apply least privilege to service credentials as well. A read-only search tool should not run with a token that can delete records. A repository inspection tool should not automatically inherit production deployment rights. Smaller permission envelopes reduce the consequence of prompt injection, tool-selection mistakes, and implementation bugs.
For agents that operate across many systems, the broader Claude operations layer should document credential boundaries, audit requirements, and approval policy. Tool safety becomes much harder when every connector silently uses the same powerful service account.
Validate inputs at the tool boundary
Treat model-generated arguments like any other untrusted input. Validate identifiers, enum values, sizes, paths, URLs, date ranges, and object ownership. Normalize data before use and reject ambiguous or oversized requests. If a tool writes files, enforce an allowed root and block traversal. If it makes network calls, restrict destinations instead of accepting arbitrary URLs.
Tool outputs need validation too. External systems can return malformed, stale, or adversarial content. Separate data from instructions when possible, cap response size, and strip secrets that the model does not need. A tool result should contain enough context to make the next decision, not an uncontrolled dump of an internal system.
The same principle appears in security automation: automation should make trusted boundaries more explicit, not bypass them. When a workflow becomes agentic, the need for input and output validation increases because the sequence can adapt dynamically.
Make side effects deliberate and recoverable
Read operations are easier to automate than writes because they are naturally lower impact. For write tools, decide whether the action is idempotent, reversible, and safe to retry. Creating the same deployment twice, charging a card twice, or sending duplicate notifications are very different failure modes from repeating a search query.
Use idempotency keys or application-level deduplication for operations that might be retried. Return stable identifiers so the model can refer to the action that actually happened. When a tool starts a long-running job, prefer a start-and-check pattern over keeping a single call open indefinitely; it gives your application clearer timeout and recovery behavior.
For destructive or high-impact actions, require explicit approval or a staged plan. The model can prepare a change, show the exact target and effect, and wait for a human or policy engine before execution. Human-in-the-loop is most valuable at irreversible boundaries, not as a confirmation popup on every harmless read.
Handle failures as part of the protocol
A tool can fail because of validation, authorization, rate limits, timeouts, downstream outages, conflicts, or business rules. Return errors in a structured way that distinguishes retryable from permanent conditions. Anthropic’s tool protocol supports marking tool results as errors, allowing Claude to adapt rather than pretending the operation succeeded.
The companion guide on Claude API errors covers transport-level failures, but tool execution has its own failure domain. A 200 response from the Claude API can still contain a tool request whose downstream action fails. Keep those layers separate in logs so operators can tell whether the model service, the tool runtime, or the target system caused the incident.
Error messages should be instructive without leaking secrets. “Permission denied for project 123; request access or choose a project you can administer” is more useful than “failed,” while a raw database stack trace is too much. Give the model information that supports recovery, and keep sensitive diagnostics in your internal telemetry.
Defend against untrusted content steering tools
Prompt injection matters most when an agent reads content and can act. A document, web page, ticket, or repository file may contain text that tries to redirect the model. Your application should treat retrieved content as data, keep policy in trusted instructions, and restrict tools so an injected sentence cannot escalate privileges.
Separate discovery from execution for sensitive workflows. One tool can search or inspect; another can perform the change after validation and approval. This architecture creates a natural checkpoint where the application can compare the proposed action with user intent and policy before side effects occur.
Do not assume that hiding a tool name provides security. The enforcement point is the executor. Even if the model never sees a dangerous operation directly, a broad proxy tool can expose the same capability. Review what each tool can actually do with its credentials and network access.
Instrument tools like production APIs
Log the tool name, normalized inputs, caller identity, authorization result, latency, outcome, and correlation IDs. Redact secrets and personal data according to policy. Traces should make it possible to reconstruct why an action happened without requiring the original model response to be the only audit record.
Use metrics to find design problems. A high rate of invalid arguments may indicate a weak schema or description. Frequent retries may point to poor timeout choices. A tool that is called often but rarely changes the outcome may be adding context cost without value. Operational data should feed tool redesign, not just incident response.
The patterns in tool calling and the broader production engineering hub become safer when observability is part of the interface from day one. The objective is not to prevent Claude from acting; it is to make each action bounded, authorized, observable, and recoverable enough that an adaptive system can be trusted in production.
Design explicit trust zones between tools
An agent often crosses several trust zones in one workflow: user input, retrieved documents, internal databases, third-party APIs, and privileged administrative systems. Map those zones before connecting tools. A result from one system should not automatically become authorization for another, and data retrieved from an untrusted source should not be able to widen the next tool’s permissions.
Use separate credentials and executors when the trust levels differ materially. A public web lookup, an internal knowledge search, and a production change tool should not share the same network reach or secret store. Segmentation limits blast radius and makes audit trails easier to interpret because each executor has a purpose.
Trust-zone mapping also clarifies where sanitization belongs. Structured records may need field-level filtering; documents may need size and content controls; shell or database tools need strong parameterization. Security improves when the boundary is architectural rather than a prompt instruction.
Test tools independently from the model
Every tool should have ordinary unit and integration tests that prove validation, authorization, idempotency, timeout behavior, and error mapping without involving Claude. This keeps application correctness from depending on stochastic model behavior and lets the team reproduce failures quickly.
Then test the model-facing layer with representative tool schemas and synthetic results. Include ambiguous requests, invalid arguments, permission denials, partial downstream outages, and requests that should not call a tool at all. The goal is to verify both halves of the contract: the tool is safe when called, and the agent chooses it appropriately.
Production reviews should sample real tool traces as well. Look for unnecessary calls, repeated failures, oversized results, and actions that consistently need human correction. Those observations can improve descriptions, schemas, or workflow design before they become incidents.
Review the tool catalog periodically as well. Remove capabilities that no workflow uses, split broad tools that repeatedly require special-case authorization, and retire credentials that outlive their integration. A smaller, clearer capability surface improves both model selection and security because there are fewer ambiguous paths to the same side effect.