Guardrails for Claude applications are the controls that keep untrusted language from becoming untrusted behavior. They include input screening, hardened instruction hierarchy, data boundaries, structured outputs, tool authorization, content moderation, output checks, human approval, rate limits, and monitoring. No single prompt can provide all of these guarantees because the application\u2014not Claude\u2014owns credentials, APIs, customer data, and external effects.<\/p>\n
Anthropic’s current guardrail guidance distinguishes direct jailbreaks from indirect prompt injection. Direct attacks come from a user trying to bypass policy; indirect attacks arrive through content Claude is asked to process, such as webpages, documents, email, tool results, or retrieved knowledge. The defenses overlap, but indirect attacks require special care because the user may be trusted while the data is not.<\/p>\n
Guardrails are therefore an architecture layer inside Claude Production Engineering<\/a>.<\/p>\n A lightweight classifier or deterministic rule can reject known abuse patterns before expensive model or tool work begins.<\/p>\n Anthropic’s current guidance explicitly shows harmlessness screening with a smaller Claude model and structured output for a simple safe\/unsafe decision.<\/p>\n Screening should reduce obvious risk without becoming the only defense; attackers can phrase the same intent in many ways.<\/p>\n System policy, user requests, retrieved documents, and tool results should be clearly delimited so Claude can distinguish instructions from evidence.<\/p>\n Agent boundaries<\/a> are especially important when documents or web pages can contain text that attempts to override the application’s rules.<\/p>\n Retrieved content should be treated as data to analyze, not as a new source of execution authority.<\/p>\n Limit request size, file type, allowed URLs, tenant identifiers, numeric ranges, and structured fields before invocation.<\/p>\n Known injection patterns can be filtered where that provides value, but validation should focus on enforceable application constraints rather than trying to enumerate every adversarial sentence.<\/p>\n Malformed or unauthorized requests should fail before they consume Claude context or trigger tools.<\/p>\n When downstream software needs JSON, use the Claude API’s structured-output feature instead of relying only on a prompt that asks for JSON.<\/p>\n Reliable JSON<\/a> becomes possible through schema-constrained generation, while strict tool use<\/a> can constrain tool parameters separately.<\/p>\n Strong formatting reduces parser ambiguity but does not replace business validation of the values inside the schema.<\/p>\n Claude never executes a client tool by itself; it requests a tool and the application performs the operation.<\/p>\n Tool control<\/a> should verify caller identity, target, permission, parameter range, and current resource state before execution.<\/p>\n Tool descriptions guide selection, but IAM, API authorization, and domain rules are the actual security boundary.<\/p>\n Destructive, privileged, financial, or external actions can pause for a reviewer after the agent has prepared the exact payload.<\/p>\n Claude approval<\/a> should show meaningful consequences and use fresh authorization at execution time.<\/p>\n Do not use human review as a substitute for narrow tool scope; approvals work best when the proposed action is already technically constrained.<\/p>\n Content moderation, sensitive-data detection, allowlisted links, code scanning, or domain-specific validators can inspect the result before it reaches a user or another system.<\/p>\n Anthropic’s guidance on reducing prompt leak recommends monitoring and output screening before adding increasingly complex prompt-only defenses.<\/p>\n Output checks should target concrete risk such as secrets, prohibited data, or executable content rather than trying to \u201cprove\u201d the model is generally safe.<\/p>\n Current Claude models can return Applications should detect these states explicitly instead of parsing a refusal or truncated response as normal business output.<\/p>\n Fallback, retry, or user messaging should be defined according to the use case and Anthropic’s current refusal guidance.<\/p>\n Red-team prompts should attempt direct jailbreaks, indirect injection, tool misuse, sensitive-data extraction, path manipulation, and policy-conflicting requests.<\/p>\n Adversarial tests<\/a> should become release regressions once a meaningful weakness is found.<\/p>\nScreen obviously unsafe input early<\/h3>\n
Keep instructions separate from untrusted content<\/h3>\n
Validate input outside the model<\/h3>\n
Use structured outputs for machine boundaries<\/h3>\n
Put authorization around tools<\/h3>\n
Use human approval for high-impact work<\/h3>\n
Screen output where consequence requires it<\/h3>\n
Handle refusals and incomplete outputs<\/h3>\n
stop_reason: refusal<\/code> for safety-classifier decisions, and responses can also stop at max_tokens<\/code>.<\/p>\nContinuously test the guardrails<\/h3>\n