Validate everything crossing the LLM/tool boundary, not just user input

The problem

When an LLM decides to call a tool (run a query, hit an API, write a file, shell out), it's common to pass the model's generated arguments straight through, on the assumption that "the model generated it, so it's safe." That assumption fails in two ways: the model can be manipulated by injected instructions (in a document it read, a web page it fetched, a prior tool result) into generating malicious arguments, and the model can simply hallucinate or malform an argument that happens to be dangerous when interpreted literally — an unescaped path, an unparameterized SQL fragment, a shell metacharacter.

Model output is not a trusted internal value. It is untrusted input arriving from a probabilistic process that can be steered by anything it has read.

The methodology

1. Treat the LLM's tool-call arguments as crossing a trust boundary, exactly like a network request or user form submission. Validate type, shape, and allowed values before use — don't rely on the tool's own JSON schema description as enforcement, since the model can still emit something the schema didn't prevent.
2. Never string-concatenate model output into SQL, shell commands, or file paths. Use parameterized queries, an allowlisted set of shell subcommands with argument arrays (not a shell string), and path resolution that rejects traversal (../) and confines writes to an allowed directory.
3. Allowlist over denylist for anything executable. If a tool call can select an action (a command to run, an endpoint to hit), prefer an enum of pre-approved actions over free-text that gets interpreted.
4. Sanitize before the second hop. If a tool's output (e.g. a scraped web page, a file read) is fed back into the model or into another tool, sanitize it the same way — this is where injected instructions most often ride in, disguised as "content."
5. Cap and rate-limit anything with real-world side effects that an LLM can trigger autonomously (sending messages, spending money, deleting data) — a validation bug should be able to fail loudly, not fail expensively.
6. Log the actual arguments used, not just the model's stated intent, so an incident can be traced to what the tool actually received and did.

Why this holds up project-agnostically

This is the same "validate at the trust boundary" principle from classical web security (never trust client input), applied to a new boundary: the point where a language model's output becomes a program's input. It applies regardless of model vendor, agent framework, or programming language — any system where an LLM's generated text results in a tool call, query, or command needs this same boundary check.

When this doesn't apply / limits

• Validation reduces the mechanism of injection/misuse; it doesn't prevent a well-formed but semantically wrong action (e.g. a validly-typed but incorrect file path within the allowed directory). Pair it with the scoping in Scope your coding agent to least privilege so the blast radius of a "valid but wrong" action stays small.
• Extremely open-ended tools (a general shell, arbitrary code execution) are hard to fully allowlist — for those, lean harder on sandboxing and confirmation gates rather than input validation alone.