A support agent I audited last year issued a $4,300 refund to a customer who had already been refunded twice. Its system prompt was 40 words long: “You are a friendly support assistant. Be helpful, resolve customer issues, and keep the customer happy.” Every one of those words was a small liability. “Helpful” gave the agent permission to act. “Keep the customer happy” gave it a reason to ignore the refund policy sitting in its retrieval store.
That story captures most of what goes wrong with ai agent system prompts. They get read on every single turn, they outrank almost everything else in the context window, and they are usually written in five minutes by whoever created the account.
What a system prompt controls that a user message doesn’t
A system prompt is the only instruction that survives the whole conversation. User messages arrive once and get buried. Tool results pile up. Context windows get trimmed. The system prompt sits at the top of every request, quietly shaping how the agent reads everything below it.
That persistence is the point, and it’s also the danger. A badly worded user message costs you one bad reply. A badly worded system prompt costs you every reply for as long as it’s deployed. I’ve seen a single ambiguous line (“use your judgement on discounts”) produce a 12% margin hit across a quarter before anyone traced it back.
Three things belong in that persistent layer: identity, authority, and boundaries. Personality is optional. Most agents would be better off without it.
Write the failure list before you write the prompt
Before drafting anything, spend twenty minutes listing ten ways the agent could embarrass you. Not abstract risks. Specific ones: quotes a price from memory instead of the pricing tool, sends an email before the draft is approved, keeps retrying a failed API call until it burns $40 in tokens, agrees with a customer who claims a policy that doesn’t exist.
Now write the prompt to prevent those ten things. This flips the usual approach, where people start with a personality sketch and patch holes after launch. The failure list also gives you something most teams skip entirely: a test set. Every item on it becomes a case you run against each new version.
The five blocks worth including in every agent prompt
A workable system prompt has a predictable shape. It doesn’t need to be elegant, but it does need these parts:
- Identity and authority. Who the agent is, who it answers to, and what it may decide alone. “You are a billing assistant for a B2B SaaS company. You may discuss invoices, payment methods, and dunning timelines. You may not discuss contract terms or renewal pricing.”
- Tool contracts. Which tool to call for which question, when not to call it, and what to do with the result. Name the tools exactly as they appear in your schema.
- Hard constraints. The non-negotiables, phrased as prohibitions. “Never state a dollar figure that did not come from the billing tool in this conversation.”
- Stop conditions. When to hand off to a human, when to ask a clarifying question, and when to say “I don’t know.” Agents default to forging ahead, so this section earns its tokens.
- Output shape. Length, format, tone, and what to do when the requested format doesn’t fit the answer.
Order matters. Put identity first, constraints second, and repeated critical rules near the end. Models weight the opening and closing of a long prompt more heavily than the middle, so a refund rule buried in paragraph nine is functionally weaker than the same rule in line two.
Constraints beat adjectives every time
“Be concise” tells the model nothing measurable. “Answer in three sentences or fewer unless the user asks for a walkthrough” is a rule it can follow and you can check. The same applies across the board:
- “Be careful with money” becomes “refunds above $200 require a human approval step before the tool call.”
- “Stay on topic” becomes “if the question is outside billing, reply with one sentence and offer the support handoff.”
- “Don’t hallucinate” becomes “if a policy detail is not in the retrieved documents, say so and escalate.”
Numbers, named tools, and explicit fallbacks do the work that adjectives only gesture at. If you find yourself writing “always” or “never,” that’s usually a good sign, as long as the clause that follows is concrete.
Tool descriptions do half the work
Teams spend hours polishing the instruction text and thirty seconds on the tool definitions, then wonder why the agent calls the wrong endpoint. The model picks tools by reading their names and descriptions, and a vague description like “handles customer data” invites creative interpretation.
Write tool descriptions the way you’d write a good docstring: what it does, what it needs, what it returns, and one example of a valid call. Where the wrong call is expensive or irreversible, say so in the description itself. This is the same distinction between a reliable agent and a confident liar that shows up in any serious look at AI agent prompt engineering, and it rarely gets fixed by adding more paragraphs of instructions.
Decide up front what the agent does when it’s stuck
Every agent hits a wall eventually: a tool times out, a search returns nothing useful, a user gives contradictory information. Without guidance, most models will try again in slightly different wording, then again, then hallucinate a plausible answer.
Give it an explicit ladder. One retry with a different query, then a fallback path, then a handoff. Cap the number of steps it can take without a human checkpoint, and tell it what to say when it gives up. If you haven’t thought about this yet, the failure loop described in how to build an AI agent from scoping to tool selection is worth reading before your first deploy rather than after.
Keep it long enough to be complete, short enough to be read
Production prompts I’ve reviewed cluster between 400 and 1,500 tokens. Below 400, you’re usually missing stop conditions and tool rules. Above 1,500, you’re often restating the same constraint in four ways, and the model starts treating all rules as equally optional.
If the prompt keeps growing, the extra bulk usually belongs somewhere else. Company background goes in retrieval. Long formatting examples go in a tool or a template. Product catalogues go in a lookup. The system prompt should hold decisions and boundaries, not reference material.
Version and test prompts like code
A prompt change is a deployment. Treat it that way. Keep each version in source control with a one-line note on why it changed, and run your failure list against the new version before it reaches users. Two or three real transcripts from last week’s logs make a decent regression suite on their own.
If you’re building this in code rather than a no-code canvas, the plumbing is straightforward: prompts as files or constants, a test harness that replays recorded conversations, and a diff view before merge. The patterns in building AI agents with JavaScript for production cover that setup in more depth than most teams bother with, which is exactly why it’s useful.
Don’t copy a prompt between platforms
The same 900-word prompt behaves differently on different stacks. Some frameworks inject their own preamble, some wrap tool outputs in XML, some truncate aggressively, and the reasoning models will reinterpret instructions that a faster model follows literally. A prompt tuned on one platform can look broken on another without a single word changing.
Before you port anything, check how each platform handles instruction hierarchy and context trimming. The differences that matter here are the ones covered in this AI agent platform comparison, and they’re worth checking before you carry a prompt across rather than after.
The weekly review that catches drift
Prompts don’t decay, but the world around them does. A pricing page changes. A tool gets renamed. A policy tightens after a legal review. The agent keeps following instructions that are now subtly wrong, and nothing in your monitoring flags it because the outputs still look confident.
Book thirty minutes a week to read five real transcripts, chosen at random rather than by error rate. Look for the agent doing something technically permitted and practically unwise. When you find one, resist the urge to add a paragraph. Ask whether a constraint, a tool description, or a missing stop condition caused it, then fix that specific thing and re-run your failure list.
Two or three cycles of this and the system prompt stops being a document you wrote once. It becomes a record of every mistake your agent has made and been taught not to repeat, which is far more useful than anything you could have drafted on day one.