How to Write a System Prompt That Holds in Production

A model that keeps going off-script usually doesn't have a model problem. It has a system prompt problem. The instruction that was supposed to frame every answer is either missing, vague, or buried in the user message where it competes with the actual request. Learning how to write a system prompt is mostly learning to move the standing rules out of the chat and into the frame, where the model treats them as policy instead of one more request.

Most guides that rank for this either run thin or run tool-flavored. The system-prompt walkthrough on dev.to covers the principles well in about 800 words but hands you no reusable template with variables. The longer guide on saharaai.com gives toy examples, then pivots to its own agent builder, and never says how the three major models differ on the same instruction. So here's the gap-filler: a five-part structure, a copyable template with named slots, and a table of where each model needs a nudge.

What a system prompt is, precisely

A system prompt is the standing instruction that frames every turn, passed separately from the user's message. It's not the request. It's the policy the request is answered under: who the model is, what it may and may not do, which tools it can call, and what shape its answers take.

Here's what a working system prompt earns its place doing:

• Setting a role specific enough to change behavior, not just decorate it
• Stating the rules the model must hold across every turn, not just the current one
• Declaring which tools exist and when to use them, so the model stops inventing capabilities
• Locking the output format once, so every response comes back the same shape
• Spelling out precedence, so the model knows system rules beat a conflicting user ask
• Handling the edge cases up front instead of discovering them in production

The dividing line is simple. If an instruction should apply to every answer, it's a system instruction. If it applies to one request, it's a user prompt. People blur the two and then wonder why "please always cite sources" gets forgotten three turns later.

The five-part structure

Effective system prompts share a skeleton. Identity, rules, tools, format, edge cases. You adapt it, you don't copy it blindly, but the parts stay the same.

System prompt structure

  1. Identity   - You are a {{role}} for {{domain}}. You help {{audience}}.
  2. Rules      - Do X. Do Y. (positive instructions, one per line)
  3. Tools      - Available: {{tool_list}}. Use {{tool}} only when {{condition}}.
  4. Format     - Every answer: {{output_shape}}. State this last.
  5. Edge cases - If {{missing_input}}, ask for it. Never invent {{sensitive_field}}.

Two details do the heavy lifting. The identity has to constrain something. "You are a helpful assistant" changes no behavior, so it's not identity, it's filler. "You review commercial contracts and cite the exact clause number for every claim" actually narrows what the model does. And the format rule goes last, because the model weights its most recent tokens most heavily when it starts generating.

Step-by-step: writing one that holds

1. Write an identity that constrains

Name the role, the domain, and the audience in one or two lines. The test: read it and ask whether it rules anything out. If it doesn't, it's decoration. A good identity makes some responses obviously wrong, which is the point.

2. State rules as things to do

Prefer "Cite the clause number" over "Don't make unsupported claims." Positive instructions give the model an action to take; prohibitions make it invert a rule first, which it does less reliably. When a boundary is genuinely a "never," state it once and plainly, then move on. Repeating a prohibition five times doesn't make it more obeyed.

3. Declare the tools explicitly

If the model can call tools, list them, and say when each applies. An undeclared tool gets either ignored or hallucinated. A declared one with a usage condition ("search only when the answer depends on current data") gets used when it should be and left alone otherwise.

4. Lock the format on the last line

Whatever shape you need, state it after everything else. This placement matters more than any wording tweak. A format rule at the top of a long system prompt gets diluted by the time the model generates; on the last line, it survives. Test it by moving a drifting rule to the end and watching the drift drop.

5. Handle edge cases before they happen

List the inputs that might be missing and say what to do with each. "If the user gives no date, ask for one, don't assume today." This is where most production failures actually live, and it's the section thin guides skip entirely.

How the three models treat system prompts

The skeleton carries across Claude, ChatGPT, and Gemini. The placement and phrasing don't. A system prompt tuned on one can behave differently on another.

The rule that helps all three: spell out precedence. State that system instructions outrank user requests when they conflict (system > developer > user > retrieved content). Without it, a model faced with a user message that contradicts the system prompt sometimes sides with the more recent user text. Anthropic and OpenAI both document versions of this ordering; making it explicit removes the ambiguity.

Prompt-craft patterns for system prompts

Separate the constant from the variable

The identity, rules, and format are constant. The specific request is variable. Keep them in different places, the standing parts in the system prompt, the per-request parts in the user turn. Mixing them is how a rule meant for every answer gets treated as a one-time ask.

One concern per rule line

Don't stack tone, length, and format into one sentence. "Be friendly, formal, and technical" pulls three directions and the model splits the difference badly. One rule per line, each doing one thing, gets followed. Conflicting rules crammed together get averaged into mush.

Version the frame, not the message

When behavior needs fixing, change the system prompt and every future turn inherits the fix. Patching individual user messages fixes one conversation and forgets the lesson. A system prompt is the place a fix compounds.

Variables you'll set

The {{role}} slot is where most system prompts go wrong. Fill it with a constraint, not a compliment, and the rest of the prompt gets easier to write.

Getting started

1. List every "always do this" instruction you keep repeating in chats.
2. Move them into a frame with the five parts: identity, rules, tools, format, edge cases.
3. Rewrite prohibitions as positive actions wherever you can.
4. Put the output format on the last line and state precedence explicitly.
5. Name the inputs that might be missing and say what to do with each.
6. Test on a conversation that tries to override the rules, not just a cooperative one.
7. When behavior drifts, fix the system prompt, not the individual message.

For a production-grade version with the guardrails and precedence already written, the Agent Guardrails Policy System Prompt ships the frame built for agent work.

A system prompt is the frame; what you feed it still matters. For assembling the inputs a framed model needs, context engineering for LLMs covers the ordering that keeps the format rule from getting buried. And when the system prompt drives subagents, Claude Code subagents and their system prompts shows the same structure applied to delegated work. Pair either with the Code Review Policy System Prompt when the standing rules govern a review, not a chat.