An Agent Self-Reflection Prompt That Actually Catches Errors

The reflection pattern promises a tidy story: the agent writes something, looks at it critically, fixes the flaws, and ships better work. Sometimes it does. Often it writes a confident answer, grades that answer "looks good," and ships the same mistake with a stamp of approval. An agent self-reflection prompt is only as good as the critic behind it, and a critic that shares the generator's blind spots is theater.

The pattern is described everywhere as generate-critique-refine. What's rarely shipped is the part that makes it work: a critic prompt with an external pass/fail contract, plus the warning about when reflection quietly fails.

Why self-reflection often rubber-stamps

The uncomfortable finding from the research is that if a model confidently produces a wrong answer, the same model often confidently says that wrong answer is fine. The critic and the generator are the same weights with the same gaps. Asking a model to check its own reasoning is asking it to notice an error it couldn't notice the first time.

Reflection still improves results, but the gain comes from a specific place: the critique has to carry information the generator didn't use. A critic that re-runs the code, checks against a rubric, or compares to retrieved facts catches real errors. A critic that just re-reads and "thinks harder" mostly agrees with itself.

The three feedback sources, ranked

Not all reflection is equal. The feedback source decides whether the loop catches errors or launders them.

External verification is the strongest. The agent runs a tool to check its own work: execute the code, run the tests, query the source. The verdict comes from reality, not opinion. This is the version that reliably catches mistakes.

A separate rubric is the middle tier. A critic prompt grades the output against explicit, written criteria it didn't generate. Because the criteria are fixed and external to the generation, the critic has something concrete to check against instead of vibes.

Pure self-critique is the weakest. The same model re-reads its output and comments. It helps with obvious slips and formatting, and it's nearly useless on the confident errors that matter most. Use it as a polish pass, never as a correctness gate.

What you can do with a critic prompt

• Grade an agent's output against explicit criteria instead of a vibe
• Return a per-criterion verdict with evidence, not a single "looks good"
• Gate on pass/fail so a failing output loops back with specific fixes
• Ground the check in tests or retrieved facts so it catches real errors
• Keep the critic separate from the generator to break the shared blind spot
• Log critiques so you can see what the loop actually caught over time

Anatomy of the critic contract

The reusable piece is a critic prompt, kept deliberately separate from whatever generated the output.

Variables → {{output_to_check}}, {{criteria}}, {{evidence}}

Prompt → Role: you are an external reviewer, not the author.
         Check {{output_to_check}} against each item in {{criteria}}.
         Use {{evidence}} (tests, facts) where available.

Output contract (locked):
  per_criterion: [ { criterion, verdict, evidence } ]
  verdict:       PASS | FAIL
  fixes:         [ specific, actionable, only if FAIL ]

The evidence column per criterion is the whole point. A verdict without evidence is the generator agreeing with itself in a different font. A verdict that points to a failing test or a contradicting fact is a real catch. The fixes block, populated only on FAIL, is what the generator consumes on the next pass, so the loop converges instead of circling.

Where models differ

Critic behavior varies, and the differences matter when reliability is the goal. Claude is comfortable returning a hard FAIL and tends to give specific, actionable fixes rather than hedging. GPT-4o is a capable critic but leans agreeable, so the prompt has to explicitly license it to fail the output and restate that near the end. Gemini grades criteria cleanly but sometimes merges the verdict and the fixes unless they're separate sections. Whichever you use, run the critic as a different call from the generator, and ideally a different model, so the blind spots don't line up. Pin the versions, since critic strictness drifts across updates.

An opinionated take: stop using self-reflection as a correctness gate when you have tests available. A critic grounded in a passing or failing test suite is worth more than ten introspective passes, and it's cheaper. Reserve pure self-critique for style and clarity, where the model's judgment is actually reliable, and gate correctness on something external. The Agent Code Output Verification Rubric is built this way: a {{criteria}} variable drives a per-criterion check with an evidence column and a hard pass/fail contract, so the critique catches more than a vibe.

Step-by-step: building a reflection loop that catches errors

1. Separate the critic

Run the critique as a different call from the generation. Same prompt, same context, same blind spot — keep them apart.

2. Ground the check in evidence

Wire {{evidence}} to tests, retrieved facts, or a tool result. A critic with evidence catches errors; a critic without it agrees with itself.

3. Use explicit criteria

Write the criteria down. A critic checking against a fixed rubric beats one improvising what "good" means each time.

4. Gate on pass/fail

A FAIL loops back with specific fixes. A PASS ships. Don't let "mostly good" through; that's how the confident error survives.

5. Cap the loop

Limit retries. If the critic FAILs three times, escalate to a human rather than circling. Reflection that never converges is just expensive.

Variables you'll set

When reflection makes things worse

There's a failure mode worse than rubber-stamping: the critic "fixes" a correct output into a wrong one, confidently. It happens when the criteria are vague enough that the critic invents standards the output never needed to meet. Tight, specific criteria prevent most of it, and so does grounding the critique in evidence rather than taste. The other trap is cost: a reflection loop with no cap can run the generator and critic in circles, doubling or tripling token spend for a marginal gain. Cap the retries, gate correctness on external signals, and treat pure introspection as a polish pass, not a guarantee.

This pairs naturally with verification work; for grading an agent's diff against outcome and process, see verify AI coding agent output, and for catching invented facts before they enter the loop, see agent memory design patterns that hold up.

Getting started

1. Split the critic into a separate call from the generator.
2. Wire {{evidence}} to tests or retrieved facts.
3. Write explicit criteria the critic checks against.
4. Gate on a hard pass/fail verdict.
5. Cap the retry count and escalate past it.
6. Reserve pure self-critique for style only.
7. Start from the free Hallucination Spot Checker to catch invented facts first.

For the reasoning loop that produces the output a critic checks, see ReAct vs plan-and-execute agent prompts, and browse the verification packs for the rest of the quality story.