LLM Over-Refusal Audit Rubric
Audit your LLM for over-refusal: classify false rejections across a 7-category taxonomy, score borderline cases with calibrated anchors, map helpfulness-safety tradeoffs, and get a prioritized remediation plan — for ML and product teams in healthcare, legal, and customer support.
A 4-step agentic workflow pack for other built to run with ChatGPT, Claude, and Gemini. Open the Markdown files, fill the variables, and paste into your model. Most buyers get a reviewable result in about 45 minutes.
- Classify every false refusal across a 7-category taxonomy with severity and frequency scoring
- Score ambiguous borderline cases with calibrated 1–5 anchors that encode domain-specific judgment
- Map each refusal pattern onto a helpfulness-safety tradeoff matrix to find systematic policy miscalibration
- Receive a prioritized remediation plan with specific prompt-engineering, fine-tuning, and policy levers
- Designed for high-stakes domains: healthcare triage, legal research, customer support, and dual-use scenarios
- Encodes judgment from FalseReject literature and 2025 over-refusal benchmarks — not producible by a naive ask
Prompt Customization Service — optional help adapting variables and output to your brand voice. Choose your tier at checkout (not tied to this prompt's price).
This pack is $8 on its own. Buying every pack separately costs $935. The Lifetime Bundle is $149 one-time — you save $786 (84% off) and unlock every future pack free.
Get the Lifetime Bundle — $149Paste the license key from your receipt. It must match this prompt pack.
What ships with your purchase
Prompt files
Plain Markdown files with `{{variables}}` you fill in, ready to paste into ChatGPT, Claude, or Gemini. No setup, no tooling required.
Usage guide
Variable reference, model compatibility, examples, and customization tips so you can adapt the pack to your brand voice.
Lifetime updates
When we improve the pack, you get the new version automatically. Email support included with every purchase.
Models tested: ChatGPT, Claude, Gemini.
The workflow inside this pack
4 composable prompts you run in order — each one picks up where the last left off.
- Step 1
False-Refusal Classifier Rubric
Paste your sampled refusal log and domain context — the rubric classifies each refusal into one of 7 false-refusal categories with severity and confidence scores.
- Step 2
Borderline Case Scorer
Feed the borderline cases surfaced by the classifier rubric plus your domain policy — each case is scored 1–5 with explicit anchor justification and a recommended label.
- Step 3
Helpfulness-Safety Tradeoff Mapper
Feed the classifier findings and borderline scores — the mapper plots every refusal pattern across a helpfulness-safety matrix and names the dominant miscalibration zone.
- Step 4 · optional
Remediation Recommendation Prompter
Feed the tradeoff mapper output and your team's constraints (model access level, fine-tuning availability, deployment timeline) — the prompter produces a ranked remediation backlog.
Perpetual (lifetime) use license
Your one-time purchase includes an ongoing right to use this prompt pack with the AI tools and models you control for your own and your clients' work — not for resale or public redistribution of the files as a product.
We keep the copyright
The prompt files, guides, examples, and bundled assets stay our copyrighted works (or our licensors'). Payment grants the limited license in our Terms only — it does not transfer ownership.
Need help adapting this prompt to your team? Add Prompt Customization Service at checkout.