RAG Quality Audit Playbook
Stop guessing why your RAG system answers wrong — classify each failure into a nine-family taxonomy, isolate the broken layer with probe queries and a five-lens corpus audit, and ship grounded prompt fixes with verification probes that prove the fix.
A 4-step agentic workflow pack for coding built to run with ChatGPT, Claude, and Gemini. Open the Markdown files, fill the variables, and paste into your model. Most buyers get a reviewable result in about 15 minutes.
- Classify every wrong answer into a nine-family root-cause taxonomy (missed retrieval, ranking, chunk splits, stale corpus, ignored context, fabrication) instead of guessing
- Isolate recall vs ranking vs vocabulary failures with four-variant probe queries and expected-document assertions you write before running them
- Follow a five-branch decision tree that names the broken stage and its fix owner — so you stop swapping embedders when the index is the problem
- Audit chunk boundaries, structure, metadata, duplication, and staleness with 1–5 severity anchors tied to your actual failing queries
- Get a before/after re-chunking spec only when evidence demands it — 'leave it alone' is a valid audit outcome
- Fix generation-side failures with minimal prompt edits, a grounding checklist, and regression probes that guard against over-refusal
Prompt Customization Service — optional help adapting variables and output to your brand voice. Choose your tier at checkout (not tied to this prompt's price).
This pack is $10 on its own. Buying every pack separately costs $935. The Lifetime Bundle is $149 one-time — you save $786 (84% off) and unlock every future pack free.
Get the Lifetime Bundle — $149Paste the license key from your receipt. It must match this prompt pack.
What ships with your purchase
Prompt files
Plain Markdown files with `{{variables}}` you fill in, ready to paste into ChatGPT, Claude, or Gemini. No setup, no tooling required.
Usage guide
Variable reference, model compatibility, examples, and customization tips so you can adapt the pack to your brand voice.
Lifetime updates
When we improve the pack, you get the new version automatically. Email support included with every purchase.
Models tested: ChatGPT, Claude, Gemini.
The workflow inside this pack
4 composable prompts you run in order — each one picks up where the last left off.
- Step 1
Failure Symptom Classifier
Paste your wrong answers and architecture description — each case gets a primary failure family from a nine-family taxonomy, with confidence and the evidence behind it.
- Step 2 · optional
Retrieval Layer Prober
Each failing case becomes a four-variant probe group — original query, canonical phrasing, bare keywords, and a near-verbatim quote — each with a written expected-document assertion.
- Step 3 · optional
Chunking & Corpus Auditor
Your chunking config and failing evidence get audited across five lenses — boundary integrity, structure preservation, metadata, duplication, staleness — each scored 1–5 with anchored severity definitions.
- Step 4 · optional
Grounding & Prompt Fixer
Your generation prompt is audited against a six-item grounding checklist; each failing case is tied to the exact gap that permits it, with the minimal before/after edit that closes it.
Perpetual (lifetime) use license
Your one-time purchase includes an ongoing right to use this prompt pack with the AI tools and models you control for your own and your clients' work — not for resale or public redistribution of the files as a product.
We keep the copyright
The prompt files, guides, examples, and bundled assets stay our copyrighted works (or our licensors'). Payment grants the limited license in our Terms only — it does not transfer ownership.
Need help adapting this prompt to your team? Add Prompt Customization Service at checkout.