Your support bot looks greaton the dashboard. Is it faking it?
A high deflection number can hide a bot that improvises answers and lets customers abandon in silence. Score each live bot on six fitness signals and get one honest verdict — KEEP LIVE, RESTRICT SCOPE, or PULL BACK — with a gate that catches vanity deflection the dashboard never will.
Containment isn't resolution — and the gap is where bots quietly fail.
of the time ungrounded support bots hallucinate; grounded ones drop under ~1.5%. Whether the bot is grounded is the single biggest accuracy decision.
if bot CSAT drops more than ~10 points below human CSAT, the deflection number is lying to you — customers are being handled worse, not better.
a chat that stayed with the bot. It is not the same as solved. A frustrated customer who abandons without escalating still counts as a contained win.
This grades whether a live bot is actually fit — not whether its dashboard looks good. The verdict is the bot's weakest fitness signal, and a vanity-deflection gate refuses to let a bot that's both improvising and faking resolution read as merely “needs work.”
Score a bot and watch the vanity-deflection gate fire.
Mark each fitness signal. The verdict is the weakest one — same math as the workbook.
Answers only from your knowledge base, or improvises from training data?
Bot CSAT within ~10 pts of human-handled CSAT?
Issue actually solved, or just the chat held without escalation?
Reach a human in one step, with context carried over?
Billing, refund, policy held to grounding or routed to a human?
Track chat abandonment and 7-day repeat-contact?
The weakest signal alone would read RESTRICT SCOPE — but grounding and verified resolution are both only partial, so the vanity-deflection gate forces PULL BACK. This bot is improvising some answers and counting some contained chats as resolved at the same time.
Fix first: Grounded answering (RAG, no guessing)
Your marks only · no log access · grades the bot, not people
One command, every bot, an honest fleet read.
The zero-dependency Python engine reads your bot list and prints the same verdict the workbook and demo produce. Note the Account-help bot: a mean of 83, and still PULL BACK — because grounding and verified resolution are both only partial.
Live Support-Bot Fitness Gate
====================================================
Billing FAQ bot weakest 1/2 (mean 58) PULL BACK [GATE -> PULL BACK]
fix first: Grounded answering (RAG, no training-data guessing)
Order-status bot weakest 2/2 (mean 100) KEEP LIVE
Returns assistant weakest 1/2 (mean 75) RESTRICT SCOPE
fix first: CSAT within range of human-handled CSAT
Tech-troubleshoot bot weakest 1/2 (mean 92) RESTRICT SCOPE
fix first: Verified resolution, not self-declared containment
Policy Q&A bot weakest 0/2 (mean 75) PULL BACK
fix first: Grounded answering (RAG, no training-data guessing)
Account-help bot weakest 1/2 (mean 83) PULL BACK [GATE -> PULL BACK]
fix first: Grounded answering (RAG, no training-data guessing)
----------------------------------------------------
Fleet: PULL-BACK FIRST
3 of 6 bot(s) read PULL BACK.Six fitness signals — the weakest one is the verdict.
A live bot is only as fit as its weakest dimension. The verdict is the lowest signal, never the flattering average — the mean is shown for context only.
When grounding AND verified resolution are both only partial, the verdict is forced to PULL BACK even when the weakest single signal reads RESTRICT SCOPE — the vanity-deflection trap a weakest-signal rule alone would miss.
Bring grounding or verified resolution to full and the gate releases. The fix-first signal is always named, so you know the one thing to work first.
A go/no-go fitness call, not a chatbot dashboard.
- A deterministic keep/restrict/pull-back verdict from your own marks.
- A vanity-deflection gate that catches the wrong-but-contained pattern.
- A fleet rollup that names the worst bot to act on first.
- Offline — engine, workbook, and demo agree to the verdict.
- Not connected to your bot or your chat logs — nothing is read live.
- Not an AI that grades you; your marks drive the verdict.
- Not a compliance certification, and it scores no person or customer.
- Not a replacement for human support — it keeps a human in reach.
Not legal, safety, or compliance advice. This is a go/no-go decision aid that grades a bot's fitness from your own marks. It connects to nothing, reads no logs, and scores no person or customer — the decision to keep, restrict, or pull back a live bot is yours.
Anyone accountable for a live support bot.
Check the number, gate the fitness, watch the liability.
Grades whether a deflection number is real, soft, or false before you ever trust it.
ViewThe liability lane: triages every customer-facing AI surface on legal exposure.
ViewReveals the end-to-end success rate a per-step containment metric hides.
ViewThe honest answers.
The Risk Triage grades liability exposure on any customer-facing surface — its gate fires when a surface can state policy with no grounding and offers no human exit ramp. This grades a live support bot's fitness to keep running, and its gate fires on vanity deflection: a bot that is both partly ungrounded and only counts containment as resolution. One is about legal exposure; this is about whether the bot is quietly failing customers while looking successful on a deflection dashboard.
Because the verdict is the weakest signal, not the average — and the vanity-deflection gate is sharper still. If grounding and verified resolution are both only partial, the bot is improvising some answers and counting some contained-but-abandoned chats as resolved at the same time. That combination forces PULL BACK even when the weakest single signal would read RESTRICT SCOPE. The sample's Account-help bot scores a mean of 83 and still pulls back for exactly this reason.
Grounded answering (RAG, no training-data guessing) [gate], CSAT within range of human-handled CSAT, verified resolution rather than self-declared containment [gate], one-click human escalation with context transfer, a guardrail on regulated and dollar-amount answers, and abandonment plus repeat-contact monitoring. You mark each 0, 1, or 2 from your own read of the bot.
Containment measures whether a chat stayed with the bot; resolution measures whether the customer's issue was actually solved. A bot can show high containment while customers abandon in frustration without ever escalating — that is vanity deflection, and it tanks CSAT while the deflection dashboard still looks green. Ungrounded bots hallucinate far more often than grounded ones, so a bot that is both partly ungrounded and only tracks containment is the documented failure mode. The gate refuses to let that bot read as merely 'needs work.'
No. It is deterministic and offline. You enter your own marks and it computes the verdict — it never connects to your bot, reads logs, or scores anything live. The same logic runs in the workbook, the Python engine, and the on-page demo, so all three agree to the verdict.
No. It is a go/no-go decision aid that grades a bot's fitness from your own marks — it scores the bot, not any person or customer, and asserts no statute. Use it to decide whether a live bot stays live, gets its scope narrowed, or comes down. Not legal, safety, or compliance advice.
Stop trusting the dashboard.
Test the bot.
One purchase, lifetime access, 12 months of updates. $79, once.
Not legal, safety, or compliance advice. This is a go/no-go decision aid that grades a bot's fitness from your own marks. It connects to nothing, reads no logs, and scores no person or customer — the decision to keep, restrict, or pull back a live bot is yours.
Sold by RedHub AI LLC · Secured by Stripe · redhub.ai