The prompt text didn’t change.The agent talks over callers now.
A text diff tells you the words are the same. It can’t tell you the agent stopped yielding when callers interrupt, or that its latency tail doubled, or that it quit escalating. This library generates fixed canary calls and scores every release against your own baseline on the behaviors that actually decide a call.
A green text diff hides a voice regression.
Same correct answer. Drop the barge-in. Watch it break.
CN-002 is already below baseline in both states — same correct answer, but p95 latency jumped from 950ms to 1250ms. Correctness held; the call got slower. The library grades the behavior, not just the words.
Generate the library once. Score every release against baseline.
A zero-dependency Python tool generates the canary library from your topics and scores a run against your baseline. Here is its actual score output on the shipped seven-canary sample — a release with two slips and two breaks:
CANARY CALL LIBRARY - RELEASE SCORE ==================================================== PER-CANARY (this run vs baseline) ---------------------------------------------------- CN-001 ON BASELINE CN-002 BELOW BASELINE latency_p95_ms 1250ms > baseline 950ms + slack CN-003 ON BASELINE CN-004 CANARY BROKE barge_in 4 < floor 5 CN-005 BELOW BASELINE dead_air 2 > baseline 0 CN-006 ON BASELINE CN-007 CANARY BROKE escalation 0 < floor 1 ROLLUP ---------------------------------------------------- Canaries scored .......... 7 On baseline .............. 3 Below baseline ........... 2 Canary broke ............. 2 Review first ............. CN-004 VERDICT: REGRESSIONS FOUND Scores a run against your own baseline run. It places no calls and grades no person. Not legal advice.
Three rules the library holds every release to.
Same calls, every release
A canary is only a signal if it never changes. The generator builds a fixed library from your topics — hours, booking, a grounding trap, a barge-in test, an escalation demand — so any difference between runs is your agent, not the test.
Your baseline, not a benchmark
A regression is your agent getting worse than the run you were happy to ship. Every later run is scored against that baseline with tolerance — industry latency numbers are context, not a pass mark.
Some drills don’t bend
Barge-in below 5 of 5, or a scripted escalation that didn’t fire, forces CANARY BROKE regardless of how correct the answers were. An agent that talks over callers fails the call — the words being right doesn’t save it.
A voice-behavior regression check. Not a text diff, not a dialer.
- A generator for a fixed canary-call library from your topics.
- A scorer of voice behavior — barge-in, latency, dead-air, escalation.
- Baseline-relative, with a dispositive barge-in / escalation gate.
- Deterministic — the same two runs always score the same.
- A prompt-text diff — that’s the Prompt Regression Lab.
- A dialer or call simulator — it scores results you record.
- A benchmark against other companies’ numbers.
- A judge of people — it grades runs, never person.
Scope note. The Canary Call Library scores runs you record against your own baseline; it places no calls, connects to nothing, and grades no person. It is a release-quality aid, not a certification, and not legal advice.
Anyone who ships changes to a voice agent that’s already live.
The voice half of a release-safety stack.
Diffs the prompt text and fails CI on a change. This scores the voice behavior around it.
ViewBuilds the safety blocks in up front. The canary library proves they still hold after a change.
ViewGrades real calls after the fact. The canary library grades a release before it ships.
ViewThe parts worth pinning down before you buy.
Ship the change.
Know what else it moved.
Run the same canary calls every release and score them against the version you trusted. One purchase, lifetime access, 12 months of updates. $79, once.
Sold by RedHub AI LLC · Secured by Stripe · redhub.ai