How the gate works

How the Agent Reliability Harness gate works

This tool grades one AI-agent trajectory across six evaluators before it ships. Each signal is scored 0–0; the verdict is the weakest one — then a gate that can only make it worse.

The 6 signals

01task_success

02tool_choice

03argument_validity

04step_efficiency

05cost_budget

06policy

SHIP

every evaluator passes (weakest = 1)

FIX

any evaluator fails (weakest = 0) - one bad axis overrides five good ones

All-or-nothing ship gate · worsen-only

The case verdict is SHIP only if every one of the six evaluators passes. Any single failing check - however minor - drops the case straight to FIX, because a trace that fails on one axis isn't shippable even when the final answer was right.

FIX

Why: The looping-agent trajectory reaches the right final answer - task_success still passes - but it takes 6 steps against a budget of 5, repeating the same call 5 times. step_efficiency scores 0.40 and is the only failing check, so the case-level verdict is FIX even though five of six evaluators are clean. weakest signal 0/0 · 5 of 6 checks clean.

Fix firststep_efficiency - 6 steps > budget 5; repeated identical call x5 (possible loop)

A clean sheet didn’t save it. The verdict is the weakest signal — and a top trigger is dispositive on its own — so it names the one thing to fix first. Grades the guardrails and trajectory you describe. It does not run or change any agent, and it never scores a person.

RedHub AIthe AI that tells you when to doubt it redhub.ai/systems/agent-reliability-harness