Tool Gate · How it works
How the Agent Reliability Harness gate works
How the Agent Reliability Harness scores one agent trajectory across 6 evaluators — task success, tool choice, argument validity, step efficiency, cost, policy — and ships only if every one passes: the default looping agent reaches the right answer but repeats a call, so step efficiency alone drops it to FIX with the other five clean.
How the gate works
This tool grades one AI-agent trajectory across six evaluators before it ships. Each of 6 signals is scored 0–0 from your own evidence; the verdict is the weakest one — and then it is run past a gate that can only make the verdict worse.
The 6 signals
| Signal | Role |
|---|---|
| task_success | signal |
| tool_choice | signal |
| argument_validity | signal |
| step_efficiency | signal |
| cost_budget | signal |
| policy | signal |
The verdict bands
The case verdict is SHIP only if every one of the six evaluators passes. Any single failing check - however minor - drops the case straight to FIX, because a trace that fails on one axis isn't shippable even when the final answer was right.
In the worked sample 5 of 6 checks clean (weakest 0) and it still reads FIX — The looping-agent trajectory reaches the right final answer - task_success still passes - but it takes 6 steps against a budget of 5, repeating the same call 5 times. step_efficiency scores 0.40 and is the only failing check, so the case-level verdict is FIX even though five of six evaluators are clean. The one thing to fix first: step_efficiency - 6 steps > budget 5; repeated identical call x5 (possible loop).
Common questions
Can a profile look clean on the How the Agent Reliability Harness gate works and still fail?
Yes. The verdict is the weakest of 6 signals, then run past a worsen-only gate — the All-or-nothing ship gate — that can only make it worse. In the worked sample 5 of 6 checks clean (weakest 0) and it still reads FIX, because the looping-agent trajectory reaches the right final answer - task_success still passes - but it takes 6 steps against a budget of 5, repeating the same call 5 times. step_efficiency scores 0.40 and is the only failing check, so the case-level verdict is FIX even though five of six evaluators are clean.
What is a worsen-only gate?
A dispositive rule that can only lower a verdict, never raise it. One fatal flaw overrides an otherwise-clean sheet, because a single disqualifying gap shouldn't hide behind everything that passed. The tool also names the one thing to fix first — here, "step_efficiency - 6 steps > budget 5; repeated identical call x5 (possible loop)".
Grades the guardrails and trajectory you describe. It does not run or change any agent, and it never scores a person.
Embed this diagram
Free to share and embed with attribution (CC BY 4.0) — keep the link to redhub.ai.
Interactive — renders the live diagram
<iframe src="https://redhub.ai/visuals/tool/agent-reliability-harness.html" title="How the Agent Reliability Harness gate works — RedHub AI" width="760" height="1609" loading="lazy" style="border:0;width:100%;max-width:760px"></iframe>
<p style="font:14px/1.5 system-ui,sans-serif"><a href="https://redhub.ai/visuals/tool/agent-reliability-harness">How the Agent Reliability Harness gate works</a> — by <a href="https://redhub.ai">RedHub AI</a>, the AI that tells you when to doubt it.</p>Image + link — a static picture for any blog
<a href="https://redhub.ai/visuals/tool/agent-reliability-harness"><img src="https://redhub.ai/visuals/tool/agent-reliability-harness-share.png" alt="Diagram of the Agent Reliability Harness: six pass/fail evaluators and a FIX verdict driven by the single failing step-efficiency check on a looping agent whose final answer was correct." width="760" loading="lazy" style="max-width:100%;height:auto;border-radius:16px"></a>
<p style="font:14px/1.5 system-ui,sans-serif"><a href="https://redhub.ai">RedHub AI</a> — the AI that tells you when to doubt it.</p>
This is how Agent Reliability Harness works. More diagrams in the Visual Field Guide.