Your AEO can be perfectand the AI still can't read you.
A 2023-era robots.txt rule, a WAF challenge, or JS-only rendering silently walls AI crawlers out — nothing errors, no dashboard flags it, and every dollar spent on the content layer is wasted. This kit audits the door itself: six access signals per engine, a per-engine verdict of CRAWLABLE / PARTIALLY BLOCKED / INVISIBLE TO AI, and the one engine and signal to unblock first.
The failure is invisible: the site looks perfect, and the engines never fetched it.
SaaS companies were found blocking at least one major AI crawler in 2026 research — often an inherited 2023-era rule nobody has re-read since. The block that stopped training bots now stops citations.
Most AI crawlers do not execute JavaScript. A React page that renders its price and answers client-side is a blank document to GPTBot, ClaudeBot, and PerplexityBot — while Googlebot, which renders, sees it fine. Same page, two realities.
ChatGPT citations come from OAI-SearchBot — not GPTBot, the training crawler. Blanket "block the AI bots" rules catch both, and WAF rules override robots.txt entirely. The kit checks the bot that actually carries your citations, per engine.
Six signals per engine, weights summing to 100, and a gate that outranks the score.
The access gate is forcing this verdict: a robots.txt disallow or a WAF block means the crawler never fetches the page — content that is never fetched cannot be cited, whatever the score says. Lift the blocking mark off 0 and the gate releases.
Same weights, bands, and gate as the workbook and the runnable engine — one scoring model, three surfaces. Deterministic and offline: it grades the marks you record, fetches nothing, and no citation outcome is guaranteed. Refresh resets the demo; the kit is yours to keep.
One command reproduces the whole audit — this output is from the shipped sample, run and verified.
AI CRAWLER ACCESS & RENDER AUDIT
==========================================================================
Signals (weights): robots.txt access 25, WAF/CDN passage 20, Render visibility 20, Content extractability 15, Fetch health 12, Log-verified access 8
--------------------------------------------------------------------------
ChatGPT Search 67 INVISIBLE TO AI [ACCESS GATE: forced]
crawler: OAI-SearchBot
marks: robots_access=0 waf_passage=2 render_visibility=2 extractability=2 fetch_health=2 log_verified=0
fix first: robots.txt access
Perplexity 86 CRAWLABLE
crawler: PerplexityBot
marks: robots_access=2 waf_passage=1 render_visibility=2 extractability=2 fetch_health=2 log_verified=1
fix first: WAF/CDN passage
Claude 72.5 PARTIALLY BLOCKED
crawler: Claude-SearchBot / ClaudeBot
marks: robots_access=2 waf_passage=2 render_visibility=0 extractability=1 fetch_health=2 log_verified=2
fix first: Render visibility
Google AI (AI Overviews / AI Mode / Gemini) 100 CRAWLABLE
crawler: Googlebot + Google-Extended token
marks: robots_access=2 waf_passage=2 render_visibility=2 extractability=2 fetch_health=2 log_verified=2
fix first: robots.txt access
Microsoft Copilot 66 PARTIALLY BLOCKED
crawler: Bingbot
marks: robots_access=1 waf_passage=2 render_visibility=1 extractability=1 fetch_health=2 log_verified=1
fix first: robots.txt access
--------------------------------------------------------------------------
SITE VERDICT: WALLED OFF
Reachability rate: 40% (2 of 5 engines CRAWLABLE)
Fix first: ChatGPT Search -> robots.txt access
==========================================================================
Deterministic and offline. Grades the access posture you recorded;
fetches nothing, scores no people. Crawler names shift -- verify
against vendor bot docs before each audit. No citation guaranteed.Read the ChatGPT row: it scores 67 — and reads INVISIBLE TO AI anyway, because the access gate found OAI-SearchBot disallowed in robots.txt. Content that is never fetched cannot be cited, whatever the score says. Zero dependencies, Python 3.8+, fully offline; the workbook reproduces the same scoring cell by cell, and a 729-combination verifier ships in the zip so you can prove it yourself.
The weakest layer decides, the gate never lifts, and the verdict comes from what you observed.
The gate is worsen-only and dispositive: a robots.txt disallow or a WAF block on an engine's crawler forces INVISIBLE TO AI regardless of the weighted score. It releases the moment the blocking mark lifts off zero — and it can never improve a verdict.
Every engine gets its own row and verdict, because the facts are engine-shaped: Googlebot renders JavaScript, most AI crawlers do not; each robots.txt block names a specific bot. The rollup names the weakest engine — ties break to the lower score, then input order.
No scraping, no API, no baked-in benchmark. You record what your own robots.txt, WAF console, page source, and access logs actually show; the engine turns those marks into the verdict deterministically. The verdict is only as honest as the audit — the playbook keeps both honest.
The infrastructure gate under the AEO line, not another content tool.
- A deterministic, offline audit of whether AI crawlers can fetch, render, and extract your money pages — per engine, with the fix to make first.
- The technical prerequisite check that explains why AEO work isn't landing: is the content bad, or can they just not see it?
- One scoring model on three surfaces — workbook, Python engine, and this page's demo — with a shipped verifier that proves they agree.
- Editable by design: engine rows and crawler names live in config, because those names shift.
- Not a crawler, scanner, or monitoring service — it fetches nothing and connects to nothing. You do the looking; it does the judging.
- Not a citation guarantee. It removes the technical reasons you can't be cited; earning the citation is the content layer's job.
- Not affiliated with OpenAI, Anthropic, Perplexity, Google, or Microsoft — crawler behavior is theirs to change, which is why you re-audit quarterly.
- Not legal advice, and it scores your site's access posture — never people. Decisions about training crawlers are content-policy decisions; make them deliberately.
Anyone whose AI-visibility spend depends on the engines being able to read the site.
Unblock here, then optimize there — this kit is the bottom layer of the AI-visibility line.
The content layer above this kit: once the engines can fetch you, it grades whether what they fetch is citable.
ViewThe treatment: the CLEAR framework, paste-ready templates, and the 30/60/90 prompt pack to earn the citations.
ViewThe structured-feed lane: readiness for AI shopping panels and product carousels, where feeds outrank pages.
ViewDirect answers on scope, the gate, and the crawler facts that trip people up.
No. It is deterministic and fully offline. You check your own robots.txt, CDN/WAF console, page source, and access logs, mark six signals 0/1/2 per engine, and the workbook and Python engine turn those marks into the verdict. It fetches nothing and sends nothing.
Different layer. The AEO Audit Kit grades whether your content is citable; this kit grades whether the engines can fetch and read that content at all. Unblock here first — a page an AI crawler never receives cannot be cited no matter how well it is written.
Because of the access gate. A robots.txt disallow or a WAF/CDN block on that engine's crawler forces INVISIBLE TO AI regardless of the weighted score — content that is never fetched cannot be cited. The gate is worsen-only: it can never lift a verdict, and it releases the moment the blocking mark rises above zero.
No — and that distinction is half the kit. GPTBot is OpenAI's training crawler; ChatGPT Search citations come from OAI-SearchBot. The common failure is a 2023-era blanket rule that blocks both. The playbook walks the current crawler names for each engine, and every engine row is editable because those names shift.
That is exactly what the render-visibility signal catches. Most AI crawlers do not execute JavaScript, so JS-only money content is invisible to them even when Google sees it fine — Googlebot renders, GPTBot and ClaudeBot do not. The kit marks each engine against its own crawler's capability, and the Unblock Runbook covers the SSR, pre-render, and inline-content fixes.
No. It removes the technical reasons you cannot be cited; earning the citation is the content layer's job. Verdicts come from your own recorded marks — no scraping, no baked-in benchmark — and crawler behavior changes, so re-run the audit quarterly and after any CDN or framework change. It grades your site's access posture, never people, and it is not legal advice.
Find out if the door
is even open.
One audit tells you whether your AEO spend has a chance. One purchase, lifetime access, 12 months of updates. $79, once.
Sold by RedHub AI LLC · Secured by Stripe · redhub.ai