Cloudflare Radar AI Insights adds three new AI bot/crawler visibility features
Source: https://developers.cloudflare.com/changelog/post/2026-04-17-radar-ai-insights-updates/
What changed
Cloudflare Radar’s AI Insights page gained three new features (announced 2026-04-17): (1) an Adoption of AI Agent Standards widget tracking website adoption of agent-facing standards (filterable by domain category, updated weekly), backed by a new Agent Readiness API; (2) a Markdown for Agents savings gauge showing median response-size reduction when serving Markdown vs. HTML to AI bots, with a corresponding Markdown for Agents API; and (3) a Response Status widget showing HTTP status code distribution (200/403/404 or 2xx–5xx groupings) for AI bot/crawler traffic, also surfaced on individual verified bot detail pages. The URL Scanner also gained an Agent Readiness tab evaluating scanned URLs against the isitagentready.com scoring criteria.
Implication
Operators and researchers can now use Cloudflare Radar to audit how broadly AI agent standards (e.g., llms.txt, robots.txt AI directives) are being adopted across the web, quantify bandwidth/token savings from Markdown serving, and inspect how sites are responding (blocking vs. allowing) to specific AI crawlers — all via both the dashboard UI and new API endpoints.