The Crawl Ledger

Method, and what the numbers are not

• Crawler counts come from Cloudflare's request analytics for this zone, grouped by user-agent string, one day at a time, on a plan that samples: the figures are Cloudflare's adaptive estimates, not raw log counts, and a user-agent string is a claim, not a proof — forged GPTBot traffic exists and is counted here as GPTBot, because that is what the site sees.
• Referrals come from Cloudflare Web Analytics, a browser beacon. Bots do not fire it; people do. "Visits referred" is page loads whose referrer was an AI answer engine's domain. Engines that strip referrers, and app traffic, are invisible here, so the referral count is a floor.
• Purpose follows each operator's own documentation as recorded in the crawler registry: training, search, or an on-demand fetch for a user's request.
• Scale. This is one two-month-old news site. The ratio is real for this site and says nothing about any other; the point of publishing it is that the ratio is usually asserted about all sites from data nobody outside Cloudflare can check, and here is one that anyone can.
• Retention on the plan is eight days, so a week that is not captured is lost. Captured on Sundays; the week's line goes into the Monday briefing.