People Inc. sites and The Atlantic block GPTBot, still bill Claude, Perplexity
What this answers
- Is GPTBot now blocked at Allrecipes, Investopedia, People, or The Atlantic?
- Yes — as of 2026-09-13 all four returned a Cloudflare-managed 403 to both GPTBot and a browser user-agent, per the wire's crawler access panel, while continuing to serve Anthropic's ClaudeBot and Perplexity's crawler the same 402 Pay Per Crawl response as before.
- Does Quora block AI crawlers?
- Yes — Quora's robots.txt disallows GPTBot, ClaudeBot, PerplexityBot and Google-Extended by name, and as of 2026-09-13 its homepage returns a flat 403 to all of them rather than the redirect it served the day before.
The wire’s crawler access panel — 55 publisher domains probed daily as GPTBot, ClaudeBot, PerplexityBot and a browser — caught three People Inc. properties (Allrecipes, Investopedia and People) and The Atlantic making an identical change between the September 12 and 13 captures: GPTBot and browser-UA requests that drew a 200 the day before now draw a Cloudflare-managed challenge (403), while ClaudeBot and PerplexityBot still get the same Cloudflare Pay Per Crawl response (402) as before. Quora’s homepage hardened the same day, from a redirect to a flat 403 for all four probes, and its robots.txt — readable to the panel for the first time — disallows GPTBot, ClaudeBot, PerplexityBot and Google-Extended by name alongside a notice barring AI-training use outright. Reddit tightened separately, on its own infrastructure, adding a 403 to browser and PerplexityBot requests that had drawn a 200 the day before.
Why it matters: It shows AI-crawler access splitting into two tracks at some of the web's largest publishers — pay under Cloudflare's Pay Per Crawl, or get blocked outright — with OpenAI's crawler landing in the blocked track on 2026-09-13 while Anthropic's and Perplexity's kept paying their way in.
The record: CloudflareOpenAIReddit
Posted to the wire September 13, 2026. Edited by Joe Balewski.