# El País blocks GPTBot, ClaudeBot and PerplexityBot

Published: 2026-09-16T23:27:39.510Z · Source: Crawler access panel (https://anythingengineoptimization.com/crawl-ledger/)
Source date: 2026-09-16
Entities: cloudflare, anthropic, openai

The wire's crawler access panel caught El País's homepage flip from a 200 to a 403 for GPTBot, ClaudeBot and PerplexityBot alike between the September 13 and 16 captures, verified still live at curation time. The Spanish daily's robots.txt already disallowed ClaudeBot and PerplexityBot by name; GPTBot was not named there, so its block is new at the enforcement layer even though the wildcard rule never covered it explicitly. Separately, Stack Overflow moved Anthropic's ClaudeBot off Cloudflare's Pay Per Crawl track — a 402 response offering paid access — onto a flat 403 block, while GPTBot and PerplexityBot stayed behind the same Cloudflare challenge they hit before.

Why it matters: AI-crawler access keeps moving at the enforcement layer, not just in robots.txt — El País shut out all three major crawlers at once, and Stack Overflow ended a paid-access arrangement for one of them, changes neither site's robots.txt alone would show.

## What this answers

**Does El País block GPTBot, ClaudeBot, or PerplexityBot?**

Yes — as of the wire's September 16 capture, El País's homepage returns a 403 to GPTBot, ClaudeBot and PerplexityBot alike, a change from the 200 all three drew on September 13; its robots.txt already disallowed ClaudeBot and PerplexityBot by name but did not name GPTBot.

**Does Stack Overflow charge AI crawlers to access its site?**

It still returns Cloudflare's Pay Per Crawl 402 to some bots, but as of September 16 it moved Anthropic's ClaudeBot off that track onto a flat 403 block, per the wire's crawler access panel, while GPTBot and PerplexityBot remain behind a Cloudflare challenge as before.


Canonical: https://anythingengineoptimization.com/item/2026-09-16-el-pa-s-blocks-gptbot-claudebot-and-perplexitybot/
From Anything Engine Optimization (AEO Wire) — https://anythingengineoptimization.com/ · Standards: https://anythingengineoptimization.com/standards/