# Forbes blocks PerplexityBot; El País reopens to GPTBot and ClaudeBot

Published: 2026-10-25T21:36:22.203Z · Source: Crawler access panel (https://anythingengineoptimization.com/crawl-ledger/)
Source date: 2026-10-25
Entities: openai, anthropic, perplexity

The wire's crawler access panel caught shifts at three publishers between its October 18 and October 25 captures. Forbes's homepage began returning a 403 to PerplexityBot, where it had drawn a 200 a week earlier, now served by the bot-detection vendor DataDome in place of the Istio proxy that answered a week before; a plain-text probe sent with a browser's user-agent string, not a rendered browser, drew the same new 403. GPTBot and ClaudeBot were unaffected, continuing to draw the 402 Payment Required response they got a week earlier. Forbes's robots.txt names no AI crawler, unchanged.

El País's homepage returned to a 200 for GPTBot and ClaudeBot, both of which the panel found at 403 a week earlier — its third flip there since mid-September, when El País first blocked all three bots, then reopened to ClaudeBot alone four days later. El País's robots.txt still disallows ClaudeBot, its Claude-SearchBot counterpart and PerplexityBot by name; PerplexityBot's homepage has returned a 200 throughout, despite that rule.

Nerdwallet's robots.txt dropped the line disallowing Google's AI-training crawler, Google-Extended, that the wire's October 19 robots.txt survey had found there. Nothing else in its file or its homepage access changed.

Search Engine Land's robots.txt, readable a week earlier, now returns a 403 to the panel's fetcher; its homepage was already challenged for all four probed agents in both captures, so the panel can no longer compare what the file says. As with any single-probe reading, a flip that doesn't hold on the next capture would count as transient rather than a change.

Why it matters: Enforcement keeps moving independently of stated policy on this beat: Forbes's new bot-detection layer blocks Perplexity's crawler though its robots.txt names none, El País reopened to two crawlers its file still disallows, and Nerdwallet rewrote its file to let Google's AI-training bot back in.

## What this answers

**Does Forbes block PerplexityBot?**

As of the wire's October 25 capture, yes at the enforcement layer — Forbes's homepage returns a 403 to PerplexityBot, now served by the bot-detection vendor DataDome, though Forbes's robots.txt still names no AI crawler. OpenAI's GPTBot and Anthropic's ClaudeBot continue to draw a 402 Payment Required response there, unchanged from a week earlier.

**Does El País block GPTBot or ClaudeBot?**

Not as of October 25 — both now draw a 200 from El País's homepage, reversing a 403 the wire's panel caught a week earlier. El País's robots.txt still disallows ClaudeBot by name; it's the third flip the panel has recorded there since mid-September.

**Does Nerdwallet block Google's AI-training crawler?**

No — as of October 25, Nerdwallet's robots.txt no longer disallows Google-Extended, a rule the wire's October 19 robots.txt survey had found there.


Canonical: https://anythingengineoptimization.com/item/2026-10-25-forbes-blocks-perplexitybot-el-pa-s-reopens-to-gptbot-and-claudebot/
From Anything Engine Optimization (AEO Wire) — https://anythingengineoptimization.com/ · Standards: https://anythingengineoptimization.com/standards/