# Cloudflare's AI-crawler default did not reach this site's existing zone

Published: 2026-10-09T03:34:22.727Z · Source: Cloudflare Blog (https://blog.cloudflare.com/content-independence-day-ai-options/)
Source date: 2026-08-30
Flags: wire's own measurement — one zone, this site's own, which already had every bot protection off; snapshots taken five days before and three weeks after the change
Entities: cloudflare

Cloudflare's September 15 change to its AI-crawler defaults did not reach this publication's own site, an existing domain on Cloudflare's Free plan, according to snapshots of the zone's settings taken through Cloudflare's API on September 10 and October 9. Cloudflare said Training and Agent crawlers would be "blocked by default on the pages that display ads" for "all new domains onboarding to Cloudflare"; several third-party summaries said the change reached existing free customers as well, and Cloudflare did not address the difference.

The API records the choice in more than one place, and each accepts a value of onlyonadpages. aibotsprotection is the rule that blocks AI crawlers; aitraining, aisearch and aiuser set the robots.txt policy for training, search and assistant-or-agent bots. A zone that took the ad-page default would show it in those fields whether or not it runs ads. On anythingengineoptimization.com all four read disabled on both dates. The zone never set the flag Cloudflare's schema describes as tracking "zones opted out of AI bots managed-rule updates," and received no update anyway.

This is one zone, and it already had every bot protection switched off. The second snapshot came three weeks after the change, so a setting changed and reversed in between would not show. The result is consistent with Cloudflare's own wording, new domains only, and says nothing about any other existing zone.

Why it matters: Cloudflare's post said the September 15 default covered new domains, third-party summaries said it reached existing free customers too, and Cloudflare never settled it; a before-and-after of one real existing zone is a first data point, and it matches Cloudflare's own wording.

## What this answers

**Did Cloudflare's September 15 AI-crawler default apply to existing domains?**

Not to this site's. AEO Wire's own zone, an existing domain on Cloudflare's Free plan, showed identical AI-bot settings in API snapshots taken September 10 and October 9: the blocking rule and the robots.txt policies for training, search and agent bots all read disabled on both dates. Cloudflare's post scoped the default to new domains; this is one zone.

**How do I check whether Cloudflare changed my site's AI-crawler settings?**

Read the zone's bot management settings. ai_bots_protection is the rule that blocks AI crawlers; ai_training, ai_search and ai_user set the robots.txt policy for training, search and assistant-or-agent bots. A zone on the ad-page default reads only_on_ad_pages in those fields, whether or not it runs ads.


Canonical: https://anythingengineoptimization.com/item/2026-10-09-cloudflare-s-ai-crawler-default-did-not-reach-this-site-s-existing/
From Anything Engine Optimization (AEO Wire) — https://anythingengineoptimization.com/ · Standards: https://anythingengineoptimization.com/standards/