Google plans to support shared AIPREF AI training opt-out in robots.txt
What this answers
- Will Google support the IETF's shared AI training opt-out in robots.txt?
- Yes. Google said on October 30 that it plans to support a shared AI training control in robots.txt, based on the IETF's AIPREF work, and aims to do so alongside other AI providers in 2027. Its crawler documentation says early 2027.
- Does a shared robots.txt AI training opt-out replace Google-Extended?
- No. Google says existing company-specific controls keep working alongside it. If a site opts out of training with the shared token and does not name Google-Extended, Google plans to treat that as a Google-Extended disallow.
Google said on October 30 that it plans to support a shared AI training control in robots.txt, based on the IETF’s AIPREF working group, in 2027 and alongside other AI providers (per the Google Search Central Blog). A site that uses the shared “policy token” to opt out of training, and does not name Google-Extended, would be treated as if it had disallowed Google-Extended. Google says existing company-specific controls will keep working beside the shared signal.
Google’s crawler documentation, changed the same day, adds a note that the company intends to support the signal (user-agent: __usage_ai-training) in early 2027. Google adds that sites which explicitly allow Google-Extended stay eligible to appear in Gemini Apps responses through grounding. The post gives no date for the other providers it expects to join and names none.
Why it matters: Google adopting the IETF's training definition, even as a plan for 2027, is the first sign a major engine will honor one robots.txt signal that other AI providers can also read, after a record showing almost no sites using the IETF vocabulary.
Glossary: Grounding
Via Google Search Central Blog ↗ · Google common crawlers list ↗
Posted to the wire October 30, 2026. Edited by Joe Balewski.