Reference
Current Best Practices
What the record supports, practice by practice. Every entry carries the date it was last verified and the references behind it — because in this field, an undated best-practices page is fiction within a quarter.
Maintained by the editor; practices are re-verified against the record and re-dated, revised, or retired. This page sells nothing — the wire offers no optimization services.
-
Decide your AI crawler policy deliberately — and per bot
AI companies run separate crawlers for model training, search indexing, and live retrieval, and the trade-offs differ: blocking a training bot withholds content from future models, while blocking an index or retrieval bot removes you from today's answers. Set robots.txt per user-agent with that distinction in mind, and re-check the strings periodically — companies rename crawlers as products rebrand, and a rule pointed at yesterday's string silently stops applying.
-
Put the answer where a machine can lift it
Answer engines quote passages, not pages. Content that answers a searchable question in a self-contained, quotable block — near the top, in plain declarative language — is retrievable and citable in a way that the same facts diffused across twelve paragraphs are not. Question-shaped headings and explicit Q&A blocks make the mapping from query to passage trivial.
-
Ship structured data for what the page actually is
Schema.org markup — NewsArticle for news, FAQPage for question-and-answer content, DefinedTerm for definitions, Organization and Person for who stands behind it — is machine-readable self-description, and it is cheap. Mark up what the page genuinely is; structured data that misdescribes a page is a trust liability, not an optimization.
-
Be unambiguous about your entities
Knowledge graphs and answer engines resolve content to entities — companies, products, people. Give each entity you own a canonical hub page, keep names consistent, and declare external identities with sameAs links to official sites and reference pages. Ambiguity about which thing you mean is a retrieval failure you inflicted on yourself.
-
Wire the freshness signals: sitemaps, IndexNow, WebSub
Engines can only be as current as their discovery. Maintain an XML sitemap (a news sitemap if you publish news), ping IndexNow on every publish for the engines that accept it, and declare a WebSub hub on your feeds so subscribers are pushed rather than left to poll. None of this improves content; all of it shortens the gap between publishing and being retrievable.
-
Publish llms.txt — and the full-content companion
llms.txt is a proposed standard: a machine-readable index of what your site offers, with llms-full.txt carrying the content itself for systems that want the corpus. Whether the major engines read it is still an open question the record tracks — but adoption tooling now exists, the cost is near zero, and the audience it addresses is the one this discipline optimizes for.
-
Get your first-party AI numbers where they exist
The engines have started shipping first-party AI-surface data: Search Console added generative-AI performance reports, and Bing Webmaster Tools previews citation-share and AI-visibility insights. First-party beats scraped estimates wherever both exist — verify your site in both consoles and take the numbers the engines will actually give you.
-
Date everything, visibly and in markup
Answer engines weigh freshness and increasingly surface dates alongside citations. Publish visible dates, keep dateModified truthful in markup, and never launder old content as new — an engine that catches a laundered date has a durable reason to distrust the domain. Honest dating is both a trust signal and a defense.
-
Watch the engines' documentation, not just their announcements
The engines' most consequential changes ship quietly — help-page edits, crawler renames, policy language — hours or days before anyone announces them, if anyone ever does. Watch the primary documentation pages that govern your visibility and diff them on a schedule. The announcements are marketing; the documentation is policy.