41-day test finds AI crawlers miss JavaScript-only links entirely
An SEO tested crawler discovery on a 2,400-page site by splitting internal links between hard-coded HTML and JavaScript-injected versions, then logging bot requests for 41 days, per Search Engine Land. GPTBot, ClaudeBot and Bingbot found none of the JavaScript-linked pages, while Googlebot reached just 2% of them versus 48% for Google’s separate GoogleOther crawler. After the links were converted to HTML, GPTBot recovered 250 of the previously unreachable pages within 48 hours.
Why it matters: Sitemaps and internal-link audits are usually judged by Googlebot's behavior, but this experiment shows the AI crawlers publishers now court most — GPTBot, ClaudeBot — can behave more strictly than Googlebot itself, silently excluding JavaScript-linked content from citation eligibility.
Glossary: SEOAI crawlers
Posted to the wire August 19, 2026.