New benchmark: AI guardrails miss most GEO misinformation attacks

A new arXiv preprint introduces Counter-GEO-Bench, a benchmark for testing whether AI safety guardrails catch misinformation delivered through generative engine optimization (GEO) — ordinary-looking documents crafted so large language models retrieve and repeat distorted claims. Across 247 human-verified queries and three victim LLMs the paper does not name, the authors found three off-the-shelf guardrails (Granite Guardian, Llama Guard 3, NeMo Self-Check Fact-Checking) cut attack success by at most 5.7% relative, with Granite Guardian’s reduction not statistically significant — because the guardrails are built to catch policy violations, not fluent but factually wrong text. The authors’ own proposed defense, C-GEO Guard, cut attack success by 47.6% relative with near-zero utility loss. The paper is accepted to EMNLP 2026.