Free Audit
Insights · AI Readiness

The Quiet Shift: Why Your Website Needs to Become AI-Ready

Search is shifting from blue links to AI answers. We audited 300 sites — Tranco top-100, a random 100, and South Africa's top 100. Most aren't ready, and here's what to do about it.

By Interon · 12 May 2026

Search is quietly migrating from blue links to AI answers. The sites that get cited in those answers are the ones AI can actually read. Most can’t.

Based on an audit of 300 sites — Tranco top-100, a random 100 from the global million, and South Africa’s top 100.

For 25 years, “being on the web” meant one thing: show up in Google’s blue links. Optimise a few tags, keep your sitemap tidy, win.

That game is ending. ChatGPT, Claude, Perplexity, and Google’s own AI Overviews are increasingly the first answer people see. And the answer they generate depends entirely on whether a model can find, parse, and trust your content. If it can’t, you don’t get cited — and increasingly, you don’t get clicked.

The question isn’t whether your site is on the web. It’s whether the web’s new readers can read it.

The state of play, May 2026

We audited 300 sites across three samples. The results are stark: even the world’s most-visited 100 websites are barely AI-ready by any modern measure.

90%
of the global top 100 have no llms.txt — the emerging “site map for AI”
76%
have no Schema.org structured data — the format AI uses to understand pages
96%
of South Africa’s top 100 have no AI content map at all

Current state vs. where we’ll be in 12 months

AI-readiness signals — today and projected (May 2027)

Pick a sample. Solid bar = adoption today. Hatched extension = projected adoption in one year.
Today (May 2026) Projected (May 2027)
Has llms.txt (AI content map)10% → ~35%
Has Schema.org structured data24% → ~33%
Has /sitemap.xml22% → ~27%
Has Open Graph meta tags44% → ~48%
Blocks at least one AI bot10% → ~32%
Has llms.txt (AI content map)6% → ~28%
Has Schema.org structured data24% → ~30%
Has /sitemap.xml29% → ~33%
Has Open Graph meta tags35% → ~40%
Blocks at least one AI bot12% → ~30%
Has llms.txt (AI content map)4% → ~22%
Has Schema.org structured data30% → ~36%
Has /sitemap.xml32% → ~36%
Has Open Graph meta tags41% → ~45%
Blocks at least one AI bot16% → ~38%

Two trends jump out.

First, llms.txt adoption is about to spike. The standard barely existed 18 months ago. Today it sits below 10%. By next year, expect a third of leading sites to ship one by default — agencies and CMS plugins are racing to make it a checkbox.

Second, AI bot blocking is climbing fast. Cloudflare alone has shifted from opt-in to opt-out blocking of AI crawlers. Sites that don’t want to be scraped are saying so loudly. The flip side: sites that do want to be cited need to actively signal that — silence will increasingly be read as “no.”

What this means for search

Traditional SEO isn’t dying, but it’s being joined by a new layer often called GEO — Generative Engine Optimisation. The mechanics are different:

Old game: rank in the top 10. Get the click. Convert.
New game: get cited in the AI answer. Get the brand mention. Convert later — or never see the user at all, but still influence them.

A user who asks Claude “what’s the best accounting software for a small South African business?” doesn’t see ten links. They see a paragraph. Maybe with two citations. If your site isn’t one of them — and isn’t legible enough for the model to summarise confidently — you’re not in the conversation.

In 2010, not having a mobile site meant losing traffic. In 2026, not being AI-readable means losing visibility before traffic is even on the table.

The South African angle

Local sites are strong on the basics — 81% serve HTTPS properly, better than the global average. But on AI-specific signals they trail by a meaningful margin: only 4% have an llms.txt, less than half the global top-100 rate. This is the typical pattern for emerging markets — fundamentals adopted, cutting-edge optimisations lag by 12–24 months.

That gap is also an opportunity. Being one of the few SA sites that’s properly AI-readable in 2026 is a low-cost way to dominate AI citations in a category before competitors catch up.

What to do this quarter

  • Publish an llms.txtA short markdown file at /llms.txt that summarises what your site offers and links to your most important content. Takes an hour. Almost nobody has one.
  • Add Schema.org structured dataMark up products, articles, FAQs, organisations. This is how AI knows your content is a recipe vs. a review vs. a price.
  • Decide your AI bot policy — explicitlyBlock them in robots.txt, allow them, or set conditions. Don’t leave it ambiguous.
  • Keep the boring basics tightHTTPS, sitemap, Open Graph. None of this is new, but a quarter of top sites still fail on at least one.
  • Write for both audiencesClear summaries, strong headings, factual claims with sources. What helps a model cite you also helps a human trust you.

The bottom line

The web’s new readers don’t scroll, don’t click ads, and don’t care about your hero image. They care whether your content is structured, signed, and easy to summarise. The sites that adapt early to this won’t just keep their traffic — they’ll inherit a citation surface that didn’t exist five years ago.

The audit data is clear: that adaptation hasn’t happened yet. Which means the window to get ahead is still open. For now.

Methodology: 300 sites audited May 2026 — Tranco top-100, random 100 from Tranco 1M, and the top 100 South African sites. Projections are author estimates based on adoption trajectories from Cloudflare Radar, the llms.txt directory, and Schema.org usage trends.

Tags: AI Readiness GEO llms.txt Schema Markup AI Search