llms.txt

llms.txt is the AI-era counterpart to robots.txt — a plain-text index at the root of your site telling ChatGPT, Claude, and Perplexity which pages actually matter.

llms.txt is a proposed plain-text file — modelled on robots.txt — that sits at the root of a website and tells large-language-model crawlers which pages to prioritise, how they are structured, and what the site is actually about. It was introduced by Jeremy Howard in September 2024 as a convention, not a standard: no single body enforces it, but OpenAI, Anthropic, Perplexity, and other AI companies have signalled informal support. The format is Markdown with a title heading, a short blurb, and lists of canonical URLs grouped by section. Because AI crawlers already pull HTML from the public web, llms.txt is less about access control and more about curation — a way for the site owner to say 'these are the pages that actually matter, in plain English, without the navigation chrome'. In 2026 the file is on perhaps one in a thousand sites, mostly AI-adjacent companies, and is quietly spreading.

What this site ships

This site ships a llms.txt at /llms.txt listing the homepage, key writing essays, location pages, industry pages, comparisons, resources, and the audit tool — each with a one-line description. AI crawlers from Anthropic, OpenAI, and Perplexity are explicitly allowed in robots.txt. The file is static, under 10 KB, and regenerated whenever the typed content layer changes, which keeps it canonical with the rest of the site.

Why citation in AI overviews is worth a ten-minute setup

Citation from AI overviews is the new ranking signal nobody budgets for. Google's AI Overview, ChatGPT, Perplexity, and Claude are already surfacing specific pages with specific quotes in answer boxes — and the pages that get cited are the pages with clear, quotable passages and discoverable structure. llms.txt is a free, cheap-to-maintain hint to those crawlers that pays off for years and costs an afternoon to ship.

Related terms


— From the studio

Bespoke, live in seven days.

£1,000 flat. Paid upfront. Full refund if I miss day seven. Code, hosting, and domain all in your name.