How to create an llms.txt: structure, example & guide

An llms.txt is a small Markdown file in the root directory of your website that tells AI systems such as ChatGPT, Claude and Perplexity what your site is about and which pages matter. This article explains what the file is, how it is structured, shows a complete example and walks you through both ways of creating one: by hand, or in two minutes with the generator. It ends with an honest assessment of what you can expect from it – and what you cannot.

What is an llms.txt?

The proposal was published by Jeremy Howard (Answer.AI) in September 2024 and is documented at llmstxt.org. The idea is simple: HTML pages are hard for language models to read – navigation, ads, cookie banners and scripts bury the actual content, and a model's context window is limited. An llms.txt instead provides a curated, clean overview: what this website is, which pages count, and where an AI system finds the content in a readable form.

Three things matter to put the file in perspective:

• It is a proposal, not a standard. No AI provider has committed to reading the file. It is, however, used by documentation platforms, developer tools and a growing number of company websites – and AI crawlers demonstrably fetch it.
• It controls nothing. Unlike robots.txt, it neither allows nor blocks access. It delivers content – whether and how it is used is up to each AI system.
• It is Markdown. No programming language, no tooling required. Any text editor will do.

Structure of the file

The specification defines a fixed order. Only the heading is required; everything else is optional but recommended:

1. H1 heading (# Name of the website) – the only required element. The name of the project, company or store.
2. Blockquote (> Short description) – one or two sentences explaining what the site is about. This is the most important text in the file: it is the first thing an AI system learns about you.
3. Free text (optional) – paragraphs, lists, notes. Anything except further headings. This is the place for audience, region, specialities.
4. H2 sections with link lists (## Section) – the actual overview. Each entry follows the pattern - [Title](URL): description. Typical sections: Products, Categories, Guides, Documentation, Contact.
5. "Optional" section (## Optional) – a reserved keyword. Everything below it may be skipped by an AI system when context is limited. Ideal for legal pages, archives or secondary pages.

What does not belong in the file: cart, login, checkout, search result pages, tag archives, author pages. They dilute the overview and tell an AI system nothing about your offering.

Complete example

This is what an llms.txt for a small online store looks like. The example is fictional but follows the specification exactly and can be used as a template:

Note the details: every description contains what a customer would ask about – roast level, origin, quantity, price. These are exactly the facts an AI system needs to mention your product at all in an answer to "Which espresso for a bean-to-cup machine under €10?". A list of bare URLs helps nobody.

llms.txt, llms-full.txt and robots.txt – the difference

Three files, three jobs. They do not replace each other; they complement each other:

• robots.txt governs access: which crawler may load which areas. If you block GPTBot or ClaudeBot there, any llms.txt is useless – the crawlers never get as far as the file. The guide to robots.txt for AI crawlers shows how to check this.
• llms.txt is the overview: compact, curated, with links and short descriptions. It should stay small – a few kilobytes, so that it fits in any context window.
• llms-full.txt is the full text: the content of the most important pages as Markdown in a single file, so an AI system does not have to fetch every page individually and strip the HTML. It may be considerably larger and is optional.

For most websites the llms.txt alone is enough. The llms-full.txt pays off when your pages contain a lot of explanatory content – guides, documentation, detailed product descriptions.

Creating an llms.txt by hand

For a website with a handful of important pages, the manual route is perfectly sufficient and takes about half an hour:

1. Open a text editor – Notepad, TextEdit, VS Code, Notepad++. Not Word, not Google Docs (they insert invisible formatting).
2. Lay out the structure following the pattern above: H1, blockquote, sections with link lists. Use the example as a guide and replace the content.
3. Save as llms.txt with UTF-8 encoding. The file name is exactly that, lowercase, with no additional extension.
4. Upload to the root directory – via FTP or your host's file manager, into the same folder as robots.txt or index.php.
5. Verify: open https://your-domain.com/llms.txt in a browser. If the text shows up, the file is in the right place. Then test format, structure and links with the llms.txt Validator.

Why the file pays off for stores

The following part is aimed mainly at store operators who want to use the generator – no technical background required. When a customer asks ChatGPT, Claude or Perplexity "Which espresso machine under €500 is recommended?", the AI searches the internet for relevant sources. With an llms.txt file, you help AI systems better understand your products and offerings and consider them in answers.

With an llms.txt file in your website's root directory, you provide AI with structured product data: real prices, availability, ratings and direct links. The goal: the AI gets the chance to mention your products with correct information and a link to your store.

With the generator in 2 minutes: prerequisites

You only need two things:

• Your shop URL (e.g. www.my-store.com)
• FTP access or file manager at your host (for uploading the finished files)

No technical knowledge required, no login, no installation. The generator runs entirely in the browser.

Step 1: Enter Shop URL

The three trial generations already include all Pro features: product drill-down, intent routing and all 5 file formats.

Step 2: Analysis & Drill-Down

This process typically takes 30–90 seconds, depending on the size of your shop. You can see the progress live in the log.

Step 3: Check results

Step 4: Download files

llms.txt – The main file of the standard. Compact overview of your store with categories, selected products and intent routing. This is the file that AI systems read first.

llms-full.txt – Extended version with complete descriptions and all product details. Recommended, as AI systems find deeper information here.

llms-data.json – Machine-readable version with semantic tags, product comparisons and structured intents. For AI APIs and advanced applications.

llms-meta.yaml – Metadata about the generation. Optional, for developers and build systems.

robots-llms.txt – Snippet for your robots.txt. Only necessary if your robots.txt actively blocks AI crawlers.

Step 5: Upload to web root

After uploading, the files should be accessible at these addresses:

Does it work? How to test

Simply open https://www.your-store.com/llms.txt in your browser. If the content is displayed, everything is set up correctly. Additionally, you can test your AI visibility:

1. Ask ChatGPT or Claude about one of your products
2. Monitor over the next weeks whether AI correctly reflects your product data
3. Check your server logs to see if GPTBot, ClaudeBot or PerplexityBot are fetching your llms.txt – or, more conveniently, use AI Bot Tracking

Formal errors are caught by the llms.txt Validator: 10 checks for structure, tokens, links and sections – free, no login.

Honest assessment: what does an llms.txt deliver?

A lot is promised around llms.txt. Most of it cannot be substantiated, and we stick to what can be measured.

What is certain: AI crawlers fetch the file. We see it in the logs of websites that use bot tracking – PerplexityBot, ChatGPT-User, ClaudeBot and others request llms.txt and llms-full.txt regularly. So the file is being read.

What nobody can guarantee: that an AI system will therefore mention or cite your site in an answer. Whether and how the content flows into answers is decided by each system, and no provider has publicly committed to it. Anyone promising you rankings, citations or revenue growth from an llms.txt is selling something they cannot deliver.

Why we recommend it anyway: The effort is half an hour, or two minutes with the generator, the risk is zero, and the file is the only way to hand AI systems your content in the form they process best. There is also a side effect many underestimate: if you have to summarise your website in 30 lines, you quickly notice whether your offering is clearly stated. That helps humans too.

If you want to check the effect yourself, use AI Bot Tracking for the question "is the file being fetched?" and the AI Visibility Check for the question "is my brand mentioned unprompted?". More on measurability in the guide Does llms.txt work? What can be measured.

How often to update?

The freshness of your llms.txt affects the quality of AI recommendations:

• Shops with frequent price changes: Update weekly
• Shops with stable inventory: Monthly is sufficient
• Static websites: When major changes are made

With a Pro or Agency subscription you can automate updates via cron job – your llms.txt will always stay up to date.

Frequently asked questions

What is an llms.txt?

An llms.txt is a plain-text file in Markdown format that lives in the root directory of a website at /llms.txt. It gives AI systems such as ChatGPT, Claude or Perplexity a curated overview of the most important pages with short descriptions. The proposal was published by Jeremy Howard (Answer.AI) in September 2024 and is documented at llmstxt.org.

Where does the llms.txt have to be located?

In the root directory of the domain – the same place as robots.txt and sitemap.xml. It must be directly accessible at https://your-domain.com/llms.txt. Subfolders or subdomains do not count as a root file.

How is an llms.txt structured?

An H1 heading with the name of the website (required), followed by a blockquote with a short description, optionally a paragraph of free text, then H2 sections containing link lists in the format "- [Title](URL): description". A section named "## Optional" marks content that AI systems may skip when context is limited.

What is the difference between llms.txt and llms-full.txt?

The llms.txt is the compact overview with links and short descriptions. The llms-full.txt contains the full content of the most important pages as Markdown, so an AI system does not have to fetch each page individually. The two files complement each other; robots.txt, by contrast, only controls whether crawlers may access the site at all.

Does an llms.txt really improve AI visibility?

Honestly: there is no guarantee. No AI provider has publicly committed to reading the file, and nobody can promise a citation in an AI answer. What is measurable is whether AI crawlers fetch the file – bot tracking shows that. The effort is small, the risk is zero, and a clean llms.txt forces you to state clearly what your website is about. Nobody should promise more than that.

How often should I update the llms.txt?

Whenever your offering, structure or key pages change. Weekly for stores with frequent price changes, monthly for stable catalogues, and only after major changes for static websites. With a Pro subscription, updates can be automated via cron job.

CMS-specific quickstarts

Jump to the matching guide: llms.txt for Shopware, WooCommerce, Shopify, PrestaShop, Magento, JTL-Shop, Gambio, Drupal and the WordPress plugin.

Further Reading

• Improve AI visibility — The complete strategy from robots.txt to llms.txt
• robots.txt for AI crawlers — GPTBot, ClaudeBot & 11 more crawlers properly configured
• Schema.org for online stores — Structured data for Rich Snippets and KI-Recommendationen
• llms.txt for Shopify, WooCommerce & Gambio — Plattform-spezifische guideen