llms.txt generator: how to create and validate your file
By Abhijay Tondak, Founder · Updated July 3, 2026 · 6 min read
An llms.txt generator creates the AI-crawler guidance file that lives at your website's root (like robots.txt) and tells AI engines — ChatGPT, Perplexity, Claude, Gemini — which of your pages are most important to read and cite. The file lists your key pages with titles and descriptions in a simple, machine-readable format. While AI engines don't require llms.txt to discover your content, providing one makes discovery faster, more accurate, and more likely to surface your best pages.
Key takeaways
- llms.txt is an AI-crawler guidance file at your root domain (example.com/llms.txt).
- It lists your most important, citation-ready pages with titles and descriptions.
- The format is plain text with Markdown-style links — simple to create and maintain.
- llms-full.txt is the companion file with full article text for deep reading by AI crawlers.
- Generate it from your sitemap or CMS, then validate it by fetching the live URL.
What llms.txt is and why it exists
llms.txt is the AI-search equivalent of robots.txt. Where robots.txt tells crawlers what they can and can't access, llms.txt tells AI crawlers what they should prioritize reading. It's a plain text file at your root domain that lists your most important pages with their titles and one-line descriptions.
The standard was proposed to solve a discovery problem: AI crawlers visit millions of pages but have limited context about which ones are most valuable to read deeply. Your sitemap lists every page; your llms.txt highlights the ones that are most worth citing. Think of it as a curated reading list for AI engines.
The llms.txt format
The format is intentionally simple. The file opens with your brand name (prefixed with #) and an optional tagline (prefixed with >). Then list your pages using Markdown-style links with descriptions. Group pages by section using ## headings.
- Line 1: # Your Brand Name
- Line 2: > Your brand tagline or value proposition
- Blank line, then ## Pages
- Each page: - [Page Title](https://yoursite.com/page): One-line description of what this page covers.
- Optional: ## Guides section for resource/blog content.
- Optional: ## Contact section with contact details.
How to generate your llms.txt
The fastest approach is to derive it from your sitemap. Pull the URLs from sitemap.xml, add titles and meta descriptions for each, and format them as Markdown links. For sites with hundreds of pages, curate rather than list everything — llms.txt should highlight your 20–50 most important, citation-ready pages, not replicate your entire sitemap.
For the companion llms-full.txt file (which includes the full text of each page for AI deep reading), you'll need to strip HTML and output clean text. This file can be larger — 500KB or more for a substantial content library — and gives AI engines the full context without requiring them to crawl and parse each page individually.
Validating and maintaining your llms.txt
After deploying, validate by fetching the live URL (curl https://yoursite.com/llms.txt) and checking that it returns plain text with the correct format. Common issues: the file returns HTML instead of text/plain (fix your Content-Type header), the file is empty or cached stale (check your deployment), or links are broken (verify each URL resolves).
Maintain it as part of your deploy process. When you publish new content, add it to llms.txt. When you remove or unpublish content, remove it. Citensity generates and updates llms.txt automatically as part of the publishing workflow.
Frequently asked questions
Is llms.txt required for AI citation?
No — AI engines can discover your content through sitemaps, internal links, and crawling. llms.txt makes discovery faster and more focused by highlighting your best pages, but it's not a hard requirement.
Should llms.txt include every page on my site?
No. Curate it to your 20–50 most important, citation-ready pages. AI engines have limited attention — giving them a focused reading list is more effective than dumping your full sitemap.
What's the difference between llms.txt and llms-full.txt?
llms.txt lists pages with one-line descriptions (like a table of contents). llms-full.txt includes the full text of each page — a complete reading package AI engines can ingest without crawling individual URLs.
Put this into practice — free.
Get your free AI-visibility audit and see where engines find you today.
More from this topic
Keep building your expertise with related GEO content in the same cluster.
Structured data (JSON-LD) for AI search
Structured data helps AI engines understand and cite your pages. Here are the JSON-LD schema types that matter for AI search and how to implement them.
ReadHow to write a TL;DR that gets cited
A citable TL;DR answers the page's core question in 1-3 self-contained sentences at the top. Here's how to write one AI answer engines will lift verbatim.
ReadWhy original data and statistics win AI citations
Original statistics and data give AI answer engines something concrete and attributable to cite. Here's why proprietary data outperforms recycled claims in GEO.
Read