Definition
The llms.txt is a Markdown text file in the root directory of a website. It summarises for language models what the website is about and points to the most important pages.
Also: LLMs.txt, llms-txt file
Where the llms.txt comes from
Jeremy Howard published the proposal on llmstxt.org in September 2024. The idea: language models have a limited context window, and an HTML page with navigation, scripts and advertising is laborious for them to process. A short, well-structured text file at /llms.txt provides the most important information in compact form. So far, the file is not an official standard of any standards body.
Structure
The file follows a simple Markdown scheme:
- a first-level heading with the name of the website or project,
- a short summary as a blockquote,
- optionally a few paragraphs with key facts,
- sections with second-level headings, each containing lists of links with a short description per page.
A section headed “Optional” can include pages that a language model may skip when context is tight. In addition, the proposal suggests offering individual pages as a plain Markdown version at the same address with the extension .md.
A shortened example for a fictitious shop service provider:
# Example Ltd
> Example Ltd develops Shopware 6 online shops for retailers in mechanical engineering.
## Services
- [Shopware development](https://www.example.com/shopware/): shops, plugins and integrations
- [Contact](https://www.example.com/contact/): contacts and enquiry formWhat the file can and cannot do
The llms.txt mainly helps tools and agents that need to explore a website in a targeted way, such as coding assistants reading documentation. According to Google, it is not required for Google’s AI search features. It does not replace helpful content and clean data.
Lighthouse now checks in its Agentic Browsing category whether an llms.txt is present. This is an indication of usability by AI agents, not a ranking factor.
Maintenance
An llms.txt quickly goes out of date if it is maintained by hand. It makes sense to generate it automatically on publication from the same data as the sitemap. That is what we do for this website too: our llms.txt is generated on every build from the list of all indexable pages.
You will find more on visibility in AI search under GEO, AEO and AI visibility.