llms.txt: what this small file really does for your website

An llms.txt is a simple text file that sits in the root directory of a website, so at yourdomain.com/llms.txt. It isn't aimed at people but at language models. LLM stands for large language model, exactly the technology behind ChatGPT, Claude or Perplexity.

The idea comes from Jeremy Howard, who proposed it in autumn 2024. The thinking behind it makes sense. A normal website is pretty messy for machines: navigation, cookie banners, scripts, nested menus. A language model has to work its way through all of that to find a few useful sentences at the end. The llms.txt is an attempt to hand it those sentences directly.

Think of it like the note you hand a new colleague on their first day. Not a manual, but the most important things on one page.

It's written in Markdown, a plain text format that people and machines can read equally well. The structure follows a fixed pattern:

• At the very top, the name of the website or company as a heading
• Below that, a short summary in one or two sentences
• Then sections with further information, for example on services or ways of working
• Each section can contain links to the most important subpages, each with a short description

Our own file looks like this: HWM Agentur at the top, below it two sentences on who we are and where we're based. Then come our profile, the most important pages, our services, the technology we work with and the articles from this blog. The whole thing fits on one screen and took a good hour to write.

Some sites additionally create an llms-full.txt, which contains not just links but the complete content. That's mainly interesting for software documentation. A normal company site rarely needs it.

Because all three files sit in the same directory and sound similar, they often get mixed up. Yet they have very different jobs:

• The robots.txt controls which areas of a website a crawler may visit and which it may not
• The sitemap lists all pages so that search engines don't miss anything
• The llms.txt explains what the website is about and what matters on it

The first point is the most important. If you want to prevent AI providers from using your content, the llms.txt won't do it. That's still the job of the robots.txt. The llms.txt is an invitation, not a locked door.

This is where the promises online and reality are furthest apart. The llms.txt is not an official standard. No committee has adopted it, and none of the major AI providers has made a binding commitment to use it.

Google has even been fairly clear about it. People from the search team have said several times that Google doesn't use the file and compared it to the old keywords meta tag that search engines have long ignored. And many site owners who look at their access logs report that the well-known AI crawlers have so far rarely requested the file.

So if someone tells you that an llms.txt will get you to the top in ChatGPT, they're selling you something nobody can prove. We say this so clearly because we're hearing exactly these promises quite often right now.

If the file achieves so little, why is there one on our site? For three reasons:

• It costs almost nothing. An hour of work, created once, updated occasionally
• It is used, just elsewhere. Some programming tools and AI assistants that are meant to read a specific website fall back on it, especially for technical documentation
• It forces you to describe your own business in a few sentences

In our view, the last point is the most valuable. While writing our own file, we noticed how hard it is to say in two sentences what you do and for whom. If you can't do that, you usually have a problem with it on your website too. In that sense the llms.txt is a pretty good exercise, even if not a single language model ends up reading it.

And if the idea does catch on, you're prepared. The risk is zero, the potential benefit somewhere between small and interesting.

If your goal is to appear in the answers of ChatGPT, Perplexity or Google's AI overviews, it's mainly your website itself that decides. These systems access the same content as search engines, and they like the same things:

• Clear copy that answers concrete questions instead of generic advertising language
• A dedicated subpage for each service, so it's clear what you offer
• A clean structure with real headings that makes sense even without design
• Consistent name, address and service details everywhere online
• Mentions on other sites, such as directories, industry portals or the local press
• Structured data that lets machines clearly identify address, opening hours and reviews

That's basically the same work that helps with Google. We've written about this in more detail in our article on why the competitor with the worse website often ranks higher. An llms.txt can complement this work. It can't replace it.

If you want to try it, a simple text editor is enough. This is how we go about it:

• Write the name of your company in the first line with a hash in front of it
• Below that, summarise in two sentences who you are, what you do and for whom
• List the five to ten most important pages of your website, each with a one-sentence description
• Add short sections on services, location and contact
• Save the file as llms.txt and upload it to the root directory of your website
• Open it in the browser at yourdomain.com/llms.txt and check that everything is readable

Write it the way you'd explain it to a person. No list of search terms, no exaggerations. What's in the file should also be found on your website in exactly the same way, otherwise the picture doesn't add up in the end.

And remember to maintain it. An llms.txt that still lists services you stopped offering long ago is worse than none at all.

The llms.txt is a good idea that is still waiting to be truly needed. It doesn't hurt, it costs hardly anything, and it helps you get your own message to the point. Today it isn't a lever for more enquiries.

If you want to know how search engines and AI systems currently read your website, feel free to get in touch. We'll look at it together with you and tell you honestly where work pays off and where it doesn't. And if you like, we'll write your llms.txt along the way.