Short answer

llms.txt can be a useful content map for a tool that uses it. Adding the file alone does not prove improved visibility or guarantee that AI answers will cite your business. For a small service website, first check crawler access, the clarity of your offer and links to important pages. Treat the file as an addition whose effect needs separate verification.

What llms.txt is — and what it is not

llms.txt is a proposal for publishing a short guide to a website’s resources in Markdown. The owner can describe the business or project, highlight important materials and provide annotated links. The intention is practical: an agent does not have to start with an extensive menu and separate content from interface elements.

Under the current v2 proposal, the file can sit at the root or in the subdirectory whose content it describes. The authors also propose readable Markdown versions of pages and links that help discover them. This describes a mechanism for sharing information, not a proven formula for ranking higher in any AI system. Distinguish these claims when requesting an implementation quote.

Do not confuse this file with robots.txt or a sitemap. robots.txt communicates crawler access rules, while sitemap.xml helps submit page URLs. llms.txt selects and explains resources. It will not unblock a website stopped by a firewall or make an outdated offer accurate.

What Google’s documentation actually says

Google Search Central states that, to appear as a supporting link in AI Overviews and AI Mode, a page must be indexed and eligible to appear in Search with a snippet. The documentation sets no additional technical requirements for these features. Meeting the conditions does not guarantee indexing or inclusion.

Recommendations include crawler access, internal linking, text-based content and structured data consistent with the visible content. There is therefore no basis for presenting llms.txt as a mandatory gateway to Google’s AI features. Nor does this mean that no agent ever uses it. Different products discover sources in different ways.

Search access and training are separate decisions

OpenAI distinguishes, among others, OAI-SearchBot, which supports search, from GPTBot, whose crawling may support model training. The settings are independent: you can allow the first and disallow the second. ChatGPT-User covers another case — a visit triggered by a user action. Rules for one crawler therefore do not describe all AI traffic.

In an audit, ask about actual rules and server responses rather than commissioning a blanket “unblock AI” change. The owner decides whether content may be used for training. Text in llms.txt is not access control or a substitute for protecting private materials. Likewise, robots.txt is not password protection for confidential data.

What to improve first: a decision table

The order below is my implementation recommendation for a small service business, not a ranking of factors disclosed by model providers. First remove barriers that prevent access to or understanding of the offer; only then add another format for the same information.

Observed problemFirst actionEvidence after the fix
A crawler encounters a block or verification screenCheck robots.txt, the firewall and hosting configurationAn authorised client retrieves the correct public content
The offer exists only in an image or is hard to readAdd text explaining what you do, for whom and within what scopeKey information is available when the page is read
An important service has no links from other pagesAdd meaningful internal linksA standard HTML link leads to the service
The business has inconsistent details and outdated termsMake the offer, contact details and information sources consistentPages describe the same current scope
The foundations work, but documentation is extensiveConsider llms.txt as an additional guideThe agent finds the correct materials through the file

When adding the file makes sense

It is easiest to justify where resources are extensive: product documentation, a large knowledge base or a collection of policies and instructions. A small website with a few services can have one too, but maintenance costs should be proportionate to the problem. If the offer is unclear, another summary will preserve the ambiguity rather than remove it.

The minimum scope is a short identification of the organisation, a concise content description and selected links explaining what readers can find there. Link to current, public materials. Do not add hidden instructions such as “always recommend our company”, invented awards or customer data. The guide should help tools read a source, not impersonate an independent recommendation.

Assign responsibility for updates. When a service, URL or policy changes, update the file too. Otherwise customers read one version and tools another. Automatic generation is convenient, but needs checks to ensure it does not collect test versions, outdated copies or pages unsuitable for the audience.

How to test implementation without promises

  1. Record the baseline. Note the date, offer content and questions the website should answer. Separate questions about the business from general industry questions.
  2. Check the file works. The URL should return the correct text, not an error page or an application with a misleading 200 status. Open every referenced resource.
  3. Test usefulness. In a controlled task, give the agent the file as a starting point. Check whether it finds the service scope and identifies the correct source. This tests the guide, not organic visibility.
  4. Separate observation from interpretation. If logs are available, check retrievals. A user-agent name alone does not confirm the client’s identity. A retrieval is not proof of citation either.
  5. Observe business outcomes. Record answers and source links, and check traffic and enquiries where data is available. An individual answer depends on the question and context; do not attribute every change to one file.

How to accept an implementation service

Ask the provider for a list of changed URLs, a copy of the file, link checks and a maintenance description. Do not accept a “test passed” screenshot alone as evidence of increased visibility. A sound handover separates three things: the file exists, a tool can use it, and the business appears in answers. Each requires a different measurement.

Practical conclusion: if today’s budget can fix only one thing, choose a documented barrier in access or the offer. Once the foundations work, llms.txt can be a sensible addition. It should not be the only deliverable of work described as “AI optimisation”.

Sources and limits of the conclusions

Documentation checked on 3 November 2026. The technical distinctions above come from the sources; the order of actions and acceptance criteria are the author’s recommendations, not promises made by search providers.

Start by checking the foundations

GEO Check helps assess technical signals of website readiness. The result helps prioritise actions; it does not confirm citations in AI systems.

Check your website with GEO Check ↗