Why llms.txt Files Won't Help You Sell More

97% of published llms.txt files received zero HTTP requests in a month - that's the finding of a 2026 Ahrefs study of 137,000 domains, and yet this file still keeps landing in AI marketing plans as a visibility item that someone later has to defend to the board. An llms.txt file is a single technical artifact with negligible measurable effectiveness, while an AI visibility strategy requires process, data, and measurement - and it's usually exactly this that's missing under this item, and this exact gap decides whether you can defend the decision when someone asks about impact.

Context: shortlists are moving into AI

In B2B, especially in the mid-market and enterprise segments, purchasing decisions increasingly begin inside AI systems, and classic Google Search is losing its status as the first point of contact. The sentence "51% of buyers start in AI. I don't know whether we're even there" captures a growing sense of risk: if a brand doesn't appear in the answers of ChatGPT, Claude, Perplexity, or Google AI Overviews, it disappears from the shortlist before any lead ever reaches the website. This process links to trends visible in SEO for years - a rising share of queries ending on the results page, rising CPC in paid campaigns, and drops in organic traffic despite maintained rankings - and each of these trends adds pressure to the pipeline.

In AI-based search, success increasingly means appearing in the answer without a click: the brand name mentioned in the answer, the page cited as a source, a recommendation in a generated shortlist of vendors. This is a zero-click case with more weight than a classic featured snippet, because the user's decision can be made entirely inside the model's interface - and that's why the lack of presence in answers hurts faster than a drop in Google rankings.

Pressure on pipeline and quota makes every shortcut to AI visibility sound attractive, and boards expect CMOs and VPs of Sales to justify every budget line with hard data - and this happens exactly at a time of content budget cuts and hiring freezes across teams. The question "does this item in the AI marketing plan have a real impact on pipeline" comes up almost every time then, and the person who put llms.txt in the plan and checked off the AI visibility topic is left with a decision they can't defend. AI visibility without measurement is the most expensive form of peace of mind: the item looks closed, and nobody can say exactly what it changed.

The trap mechanism: how the file gets into the plan

The trap rests on three elements that work together.

• Where the item comes from: vendor narratives sell llms.txt as "robots.txt for AI" and a "source of truth for models," and deploying the file is supposedly enough to open the company up to generative answers.
• Why it looks like action: the file can be generated, uploaded to the server, and shown in a project report as a closed item, so in a presentation it looks like a concrete result next to tasks that carry real weight.
• Why the lack of impact doesn't hurt: nobody assigned a measurement to this item, so there's never a moment when the absence of traffic on the /llms.txt path becomes visible, and the question about impact only comes up when the budget gets cut.

The only way to treat the file that will hold up before the board is as a cheap hedge for the future: publish it as a minor add-on with zero maintenance cost and account for it outside your AI visibility strategy, with the clear assumption that its current impact is zero. Any other framing of this item - a visibility item, a GEO element, "securing presence in AI search" - amounts to a promise without measurement that you won't be able to defend in a budget conversation.

What llms.txt is

The llms.txt file is a community proposal by Jeremy Howard of Answer.AI, announced in 2024, meant to give AI models a model-friendly summary of a website's key content. In its basic form it's a text file in Markdown format placed in the domain's root directory at the path /llms.txt, containing a short description of the project or company and organized lists of links to the most important resources and documentation. The specification defines structural requirements: an H1 heading with the company name, a short description in blockquote form, and sections with links under H2 headings.

The extended specification for business use cases emphasizes that the file should present a structured picture of the organization - who the company is, what services it provides, where it operates, how it can be contacted, and what categories of information models should understand to correctly represent this entity. In the creators' vision, llms.txt acts as a map and reading list for AI agents, sparing them the costly need to crawl the entire site - an intuitive and attractive argument, because it promises a large effect at low cost. The key caveat concerns reading: the specification does not guarantee that models will read the file at all, let alone that they will prioritize the page over others.

llms.txt nonetheless remains a community convention with no formal support from major model vendors, and the difference relative to robots.txt and ai.txt is crucial for a budget decision. Robots.txt has decades of established practice and de facto IETF standard status: all major crawlers respect it, and the interpretation process is widely documented. Ai.txt is an emerging standard, under active work within the IETF - it lets site owners define policies for AI content use across three modes (train, retrieve, cite) and has growing declared adoption from GPTBot, ClaudeBot, Google-Extended, and PerplexityBot. The conclusion for a budget decision is simple: today only robots.txt has decades of established practice, ai.txt is only beginning to build it, and llms.txt operates on the goodwill of the agent.

Query volume: 97% of files with no requests

The strongest evidence against treating llms.txt as a strategy comes from server logs and three independent studies that measured the same phenomenon using different methods.

• Ahrefs, a 2026 study of 137,000 domains: about 28% of domains, i.e. 38,000 sites, had a correctly published llms.txt file, and 97% of them received zero HTTP requests in the analyzed month. Of the 3% of files that received any traffic at all, 96% of requests were generated by bots, but among them SEO tools, auditors, and research systems focused specifically on llms.txt dominated - the share of named AI bots (GPTBot, Claude-Code) and citation-generating crawlers (ChatGPT, Perplexity, Google-Extended) was statistically negligible. So even this group doesn't prove that models actually use the file.
• OtterlyAI, a 90-day experiment: monitoring over 62,100 AI bot visits found that only 84 visits, about 0.1% of all AI bot activity, hit the /llms.txt path, and that path's performance was three times worse than the average performance of other pages on the same sites.
• SE Ranking, an analysis of nearly 300,000 domains: no correlation between the presence of llms.txt and the frequency of domains being cited in AI answers.

Three independent methodologies - server logs (Ahrefs), bot monitoring (OtterlyAI), and citation correlation modeling (SE Ranking) - lead to a single conclusion: none of the studies found a statistically significant increase in AI bot visits or citations linked to llms.txt. This sentence is worth remembering in a board-ready version, because the convergence of three methods is the strongest argument in a budget conversation. llms.txt query volume is close to zero, and belief in this file rests on vendor narratives, while measurement shows a picture disconnected from those promises.

For CMOs and VPs of Sales, these numbers have practical implications: if you list "llms.txt implementation" in your AI marketing plan as an item that improves brand visibility in AI, the statistical chance that any AI model crawler will query this file is about 3%, and the chance that it's the crawler responsible for citations is even lower. That's an activity profile you can't defend to the board as an investment in pipeline - and that's exactly the kind of defense every item in this quarter's plan requires, especially amid content budget cuts.

Google's and model vendors' position

Google has addressed this issue several times, and very clearly. In its AI optimization documentation from May 2026, Google states that you don't need to create new machine files or special formats, such as llms.txt, to appear in Google Search results, including its generative capabilities. In June 2026, the position was clarified further: llms.txt files "neither help nor hurt" visibility or rankings, because the search engine ignores them, and creating them for other services that might use them is completely fine.

John Mueller of Google's search team explained why the file isn't a ranking signal: it's self-reported by the site owner and contains self-promotional claims that can't be trusted as a way to differentiate sites. Google doesn't endorse the llms.txt standard, and mentions of the file in developer documentation came from automated CMS support and were removed by the search team.

From a budget-decision perspective, the search engine's stance on llms.txt remains neutral: you can keep the file as a cheap add-on with zero impact on Search visibility, but tying it to improved visibility in this key channel has no basis in Google's documentation. This distinction is worth keeping handy when someone in a presentation cites "Google's support for llms.txt."

Other model vendors hold a similar position: as of mid-2026, none of them has publicly stated that its production systems read llms.txt and use the file's content when generating answers. Log studies also show no citation-generating crawler systematically querying this file.

The one clear case of llms.txt being used in practice is Perplexity's documentation, which recommends that agents use the /llms.txt file as a resource map while discovering the full documentation index. This is an example limited to Perplexity's own documentation: the company controls both the site and the agent, and the file acts here as part of an internal architecture for agents that have been explicitly directed to it - without such an instruction, AI model crawlers don't reach for this file.

Why crawlers bypass llms.txt

To understand the low query volume, it's worth looking at the architecture of AI systems. Most modern models acquire knowledge of the world through two channels: pre-training on massive text corpora (Wikipedia, Wikidata, Common Crawl, books, code, and open sources) and retrieval at inference time, when the model queries the web to enrich the answer with current information. In both cases, what matters is the breadth and quality of the corpus and the consistency of entities, and a single file on a site remains irrelevant to these mechanisms.

In pre-training, whether content is present depends on whether it made it into the corpus, and in retrieval, entry to a site is decided by links, signals, and instructions - the /llms.txt path lies outside these mechanisms, which the logs confirm: without an explicit agent instruction, AI model crawlers skip this file. Data from all three studies confirms this mechanism in practice: the file works only when someone explicitly points an agent to it.

What works instead

Instead of relying on an artifact that goes unread in 97% of cases, CMOs and VPs of Sales build confidence on a set of actions with proven impact on AI visibility. Each action has an assigned measurement, and together they form a steady process for measuring content impact that can be shown to the board as numbers:

• Working on entities and consistent sources about the brand. Measurement: presence and consistency of the company's entity in Wikidata, Wikipedia, and external sources.
• Schema.org and Schema Graph. Measurement: an entity graph readable by models, i.e. the completeness and interconnection of structured data on the page.
• Controlling AI robots via robots.txt and ai.txt. Measurement: train, retrieve, and cite policies and declared adoption by GPTBot, ClaudeBot, Google-Extended, and PerplexityBot.
• Monthly monitoring of citations in AI model answers. Measurement: retesting the same queries and the difference in answers between consecutive months.
• The 20-query test. Measurement: a list of category queries and the answer to whether the brand appears in the results. At AGAPE, I run this test as a repeatable retest - the same questions come back every month, and the difference in the answers is my reading of the brand's visibility in AI.
• Content atomization, 1 into 8. Measurement: the number of content variants present in zero-click channels.

This set has one shared trait: every item can be shown to the board as a number or a difference between two readings. And that trait is what separates an AI visibility strategy from a list of tasks whose impact ends in a project report.

The question you ask the vendor

Every item in an AI marketing plan should have an assigned way of measuring its impact and a link to pipeline, and before approving the next item for this quarter, it's worth asking the vendor one question: how will you measure the impact of this item and tie it to pipeline? Ask for a number, not a description.