Did AI Crawlers Read Our llms.txt? Five Months of Data

On our own site, intrix.kr, AI crawlers made zero requests for llms.txt or llms-full.txt over about five months. The same recorded AI traffic fetched robots.txt 1,586 times and sitemap.xml 619 times. For marketers deciding where to spend their next development budget, those results support prioritizing the files crawlers actually requested.

Data as of October 28, 2026 (our measurement). The extended measurement covers May 31 through October 28, 2026, on intrix.kr alone.

What did AI crawlers actually request?

We counted requests from AI crawlers in our server logs. Across 22,514 recorded AI requests from 23 crawler types, neither llms.txt nor llms-full.txt received a crawler request. Both files were available throughout the measurement period.

Robots.txt and sitemap.xml had a different record. The same AI traffic repeatedly requested them while leaving the llms.txt files untouched. This was a site with active AI crawler traffic, rather than a site where the absence of requests could be explained by crawlers never arriving.

The distinction matters when evaluating a recommendation. Publishing a file and having a crawler retrieve it are different outcomes. On our site, the request record supports the use of robots.txt and sitemaps as crawler-facing resources. It does not provide the same support for llms.txt.

Did the result change when the measurement window grew?

Our earlier measurement covered 75 days, from May 28 through August 10, 2026. Across 1,622 AI requests from 10 crawler types, llms.txt and llms-full.txt received zero crawler requests. We use that earlier result here only to show that the finding held when we extended the measurement.

The later measurement covered about five months and substantially more AI traffic. Requests for the llms.txt files remained at zero. The date ranges differ slightly, so these are separate reporting windows, rather than figures to add together.

For a team reviewing an existing llms.txt recommendation, the longer window makes the finding more useful. The absence of crawler requests persisted as traffic accumulated. It still describes the behavior recorded on our own site.

Does this mean llms.txt is useless?

No. These figures answer a narrower question: did the AI crawlers captured in our logs fetch these files from intrix.kr during the measurement period? They did not. The figures do not establish whether the content would help an AI system if that system received it.

They also do not measure the effect of a person directly giving an llms.txt link to an AI tool. That is a different use case from a crawler independently requesting the file. A document might be useful when deliberately supplied without being part of a crawler's normal visits.

Our logging coverage is another boundary. These figures include requests classified as bot traffic. Requests that were not classified that way are outside this count. The result should not be extended to every possible visitor, every AI service, or every website.

Calling llms.txt universally useless would go beyond the evidence. So would telling a customer that publishing it means AI crawlers will read it. Our measurement supports a specific statement: the crawlers recorded on intrix.kr did not fetch it during this period.

What should marketers and founders prioritize instead?

Give robots.txt and sitemaps attention in the website work you already fund. On intrix.kr, these were resources that AI crawlers repeatedly requested. Their recorded use gives them a firmer basis for priority than an assumed benefit from publishing llms.txt.

For a founder, this changes the question to ask about a proposed deliverable. A file appearing on your domain is a completed publishing task. Evidence that an intended reader uses it is a separate business consideration. A proposal should make clear which outcome it is offering.

For an agency, the implication is similar. Describe llms.txt as the document you are providing, and explain its intended use without promising crawler adoption. If it is included in a broader engagement, its presence should not stand in for evidence that AI systems find or represent the client's company.

Maintaining robots.txt or a sitemap does not, by itself, establish that an AI answer will cite your company. Our data concerns requests for files. A request is evidence of retrieval, rather than evidence that a company was recommended or that an answer described it accurately.

How should a business judge an AI visibility proposal?

Look for a clear connection between the work being sold and the result being discussed. An llms.txt deliverable addresses publishing. A crawler request addresses retrieval. An answer that mentions your company addresses how the business appears to a potential customer.

Keeping those outcomes distinct makes budgets easier to assess. A technical task may be reasonable without proving a sales benefit. When the proposed benefit is visibility in AI answers, the review should reach those answers rather than stop at the existence of a file.

That is the practical lesson from our site. The files that attracted repeated crawler requests deserve attention. Claims about llms.txt adoption need evidence from the setting in which the claim is being made. The next business question is how AI answers actually describe your company.

Frequently asked questions (FAQ)

Q. Should we delete an existing llms.txt file?
This measurement does not establish a reason to delete it. It establishes that the AI crawlers recorded on intrix.kr did not request it during the measurement period. Review the file's intended use before assigning more work or budget to it.

Q. Do robots.txt and sitemap requests prove AI will cite us?
No. They show that crawlers requested those files on our site. They do not establish citation, recommendation, or an accurate description of a business in an AI answer.

Q. Does this result apply to every website?
No. It comes from intrix.kr alone and covers the AI crawler traffic captured in our logs. It is evidence for questioning an assumed benefit, rather than a universal claim about llms.txt.

To check how AI answers describe your own company, contact Intrix.