llms.txt file with “File does not appear to contain any links” when the file uses plain URLs instead of Markdown link syntax. Treat this as an audit-compatibility check, not proof that llms.txt drives AI-search visibility. If you keep an llms.txt file, use simple Markdown links such as [Guide title](https://example.com/guide/), keep the file short, and verify it with Lighthouse; do not build a separate Markdown mirror just to satisfy AI tools. Source: Search Engine Journal.llms.txt, special AI text files, Markdown versions, or new machine-readable markup to appear in Google Search, AI Overviews, or AI Mode. Google also says maintaining an llms.txt file for other systems is fine, but it will not help or hurt Google rankings. The practical takeaway remains: keep llms.txt lightweight if useful for non-Google discovery, but prioritize crawlable HTML, clear answers, strong sources, and internal links. Source: Google Search Central generative AI guidance.Short answer: do not treat llms.txt as a primary AI search ranking lever yet. Ahrefs’ new crawl-data study says 97% of observed llms.txt files were never requested, so SEO teams should keep the file lightweight, but put most effort into crawlable pages, clear entities, original evidence, internal links, and measurement.
Editorial update: Ahrefs shipped a free llms.txt generator
Updated July 24, 2026: Ahrefs published a free llms.txt generator and said it was based on analysis of 137,000 domains. This does not make llms.txt a ranking factor, but it is useful as a governance artifact: teams can document which pages, docs, and resources they want AI crawlers and retrieval systems to discover first. Source: Ahrefs.
SGO action: keep robots.txt and XML sitemaps as the crawl-control foundation, then use llms.txt as a concise content map for high-value guides, tools, pricing/comparison pages, policies, and source material. Review it monthly so it does not become a stale mirror of the sitemap.
Editorial update: llms-author.txt is not the next llms.txt priority
2026-07-06 llms-author.txt update: Google has now addressed questions about an experimental llms-author.txt file, and the practical SEO takeaway is conservative: do not treat unofficial LLM-specific files as a replacement for crawlable pages, clean source pages, internal links, sitemaps, and a valid llms.txt. Search Engine Journal also reported that some llms.txt checks fail when files do not use normal Markdown links. For SGO teams, the useful action is not to add another speculative file; it is to keep llms.txt simple, source-linked, and consistent with the pages you actually want answer engines to cite.
Sources: Search Engine Journal on Google and llms-author.txt; Search Engine Journal on Markdown links in llms.txt checks.
Update: Google cautions against Markdown mirror sites for AI SEO
Editorial update, June 24, 2026: Search Engine Journal reports that Google cautioned against publishing separate Markdown versions of websites purely for AI SEO. This reinforces the practical reading of the Ahrefs llms.txt study: focus first on crawlable HTML, clear page structure, canonical URLs, internal links, and source-quality signals. An llms.txt file can be a discovery aid, but it should not become a parallel content system or a substitute for a well-structured site.
Action for site owners: if you maintain Markdown mirrors, audit canonicalization, duplication risk, freshness, and whether the mirror is actually used by target AI systems. Source: Search Engine Journal.
Source: Ahrefs, “97% of llms.txt Files Never Get Read”. This is timely because many GEO and AI-search checklists have started treating llms.txt as if it were already equivalent to robots.txt, sitemaps, or structured data. The evidence is not there yet.
SGOinsights action view
Where llms.txt fits now
Use a simple file as an experimental discovery aid.
No current evidence that major answer engines rely on it broadly.
Indexable content, source clarity, sitemaps, schema, and citation-worthy evidence.
Related: If your AI visibility work includes tools, APIs or MCP-style resources, read our tactical guide to Agentic Resource Discovery and SEO readiness.
What changed
Ahrefs analyzed 137,000 sites with llms.txt files and reported that 97% of those files were never requested. Among requests that did happen, Ahrefs found a mix of identifiable AI bots, auditing tools, and other crawlers rather than broad proof that answer engines are using the file as a standard input.
That matters for generative engine optimization because the file has become a popular “AI visibility” talking point. It may eventually matter more, but right now it should be handled as an experimental signal, not as the center of an AI search program.
What SEO teams should do next
- Maintain the basics first: make sure important pages are indexable, internally linked, included in XML sitemaps, and easy for crawlers to understand.
- Use
llms.txtonly as a lightweight directory: link to canonical guides, methodology pages, documentation, pricing, comparison pages, and important resources. - Do not block strategic crawlers accidentally: review robots.txt, CDN firewall rules, bot protections, and server logs before assuming “AI search cannot see us.”
- Measure actual access: look for requests to
/llms.txt, AI crawler user agents, and referral patterns, but separate crawler activity from visibility and conversions. - Invest in citation assets: original research, clear definitions, comparison tables, process diagrams, and named methodology pages are more defensible than a file alone.
How this affects GEO and AEO work
For GEO, the practical takeaway is prioritization. A clean llms.txt file can be part of a technical discovery layer, but it does not replace content quality, retrieval-friendly structure, author/source signals, or entity clarity. For answer engine optimization, teams should still focus on concise answer blocks, well-labeled sections, FAQ-style coverage where appropriate, and pages that directly answer the query.
A practical llms.txt policy for now
SGOinsights’ recommendation: create or maintain llms.txt if it takes little effort, but cap the investment until stronger platform evidence appears. Treat it like a discovery hint, not a ranking factor. If you have limited time this week, update your strongest content and measurement workflow before polishing the file.
Useful next steps: run the AI Search Optimization Checklist, review the SGO Playbook, and test key pages with the GEO Readiness Scanner.
FAQ
Should every site have an llms.txt file?
It is reasonable to have one if it is simple to maintain, but the Ahrefs study suggests it should not displace higher-impact SEO and AI-search work.
Does llms.txt improve AI citations?
There is not enough public evidence to say that it improves citations. Citation potential still depends more on useful, accessible, trustworthy content.
What should llms.txt include?
Include a short list of canonical, high-value URLs: core guides, product or service pages, methodology pages, help documentation, research, and key resources.
How to use this analysis
This article is most useful when it turns into a short action list. For SEOs deciding whether llms.txt deserves workflow time, the practical question is not only what happened, but which pages, templates, measurements, and publishing habits should change because of the Ahrefs llms.txt study and what SEO teams should do next.
Start by mapping the idea to one live page or workflow. Check whether the page explains the topic clearly, supports important claims, gives readers a next step, and connects to related guides or tools. If the article points to a platform shift, add a follow-up review date because AI search behavior can change quickly.
What to monitor next
Monitor whether the same pattern appears in Search Console queries, analytics referrals, AI answer citations, brand mentions, and competitor source appearances. One observation is rarely enough. Repeated appearances across queries and answer engines are stronger evidence that the topic deserves a content update, technical fix, or new resource.
- Record the queries or prompts affected by the change.
- Compare cited sources against your own page structure and evidence.
- Update internal links when a related guide or resource gives readers the next useful step.
- Refresh the article if platform documentation or visible behavior changes.