Google updated its guidance on LLMs.txt for AI SEO & GEO, presenting a notably less discouraging tone than before. The company now states it’s completely fine to create and maintain LLMs.txt files for other services or systems, though doing so won’t harm or help your visibility or rankings in Google Search.
User behavior is shifting fast as people are gravitating toward generative AI experiences to find information, and the pressure to “do something” for AI SEO has never been higher. The data, though, tells a different story than the hype.
We’re going to cut through the noise with evidence, look at what Google actually says, and give you a clear verdict on whether LLMs.txt deserves a place in your workflow.
Key Takeaways
LLMs.txt is a proposed standard for AI-friendly content curation, but the data paints a very different picture than the hype suggests:
- 97% of LLMs.txt files receive zero requests — Ahrefs analyzed 38,000 domains and found no major AI provider currently supports the standard, making it largely ineffective for most sites.
- Google explicitly states LLMs.txt won’t help rankings — June 2026 guidance confirms these files provide no SEO benefit, with John Mueller calling it “purely speculative.”
- Proven GEO strategies deliver real results — Vercel achieved 10% signups from ChatGPT through calculated optimization efforts, not LLMs.txt implementation.
- LLMs.txt only makes sense for technical documentation — The file has a legitimate use case for frequently-updated docs with AI-assisted discovery needs, but skip it for general website visibility.
- Structured content and clear markdown actually move the needle — Simplifying content format, using proper schema markup, and providing authoritative answers deliver measurable results where LLMs.txt falls short.
Unless you’re managing complex technical documentation with specific AI agent integration requirements, your time and budget are better spent on content optimization that actually influences how AI systems discover and cite your content.
What Is LLMs.txt and How Does It Compare to Robots.txt?
Jeremy Howard from Answer.AI proposed the llms.txt standard in September 2024. The idea was straightforward: a markdown file sitting at the /llms.txt path of a website, offering curated overviews, brief background information, and links to detailed markdown documents that language models can consume during inference.
The comparison to robots.txt really only holds up at the file location level. Robots.txt controls what automated tools can access and restricts crawling behavior through disallow directives. LLMs.txt works in the opposite direction — it highlights which content AI should focus on rather than blocking access. There are no disallow rules in llms.txt files.
The format uses markdown with H2 headers to organize resource links by type. Website owners create sections pointing to API documentation, product information, FAQs, or other high-value content they want language models to understand and potentially cite. The expectation centers on inference use cases, specifically when users explicitly request information through AI assistants or development environments that incorporate documentation.
This is also where sitemap.xml enters the picture. A sitemap lists every indexable page on a site, often totaling documents too large to fit in LLM context windows. LLMs.txt provides a more selective alternative by curating only essential resources and can include external URLs that help put site information in context. The distinction is worth keeping in mind: robots.txt manages exclusion, sitemap.xml enables discovery, and llms.txt focuses on curation.
The standard remains a proposal rather than an adopted convention that AI platforms are actively using in their data pipelines.
What Google’s Updated Guidance Actually Says About LLMs.txt
Google’s position on llms.txt depends entirely on which product team you ask. The Search team published guidance in June 2026 stating that websites do not need to create llms.txt files or other special markup to appear in generative AI search features. The optimization guide groups llms.txt with content chunking and AI-specific schema, explicitly listing them under tactics to skip.
Then, days later, Google shipped Lighthouse version 13.3 with a new Agentic Browsing category that includes an llms.txt audit. The tool checks whether sites provide the file and flags server errors when retrieving it. Lighthouse documentation describes llms.txt as providing a machine-readable summary specifically designed for LLMs and AI agents, noting that without the file, agents may spend more time crawling to understand site structure.
So Which Team Is Right?
John Mueller addressed the apparent contradiction head-on. He called llms.txt “purely speculative for now” and noted that, despite existing for years, no AI systems actually use it. His comparison is worth paying attention to: he likened the file to the keywords meta tag, which search engines have ignored for over a decade because site operators control the signal and can manipulate it. Mueller went further and called building separate Markdown pages for bots “a stupid idea”.
The internal signals weren’t encouraging either. At Search Central Live Deep Dive Asia Pacific, Gary Illyes and Amir Taboul confirmed Google was not pursuing llms.txt. When an llms.txt file briefly appeared on Google’s own Search Central developer documentation in December 2025, Mueller responded on Bluesky with “hmmn :-/”, before the file was removed within hours.
That’s not the reaction of someone who endorses the standard. The likely explanation is that an internal CMS platform update generated the file automatically, not that the Search team changed its position.
This doesn’t mean Google is sending mixed signals without reason. Search Central focuses on discovery through Google Search, where llms.txt provides no ranking benefit. Lighthouse evaluates machine interaction for browser-based agents, which is a separate concern from search visibility entirely. The two teams are solving different problems, and it shows in their guidance.
Mueller’s broader point is worth keeping in mind: websites have more important priorities than preparing for a potential future situation that may never materialize. He recommended WebMCP as a clearer alternative with defined goals and processes for agent functionality.
The takeaway here is simple. If your goal is visibility in Google Search or AI-powered search features, llms.txt is not part of that equation.
The Adoption Data Nobody Mentions: Does LLMs.txt Actually Work?
Ahrefs Study: 97% of LLMs.txt Files Get Zero Requests
The numbers are hard to ignore. Ahrefs analyzed 137,000 domains in their Web Analytics platform and found that 28% already publish an llms.txt file. That sounds encouraging until you look at what actually happens to those files. Of the approximately 38,000 domains with valid llms.txt files, 97% received zero requests for them in May 2026. No bots. No humans. The files sat untouched despite proper implementation.
SE Ranking ran a separate analysis across 300,000 domains and found a 10.13% adoption rate. Their team went further and built an XGBoost machine learning model to test whether llms.txt presence correlates with AI citation frequency. The result? The model’s predictions actually improved when the llms.txt factor was removed entirely. The file introduced noise rather than signal into citation behavior patterns. That’s not a neutral result — that’s a negative one.
AI Retrieval Bots Account for Only 1% of Total Requests
Otterly.ai tracked AI bot activity across 90 days. Total AI bot visits to their site reached 62,100+. Total visits to the /llms.txt file during that same period: 84. That’s 0.1% of all AI bot traffic. The average content page on the site pulled approximately 265 AI bot visits over the same timeframe, meaning llms.txt performed three times worse than a standard page.
Search Engine Land monitored 10 sites for 90 days before and after llms.txt implementation. Eight sites saw no measurable change in AI traffic. One site declined by 19.7%, though unrelated factors caused the drop. Only two sites showed increases of 12.5% and 25%, but those gains traced back to content improvements, not the file itself.
Real-World Performance of llms-full.txt Implementations
There is one scenario where the picture looks slightly different. Mintlify reports that data from Profound shows LLMs accessing llms-full.txt more frequently than standard llms.txt. Anthropic specifically requested that Mintlify implement both files for their documentation. The full-text format reduces token costs and parsing complexity, but the use case is narrow — agent-to-agent communication for technical documentation, not general search visibility.
That distinction matters. The data doesn’t say llms.txt is useless in every situation. It says llms.txt is useless for most sites pursuing AI visibility in search.
What Actually Drives AI Visibility: GEO and AEO Strategies That Work
Traditional SEO practices are insufficient for AI-driven discovery. LLMs have fundamentally different needs than traditional search engines, benefiting from clarity, context, and structure in ways that conventional search doesn’t.
Vercel’s 10% signup rate from ChatGPT came from calculated GEO efforts, not SEO. The data demonstrates that optimizing for AI requires a different approach: a simpler format reduces the computational effort required for LLMs to extract meaning from your content.
The emerging best practices center on simplicity. Simplification into markdown works, and so do other methods like providing individual pages in markdown for easier ingestion into LLMs. Think about not just how AI indexes your content, but how users will interact with it through AI interfaces.
Should You Use LLMs.txt?
LLMs.txt can reference structured data markup used on the site, helping LLMs understand how to interpret that information in context. The file is designed to coexist with current web standards rather than replace them. Sitemaps list all pages for search engines, while llms.txt offers a curated overview for LLMs specifically.
The use cases are broader than most people realize: helping developers navigate software documentation, giving businesses a way to outline their structure, breaking down complex legislation for stakeholders, answering questions about someone’s CV, explaining e-commerce products and policies, or providing quick access to course information.
There’s an underrated benefit to creating the file that has nothing to do with whether AI platforms actually parse it. The act of building an llms.txt forces a content audit that most sites desperately need. Which pages represent your deepest expertise? Which content would you want an AI system to cite? Which pages are authoritative enough to stand as primary references?.
That editorial discipline often uncovers content gaps, outdated pages, and structural weaknesses that strengthen your broader LLM optimization regardless of whether any AI platform ever reads the file.
So when does an llms.txt file actually make sense? Create one when you publish documentation that changes frequently, spans multiple sections, or needs to support AI-assisted discovery. REST or GraphQL references benefit from pointing crawlers to canonical endpoints, versioned paths, and Markdown exports. Product guides with frequent feature launches need it to steer AI tools toward current releases and away from outdated content.
LLMs.txt for AI SEO & GEO Conclusion
The evidence speaks clearly: llms.txt won’t move the needle for most sites. Google’s guidance confirms it provides no ranking benefit, and the adoption data shows 97% of files receive zero requests. Skip it unless you maintain technical documentation with specific agent-integration needs.
Your resources deliver better returns when focused on proven GEO fundamentals that include structured content, clear markdown, and authoritative answers with clear search intent.
Are you interested in seeing where AI search is citing your competitors instead of you? Our GEO visibility audit maps exactly where you’re cited, where you’re invisible, and how to close the gap. Our team stands ready to talk and provide a complete audit that will cite you in AI searches.
LLMs.txt for AI SEO FAQs
Is Google’s guidance on LLMs.txt for AI SEO contradictory between different teams?
No, it’s not contradictory—it’s addressing two different use cases. Google Search Central states you don’t need LLMs.txt to appear in AI search results or improve rankings. Meanwhile, Chrome Developers discusses LLMs.txt in the context of agentic browsing, where AI agents navigate your site to complete tasks. The distinction is: LLMs.txt won’t help you get cited in AI answers, but it might eventually help AI agents understand your site structure if agentic browsing becomes mainstream.
Do any major AI platforms actually use LLMs.txt files?
Currently, no major LLM provider actively supports or crawls LLMs.txt files. This includes OpenAI, Anthropic, and Google. Research shows that 97% of websites with valid LLMs.txt files received zero requests for them in May 2026. While Google included LLMs.txt in their Agent2Agent protocol, they haven’t committed to actually crawling these files, making it a speculative standard rather than an adopted convention.
Should I create an LLMs.txt file for my website?
For most websites, LLMs.txt isn’t worth the effort right now. Skip it unless you maintain technical documentation with specific agent-integration needs, or unless an AI platform that brings you clients specifically requests it. The file provides no ranking benefit in Google Search and shows no correlation with improved AI citation rates. Focus instead on proven strategies like structured content, clear formatting, and authoritative answers.
What’s the difference between LLMs.txt and robots.txt?
They serve opposite purposes. Robots.txt controls what automated tools can access by blocking or restricting crawling behavior through disallow directives. LLMs.txt does the reverse—it highlights which content AI should focus on by curating essential resources and providing an overview of your site’s most important pages. Think of robots.txt as exclusion and LLMs.txt as curation.
What actually improves visibility in AI search results instead of LLMs.txt?
Focus on fundamental content quality and structure. Ensure your pages answer questions clearly in the first paragraph, use clean markdown formatting where possible, implement proper structured data and schema markup, and build authoritative content that gets cited on trusted third-party sites. Companies seeing real results from AI search, like Vercel’s 10% signup rate from ChatGPT, achieved it through calculated GEO (Generative Engine Optimization) efforts focused on content clarity and authority, not through LLMs.txt files.