llms.txt and MCP Endpoints
What is a .well-known AI file?
A .well-known AI file is a machine-readable document served from a site's standardized /.well-known/ directory, the path prefix defined by RFC 8615, that advertises AI capabilities to agents and crawlers. Common examples declare a callable plugin manifest or an MCP discovery endpoint. Bassethound treats a .well-known AI file as a machine-facing signal in the AI-readiness layer, because publishing one means a site is building for agents to consume, not just humans.
Best move: request the fixed /.well-known/ path directly. Do not infer AI-readiness from marketing copy.
Why it works: the /.well-known/ prefix is a standard (RFC 8615), so an AI file lives at a predictable URL. You fetch it, you parse it, you know. No render, no guessing.
Key takeaways
- A .well-known AI file is machine-readable metadata served from the RFC 8615 /.well-known/ path, aimed at agents rather than human readers.
- The AI variants declare either a callable capability (a plugin or API manifest) or a discovery endpoint (MCP), so an agent knows what it can invoke.
- Because the path is fixed, detection is a direct static fetch with no JavaScript rendering required.
- One machine-facing endpoint flips a site to shipping in Bassethound’s verdict logic, and a .well-known AI file is one of them.
- Auth-gated, geo-fenced, or fully server-side files stay invisible to an outside crawl, and Bassethound reports those as gaps instead of guessing.
What lives in a site’s /.well-known/ directory?
RFC 8615 reserves the /.well-known/ path prefix so software can find metadata at a predictable location instead of scraping a page for it. You already depend on this. security.txt tells researchers where to report a vulnerability. openid-configuration bootstraps an auth flow. apple-app-site-association wires up deep links. IANA keeps a registry of those suffixes, and the pattern behind all of them is one rule: put machine-readable config where a program can fetch it by convention, not by luck.
An AI file follows the same contract. Rather than a human reading your docs and hand-writing an integration, an agent requests one fixed URL and gets structured instructions back. The directory is deliberately boring. That is the value. A boring, standardized path means a crawler does not have to reverse-engineer your site to learn what it exposes. It asks the address everyone agreed on and reads what returns. When Bassethound checks a domain, /.well-known/ is a first stop, because a file there is a decision a site made on purpose, not a dependency that slipped in with a build tool.
Which .well-known files signal AI readiness?
Two patterns matter. The first is a capability or plugin manifest. A file like .well-known/ai-plugin.json declares an API an agent can call, usually with an auth scheme and a pointer to an OpenAPI description. The second is a discovery file that hands an agent the address of a protocol endpoint, and MCP is the current one worth watching. Either way, the file exists for a non-human client.
Bassethound groups both under machine-facing signals. When one fires, the AI-readiness verdict moves toward shipping before you count model SDKs, orchestration frameworks, or vector stores, because a discovery file is intentional infrastructure and not an incidental import. A site that ships one has thought about agents as consumers of its surface. That intent is the signal. The exact filename matters less than the category: is this a document a browser renders for a person, or a contract a machine reads to act? A .well-known AI file is the second kind, and that is why it carries weight the homepage never will.
How is a .well-known AI file different from llms.txt?
Different jobs, different locations. llms.txt sits at the site root (/llms.txt), not under /.well-known/, and it speaks to language models reading your content. It is a curated map of what to read and in what order. A .well-known AI file speaks to agents acting on your site. It declares what they can call and how to authenticate.
Bassethound classifies llms.txt in two flavors: standard, a human-curated content map, and technical, a file that points at an endpoint or exposes an API surface. That second flavor and a .well-known AI file play the same role. Both are machine-facing, and both push the verdict toward shipping. The distinction to hold onto is content versus capability. llms.txt in its standard form helps a model understand you. A .well-known AI file and a technical llms.txt let a machine transact with you. A site can publish both, one, or neither, and each combination tells you something different about how far along its agent strategy runs.
Can you detect a .well-known AI file without running JavaScript?
Yes, and this is where the convention earns its keep. The file lives at a fixed path, so Bassethound requests it directly. No DOM to scrape, no client to render, no guessing at URLs. Static fetch, parse, done. That makes .well-known one of the cleanest signals in the whole dossier: the path either returns a valid document of the declared type or it does not.
The honest limit is response gaming. A server can gate the path behind auth, restrict it by region, or answer a request with a 200 that is really an HTML error page. Bassethound checks that the body matches the declared type rather than trusting the status code alone, and when it cannot confirm a valid file it reports the path as a gap. A guessed signal would poison the score, so an unconfirmed .well-known path counts as unknown, not as a miss and not as a hit. You get the read, and you get told where the read stopped.
Why does a machine-facing endpoint outrank a marketing page?
Anyone can type “AI-powered” into a hero section. Publishing a .well-known AI file costs engineering time and commits you to a contract an agent will hit for real. That asymmetry is the entire case for reading the stack over reading the story. A machine-facing endpoint is expensive to fake and pointless to fake.
The ai_readiness score counts distinct signal groups, capped at six, and a .well-known file counts as one of them. In the verdict logic it carries more than its share: a single machine-facing endpoint flips a site to shipping without any other signal firing. It is a load-bearing signal, and it maps cleanly to the icp_signal. A site that has published a discovery file has decided that agents are customers, and that decision predicts budget and roadmap better than any adjective on the homepage. When you rank domains by who is building for agents, a file at a fixed path beats a page full of claims every time.
Bassethound perspective
Most AI-readiness tools count adjectives. They scan the homepage for “AI,” “LLM,” and “agent,” then hand back a score built on marketing copy. We think that is backwards. A single .well-known AI file tells you more than a hundred hero-section buzzwords, because someone had to build it, host it at a fixed path, and keep it valid for a client they will never meet. That is a commitment, not a claim.
So Bassethound weights a machine-facing endpoint hard enough that one flips the verdict to shipping on its own, no SDK census required. A competitor scoring by keyword density would call that overfitting to a rare signal. We call it reading intent. The site that publishes a discovery file has decided agents are customers, and that decision predicts buying behavior better than any headline. We would rather miss a loud site with no endpoint than pass a quiet one that ships. And when we cannot confirm the file, an auth gate or a soft 404, we say so, because a guessed signal is worse than an honest gap.
Sources
- RFC 8615, Well-Known Uniform Resource Identifiers (URIs): https://www.rfc-editor.org/rfc/rfc8615
- IANA Well-Known URIs registry: https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml
- Model Context Protocol documentation: https://modelcontextprotocol.io
- llms.txt specification: https://llmstxt.org
Frequently asked questions
Is a .well-known AI file the same as llms.txt?
No. llms.txt lives at the site root and targets language models reading your content, a curated map of what to read. A .well-known AI file lives under /.well-known/ and targets agents acting on your site, declaring what they can call and how. Bassethound tracks both as machine-facing signals but classifies them separately.
Do you need JavaScript to detect a .well-known AI file?
No. The file sits at a fixed path, so Bassethound requests it directly and parses the response. If the path is auth-gated, geo-fenced, or returns a soft error page, Bassethound reports it as a gap rather than a hit.
What standard defines the /.well-known/ path?
RFC 8615 reserves the /.well-known/ path prefix for well-known URIs, and IANA maintains the registry of those suffixes. An AI file follows the same contract as security.txt or openid-configuration: predictable location, machine-readable content.
Does one .well-known AI file make a site count as shipping?
Yes. One machine-facing endpoint flips a site to shipping in Bassethound's verdict logic, and a .well-known AI file counts as one. It gets there on its own, before Bassethound tallies a single SDK or vector store.
Can a .well-known AI file be hidden from an outside crawl?
Yes. If the path is protected by auth, restricted by region, or resolved entirely server-side behind a proxy, an outside read cannot confirm it. Bassethound checks the response type, not just the status code, and marks unconfirmed paths as gaps.