Crawler evidence
Does anyone fetch llms.txt? sulayman-bowles.dev recorded 249 requests
249 recorded requests to one path; delivered content, reading, and downstream benefit remain unmeasured.

The observer recorded 249 requests to /llms.txt in 90 days: 244 GET and five HEAD. I would keep a short published file accurate, but that count supplies no evidence of reading or benefit. The observer saves request arrivals, leaving returned status and content unmeasured.
A file with working canonical links can have a modest editorial purpose. Its count doesn’t justify replacing readable HTML, ordinary links, or search discovery with an llms.txt strategy.
Method & limits
The .dev-only snapshot uses the linked observer study’s exact 90-day interval ending October 21, 2026 at 20:48:00.901287 UTC. It records accepted arrivals, leaving response status, content, and authenticated identities unmeasured. Reading and causal effects on indexing, citations, or traffic remain untested.
What is in the 249-request count?
Count the exact path. /llms.txt contributes 249 records, while /llms-full.txt contributes another 27 outside the headline. A client can probe either name even when no document exists there.
The five HEAD requests have no document body. GET requests can ask for one, but this dataset stops before recording the returned status or content. Keep those methods identifiable when choosing a delivery check.
| Observed class | GET | HEAD | Total |
|---|---|---|---|
| Self-identified bot | 128 | 5 | 133 |
| Suspected automation | 46 | 0 | 46 |
| Ordinary browser | 70 | 0 | 70 |
| All classes | 244 | 5 | 249 |
Which names did clients claim?
The named labels include bingbot, Googlebot, GPTBot, OAI-SearchBot and ChatGPT-User. They come from User-Agent classification. They are not authenticated provider identities, and some records may be diagnostic or synthetic checks.
The small named totals identify a verification sample. Without authenticated client identities and session-to-answer linkage, they leave provider adoption and use of linked content unknown.
| Claimed name | GET observations |
|---|---|
| bingbot | 28 |
| Googlebot | 14 |
| GPTBot | 7 |
| AhrefsBot | 6 |
| OAI-SearchBot | 3 |
| ChatGPT-User | 2 |
| Chrome-Lighthouse | 2 |
Why a request is a weaker result than a reader
To turn a request into a reader or outcome, independently identify the client, confirm a usable response, verify content consumption, and join that use to an answer or referral, because a client can request a missing path or fetch a file it later discards while producing the same arrival event. This study stops at that first event.
Absence of an observed provider label would also be ambiguous. Retrieval could use a different identifier, a search index, a cache or an intermediary, and the best-effort observer could lose a delivery. The broader snapshot describes the selected routes and capture limits.
The window was not a randomized deployment experiment. Other content, links, publication and search conditions can change during ninety days. Comparing traffic before and after an llms.txt edit would need a design that addresses those changes; these counts do not isolate the file’s effect.
What I would maintain, and what I would measure next
I would keep a published file accurate, concise and connected to canonical public resources. Broken links and stale claims make it a poor guide for any client. That maintenance decision can stand on its small cost and editorial purpose without claiming a measured citation benefit.
For a stronger test, define the desired outcome first: delivered file, subsequently fetched linked document, answer citation or referral. Capture the fields needed to join those stages within the site’s privacy constraints, verify client identity where possible, and retain a comparison. The server-log guide is the starting point for the attribution step.
The result remains 249 recorded requests, including five HEAD requests. A stronger test would begin at delivery and follow a verified client beyond that request.
The two paths reconcile to 276 requests
The CSV preserves exact path, method, observation class, claimed label, verification state, count and first/last event times. Sum only /llms.txt for 249; include /llms-full.txt to obtain 276 across the two names. Blank claimed labels remain unmatched.
The snapshot bundle supplies the extraction SQL, daily denominator, and method without raw visitor records or private network identifiers. I would maintain the file for its small editorial job, then require a delivery-to-outcome comparison before spending on a larger llms.txt strategy.
Sources
- Frozen .dev observer aggregates, SQL and methodSulayman BowlesVerified Oct 21, 2026
- Observer window, privacy projection and measurement limitsSulayman BowlesVerified Oct 21, 2026
- Exact-path and method aggregate CSVSulayman BowlesVerified Oct 21, 2026