A client in the Inner West asked us whether ChatGPT could “read” their site, and whether there was a file for that. There is, sort of. This is what we did about it on our own site before recommending it to anyone, and what we would tell a Sydney business considering the same.
Why we bothered: the question a Sydney client asked
We publish the file on our own site so that the advice we give about it is first-hand.
The proposal for llms.txt came from llmstxt.org, where Jeremy Howard set out a simple idea: a Markdown file at the root of a site that gives a language model the short version of what the site is and where its important pages are.
We could have answered the client’s question from the proposal alone. We preferred to publish one ourselves, keep it current, and watch what happened. Everything below comes from doing that, not from reading about it.
What our two files contain
One file is an index with a sentence per page; the other is the full text of every page in the sitemap.
Our /llms.txt opens with the site name, a one-line summary of what SEO Sydney does, and then one entry per page in the sitemap: the page title, a link, and the page’s own meta description. The entries are grouped the way the site is grouped, services, industry guides, case studies, pricing and this learning hub.
Our /llms-full.txt is the same list with the full text of each page under its heading, so an assistant that wants the detail of a pricing page or a guide can read it without parsing HTML. At the time of writing it covers every page in the sitemap, a little over 180 of them.
Both files say only what the pages themselves say. The descriptions are the ones on the pages; nothing is written specially for the assistants, because a summary that differs from the page is a trust problem waiting to happen.
How the file is generated, and why by script
A script reads the sitemap and each page, writes both files, and a check blocks any deploy where they have drifted.
The generator runs before every deploy of this site. It reads the sitemap, visits each page in the source tree, takes the title and meta description, extracts the body text for the full file, and writes both files.
A second command runs as a gate: it rebuilds the files in memory and compares them with what is about to ship. If a page was added, retired or retitled and the files do not reflect it, the deploy stops. Stale entries are the usual fault in hand-written files; the gate rules them out.
- Retired pages drop out of the file in the same deploy they drop out of the sitemap, so the assistant is never pointed at a redirect.
- Every description matches the page it describes, because it is copied from the page.
- Nobody has to remember to update it; forgetting is the failure mode we designed out.
What the proposal asks for, in one paragraph
A Markdown file at the site root: a heading, a short summary, sections of links with a line each, and an optional section that can be skipped.
The format on llmstxt.org is deliberately small. A single H1 with the site or project name. A blockquote with a one-paragraph summary. Then H2 sections, each a list of links with a short description after each. An “Optional” section marks links an assistant can leave out when it is short on context.
The full-text companion file is a convention rather than part of the proposal: a plain-text dump of the pages so an assistant can read a whole site in one request. We publish it because it costs nothing and some tools fetch it.
What the proposal does not do is tell crawlers what they may access. That remains the job of robots.txt, and the two files should agree: a page you block in robots.txt should not be advertised in llms.txt.
What we can and cannot say happened
We watch AI referrals and crawler visits, and we do not attribute either to the file.
Honesty first. We have no evidence that publishing these files changed where seosydney.com ranks or how often it is cited by an assistant, and we are not going to invent any. The files sit alongside a lot of other work on the site, and nothing isolates their effect.
Google’s own documentation on AI features in Search says there are no additional requirements to appear in them beyond the usual ones. We take that at face value: llms.txt is not a Google ranking input.
What we do see is ordinary: AI assistant domains appearing as referral sources in GA4, and AI crawlers in the server logs. Both are worth a monthly look for any Sydney business that cares about this channel, and neither needs the file to exist.
Should a Sydney business publish one?
Yes if it is generated and kept current; no if it would be written once and forgotten.
For a local service business the file is a few dozen lines: the services, the areas served, the pricing page and the FAQ, each with its real description. If your developer can generate that from the sitemap, publish it. If it would be typed by hand, it will be stale within a quarter.
| Sydney business | Worth listing | Leave out |
|---|---|---|
| Plumber, Sutherland Shire | Service pages, suburb pages, pricing, emergency page | Blog posts about the office dog |
| Law firm, CBD | Practice areas, fee structure, people, contact | Pages you would rather not be quoted from |
| Online retailer | Category pages, shipping and returns, size guides | Individual products that change weekly |
If you are doing broader work on being cited by assistants, the file is a small part of it. Our AI SEO service covers the rest, and the AI Overviews and AI Mode guide explains how the answers choose their sources.
Faults our deploy check guards against
Most llms.txt problems are the same five, and all of them are about the file falling out of step with the site.
- Entries for pages that now redirect or 404. The commonest fault, and the reason ours is rebuilt from the sitemap.
- Descriptions that differ from the page. An assistant quoting the file will contradict an assistant quoting the page.
- The file served as HTML. Some hosts wrap unknown paths in the site template; the response must be plain text.
- A firewall or robots rule blocking the crawlers you wrote the file for. Check the server log for the AI user agents after publishing.
- A file that advertises pages robots.txt forbids. Pick one answer and make both files give it.
Run those five against your own file once a quarter, or build the check into your deploy as we did and never think about it again.
