Implementation & How-To· 6 min read

Training an AI Agent on a Messy Website

Most small business websites are a bit of a mess. Here is how to get good answers out of one without rebuilding it first.


Almost nobody has a tidy website. There is a 2019 pricing page nobody deleted, a services list that stops halfway, and a PDF brochure with the old phone number.

You do not need to fix all of it. You need to fix the parts that produce wrong answers.

The one thing that actually breaks it

Contradictions. Two pages saying different things is worse than one page saying nothing.

Retrieval finds the closest match by meaning, not by date. When your site contains "from £49" and "from £65", nothing tells it which is current — so it picks one, confidently, and it will be wrong roughly half the time.

The single highest-value hour you can spend is searching your own domain for prices, phone numbers and service names, and making them agree.

Delete rather than update

An old page nobody links to is still live, still crawlable, and still retrievable. "Nobody can find it" is not true of a crawler.

If a page is out of date and not worth rewriting, delete it. This is faster than updating and it removes the contradiction permanently.

Five pages will carry you

You do not need a content programme. You need:

  1. What you do — the actual services, in the words customers use
  2. Where you cover — one sentence, specific
  3. What it costs, or what decides the cost
  4. What happens after someone gets in touch
  5. An FAQ of the ten things you are genuinely asked

That is an afternoon, and it will outperform a fifty-page site of marketing prose.

Write in the customer's words

Your website says "drainage remediation". Your customers type "blocked drain".

Retrieval matches meaning, so it will usually cope — but writing it the way people say it removes the guesswork, and it helps your search rankings at the same time.

Structure beats volume

Content is retrieved in chunks. A page where each section has a heading and answers one question retrieves cleanly. A single 2,000-word block of prose retrieves badly, because the relevant sentence arrives surrounded by four paragraphs of context that dilute it.

Short sections. Real headings. One idea each.

What to skip

Blog posts. Good for search, weak for answering. Marketing prose dilutes retrieval, which is why some products deprioritise it by default.

Your about-us history. Nobody asks.

Scanned PDFs. An image of text is not text.

The honest bit

If your website genuinely does not explain your business, no agent will fix that. But the work of fixing it is worth doing anyway — you will get better enquiries, better rankings and fewer phone calls asking things you thought were obvious.

Point one at your site and ask it your five most common questions. Where it struggles is exactly where your website is thin.

Frequently asked questions

Do I need to rebuild my website first?
No. Four or five focused pages will carry an agent perfectly well, even if the rest of the site is dated.
What hurts answer quality most?
Contradictions. Two pages saying different things is worse than one page saying nothing, because retrieval cannot tell which is current.
Should I exclude old pages from the crawl?
Yes, if they contain outdated prices or discontinued services. Better still, delete them — an unlinked page is still crawlable.
Do PDFs work?
Text-based ones do. A scanned brochure is an image with no extractable text, so nothing useful comes out of it.

Keep reading

Training an AI Agent on a Messy Website · SpideyChat