The assistant has been live for a month. It seems fine. Leads arrive, nobody has complained, and the owner has stopped opening the dashboard. This is the exact point where most website assistants start to drift.
Prices change and the PDF does not. A new service launches and the assistant has never heard of it. One particular question gets asked every other day and answered badly every time, and nobody notices because nobody is looking.
The fix is not constant monitoring. It is a short, fixed routine that you supervise an AI employee with, the same way a good manager has a regular one-to-one rather than hovering. Here is one that fits into thirty minutes.
The agenda at a glance
Book it for the same slot each week. Friday afternoon works for many firms, because the week's conversations are fresh and fixes can go in before the weekend traffic.
| Minutes | Task | What you finish with |
|---|---|---|
| 0 to 10 | Read a sample of conversations | A short list of bad or clumsy answers |
| 10 to 18 | Work through the unanswered questions | Content added or updated |
| 18 to 25 | Check handovers and actions | Notes on anything that stalled or misfired |
| 25 to 30 | Glance at the numbers and choose one change | One written action for the coming week |
If you regularly run over, sample fewer conversations. The discipline of stopping at thirty minutes is what keeps the routine alive past the second month.
Minutes 0 to 10: a sample, not the whole pile
Reading everything feels thorough and is the quickest way to abandon the habit. Pick a sample with a bias towards trouble:
- Every conversation with a poor visitor rating.
- Every conversation where the visitor asked for a person.
- Any conversation that ended without an answer and without contact details.
- Three or four chosen at random, including at least one that looks perfectly ordinary.
The random ones matter more than they seem. Problem conversations show you failures. Ordinary ones show you the slow, polite mediocrity that never generates a complaint but quietly loses enquiries: answers that are technically right, long-winded, and end without suggesting a next step.
Resist the urge to fix things as you read. Stopping to rewrite a document after the first bad answer eats the whole half hour, and you lose sight of whether that answer was a one-off or part of a pattern. Note it, move on, and fix the patterns in the next block.
For each conversation, ask one question. Would I be happy if a new member of staff had said that? If not, jot down why in a few words.
Minutes 10 to 18: clearing the knowledge gaps
The unanswered questions list is the most useful report you have, because it is written by your customers. Each entry is something a real visitor wanted to know and could not find out.
Work through it in this order. Anything asked more than once goes first. Then anything tied to money: prices, fees, deposits, cancellations. Then the rest, if time allows.
For each gap, decide where the answer belongs. Often it should go on the website, not just into the assistant, because if a visitor could not find it by asking, other visitors cannot find it by reading either. Turning unanswered questions into website pages shows how to do that without cluttering your site.
Then ask the assistant the original question, worded as the visitor worded it. Only close the gap once the answer is correct. What to do when your agent keeps saying I don't know is useful when the list is unexpectedly long.
Minutes 18 to 25: handovers and actions
This part checks the joins between the assistant and your team, which is where things most often go wrong in practice.
- Handovers: did a person join each transfer request promptly, and did they have to ask the visitor to repeat themselves? If they did, the context being passed is not good enough.
- Leads: open three captured leads at random. Could someone call each one back without re-asking the basics?
- Tickets: if you use them, were they complete, and did each get picked up by the right person?
- Webhooks: spot-check that a recent lead actually appeared in your CRM or Slack with every field filled.
If an action is causing more tidying than it saves, it is fine to switch it off while you fix it. Starting your AI employee with no permissions covers when to do that.
Minutes 25 to 30: numbers, then one change
Look at the trend lines rather than the totals: conversations, leads, ratings and unanswered questions compared with the previous few weeks. You are looking for a sudden change, not a target.
Here is a worked example with numbers you would replace with your own. Say last week showed 140 conversations, 18 leads and 11 unanswered questions. This week shows 150 conversations, 9 leads and 12 unanswered questions. Traffic is steady, gaps are steady, but leads have halved. That points away from content and towards lead capture itself: perhaps a greeting changed, an action was switched off, or the page the assistant lives on was redesigned.
Then write one change for next week. Only one. "Add the new boiler service price list" or "Shorten the welcome message on the contact page". A review that ends with seven actions usually ends with none.
When the routine tells you something bigger
Now and then the weekly review surfaces a problem that thirty minutes cannot fix. The same kind of question fails week after week despite new content. Visitors keep asking for a person about a topic where the assistant has plenty of information. Emotional or complaint conversations are going badly.
Those are usually design problems rather than content problems, and they deserve a separate session. Where AI employees fail sets out the common patterns and how to contain each one.
Set it up before you close this page
Put a recurring thirty-minute slot in your calendar now, with the agenda table pasted into the invite. Name a deputy who does it when you are away. Start a plain running document with one line per week: the date and the one change you made. After two months that document is the clearest record you will have of whether the assistant is improving, and it takes about ten seconds a week to keep.