The first time it matters is usually a small thing. A customer rings up and says, "Your website told me you'd waive the call-out fee." Someone checks the transcript and, sure enough, the assistant said something close to that, drawn from an old promotion page nobody remembered was still live. Then comes the real question: whose job was it to notice?
In most small businesses, nobody's. The assistant was set up by whoever was keenest, it worked well for a few weeks and then it became part of the furniture. Nobody reads transcripts. Nobody checks the unanswered questions. Nobody is sure who is allowed to change the instructions.
Human oversight of AI sounds like something for banks and hospitals. For a business with six staff, it just means one named person who looks after the assistant the way a good supervisor looks after a new starter.
You own what it says
Start with the uncomfortable part. When your assistant tells a visitor a price, a policy or a timescale, the visitor hears your business saying it. They will not accept "the software got it wrong" any more than they would accept "the temp got it wrong".
That is not a reason to avoid an assistant. It is a reason to manage one. A grounded assistant answering from your own content is far easier to keep accurate than one improvising, but "easier" still means someone has to do it. The content drifts, prices change and new questions appear that nobody wrote an answer for.
A lightweight ownership model
Big organisations draw diagrams for this. A small team needs to answer a handful of questions and write the answers down. Here is an example for a five-person business; change the names and roles to fit yours.
| Responsibility | Owner | Backup | How often |
|---|---|---|---|
| Overall accountability for the assistant | Owner (Helen) | None | Ongoing |
| Weekly review of conversations and gaps | Office manager (Tom) | Helen | Weekly |
| Updating prices, services and policies in the content | Tom | Helen | When anything changes |
| Deciding what actions are switched on | Helen | None | Quarterly, or when adding one |
| Taking over live conversations | Whoever is on duty | Tom | Daily, per rota |
| Handling complaints raised through chat | Helen | Tom | As they arrive |
| Checking leads and tickets are followed up | Sales lead (Amir) | Tom | Daily |
Two things matter more than the exact split. Every row has a name, and the person who can change what the assistant says is not a mystery. If three people can edit instructions and none of them tell the others, you will get contradictions and no way to trace them.
The weekly review
The manager's regular job is short and concrete. A routine that works for many small teams:
- Read the unanswered questions list. Each one is either a content gap to fill or a topic the assistant should keep declining.
- Read ten conversations at random. Not the best ones, random ones. Note anything wrong, awkward or off-brand.
- Read every handover and complaint. These show where the assistant struggled or annoyed someone.
- Check the analytics. Conversations, leads, ratings and response time. Look for sudden changes rather than obsessing over the numbers.
- Ask the team what has changed. New prices, a service dropped, a holiday closure, a supplier delay.
- Make the edits and log them. One line each: date, what changed, why.
Our guide to supervising an AI employee goes into what to look for in those transcripts.
A worked example of the time involved
Treat these figures as an illustration and swap in your own.
Say your assistant handles 150 conversations a month, roughly 35 a week. In the first four weeks the manager might spend three hours a week: reading more transcripts, writing Q&A pairs for the gaps and tuning wording. Call that 12 hours for the month.
From month two, the review settles into the six steps above: about 15 minutes on unanswered questions, 20 minutes on the random ten, 15 minutes on handovers and complaints, and 10 minutes on edits and the log. That is an hour a week, or about four hours a month.
Compare that with the time it would take to answer 150 conversations yourself, and with the cost of one customer holding you to a promise the assistant should never have made. If your volume is triple that, the review time goes up, but not threefold, because the same gaps tend to repeat.
What the manager decides, and what they escalate
A good manager for an assistant has clear authority over some things and passes others up. It is worth writing both down.
- Decides alone. Wording changes, new Q&A pairs, removing outdated pages from the content, adjusting the welcome message.
- Agrees with the owner. Switching on a new action such as webhooks or ticket creation, changing what the assistant may say about prices, changing handover rules.
- Escalates immediately. Anything that looks like the assistant gave advice it should not have, shared the wrong customer's information or made a promise about money.
Every action is off by default, which is the right starting point. Each one switched on is a decision somebody should own. Our post on deny-by-default permissions explains why that matters.
Write the one-page policy
This does not need a consultant. One page, kept somewhere the team can find it, covering:
- The manager's name and their backup.
- What the assistant is for, in two or three lines. A job description is a useful model.
- What it must never do, such as promise refunds, give professional advice or ask for payment details.
- Which actions are on, and who approved them.
- The review routine and where the change log lives.
Your first step
Put a name next to the assistant today. Not a department, a person. Give them the weekly routine above, block an hour in their diary for the same time each week and tell the rest of the team that changes go through them.
Then start the change log with a single entry describing how the assistant is set up now. When something goes wrong in three months, that log is where the answer will be. The features page lists the settings worth recording in it.