A two-week trial is the most reliable way to choose a website chatbot: one week testing setup and answer quality with your own questions, and one week in front of real visitors on a quiet page. Set pass marks before you start, run two tools side by side, and the decision on day 14 mostly makes itself. The plan below lists what to do each day and what counts as a pass.
If you only have an afternoon, our one-afternoon evaluation is a faster first filter. This plan is for the shortlist that survives it.
Before day one: set the bar
Do this once, before signing up anywhere.
- Collect 20 real questions from your inbox, contact form and phone notes, in the customer's own words.
- Add 5 questions your website doesn't answer, such as a service you don't offer or a policy you've never written down.
- Write the correct answer to each, briefly, so scoring is quick.
- Pick two tools, three at most. A shortlist from a checklist like what a small business should look for in an AI chatbot works well.
- Note your monthly conversation volume, so pricing can be checked against it later.
Pass marks for the whole trial: at least 18 of 20 answered correctly from your content, zero invented answers on the 5 unknowns, and every lead and handoff test arriving where it should.
Week one: setup and answer quality
Day 1: setup. Create the bot from your own website, not a demo site. Time how long it takes to get a working answer. Pass: under an hour, without contacting support.
Day 2: fill the gaps. Add anything the crawl missed: your price list, returns policy or opening hours, as documents, Q&A pairs or pasted text. Pass: you can see what the bot has learned and edit or delete individual entries.
Day 3: the 20 questions. Ask each one and score it right, partly right or wrong. Pass: 18 or more right. A confident wrong answer counts as wrong, not partly right.
Day 4: the 5 unknowns. Ask the questions your site doesn't answer. Pass: the bot says it doesn't know and offers a next step every time. One invented price or policy is a fail for the whole tool, because it will happen to customers.
Day 5: rewording and languages. Re-ask five questions with typos, slang or a vague phrasing like "how much?". If you have overseas customers, ask two in another language. Pass: answers stay correct, and vague questions get a clarifying question rather than a guess.
Days 6 and 7: fix and retest. Correct the content behind every failure, then re-run all 25. Pass: failures go down, and nothing that passed before now fails. Some tools help here. SpideyChat, for instance, lets you save test questions with the text an answer must contain and re-run them, flagging any that used to pass and now don't.
Week two: real visitors and the workflow
Day 8: install on one quiet page. Put the widget on a low-traffic page, such as a contact or FAQ page, using the script tag or plugin. Check it on mobile. Pass: it loads, doesn't cover your cookie banner or menu, and only runs on the domains you allow.
Day 9: leads. Play a customer who wants a quote and leaves an email. Pass: the lead reaches the right inbox with name, contact details and what they asked for, without anyone opening a dashboard.
Day 10: handoff and tickets. Ask for a person while someone on your team is available, then again when nobody is. Pass: a person can join when online, and when nobody is, the visitor gets a clear next step, such as a ticket reference or a promise of a reply, rather than a dead end.
Days 11 to 13: read real conversations. Read every transcript from the quiet page. Note questions the bot couldn't answer and any reply you would be embarrassed by. Pass: no embarrassing replies, and the unanswered list is short and fixable.
Day 14: usage and cost. Look at how much of the allowance the trial used, scale it to your monthly volume, and price the plan you would actually need, including branding removal and extra websites. Pass: the plan that covers your busiest month fits your budget.
How to score day 14
Put the results in one table, pass or fail per stage:
| Stage | Tool A | Tool B |
|---|---|---|
| Setup under an hour | ||
| 18 of 20 correct | ||
| Zero invented answers | ||
| Rewording and languages | ||
| Lead arrives complete | ||
| Handoff or ticket works | ||
| Real conversations clean | ||
| Busiest month fits budget |
Then decide in this order. First, drop any tool that invented an answer. Second, drop any tool whose leads or handoffs didn't arrive. Only then compare price and nice-to-haves. A cheaper bot that makes things up costs more than it saves.
Keep the trial honest
Two habits stop the trial drifting. Use the same 25 questions for every tool, word for word. And write down what you changed to fix each failure: if one tool needed an hour of rewriting and the other needed a Q&A pair, that is part of the result.
Most free plans are big enough for this, as long as you count usage. The free trial test plan shows how to budget a small allowance across a test set.
The quickest way to start is day zero in miniature: paste your website address into the free test, ask a handful of your 20 questions, and see whether it earns a place on the shortlist.