Your chatbot dashboard says 4,000 conversations this month. Great. Is that good? You have no idea, because the number doesn't connect to anything you care about. It could mean the bot is a hit, or it could mean your site is so confusing that thousands of people had to ask for help.
Vanity metrics like total conversations feel productive to watch and tell you almost nothing. The metrics that matter are the ones that track back to revenue earned or cost saved. Here's how to find those signals in the noise and read them without fooling yourself.
Resolution rate over raw volume
The first number worth your attention is how often the bot actually finished the job. Call it resolution rate: the share of conversations where the customer got their answer and didn't need a human afterward.
This matters because it separates helpful volume from busywork. A thousand conversations that all ended in "let me get a human" isn't support, it's a queue. A thousand that resolved cleanly saved your team real hours. Track resolution rate over time, not as a single snapshot. A number climbing week over week as you fix content gaps is the clearest sign the bot is getting healthier.
Be honest about what counts as resolved, though. A customer who gave up isn't resolved, even if they stopped typing. Look at whether they got a real answer, not just whether the conversation ended.
This is where a lot of dashboards quietly lie to you. Many tools count any conversation that didn't escalate to a human as a "success," which lumps genuine resolutions together with people who got frustrated and closed the window. Those are opposite outcomes wearing the same badge. If your resolution number looks great but your transcripts are full of dead ends, the number is measuring silence, not satisfaction. Read a sample of the conversations behind the metric every week. The number tells you the trend, but only the transcripts tell you whether the trend is real.
Qualified leads, not just captured emails
If your bot captures leads, the tempting metric is total emails collected. Resist it. An email with no intent behind it is a line in a spreadsheet, not revenue.
The metric that predicts money is qualified leads: contacts where the person showed real buying signals during the chat. They asked about pricing, described a specific need, requested a quote. Ten of those are worth more than a hundred addresses grabbed from people who just wanted your hours.
Sort your captured leads into two piles and watch the ratio:
- Qualified: asked about pricing or plans, described a project, requested a demo or quote, gave a timeline
- Unqualified: wanted support, checked policy, grabbed a coupon with no further interest
The count of qualified leads, and how it trends, is a number your sales pipeline will actually feel. In SpideyChat you'd tag or route these based on what the person asked, so the qualified ones reach a human fast while the rest get handled automatically.
Conversation-to-conversion rate
Now connect the bot to actual sales. Of the people who chatted, how many went on to buy or book? That's your conversation-to-conversion rate, and it's the closest thing to a direct revenue signal a chatbot has.
You don't need a perfect attribution model to start. Compare the purchase or signup rate of visitors who chatted against those who didn't. If chatters convert at a noticeably higher rate, the bot is doing revenue work, not just answering questions. If they don't, something in the conversation is falling flat, and the transcripts will usually tell you what.
One caution keeps this metric honest: people who start a chat are often already more interested than people who don't, so some of the gap is self-selection, not the bot's doing. That's fine. You don't need to prove the bot caused every extra sale to make decisions with this number. What you're watching is the direction and the size of the gap over time. If it widens as you improve the assistant, you're onto something. If it stays flat no matter what you change, the conversations aren't pulling their weight, and that's worth knowing before you invest more in them.
Watch this by page and topic, too. The assistant might convert well on your pricing page and poorly on a product page, which points you straight at where to improve.
Handoff quality, because a bad exit costs you
Every conversation the bot can't finish is a fork: a clean handoff that keeps the customer, or a frustrating dead end that loses them. Measuring handoff quality keeps that fork honest.
Track how often a stuck conversation turned into a real ticket with context attached, versus how often it just ended. A handoff that captures the question, grabs contact info, and reaches your team with the full transcript protects the relationship. One that leaves the customer staring at "I didn't understand" hands them to a competitor. This metric rarely makes the headline dashboard, but it's the difference between the bot saving a sale and quietly costing you one.
Put it in a single view
You don't need a data team for this. A simple table you update weekly does more than most fancy dashboards, because it forces the metric next to its meaning:
| Metric | What it really tells you | Watch for |
|---|---|---|
| Resolution rate | Real support load lifted | Trend up over time |
| Qualified leads | Pipeline the bot fed | Ratio of qualified to total |
| Conversation-to-conversion | Direct revenue effect | Chatters vs non-chatters |
| Handoff quality | Sales and trust protected | Clean tickets vs dead ends |
Leave total conversations off the top of the sheet, or at least don't celebrate it. It's a denominator, not a result.
A small business reading the right numbers
Take Northwind Bikes, a fictional shop selling online and in a storefront. Their first month with a chatbot, they proudly noted 2,200 conversations and left it there. The number felt good and changed nothing.
The next month they tracked the real four. Resolution rate was 61% and climbing as they fixed a muddled sizing page. Qualified leads came in at 40, mostly people asking about specific builds and availability. Chatters converted at a clearly higher clip than non-chatters on the custom-build pages. And their handoffs were mostly clean, because the bot captured details before passing to staff.
That view told them something actionable: sizing content needed work, custom builds were where the bot earned its money, and they should push more high-intent traffic toward chat. The 2,200 number never told them any of that.
Pick two of these metrics to start, resolution rate and qualified leads are a good pair, and check them weekly against your transcripts. The dashboard number that makes you feel busy is rarely the one that predicts what you'll bank. Follow the metrics that connect to dollars, and the bot stops being a widget you hope is working and becomes a system you can actually steer.