# What Metrics Should You Track for an AI Agent? A common mistake after launching an AI agent is measuring only: > How many conversations did the AI handle? That number is useful. It is not enough. An AI agent can handle 10,000 conversations and still create more work for the business if customers are repeatedly transferred to humans, information is wrong, or the automation triggers the wrong action. The better question is: > **Did the agent successfully perform the job it was designed to perform?** The right metrics depend on the agent's responsibility. ## Start with the job, not the dashboard Suppose the agent's job is: > Answer routine support questions. You might care about: * resolution rate * human handoff rate * unanswered questions * repeat contacts * customer satisfaction Suppose the agent's job is: > Qualify leads before sales receives them. You might care about: * qualified leads * qualification completion * handoff rate * sales acceptance * conversion after handoff The metrics should follow the responsibility. ## Metric 1: Resolution rate Resolution rate measures how often the customer's issue is resolved without requiring additional intervention. For support: ```text Resolved by AI ÷ Eligible AI conversations ``` The exact definition should be agreed upon before measuring it. Not every conversation should count. A complaint requiring a human investigation is not necessarily an AI failure. ## Metric 2: Human handoff rate Measure how often conversations are transferred to people. For example: ```text Human handoffs ÷ AI conversations ``` A high handoff rate can mean: * the agent is too limited * knowledge is incomplete * routing rules need work * the task is naturally human-heavy But a lower handoff rate is not automatically better. For some businesses, human involvement is expected. The important question is: > Are the right conversations reaching humans? ZazzyAgent supports human assignment and transfer rules that can be measured against the purpose of the workflow. See [How to Hand Off AI Conversations to Human Agents](/ai-agent-human-handoff-zazzyagent). ## Metric 3: Action success rate For agents that use tools or integrations, measure whether the required action actually worked. Examples: * order lookup success * appointment booking success * API request success * lead creation success * sequence start success For example: ```text Successful order lookups ÷ Order lookup attempts ``` This is often more useful than counting how many times the agent mentioned the order system. ## Metric 4: Incorrect-action rate This measures cases where the agent selected or triggered the wrong action. Examples: * sales lead routed to support * wrong customer record used * inappropriate sequence started * human assigned to the wrong team * action triggered without required information This metric deserves attention because a wrong action can be more damaging than a poor sentence. ## Metric 5: Knowledge accuracy Track cases where the agent gave information that was: * correct * incomplete * outdated * unsupported * contradictory This can be measured through sampled conversation reviews. You do not necessarily need a perfect automated score. A structured review process can reveal patterns quickly. ## Metric 6: “I don't know” quality A useful AI agent should know when information is unavailable. Measure how often the agent: * correctly declines to answer * escalates when appropriate * invents information * gives a misleading partial answer An agent that says: > “I don't have enough information to confirm that.” can be behaving correctly. An agent that confidently invents an answer is not. ## Metric 7: Qualification completion For a lead-qualification agent, track: ```text Started qualification ↓ Required information collected ↓ Qualification completed ↓ Lead routed ``` This lets you see where conversations are dropping. For example: 1,000 enquiries → 800 started qualification → 620 completed required fields → 400 reached sales Each stage tells you something different. ## Metric 8: Sales acceptance rate Suppose AI-qualified leads are passed to sales. Sales should be able to indicate whether those leads are actually qualified. For example: ```text AI-qualified leads accepted by sales ÷ AI-qualified leads received ``` This is far more meaningful than: > “The AI qualified 5,000 leads.” The AI's output needs to match the business's actual definition of a qualified lead. See [How Should an AI Agent Qualify a Lead When Customers Answer Questions Out of Order?](/ai-agent-leads-answer-questions-out-of-order). ## Metric 9: Conversion after AI interaction For sales workflows, measure the downstream outcome. Examples: * enquiry → booked demo * enquiry → quote * quote → purchase * lead → customer This helps answer: > Did the AI interaction contribute to the intended business outcome? Do not attribute every conversion automatically to the AI. Use the metric as a relationship to investigate, not proof of causation. ## Metric 10: Time to resolution For support: > How long did it take to solve the customer's problem? For sales: > How long did it take to reach the next meaningful sales step? AI can reduce delays even when the final outcome still requires a human. For example: ```text Customer enquiry ↓ AI collects required information ↓ Sales receives complete context ↓ Human responds ``` The AI may not close the deal. It can still reduce the time between enquiry and useful human action. ## Metric 11: Conversation abandonment Measure conversations that stop before the intended outcome. For example: * customer stops replying * qualification is incomplete * booking is abandoned * support issue remains unresolved A high abandonment rate may indicate: * too many questions * confusing responses * poor timing * excessive friction * an action that failed ## Metric 12: Repeat contact If customers repeatedly ask the same thing, something may be wrong. For example: > Customer asks about delivery. Agent answers. Customer asks again. Agent answers differently. That is a signal to investigate. Repeat contacts can reveal problems that a basic “resolved” metric misses. ## Metric 13: Follow-up conversion For agents that trigger sequences, track what happens after the follow-up. Possible metrics: * sequence start * reply rate * re-engagement * booked meeting * purchase * opt-out * human handoff ZazzyAgent supports AI-triggered sequences for follow-up workflows. See [Can AI Agents Follow Up With Customers After a Conversation Ends?](/ai-agent-customer-follow-up). ## Don't optimize every metric simultaneously Metrics can conflict. For example: > Reduce human handoffs. That might increase: > customer frustration. Or: > Increase qualification completion. That might also increase: > conversation length. A better approach is to choose: **one primary outcome** plus: **a few safety and quality metrics.** ## A useful metric structure ### Primary metric What does success mean? Example: > Qualified leads accepted by sales. ### Quality metrics Is the agent doing it correctly? Example: > Incorrect qualification rate. ### Safety metrics Is it doing anything risky? Example: > Incorrect action rate. ### Experience metrics What does the customer experience? Example: > Abandonment rate. This gives you a more balanced view. ## Different agents need different metrics ### Support Agent Track: * resolution rate * handoff rate * repeat contact * time to resolution * unresolved conversations ### Sales Agent Track: * qualified leads * sales acceptance * meetings booked * conversion after handoff * time to sales handoff ### Order Support Agent Track: * successful lookups * incorrect lookups * unresolved orders * handoffs * action failures ### Booking Agent Track: * booking completion * abandoned bookings * successful confirmations * booking errors * human intervention There is no universal AI-agent score. ## What about customer satisfaction? Customer satisfaction can be useful, but interpret it carefully. A customer may rate a conversation poorly because: * the business policy was disappointing * the product was unavailable * the agent could not make an exception That does not necessarily mean the AI behaved incorrectly. Review low scores alongside conversation outcomes. ## Review real conversations Numbers tell you where to look. Conversations tell you why. A useful operating rhythm is: ```text Metrics ↓ Find unusual pattern ↓ Review sample conversations ↓ Identify root cause ↓ Change system ↓ Measure again ``` This creates a continuous improvement loop. ## Measure the business result, not the AI activity Avoid dashboards filled with: > messages generated > tokens used > conversations started Those can be useful operational metrics. They should not become the definition of success. A business cares about: > Was the customer helped? > Was the lead qualified correctly? > Was the task completed? > Was the right action taken? > Did the workflow reduce unnecessary manual work? Those are the outcomes that matter. ## The takeaway The right AI-agent metrics depend on what the agent is responsible for. At minimum, consider measuring: * outcome or resolution * human handoff * action success * incorrect actions * knowledge accuracy * abandonment * downstream business results Then connect those metrics to real conversations. > **Measure the job the agent performs, not the number of messages the AI produces.** That one distinction makes an AI dashboard much more useful.