How to Review AI Decisions

Where to Find AI Decisions

AI decision information is available in several places:

• Conversation Detail — Every conversation shows the AI's classification (intent, confidence, sentiment) in the context panel on the right side.
• Dashboard → AI Drafts — Shows a summary of AI draft performance, including accuracy rates and guardrail activity.
• Dashboard → AI Decisions (Scale plan) — A dedicated view showing individual AI decisions with full reasoning traces.
• Dashboard → AI Observability (Growth plan and above) — Aggregate metrics showing AI performance trends over time.

Understanding Confidence Scores

Every AI action includes a confidence score between 0 and 1. This represents how certain the AI is about its decision.

• 0.90–1.00 — High confidence. The AI is very certain about its classification or response.
• 0.80–0.89 — Moderate confidence. The AI is fairly confident but may benefit from human review.
• Below 0.80 — Low confidence. The conversation is likely routed to a human agent.

Your automation confidence threshold (set in Settings → Automation) determines the minimum score required for automated actions.

Conversations below this threshold are always handled by your team.

What AI Decides

The AI makes several types of decisions for each conversation:

• Intent classification — What the customer is asking about (e.g., order status, refund request, product question).
• Sentiment detection — Whether the customer's tone is positive, neutral, or negative.
• Priority assignment — How urgent the conversation is, based on SLA proximity, customer value, and sentiment.
• Response generation — What reply to suggest or send (in draft or automated mode).
• Escalation — Whether to route the conversation to a human agent instead of handling it automatically.

Reviewing Individual Decisions

For any conversation, open the detail view and look at the context panel. You'll see:

• The detected intent and confidence score.
• The sentiment analysis result.
• The priority level and the signals that influenced it (SLA, customer value, sentiment, urgency).
• If an AI draft was generated, the draft content and whether it was sent, reviewed, or blocked.

On the Scale plan, the AI Decisions dashboard provides a timeline view of all decisions with filtering by intent, confidence range, and decision outcome.

What to Do When AI Gets It Wrong

If you notice an AI decision that doesn't seem right:

1. Review the confidence score — Low confidence means the AI was already uncertain. This is working as intended.
2. Check the intent — If the intent classification is wrong, this may indicate the AI needs more training data for that topic. Adding relevant knowledge documents can help.
3. Adjust your threshold — If the AI is making automated decisions it shouldn't, raise the confidence threshold in Settings → Automation.
4. Provide feedback — Use the helpful/unhelpful feedback buttons on AI drafts. This data helps improve future performance.
5. Reduce automation — If issues are frequent, lower the rollout percentage or pause automation until the readiness score improves.

Troubleshooting

• "I can't find AI decision data." — AI Decisions requires the Scale plan. AI Drafts and basic classification data are available on Starter and above.
• "Confidence scores seem inconsistent." — Confidence can vary based on conversation complexity. Short or ambiguous messages typically receive lower scores. This is normal.
• "AI keeps classifying the same intent incorrectly." — Add knowledge documents that cover the misclassified topic area. The AI uses your knowledge base to improve classification accuracy.
• "I want to understand why a specific conversation was automated." — Open the conversation detail and check the context panel for the decision trace. It shows the confidence score, intent, and whether guardrails were applied.