Pattern 04 · Chat or assistant, Dashboard with AI insights · Explanation pattern
Classification with confidence and reason
Practice view. What to build, the words on screen and how to measure it. Switch to Research for why it should work, the framework properties and the references.Research view. The research side is shown below the card: why it should work, the properties it targets and the references. Switch to Practice to hide it.
Every AI classification shows three things together: the label, a confidence level, and the evidence behind it, such as the sender's domain or the words that triggered the label.
The right amount of trustThe system shows how it worksCase: CNEXT Mail2Flow (company approval pending)
Done right, done wrong
Schematic screens, not real products. Click a screen to enlarge it.In practice
In Mail2Flow every incoming e-mail gets a category, an urgency and a project, each with a confidence. The Outlook add-in shows that confidence as a percentage, and the working hypothesis in the case was that people would understand the label but struggle with the number and want to know why. In the Kairos prototype tested in July the suggestions carried no confidence at all, and one participant asked for exactly that: a score, the way a junior colleague says how sure they are.
The July test added a lesson about the evidence itself. Everyone started from the AI summary, and the original e-mail sat behind a Show context button that most participants overlooked; two of them took the summary for the e-mail. Several said they need to check the AI against the source before they trust it and would rather see the original by default. So the evidence next to a label only works when the source it points to is one glance away, not one click.
Words on screen
Obligations to keep in mind (AI Act, GDPR)
- Communicate limits: show when the system is unsure rather than hiding low confidence. (O1, IUI 2026, Table 1)
- Give the evidence a person needs to overrule the classification. (O2, IUI 2026, Table 1)
Design considerations, not legal advice or a conformity assessment. What O1 to O5 mean.
Build it
Illustrative signatures, not a published library: the names below are suggestions to adapt to your stack. Thresholds are examples to tune on your own data.
States: classified, low-confidence, corrected, rerouted
Component: <Classification label={label} confidence={0.95} evidence={[...]} onChange={fn} />
Prompt snippet: Return the label, a confidence between 0 and 1, and up to three short evidence snippets quoted from the input. Do not paraphrase the evidence.
Output schema: { "label": "string", "confidence": 0.0, "evidence": ["string"] }
Measure it
Same idea, other names
Applied by: nobody yet · Replicated by: nobody yet · Used this pattern? Tell us and get listed
Try it
Research side · why it should work · framework v0.2
Why it should work
Properties targeted
Notes
How to measure: AI model certainty as calibration of the shown confidence against correctness; trust with three items plus the acceptance rate of wrong classifications; understanding model behaviour with two questions about why an item was labelled. Report accountants by seniority, since juniors and seniors verify differently.
Relations
- AI model certainty is related to trust and to the user's confidence in a decision (Donoso-Guzmán et al. 2025, section 4.11, Figure 9).
- Adding evidence raises understanding of model behaviour at a small cost in cognitive load.
Across contexts
| Context | Users | State | Result |
|---|---|---|---|
| CNEXT Mail2Flow, Kairos prototype | accountants, desktop | Seen in a case | the July prototype showed no confidence on its suggestions and one participant asked for a score; the AI summary was rated 3 to 4 of 5 by five of seven; two took it for the original e-mail, which sat behind a 'show context' button most overlooked |
| Your context | Bring your case |
References
- Zhang, Liao and Bellamy 2020, Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making, FAT*.
- Donoso-Guzmán et al. 2025, A Systematic Review of User-Centred Evaluation of Explainable AI in Healthcare.
- Amershi et al. 2019, Guidelines for Human-AI Interaction, CHI, guidelines 2 and 11.
Editors
Related patterns
All patterns
Pattern 05 · Dashboard with AI insights, Chat or assistantUrgency labels for prioritisationA small set of urgency levels on every item, so a full inbox can be scanned in seconds and the order of work is explicit.Seen in a case
Pattern 06 · Chat or assistant, Dashboard with AI insightsLow-confidence hand-off to a personBelow a threshold the AI stops, says what it could not determine, and hands the item to a person instead of guessing.Seen in a case
Pattern 07 · Dashboard with AI insights, Chat or assistantWhy is this urgentA one-line reason behind an AI judgement, available on click rather than always on screen.Seen in a case