Measure it
Nine things you can measure
Each metric card is a small kit: what it tells you, up to three questionnaire items, a few log events, and a benchmark from a case when we have one. Each takes ten minutes or less per participant; two run from your logs alone. The names come from the KU Leuven evaluation framework, the words from practice.
With users, in minutes
Questionnaire items after a task. Ten minutes per participant is the usual budget.
Confidence
Users are sure of their decisionHow certain people are about the decision they made with the AI. Set against whether each decision was right, it shows whether their certainty follows their accuracy or stays just as high when they are wrong.2 minOpen the card →Intention to use
They want to keep using itWhether people would choose the feature again tomorrow.3 minOpen the card →Cognitive load
Mental effortHow hard people had to think to use the AI feature. In the KU Leuven review, the size and structure of an explanation were found to influence it.5 minOpen the card →Information correctness
Catching made-up factsWhether people notice when generated text contains a wrong or invented fact. The KU Leuven framework suggests considering this property for any system that uses generative AI.6 minOpen the card →Controllability
Users feel in controlWhether people feel they can steer, correct and undo what the AI does. The KU Leuven framework names four forms of that control: reversing actions, correcting the system, filtering or zooming the explanation, and asking questions.8 minOpen the card →Understanding model behaviour
Users get how it decidesWhether people can predict and explain what the AI will do with an input. The KU Leuven framework defines it as the user's ability to correctly interpret the model behaviour.10 minOpen the card →Trust
Users rely on it the right amountWhether people follow the AI when it is right and check it when it is wrong. Too much trust and errors slip through; too little and the efficiency gain evaporates.10 minOpen the card →From your logs, no session needed
Counts you can take from a product that is already in use.
Efficiency
Faster, fewer errorsWhether the AI feature actually saves time and reduces mistakes, compared with doing the task without it. Efficiency is the time; performance is the error rate. Report both.from your logsOpen the card →Reliance
How often users follow the AIHow often people accept what the AI proposes, and, the part that matters, how often they accept it when it is wrong. Without the second number, an acceptance rate says nothing about whether the AI was right.from your logsOpen the card →Not sure which to pick? Find my patterns names the metrics for your context, and Plan your test explains the five steps and how many participants you need. Property definitions: Vocabulary bridge.