Metric · Information correctness
Catching made-up facts
What it tells you
Whether people notice when generated text contains a wrong or invented fact. The KU Leuven framework suggests considering this property for any system that uses generative AI.
How, in 6 minutes
Seed the test material with two known errors, such as a wrong date and an invented name, and count how many participants catch each one. Add one item on perceived accuracy.
Benchmark
No benchmark yet. Run the kit and send us yours.
Plan your test explains how to choose metrics and how many participants you need.
Questionnaire items
- The information the AI produced was accurate. (1 to 5)
Answer on a 1 to 7 scale from strongly disagree to strongly agree unless the item states its own scale. The ULTRA team wrote or adapted these items for the kits and has not checked their reliability, so report them as ULTRA kit items and name any questionnaire they were adapted from.
Log events
seeded_error_shown {task, error_id}seeded_error_caught {task, error_id}Pairs with
Research side · the property behind this metric
Property
References
- Donoso-Guzmán et al. 2025, Table 9 and figure 13: information correctness for generative AI.