It is relevant, but is it useful?
Additional information
Authors
Chakrabarti H.,
Tobia D. M.,
Allen G.,
Landoni M.,
Pera M. S.
Type
Article in conference proceedings
Year
2026
Language
English
Abstract
The traditional Information Retrieval (IR) evaluation framework—anchored in topical relevance and relevance‑based metrics—reflects a system-centred perspective. Yet for specific user groups, relevance alone is insufficient; benchmarking that relies exclusively on conventional metrics overlooks qualities intrinsic to the users IR approaches are meant to serve. Here, we draw attention to Children IR and examine the value of extending traditional evaluation with a human-centred perspective that accounts for how children interpret and evaluate information to more authentically capture performance and better reflect how well an approach truly meets children’s needs. Our empirical exploration using a child‑focused dataset, multiple ranking strategies, and traditional and extended frameworks reveals not only the limitations of relevance-based assessments but also the advantages of employing frameworks that are tailored to reflect the needs of child users, paving the way for more inclusive and effective evaluation frameworks.
Keywords
Children, Evaluation, Information Retrieval, Search
Conference proceedings
ACM Conference on User Modeling, Adaptation and Personalization (UMAP '26)
Meeting name
UMAP '26: 34th ACM Conference on User Modeling, Adaptation and Personalization
Meeting place
Gothenburg , Sweden
Meeting date
June 2026
Pages (or article number)
392-397
Diffusion
License
CC BY
Visibility
Public
Status open access
Gold
