It is relevant, but is it useful?
Hrishita Chakrabarti, Diletta Micol Tobia, Giulia Allen, Monica Landoni, and Maria Soledad Pera. 2026. #. In Proceedings of the 34th ACM Conference on User Modeling, Adaptation and Personalization (UMAP ’26), June 8–11, 2026, Gothenburg, Sweden. ACM . DOI:https://doi.org/10.1145/3774935.3806167.
Abstract
The traditional Information Retrieval (IR) evaluation framework—anchored in topical relevance and relevance-based metrics—reflects a system-centred perspective. However, for specific user groups such as children, relevance alone is insufficient. This work examines the importance of extending traditional evaluation approaches with a human-centred perspective that accounts for how children interpret and assess information. Through empirical analysis using a child-focused dataset, multiple ranking strategies, and both traditional and extended evaluation frameworks, results reveal the limitations of relevance-based metrics and the benefits of approaches tailored to children’s needs, paving the way for more inclusive IR evaluation.