Algorithmic evaluations in breast cancer: The case of Champalimaud Foundation

Abstract

Algorithmic evaluations are increasingly used to make decisions thanks to the perception of objective measures of quality and performance. However, little is known about how the current evaluation methods change with ML algorithms and with what consequences for the actors and organizations being evaluated. We conducted an exploratory case study in the breast unit of the Champalimaud Foundation in Lisbon. Gioia methodology guided the collection and analysis of semi-structured interviews and archival data. Our results show that besides generating visible and direct changes (e.g., extraction and quantification of relevant criteria with systematic approaches), algorithmic evaluations trigger indirect and less visible dynamics (e.g., adding a new dimension - aesthetic score - in the evaluation of research units), which have profound implications on how institutions operate and how resources are allocated based on the ranking lists. We contribute to digital undertow and institutional displacement and human ML collaborations by explaining the processes through which the new methods are used in medical communities and their less visible yet impactful consequences.