Artificial intelligence (AI) technologies, such as large language and deep learning models, are poised to significantly influence clinical practice. However, the real-world impact and utility of these AI applications, particularly in radiology, often remain unclear. How can their impact be measured, and can we ensure it benefits everyone? In ESR Essentials: how to get to valuable radiology AI: the role of early health technology assessment—practice recommendations by the European Society of Medical Imaging Informatics, we introduce several existing methodologies that can help comprehensively evaluate the potential impact of radiology AI applications in development.
First, impact needs to be defined. We can look at the evaluation of new drugs. They undergo extensive analysis through clinical studies to determine their cost-effectiveness and health benefits. This comprehensive assessment, known as a Health Technology Assessment (HTA), includes safety, ethical, and social considerations and is mandatory for new drugs in Europe. Efforts to achieve similar robust evaluations in radiology AI are increasing with guidelines like RADAR and FUTURE-AI that offer frameworks for the evaluation of trustworthy medical AI.
Second, we argue that early value assessments, or eHTAs, can be a crucial step for developing impactful medical AI. This alteration of the HTA is typically constrained to an early cost-effectiveness analysis. Comprehensive early evaluation can help set concrete goals and requirements of the AI application addressing specific clinical needs. To indicate how a comprehensive eHTA would look, we describe some example analyses, such as multiple criteria decision analysis.
In conclusion, to truly unlock the value of radiology AI, we must move beyond hype and focus on rigorous, early evaluation. By embedding eHTAs into the development process, we can better define impact, align innovations with real clinical needs, and ensure that these technologies deliver meaningful value to patients, providers, and healthcare systems alike.
Key points:
- Value in radiology AI should be perceived as a comprehensive term including health technology assessment domains and AI-specific domains.
- Incorporation of an early health technology assessment for radiology AI during development will lead to more valuable radiology AI tools.
- Comprehensive and transparent value assessment of radiology AI tools is essential for their widespread adoption.
Authors: Erik H. M. Kemper, Hendrik Erenstein, Bart-Jan Boverhof, Ken Redekop, Anna E. Andreychenko, Matthias Dietzel, Kevin B. W. Groot Lipman, Merel Huisman, Michail E. Klontzas, Frans Vos, Maarten IJzerman, Martijn P. A. Starmans & Jacob J. Visser
