Can training data help radiologists to open deep learning black box?

Deep learning has recently pervaded the radiology field, reaching promising results that have encouraged both scientists and entrepreneurs to apply these models to improve patient care.

However, “with great power there must also come — great responsibility” [1]! In most cases, the complexity of deep learning models forces their users, and sometimes also their developers, to treat them as black boxes. Indeed, model opacity represents one of the most challenging barriers to overcome before AI solutions can be fully translated into the clinical settings, where transparency and explainability of diagnostic and therapeutic decisions play a key role.

We know what task the model is performing and, using saliency maps, we may know where the most informative regions of the image are. But being also able to easily explain why the model returns a specific output still remains a challenge.

Several researchers are focusing on making AI models more explainable [2,3]. In our article, Prof. Baselli, Prof. Sardanelli and I hypothesized to mimic what the radiologist will do in case of diagnostic uncertainty. We proposed enriching the AI model output with some information about those annotated training cases that are near to the current case in the feature space and are, therefore, “similar” to it. Information provided to the radiologist will vary according to the performed task and may include (without being limited to) original images, ground truth and predicted labels, clinical information, saliency maps, and neighbor density in the feature space.

In our vision, this may represent a user-friendly opportunity to transmit information usually encrypted in the model, hopefully increasing radiologists’ confidence in making their final diagnostic decision. “Summarizing, if a classifier is data-driven, let us make its interpretation data-driven too” [4].

Read more about this topic in the recently published article (link below): “Opening the black box of machine learning in radiology: can the proximity of annotated cases be a way?” [4].

[1] Lee S (1962) Amazing Fantasy #15, Marvel Comics, New York City, USA
[2] Guidotti R, Monreale A, Ruggeri S, Turini F, Pedreschi D, Giannotti F (2018) A survey of methods for explaining black box models. ACM Comput. Surv. 51.5: 1-42.
[3] Arrieta AB, Díaz-Rodríguez N, Del Ser J, et al. (2020) Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. INFORM FUSION 58: 82-115.
[4] Baselli G, Codari M, Sardanelli F. (2020) Opening the black box of machine learning in radiology: can the proximity of annotated cases be a way? European Radiology Experimental 4: 1-7. DOI: 10.1186/s41747-020-00159-0

Key points

  • Clinical rules and best practice require diagnosis and therapeutic decision to be transparent and clearly explained.
  • Machine/deep learning offers powerful classification and decision tools, though in a hardly explained black box way.
  • We propose to present the current case (CC) with training and/or validation data stored in a library of annotated cases (ACs).
  • Appropriate metrics in the classification space would yield the distance between the CC and ACs.
  • Proximity with similarly classified ACs would confirm high confidence; proximity with diversely classified ACs would indicate low confidence; a CC falling in an uninhabited region would indicate insufficiency of the training process.

Article: Opening the black box of machine learning in radiology: can the proximity of annotated cases be a way?

Authors: Giuseppe Baselli, Marina Codari & Francesco Sardanelli

WRITTEN BY

  • Marina Codari

    Postdoctoral Research Fellow, Cardiovascular Imaging, Department of Radiology, School of Medicine, Stanford University

Latest posts

Become A Member Today!

You will have access to a wide range of benefits that can help you advance your career and stay up-to-date with the latest developments in the field of radiology. These benefits include access to educational resources, networking opportunities with other professionals in the field, opportunities to participate in research projects and clinical trials, and access to the latest technologies and techniques. 

Check out our different membership options.

If you don’t find a fitting membership send us an email here.

Membership

for radiologists, radiology residents, professionals of allied sciences (including radiographers/radiological technologists, nuclear medicine physicians, medical physicists, and data scientists) & professionals of allied sciences in training residing within the boundaries of Europe

  • Reduced registration fees for ECR 1
  • Reduced fees for the European School of Radiology (ESOR) 2
  • Option to participate in the European Diploma. 3
  • Free electronic access to the journal European Radiology 
  • Content e-mails for all ESR journals4
  • Updates on offers & events through our newsletters
  • Exclusive access to the ESR feed in Juisci

€ 11 /year

Yes! That is less than €1 per month.

Free membership

for radiologists, radiology residents or professionals of allied sciences engaged in practice, teaching or research residing outside Europe as well as individual qualified professionals with an interest in radiology and medical imaging who do not fulfil individual or all requirements for any other ESR membership category & former full members who have retired from all clinical practice
  • Reduced registration fees for ECR 1
  • Option to participate in the European Diploma. 3
  • Free electronic access to the journal European Radiology
  • Content e-mails for all ESR journals 4
  • Updates on offers & events through our newsletters
  • Exclusive access to the ESR feed in Juisci

€ 0

The best things in life are free.

ESR Friends

For students, company representatives or hospital managers etc.

  • Content e-mails for all 3 ESR journals 4
  • Updates on offers & events through our newsletters

€ 0

Friendship doesn’t cost a thing.

The membership type best fitting for you will be selected automatically during the application process.

Footnotes:

01

Reduced registration fees for ECR 2025:
Provided that ESR 2024 membership is activated and approved by August 31, 2024.

02
Not all activities included
03
Examination based on the ESR European Training Curriculum (radiologists or radiology residents).
04
European Radiology, Insights into Imaging, European Radiology Experimental.