July 2018

Paul M. Parizel, MD, PhD

  • Past President, European Society of Radiology (ESR)
  • Honorary President, African Society of Radiology (ASR)
  • Professor and Chair, Department of Radiology, Antwerp University Hospital (UZA) & University of Antwerp (UA)

I’ve seen the future. It started on a Saturday in Beijing, at 2:00 pm on June 30, 2018.  It was heralded by the CHAIN Cup, an event billed as “The World’s First Competition between Physicians and Artificial Intelligence in Neuroimaging”. A custom-built Artificial Intelligence (AI) system was pitted against an all-stars team of 15 specialist doctors, with experience in neuroimaging. The AI system, dubbed BioMind, was trained to accurately diagnose brain tumours and predict hematoma expansion. The programmers had fed BioMind with thousands upon thousands of image data sets of patients with brain diseases. All imaging data was harvested from the digital archives of Beijing Tiantan Hospital, affiliated to Capital Medical University. The ‘ground truth’ was provided by the neuropathology examination.

I witnessed this extraordinary event from a front row seat, as one of two international jury members[1], for the first heat, in which humans competed against BioMind in diagnosing different types of brain tumours. The competition took place in an auditorium of 2,000 people that was filled to the brim. On the left side of the huge stage, 15 senior and experienced neuroimaging specialists took their place behind 15 big-screen iMacs. They were asked to provide a diagnosis on 15 brain tumour cases each. Thus, in total, there were 225 brain tumour cases. MR images were viewed in a PACS setting, showing imaging sequences: FLAIR, T2, DWI, ADC, T2* or SWI, Gd-enhanced T1-WI in different planes, MRS for some cases, perfusion maps, in short: state-of-the-art imaging data, seen in a state-of-the art viewing set-up.

Contestants were instructed to write down a single best diagnosis (not a differential diagnosis, but only one answer).  They had 30 minutes to complete their task, i.e. 2 minutes per case. My role was to verify and double-check the scoring of the answers against the correct pathological diagnosis. On the right side of the stage stood a simple white desk, with a black box on it, containing the AI software.

And … the winner was … BioMind!  AI won, by a large margin, predicting the correct histological diagnosis in 196 out of 225 cases (87%) in just under 15 minutes, whereas the human specialist doctors got only 66% right in 30 minutes. In the second heat, a different group of doctors competed with BioMind in predicting expansion of intracerebral hematoma; and, again AI won, again by a large margin (83% versus 63%).

Obviously, there were some caveats in this competition. For example, in some cases, the AI software identified a brain tumour as a "glioma", and then on histopathology it turned out to be an astrocytoma or glioblastoma, which are subtypes of glioma; but "glioma" was scored as a correct answer. When checking the answers, the humans seemed more apt to provide a specific diagnosis, but they were more frequently wrong especially for ‘rare’ tumours.  Some contestants grumbled that 30 minutes was not enough time to make a diagnosis in 15 cases.

Others mentioned that the case mix was not representative of what they experience in daily routine. Despite these limitations, it was a clear win for the AI software, a very clear win, especially for some of the more infrequent and unusual tumours.  And all the contestants were doctors with ample experience in neuroimaging studies ... just imagine such a contest between the AI software and general, non-specialized radiologists ...

The entire event, which took 2 hours, was set up as a highly entertaining game show, with a talk show host and hostess. The CHAIN Cup was very well presented, on a huge screen, with music and video animations, strobe lighting and all the gizmos and doodads that go with modern show business. The audience greatly enjoyed it.

China Biomind AI

©China Daily/Twitter

So, what does this all mean?  First, it shows that a well-trained AI system can be trained to become proficient at diagnosing brain tumours with an accuracy rate of close to 90%; this is comparable to the accuracy of a senior doctor. Second, the AI BioMind system was not only more accurate but also very fast. It was able to finish the 225 cases in 15 minutes (averaging 4 seconds per case), whereas the team of 15 elite doctors was able to achieve an accuracy rate of 66% when diagnosing 15 brain tumours each, finishing the task in 30 minutes (2 minutes per case).

Significant breakthroughs in artificial intelligence are the result of ongoing advances in data collection and aggregation, processing power, deep-learning algorithms, and convolutional neural networks. In 1996, the chess-playing computer Deep Blue defeated the reigning world champion Garry Kasparov. In 2016, the AlphaGo computer program triumphed over Lee Sedol, a nine-dan professional master at the game of Go, proving that machines can be instructed to think like humans, and even exceed their creators. But of course, chess and Go are merely board games, and they are very, very different from medicine. Nevertheless, in our own profession, we see that AI systems are rapidly evolving though there is still a long way to go. Currently, an AI software platform such as BioMind focuses on only one aspect of the medical process (imaging) but is not yet able to provide a full diagnosis to patients.

Personally, I believe that AI will become integrated into existing medical work flow environments, more or less like a GPS navigation system guiding the driver of a car. AI software will give proposals and help the doctor to make an accurate diagnosis, thus providing a roadmap towards correct patient management and follow-up. But it will be the doctor who ultimately decides, as there are a number of factors that a machine cannot possibly take into consideration, such as a patient’s state of health and family situation.

The advent of AI systems has created a lot of anxiety and doubt among radiologists. The AI genie is out of the bottle, we cannot turn the clock back, but we do have the power to determine the future: tomorrow belongs to those who prepare for it today. As George Bernard Shaw said: “we are made wise not by the recollection of our past, but by the responsibility for our future”.

Therefore, I am not afraid, since I am convinced that (neuro)radiologists will embrace AI to help us manage ‘routine’ tasks quickly and efficiently, thus giving us more time to focus on things that really matter. As shown in the CHAIN Cup competition in Beijing, AI systems offer a unique opportunity to make a new beginning, to re-invent what we do, to boost productivity and accuracy. I am convinced that AI can take over time-consuming routine tasks, freeing up time and resources to focus our attention on individual patients, and thereby moving from volume-based radiology towards value-based radiology. So, regarding AI software, my closing message to all (neuro)radiologists is: take charge of your own future, and embrace it with confidence, courage and determination.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

[1] The other international jury member in the brain tumour competition was dr. Alexandra Golby (Director of Image-Guided Neurosurgery at the Brigham and Women’s Hospital and Professor of Neurosurgery at Harvard Medical School, Boston, USA).

© Dhr. Filip Deferme