Our study evaluated an AI-assisted double reading system for chest radiographs in two different hospital settings. The system analysed both the radiograph and the corresponding radiologist’s report to detect potential missed findings. Among 25,104 chest radiographs, clinically relevant missed findings were confirmed in 0.1% of cases, primarily consisting of unreported lung nodules, pneumothoraces, and consolidations.
What makes this approach interesting is that the AI tool operates as a second reader after report completion, rather than providing concurrent suggestions during the primary reading by the radiologist. This is particularly relevant for chest radiographs, where typical reading times are short and any unnecessary interruption of the interpretation workflow could impact radiologists’ efficiency. By analysing reports after they are finalized, this system preserves the radiologist’s reading flow.
The AI tool uses a combination of deep learning for image analysis and natural language processing to analyse reports, automatically comparing the findings to identify discrepancies. A challenge of the current implementation is the considerable number of AI-detected discrepancies, which necessitated review by an external radiologist to assess their clinical relevance. Future development should focus on reducing the number of cases requiring human review while maintaining high sensitivity for clinically significant missed findings.
I believe this type of AI implementation represents a promising direction for quality assurance in radiology. Since these systems can run in the background, they could become integrated as a safety net to mitigate diagnostic errors.
Key points:
- A commercially available double reading system supported by artificial intelligence was evaluated to detect reporting errors in chest radiographs (n=25,104) from two institutions.
- Clinically relevant missed findings were found in 0.1% of chest radiographs and consisted of unreported lung nodules, pneumothoraces and consolidations.
- Applying AI software as a secondary reader after report authorisation can assist in reducing diagnostic errors without interrupting the radiologist’s reading workflow. However, the number of AI-detected discrepancies was considerable and required review by a radiologist to assess their relevance.
Authors: Laurens Topff, Sanne Steltenpool, Erik R. Ranschaert, Naglis Ramanauskas, Renee Menezes, Jacob J. Visser, Regina G. H. Beets-Tan & Nolan S. Hartkamp
