Development and Validation of a Deep Learning–based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs
Eui Jin Hwang, Sunggyun Park, Kwang-Nam Jin et al. · Clinical Infectious Diseases · 2018
BACKGROUND: Detection of active pulmonary tuberculosis on chest radiographs (CRs) is critical for the diagnosis and screening of tuberculosis. An automated system may help streamline the tuberculosis screening process and improve diagnostic performance. METHODS: We developed a deep learning-based automatic detection (DLAD) algorithm using 54c221 normal CRs and 6768 CRs with active pulmonary tuberculosis that were labeled and annotated by 13 board-certified radiologists. The performance of DLAD was validated using 6 external multicenter, multinational datasets. To compare the performances of DLAD with physicians, an observer performance test was conducted by 15 physicians including nonradiology physicians, board-certified radiologists, and thoracic radiologists. Image-wise classification and lesion-wise localization performances were measured using area under the receiver operating characteristic (ROC) curves and area under the alternative free-response ROC curves, respectively. Sensitivities and specificities of DLAD were calculated using 2 cutoffs (high sensitivity [98%] and high specificity [98%]) obtained through in-house validation. RESULTS: DLAD demonstrated classification performance of 0.977-1.000 and localization performance of 0.973-1.000. Sensitivities and specificities for classification were 94.3%-100% and 91.1%-100% using the high-sensitivity cutoff and 84.1%-99.0% and 99.1%-100% using the high-specificity cutoff. DLAD showed significantly higher performance in both classification (0.993 vs 0.746-0.971) and localization (0.993 vs 0.664-0.925) compared to all groups of physicians. CONCLUSIONS: Our DLAD demonstrated excellent and consistent performance in the detection of active pulmonary tuberculosis on CR, outperforming physicians, including thoracic radiologists.
Cited by 240 works. doi:10.1093/cid/ciy967 Open-access PDF
Authors
- Eui Jin Hwang
- Sunggyun Park
- Kwang-Nam Jin
- Jung Im Kim
- So Young Choi
- Jong Hyuk Lee
- Jin Mo Goo
- Jaehong Aum
- Jae‐Joon Yim
- Chang Min Park
- Dong Hyeon Kim
- Woo Woo
- Choi Choi
- Inpyeong Hwang
- Yong Sub Song
- Lim Lim
- Kim Kim
- Jae Yeon Wi
- S. Oh
References
- R: A Language and Environment for Statistical Computing
- A Simple Sequentially Rejective Multiple Test Procedure
- Dermatologist-level classification of skin cancer with deep neural networks
- Advances in Neural Information Processing Systems 25
- Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs
- Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks
- Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method.
- Receiver Operating Characteristic Rating Analysis
- Automatic Tuberculosis Screening Using Chest Radiographs
- Single Reading with Computer-Aided Detection for Screening Mammography