Artificial intelligence model outperformed experienced clinicians in differentiating the aetiology of pneumonia on chest computed tomography: a retrospective study

Background Rapid and precise aetiological diagnosis is crucial for managing pneumonia. We aimed to develop and validate deep learning (DL) models for differentiating ten pneumonia aetiologies on chest computed tomography images. Methods We enrolled 1,091 pneumonia patients with 1 of 10 definite aetiological diagnoses between October 1 st , 2015 and June 30 th , 2022 in this retrospective study. We trained and validated two DL models: a classic 3D-DenseNet model (DenseNet) and a novel large vision model (LVM). The models were tested on a data from 183 nonoverlapping patients for external dataset. Model performance was assessed using the area under the curve (AUC) of the Top1 diagnosis and the accuracy of the Top1, Top2, and Top3 diagnoses. Comparisons were also performed between the DL models and eight experienced radiologists and pulmonologists. Results The LVM combined with non-imaging model (LVM+) had a greater average prediction performance than DenseNet combined with non-imaging model (DenseNet+), radiologists' results with non-imaging data (radiologists+) and pulmonologists' results with non-imaging data (pulmonologists+), with Top1 AUCs of 0.872, 0.851, 0.643 and 0.644, respectively. The Top1, Top2, and Top3 accuracies of LVM+ were 0.527, 0.701 and 0.820, respectively, similarly outperforming DenseNet+, radiologists+ and pulmonologists+. The two models performed similarly in the external test sets, with the Top1 AUCs of 0.743 for DenseNet and 0.775 for LVM. The classification-related confusion matrix of LVM/DenseNet with or without non-imaging model showed a significant advantage in identifying pulmonary non-tuberculous mycobacterium pulmonary disease (PNTM), pulmonary tuberculosis (PTB) and Pneumocystis jirovecii pneumonia (PJP). Conclusions This study presents a comprehensive classification closely aligned with pneumonia diagnosis in realistic clinical settings. We expect this method to be applied clinically to foster novel approaches to improve the accuracy in diagnosing pneumonia.