Development and validation of a transformer-based CAD model for improving the consistency of BI-RADS category 3-5 nodule classification among radiologists: a multiple center study
机构：[1]Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, China医技科室超声诊断科首都医科大学附属北京同仁医院首都医科大学附属同仁医院[2]Department of Ultrasonography, Beijing Tiantan Hospital, Capital Medical University, Beijing, China首都医科大学附属天坛医院[3]Department of Ultrasound, The Second Affiliated Hospital, Chongqing Medical University, Chongqing, China[4]Department of Medical Ultrasound, The First Affiliated Hospital, Xi’an Jiaotong University, Xi’an, China[5]Department of Ultrasound, The Southwest Hospital, Army Medical University, Chongqing, China[6]Department of Ultrasound, The Second Affiliated Hospital, Harbin Medical University, Harbin, China[7]Department of Ultrasound, The First Hospital, Shanxi Medical University, Taiyuan, China[8]Department of Ultrasound, The First Hospital, Peking University, Beijing, China[9]Department of Ultrasound, Diagnosis Center of Ultrasound, Hunan Province Cancer Hospital, Changsha, China[10]Department of Ultrasound, The Second Affiliated Hospital, Xi’an Jiaotong University, Xi’an, China[11]Department of Ultrasound, Cancer Hospital, National Cancer Center, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China[12]Department of Ultrasonography, The Third Affiliated Hospital, Guangxi Medical University, Nanning, China[13]Department of Ultrasound, The Frist Affiliated Hospital of Hebei North University, Zhangjiakou, China[14]Department of Ultrasound, Sichuan Provincial People’s Hospital, University of Electronic Science and Technology of China, Chengdu, China四川省人民医院[15]Department of Ultrasound, The First Affiliated Hospital, Zhengzhou University, Zhengzhou, China[16]Department of Ultrasound, China-Japan Union Hospital, Jilin University, Changchun, China吉林大学中日联谊医院[17]Department of Ultrasound, Qilu Hospital of Shandong University, Qingdao, China[18]Department of Ultrasound Diagnosis, The Second Xiangya Hospital, Central South University, Changsha, China[19]Department of Ultrasound, Lanzhou University Second Hospital, Lanzhou, China[20]Department of Ultrasound, The First Affiliated Hospital, Nanchang University, Nanchang, China
Background: Significant differences exist in the classification outcomes for radiologists using ultrasonography-based Breast Imaging Reporting and Data Systems for diagnosing category 3-5 (BI-RADS 3-5) breast nodules, due to a lack of clear and distinguishing image features. Consequently, this retrospective study investigated the improvement of BI-RADS 3-5 classification consistency using a transformer-based computer-aided diagnosis (CAD) model.Methods: Independently, 5 radiologists performed BI-RADS annotations on 21,332 breast ultrasonographic images collected from 3,978 female patients from 20 clinical centers in China. All images were divided into training, validation, testing, and sampling sets. The trained transformer-based CAD model was then used to classify test images, for which sensitivity (SEN), specificity (SPE), accuracy (ACC), area under the curve (AUC), and calibration curve were evaluated. Variations in these metrics among the 5 radiologists were analyzed by referencing BI-RADS classification results for the sampling test set provided by CAD to determine whether classification consistency (the k value), SEN, SPE, and ACC could be improved. Results: After the training set (11,238 images) and validation set (2,996 images) were learned by the CAD model, the classification ACC of the CAD model applied to the test set (7,098 images) was 94.89% in category 3, 96.90% in category 4A, 95.49% in category 4B, 92.28% in category 4C, and 95.45% in category 5 nodules. Based on pathological results, the AUC of the CAD model was 0.924 and the predicted probability of CAD was a little higher than the actual probability in the calibration curve. After referencing BI-RADS classification results, the adjustments were made to 1,583 nodules, of which 905 were classified to a lower category and 678 to a higher category in the sampling test set. As a result, the ACC (72.41-82.65%), SEN (32.73-56.98%), and SPE (82.46-89.26%) of the classification by each radiologist were significantly improved on average, with the consistency (k values) in almost all of them increasing to >0.6.Conclusions: The radiologist's classification consistency was markedly improved with almost all the k values increasing by a value greater than 0.6, and the diagnostic efficiency was also improved by approximately 24% (32.73% to 56.98%) and 7% (82.46% to 89.26%) for SEN and SPE, respectively, of the total classification on average. The transformer-based CAD model can help to improve the radiologist's diagnostic efficacy and consistency with others in the classification of BI-RADS 3-5 nodules.
基金：
National Key Research and Development Plan of China [2016YFC0104803]
第一作者机构：[1]Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, China
通讯作者：
通讯机构：[1]Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, China[2]Department of Ultrasonography, Beijing Tiantan Hospital, Capital Medical University, Beijing, China[*1]Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medial University, 1 Dong-jiao-minxiang, Dongcheng District, Beijing 100730, China.[*2]Department of Ultrasonography, Beijing Tiantan Hospital, Capital Medical University, No. 119 West Section of South 4th Ring Road, Fengtai District, Beijing 100070, China.
推荐引用方式(GB/T 7714)：
Hongtao Ji,Qiang Zhu,Teng Ma,et al.Development and validation of a transformer-based CAD model for improving the consistency of BI-RADS category 3-5 nodule classification among radiologists: a multiple center study[J].QUANTITATIVE IMAGING IN MEDICINE AND SURGERY.2023,13(6):3671-3687.doi:10.21037/qims-22-1091.
APA：
Hongtao Ji,Qiang Zhu,Teng Ma,Yun Cheng,Shuai Zhou...&Aiyun Zhou.(2023).Development and validation of a transformer-based CAD model for improving the consistency of BI-RADS category 3-5 nodule classification among radiologists: a multiple center study.QUANTITATIVE IMAGING IN MEDICINE AND SURGERY,13,(6)
MLA：
Hongtao Ji,et al."Development and validation of a transformer-based CAD model for improving the consistency of BI-RADS category 3-5 nodule classification among radiologists: a multiple center study".QUANTITATIVE IMAGING IN MEDICINE AND SURGERY 13..6(2023):3671-3687
