机构:[1]Shanghai Belt and Road International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai, China.[2]Ministry of Education Key Laboratory of Artificial Intelligence, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China.[3]Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China.[4]Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Hong Kong, China.[5]Department of Ophthalmology, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China.[6]School of Clinical Medicine, Tsinghua Medicine, Tsinghua University, Beijing, China.[7]School of Clinical Medicine, Beijing Tsinghua Changgung Hospital, Beijing, China.[8]Department of Ophthalmology, Shanghai Health and Medical Center, Wuxi, China.[9]Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong, China.[10]Shanghai Key Laboratory of Intelligent Information Processing, School of Computer Science, Fudan University, Shanghai, China.[11]Academy for Engineering and Technology, Fudan University, Shanghai, China.[12]Laboratoire de Traitement de l'Information Medicale UMR 1101, Inserm, Brest, France.[13]Universite de Bretagne Occidentale, Brest, France.[14]School of Computer Science and Technology, Dongguan University of Technology, Dongguan, China.[15]AIFUTURE Laboratory, Beijing, China.[16]National Digital Health Center of China Top Think Tanks, Beijing Normal University, Beijing, China.[17]School of Journalism and Communication, Beijing Normal University, Beijing, China.[18]School of Computing and Augmented Intelligence, Arizona State University, Tempe.[19]Mediwhale, Seoul, South Korea.[20]Pohang University of Science and Technology, Pohang, South Korea.[21]Department of Electrical and Computer Engineering, University of Arizona, Tucson.[22]Department of Ophthalmology, Linkou Chang Gung Memorial Hospital, College of Medicine, Chang Gung University, Taoyuan, Taiwan.中山大学附属第二医院[23]Department of Ophthalmology, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, China.[24]Singapore Eye Research Institute, Singapore National Eye Centre, Singapore.[25]Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China.[26]Department of Ophthalmology and Visual Science, Tokyo Medical and Dental University, Tokyo, Japan.[27]Institut Francais de Myopie, Rothschild Foundation Hospital, Paris, France.[28]Zhongshan Ophthalmic Center, Guangzhou, China.[29]Center for Innovation and Precision Eye Health, Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.[30]Ophthalmology and Visual Science Academic Clinical Program, Duke-National University of Singapore Medical School, Singapore.[31]Beijing Institute of Ophthalmology, Beijing Ophthalmology and Visual Sciences Key Laboratory, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China.研究所眼科研究所首都医科大学附属北京同仁医院首都医科大学附属同仁医院
Myopic maculopathy (MM) is a major cause of vision impairment globally. Artificial intelligence (AI) and deep learning (DL) algorithms for detecting MM from fundus images could potentially improve diagnosis and assist screening in a variety of health care settings.To evaluate DL algorithms for MM classification and segmentation and compare their performance with that of ophthalmologists.The Myopic Maculopathy Analysis Challenge (MMAC) was an international competition to develop automated solutions for 3 tasks: (1) MM classification, (2) segmentation of MM plus lesions, and (3) spherical equivalent (SE) prediction. Participants were provided 3 subdatasets containing 2306, 294, and 2003 fundus images, respectively, with which to build algorithms. A group of 5 ophthalmologists evaluated the same test sets for tasks 1 and 2 to ascertain performance. Results from model ensembles, which combined outcomes from multiple algorithms submitted by MMAC participants, were compared with each individual submitted algorithm. This study was conducted from March 1, 2023, to March 30, 2024, and data were analyzed from January 15, 2024, to March 30, 2024.DL algorithms submitted as part of the MMAC competition or ophthalmologist interpretation.MM classification was evaluated by quadratic-weighted κ (QWK), F1 score, sensitivity, and specificity. MM plus lesions segmentation was evaluated by dice similarity coefficient (DSC), and SE prediction was evaluated by R2 and mean absolute error (MAE).The 3 tasks were completed by 7, 4, and 4 teams, respectively. MM classification algorithms achieved a QWK range of 0.866 to 0.901, an F1 score range of 0.675 to 0.781, a sensitivity range of 0.667 to 0.778, and a specificity range of 0.931 to 0.945. MM plus lesions segmentation algorithms achieved a DSC range of 0.664 to 0.687 for lacquer cracks (LC), 0.579 to 0.673 for choroidal neovascularization, and 0.768 to 0.841 for Fuchs spot (FS). SE prediction algorithms achieved an R2 range of 0.791 to 0.874 and an MAE range of 0.708 to 0.943. Model ensemble results achieved the best performance compared to each submitted algorithms, and the model ensemble outperformed ophthalmologists at MM classification in sensitivity (0.801; 95% CI, 0.764-0.840 vs 0.727; 95% CI, 0.684-0.768; P = .006) and specificity (0.946; 95% CI, 0.939-0.954 vs 0.933; 95% CI, 0.925-0.941; P = .009), LC segmentation (DSC, 0.698; 95% CI, 0.649-0.745 vs DSC, 0.570; 95% CI, 0.515-0.625; P < .001), and FS segmentation (DSC, 0.863; 95% CI, 0.831-0.888 vs DSC, 0.790; 95% CI, 0.742-0.830; P < .001).In this diagnostic study, 15 AI models for MM classification and segmentation on a public dataset made available for the MMAC competition were validated and evaluated, with some models achieving better diagnostic performance than ophthalmologists.
基金:
This study was supported by
funding from the National Key Research &
Development Program of China
(2022YFC2502800), the National Natural Science
Foundation of China (NSFC) (82388101), and the
Beijing Natural Science Foundation (IS23096) awarded to Dr T.Wong, funding from the NSFC
(62272298) and Shanghai Municipal Science and
Technology Major Project (2021SHZDZX0102)
awarded to Dr Sheng, and funding from the
College-level Project Fund of Shanghai Sixth
People’s Hospital (YNLC201909) and the
Interdisciplinary Program of Shanghai Jiao Tong
University (YG2022QN089) awarded to
Dr XiangningWang.
第一作者机构:[1]Shanghai Belt and Road International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai, China.[2]Ministry of Education Key Laboratory of Artificial Intelligence, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China.
共同第一作者:
通讯作者:
通讯机构:[6]School of Clinical Medicine, Tsinghua Medicine, Tsinghua University, Beijing, China.[7]School of Clinical Medicine, Beijing Tsinghua Changgung Hospital, Beijing, China.[24]Singapore Eye Research Institute, Singapore National Eye Centre, Singapore.[28]Zhongshan Ophthalmic Center, Guangzhou, China.
推荐引用方式(GB/T 7714):
Qian Bo,Sheng Bin,Chen Hao,et al.A Competition for the Diagnosis of Myopic Maculopathy by Artificial Intelligence Algorithms[J].JAMA Ophthalmology.2024,142(11):1006-1015.doi:10.1001/jamaophthalmol.2024.3707.
APA:
Qian Bo,Sheng Bin,Chen Hao,Wang Xiangning,Li Tingyao...&Wang Ya Xing.(2024).A Competition for the Diagnosis of Myopic Maculopathy by Artificial Intelligence Algorithms.JAMA Ophthalmology,142,(11)
MLA:
Qian Bo,et al."A Competition for the Diagnosis of Myopic Maculopathy by Artificial Intelligence Algorithms".JAMA Ophthalmology 142..11(2024):1006-1015