高级检索
当前位置: 首页 > 详情页

A multimodal visual-language foundation model for computational ophthalmology

文献详情

资源类型:
WOS体系:
Pubmed体系:

收录情况: ◇ SCIE

机构: [1]Hong Kong Polytech Univ, Sch Optometry, Kowloon, Hong Kong, Peoples R China [2]Hong Kong Polytech Univ, Res Ctr SHARP Vis RCSV, Kowloon, Hong Kong, Peoples R China [3]Swiss Fed Inst Technol Lausanne EPFL, Lausanne, Switzerland [4]Clemson Univ, Sch Comp, Clemson, SC USA [5]Zhejiang Univ, Affiliated Hosp 2, Sch Med, Dept Ophthalmol, Hangzhou, Peoples R China [6]Wuhan Bright Eye Hosp, Wuhan, Peoples R China [7]Shanghai Jiao Tong Univ, Shanghai Gen Hosp, Natl Clin Res Ctr Eye Dis, Dept Ophthalmol,Sch Med, 100 Haining Rd, Shanghai 20080, Peoples R China [8]Royal Victorian Eye & Ear Hosp, Ctr Eye Res Australia, East Melbourne, Australia [9]Guangdong Acad Med Sci, Guangdong Prov Peoples Hosp, Dept Ophthalmol, Guangzhou, Peoples R China [10]Capital Med Univ, Beijing Tongren Hosp, Beijing Tongren Eye Ctr, Beijing, Peoples R China [11]Monash Univ, Fac Informat Technol, AIM Hlth Lab, Melbourne, Vic, Australia [12]Ctr Eye & Vis Res CEVR, 17W Hong Kong Sci Pk,Sci Pk, Hong Kong, Peoples R China
出处:
ISSN:

摘要:
Early detection of eye diseases is vital for preventing vision loss. Existing ophthalmic artificial intelligence models focus on single modalities, overlooking multi-view information and struggling with rare diseases due to long-tail distributions. We propose EyeCLIP, a multimodal visual-language foundation model trained on 2.77 million ophthalmology images from 11 modalities with partial clinical text. Our novel pretraining strategy combines self-supervised reconstruction, multimodal image contrastive learning, and image-text contrastive learning to capture shared representations across modalities. EyeCLIP demonstrates robust performance across 14 benchmark datasets, excelling in disease classification, visual question answering, and cross-modal retrieval. It also exhibits strong few-shot and zero-shot capabilities, enabling accurate predictions in real-world, long-tail scenarios. EyeCLIP offers significant potential for detecting both ocular and systemic diseases, and bridging gaps in real-world clinical applications.

基金:
语种:
被引次数:
WOS:
PubmedID:
中科院(CAS)分区:
出版当年[2025]版:
大类 | 1 区 医学
小类 | 1 区 卫生保健与服务 1 区 医学:信息
最新[2025]版:
大类 | 1 区 医学
小类 | 1 区 卫生保健与服务 1 区 医学:信息
JCR分区:
出版当年[2023]版:
Q1 HEALTH CARE SCIENCES & SERVICES Q1 MEDICAL INFORMATICS
最新[2024]版:
Q1 HEALTH CARE SCIENCES & SERVICES Q1 MEDICAL INFORMATICS

影响因子: 最新[2024版] 最新五年平均 出版当年[2023版] 出版当年五年平均 出版前一年[2022版] 出版后一年[2024版]

第一作者:
第一作者机构: [1]Hong Kong Polytech Univ, Sch Optometry, Kowloon, Hong Kong, Peoples R China [2]Hong Kong Polytech Univ, Res Ctr SHARP Vis RCSV, Kowloon, Hong Kong, Peoples R China
共同第一作者:
通讯作者:
通讯机构: [1]Hong Kong Polytech Univ, Sch Optometry, Kowloon, Hong Kong, Peoples R China [2]Hong Kong Polytech Univ, Res Ctr SHARP Vis RCSV, Kowloon, Hong Kong, Peoples R China [12]Ctr Eye & Vis Res CEVR, 17W Hong Kong Sci Pk,Sci Pk, Hong Kong, Peoples R China
推荐引用方式(GB/T 7714):
APA:
MLA:

资源点击量:28508 今日访问量:0 总访问量:1584 更新日期:2025-09-01 建议使用谷歌、火狐浏览器 常见问题

版权所有©2020 首都医科大学附属北京同仁医院 技术支持:重庆聚合科技有限公司 地址:北京市东城区东交民巷1号(100730)