高级检索
当前位置: 首页 > 详情页

Real-time tracking and inpainting network with joint learning iterative modules for AR-based DALK surgical navigation

文献详情

资源类型:
WOS体系:
Pubmed体系:

收录情况: ◇ SCIE

机构: [1]Beihang Univ, Beijing Adv Innovat Ctr Biomed Engn, State Key Lab Virtual Real Technol & Syst, Beijing 100191, Peoples R China [2]Capital Med Univ, Beijing Tongren Hosp, Beijing Tongren Eye Ctr, Beijing Ophthalmol & Visual Sci Key Lab, Beijing 100730, Peoples R China [3]Beijing Inst Technol, Sch Comp Sci Technol, Beijing 100081, Peoples R China [4]Peking Univ, Peoples Hosp, Musculoskeletal Tumor Ctr, Beijing 100044, Peoples R China
出处:
ISSN:

关键词: Joint learning Semantic segmentation Optical flow Inpainting DALK AR-based surgical navigation

摘要:
Background and Objective: Deep anterior lamellar keratoplasty (DALK) is a widely used treatment for eye diseases and requires accurate and even stitch positions during the suturing process. In this regard, the utilization of Augmented Reality (AR) navigation systems shows promising potential in enhancing the stitching process, and a clear and unoccluded view of the corneal regions would help surgeons better plan the stitching positions. Methods: In this work, we present a joint-learning and iterative network for AR-based suturing navigation. This network aims to improve the performance of the inpainting under serious occlusion in the suturing process. And it can provide both original instruments and inpainted corneal masks along with inpainted frames. The network is based on feature reuse, iterative modules, and mask propagation structures to greatly reduce the computational cost. For the requirement of end-to-end training, we also propose a novel dataset synthesis method to construct a dataset with both occluded and unoccluded image pairs, along with mask and optical flow annotations. We also develop a novel pipeline based on the grid propagation method and inpainted optical flow outputs to provide clear and stable inpainted frames. Results: Based on the synthetic datasets, compared to the recent outstanding inpainting networks, our framework reaches a better trade-off between performance and computation efficiency. Our Iter-S model finally gets a mean endpoint error (mEPE) of 1.69, a peak signal-to-noise ratio (PSNR) of 36.86, and a structure similarity index measure (SSIM) of 0.976, along with a low inpainting inference time of 16.26ms. Based on the Iter-S, we construct a novel AR navigation system with a frame rate of around 35.14ms/28FPS on average. Conclusions: The iterative modules can progressively refine the outputs while providing a favorable trade-off between visual performance and real-time computation efficiency based on the selection of iteration times. Our AR navigation framework can provide stable and accurate tracking outputs with well-inpainted results in real time under severe occlusion conditions, which demonstrates the benefits of guiding the stitching operations of surgeons in corneal surgeries.

基金:
语种:
WOS:
PubmedID:
中科院(CAS)分区:
出版当年[2025]版:
大类 | 2 区 医学
小类 | 2 区 计算机:跨学科应用 2 区 计算机:理论方法 2 区 工程:生物医学 3 区 医学:信息
最新[2025]版:
大类 | 2 区 医学
小类 | 2 区 计算机:跨学科应用 2 区 计算机:理论方法 2 区 工程:生物医学 3 区 医学:信息
JCR分区:
出版当年[2023]版:
Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Q1 COMPUTER SCIENCE, THEORY & METHODS Q1 ENGINEERING, BIOMEDICAL Q1 MEDICAL INFORMATICS
最新[2024]版:
Q1 COMPUTER SCIENCE, THEORY & METHODS Q1 MEDICAL INFORMATICS Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Q2 ENGINEERING, BIOMEDICAL

影响因子: 最新[2024版] 最新五年平均 出版当年[2023版] 出版当年五年平均 出版前一年[2022版] 出版后一年[2024版]

第一作者:
第一作者机构: [1]Beihang Univ, Beijing Adv Innovat Ctr Biomed Engn, State Key Lab Virtual Real Technol & Syst, Beijing 100191, Peoples R China
通讯作者:
推荐引用方式(GB/T 7714):
APA:
MLA:

资源点击量:28994 今日访问量:0 总访问量:1619 更新日期:2025-10-01 建议使用谷歌、火狐浏览器 常见问题

版权所有©2020 首都医科大学附属北京同仁医院 技术支持:重庆聚合科技有限公司 地址:北京市东城区东交民巷1号(100730)