Robot-assisted microsurgery is a promising technique for a number of clinical specialties including neurosurgery. One of the prerequisites of such procedures is accurate vision guidance, delineating not only the exposed surface details but also embedded microvasculature. Conventional microscopic cameras used for vascular imaging are susceptible to specular reflections and changes in ambient light with low tissue resolution and contrast. Photoacoustic microscopy (PAM) is emerging as a promising tool and increasingly used for vascular imaging due to its high image resolution and tissue contrast. This paper presents a fused microscopic imaging scheme that integrates standard surgical microscopy with PAM for improved intraoperative visualization and guidance. We propose the FPM-R2Net to Fuse Photoacoustic and surgical Microscopic imaging via cross-modality Representation and Registration Network. A MOdality Representation Network (MORNet) is used to extract unified feature representation across white-light and PAM modalities, and a Hierarchical Iterative Registration Network (HIRNet) is used to establish the correspondence between the two modalities in a coarse-to-fine manner based on multi-resolution feature maps. A synthetic dataset with ground truth correspondence and an in vivo dataset of mouse brain vasculature are used to evaluate our proposed network. Extensive validation on the two datasets has shown significant improvements compared to the current state-of-the-art methods assessed with intersection over union and Dice scores (10.3% and 6.6% on the synthetic dataset and 15.9% and 11.8% on the in vivo dataset, respectively).
基金:
Shanghai Municipal Science and Technology Major Project, China [2021SHZDZX]; Science and Technology Commission of Shanghai Municipality, China [20DZ2220400]; China Postdoctoral Science Foundation, China [2023M732246]
语种:
外文
WOS:
PubmedID:
中科院(CAS)分区:
出版当年[2025]版:
大类|1 区医学
小类|1 区计算机:人工智能1 区计算机:跨学科应用1 区工程:生物医学1 区核医学
最新[2025]版:
大类|1 区医学
小类|1 区计算机:人工智能1 区计算机:跨学科应用1 区工程:生物医学1 区核医学
JCR分区:
出版当年[2023]版:
Q1COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCEQ1COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONSQ1ENGINEERING, BIOMEDICALQ1RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING
最新[2024]版:
Q1COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCEQ1COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONSQ1ENGINEERING, BIOMEDICALQ1RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING
第一作者机构:[1]Shanghai Jiao Tong Univ, Inst Med Robot, Sch Biomed Engn, Shanghai 200240, Peoples R China[2]Shanghai Jiao Tong Univ, Tongren Hosp, Inst Med Robot, Shanghai Key Lab Flexible Med Robot, Shanghai 200240, Peoples R China
通讯作者:
通讯机构:[1]Shanghai Jiao Tong Univ, Inst Med Robot, Sch Biomed Engn, Shanghai 200240, Peoples R China[2]Shanghai Jiao Tong Univ, Tongren Hosp, Inst Med Robot, Shanghai Key Lab Flexible Med Robot, Shanghai 200240, Peoples R China
推荐引用方式(GB/T 7714):
Liu Yuxuan,Zhou Jiasheng,Luo Yating,et al.FPM-R2Net: Fused Photoacoustic and operating Microscopic imaging with cross-modality Representation and Registration Network[J].MEDICAL IMAGE ANALYSIS.2025,105:doi:10.1016/j.media.2025.103698.
APA:
Liu, Yuxuan,Zhou, Jiasheng,Luo, Yating,Chen, Sung-Liang,Guo, Yao&Yang, Guang-Zhong.(2025).FPM-R2Net: Fused Photoacoustic and operating Microscopic imaging with cross-modality Representation and Registration Network.MEDICAL IMAGE ANALYSIS,105,
MLA:
Liu, Yuxuan,et al."FPM-R2Net: Fused Photoacoustic and operating Microscopic imaging with cross-modality Representation and Registration Network".MEDICAL IMAGE ANALYSIS 105.(2025)