Precise segmentation of the visual pathway is significant in preoperative planning to prevent the surgeon from touching it during the operation. Manual segmentation is time consuming and tedious. Thus, automatic segmentation strategies are necessary to assist clinical evaluation. However, the low contrast and blurred boundary between the target and the background in the image make automatic segmentation a challenging problem. This paper proposed a spatial probabilistic distribution map (SPDM)-based two-channel 3D U-Net to make shape and position prior information available for deep learning. First, an atlas calculated by group-wise registration was used to register each training volume image for deformation field determination. Second, the deformation field was used to transform the label of the corresponding training image to the template space, and then all the warped labels were summed up to create an SPDM. Third, the region of interest of the image and SPDM were sent to the network to predict the final segmentation. The proposed method was evaluated and compared against a conventional 3D U-Net on two datasets. Experimental results indicated that our method overcame the problem of low contrast and achieved better performance than previous methods. (C) 2020 Elsevier B.V. All rights reserved.