Parameters derived from diffusion tensor imaging (DTI) have been found to be significantly altered in the optic tracts, optic nerves, and optic radiations in patients with primary open-angle glaucoma (POAG). In this study, DTI-derived parameters were further constructed into fiber connectivity, and we investigated anatomical fiber connectivity changes within and beyond the visual pathway in POAG patients. DTI and TI-weighted magnetic resonance images were acquired in 18 POAG patients and 26 healthy controls (HC). White matter tracts based on the Brodmann atlases (BA) were constructed using the deterministic fiber tracking method. The mean fractional anisotropy (FA), fiber number (FN), and mean fiber length (FL) were measured and then evaluated using two sample t-tests between POAG and HC. The fiber connectivity between regions was taken as the features for classifying HC and POAG using a machine learning method known as na ve Bayesian classification. The mean FA decreased in connections between visual cortex BA17/BA18 and cortex BA23/BA25/BA35/BA36, while it increased in the connections between cortex BA3/BA7/BA9 and BA5/BA6/BA4S/BA2S in POAG. Classification using fibers where a significant difference in FN had been identified produced better accuracy (ACC = 0.89) than using FA or FL (ACC = 0.77 and 0.75, respectively). The FN of individual fiber connections with higher accuracy and significant changes in POAG involved brain regions associated with vision (BA19), depression (BA10/BA46/BA25), and memory (BA29). These findings strengthen the hypothesis that POAG involves changes in anatomical connectivity within and beyond the visual pathway. Classification using the machine learning method reveals that mean FN has the potential to be used as a biomarker for detecting white matter microstructure changes in POAG.