Previous studies have revealed brain adaptations to injury that occurs in optic neuritis (ON); however, the mechanisms underlying the functional connectivity (FC) and gray matter volume (GMV) changes in ON have not been clarified. Here, 51 single attack ON patients and 45 recurrent attacks ON patients were examined using structural MRI and resting-state functional MRI (RS-fMRI), and compared to 49 age- and gender-matched healthy controls (HC). FC analysis with a seed in primary visual cortex (V1 area) was used to assess the differences among three groups. Whole brain GMV was assessed using voxel-based morphometry (VBM). Correlation analyses were performed between FC results, structural MRI and clinical variables. We found positive correlations between the Paced Auditory Serial Addition Test (PASAT) score and FC in V1 area with bilateral middle frontal gyrus. Disease duration is significantly negatively related to FC in V1 area with the left inferior parietal lobule. Compared to the HC, single attack ON patients were found to have decreased FC values in the frontal, temporal lobes, right inferior occipital gyrus, right insula, right inferior parietal lobule, and significant increased FC values in the left thalamus. Recurrent attacks ON patients had the same pattern with single attack ON. No significant differences were found in brain GMV among three groups. This study provides the imaging evidence that impairment and compensation of V1 area connectivity coexist in ON patients, and provides important insights into the underlying neural mechanisms of ON.