Different patterns of retinal arterioles and venules in the fundus images form an important metric to measure the disease severity. Therefore, an accurate classification of arterioles and venules is greatly necessary. In this work, we propose a novel network, named as the dual Deeply-Supervised Network (dual DSN), to classify arterioles and venules on retinal fundus images. We employ the U-shape network (U-Net) as the backbone of our proposed model. Our proposed dual DSN produces an auxiliary output of the network at every scale, which generates a loss by comparing to the manual annotation. The losses in the encoding path of dual DSN regularize the low-level features, while those in the decoding path of dual DSN regularize the high-level features. In sum, such losses in dual DSN form dual supervision to the backbone U-Net and capture the multi-level features of the input image, which improves the classification of retinal arterioles and venules. The experimental results demonstrate that the proposed dual DSN outperforms the previous state-of-the-art methods on DRIVE dataset with an accuracy of 95.0%.