Due to the lack of sufficient labeled data, semi-supervised learning is an effective scheme to boost the performance of retinal disease classification. However, traditional semi-supervised methods are restricted by the strong assumption that the labeled and unlabeled images share the same distribution. In practice, labeled and unlabeled retinal images may have different style domains (e.g. different cameras) or semantic domains (e.g. different diseases). In this paper, an out-of-distribution (OOD) semi-supervised learning method is proposed for retinal disease classification based on knowledge distillation with a teacher-student architecture. To extract the information of the OOD unlabeled images, our method leverages the consistency constrains of both spatial feature representation and the probability learned from teacher model and student model. The performance and generalization of the proposed method are evaluated on the public databases including Messidor, REFUGE and iChallenge-AMD. The experimental results show that the proposed approach outperforms the existing semi-supervised methods.