During mammalian retinal development, the multipotent progenitors differentiate into all classes of retinal cells under the delicate control of transcriptional factors. The deficiency of a transcription cofactor, the LIM-domain binding protein Ldb1, has been shown to cause proliferation and developmental defects in multiple tissues including cardiovascular, hematopoietic, and nervous systems; however, it remains unclear whether and how it regulates retinal development. By expression profiling, RNA in situ hybridization and immunostaining, here we show that Ldb1 is expressed in the progenitors during early retinal development, but later its expression gradually shifts to non-photoreceptor cell types including bipolar, amacrine, horizontal, ganglion, and Muller glial cells. Retina-specific ablation of Ldb1 in mice resulted in microphthalmia, optic nerve hypoplasia, retinal thinning and detachment, and profound vision impairment as determined by electroretinography. In the mutant retina, there was precocious differentiation of amacrine and horizontal cells, indicating a requirement of Ldb1 in maintaining the retinal progenitor pool. Additionally, all non-photoreceptor cell types were greatly reduced which appeared to be caused by a generation defect and/or retinal degeneration via excessive cell apoptosis. Furthermore, we showed that misexpressed Ldb1 was sufficient to promote the generation of bipolar, amacrine, horizontal, ganglion, and Muller glial cells at the expense of photoreceptors. Together, these results demonstrate that Ldb1 is not only necessary but also sufficient for the development and/or maintenance of non-photoreceptor cell types, and implicate that the pleiotropic functions of Ldb1 during retinal development are context-dependent and determined by its interaction with diverse LIM-HD (LIM-homeodomain) and LMO (LIM domain-only) binding protein partners.