In this study, we aimed to identify the specific microRNAs (miRNAs) that are involved in the osteogenic differentiation of bone marrow mesenchymal stem cells (BMSCs) from ovariectomized (OVX) mice, and to further explore the mechanism by which these miRNAs regulate osteogenic differentiation. Based on the existing studies, the expression of seven miRNAs in BMSCs from OVX mice was evaluated using quantitative reverse transcription polymerase chain reaction (qRT-PCR). The expression of miR-133a-3p and osteogenesis-related genes (runt-related transcription factor 2 (Runx2), Osterix, alkaline phosphatase (ALP), and osteopontin) in BMSCs treated with miR-133a-3p mimics or inhibitors was detected by qRT-PCR or Western blotting. Osteogenesis efficiency was determined using ALP and alizarin red staining. The effector-target relationship between miR-133a-3p and ankyrin repeat domain 44 (ANKRD44) was confirmed by bioinformatics and a dual luciferase assay. Among the seven selected miRNAs, miR-133a-3p expression was significantly increased in BMSCs from OVX mice. Overexpression of miR-133a-3p dramatically inhibited the expression of osteogenesis-related genes in BMSCs and reduced ALP activity and mineralization. However, these processes were markedly ameliorated upon miR-133a-3p inhibition. Moreover, miR-133a-3p appeared to target ANKRD44, and the ANKRD44 expression was negatively regulated by miR-133a3p. Furthermore, ANKRD44 upregulation eliminated the anti-osteogenic differentiation effects of miR-133a-3p in BMSCs. Thus, our results indicated that miR-133a-3p inhibits the osteogenic differentiation of BMSCs by suppressing ANKRD44.