Background There is increasing evidence that the lung is a target organ of diabetes. This study aimed to examine in detail the association between diabetes mellitus and pulmonary function using a national cohort. We also aimed to explore the non-linear association between pulmonary function and blood glucose, insulin resistance, and C-reactive protein (CRP). Methods A total of 30,442 participants from the National Health and Nutrition Examination Survey from the period between 2007 and 2012 were included. The cross-sectional association between diabetes mellitus and pulmonary function was assessed using multiple linear regression. Where there was evidence of non-linearity, we applied a restricted cubic spline with three knots to explore the non-linear association. Partial mediation analysis was performed to evaluate the underlying mechanism. All analyses were weighted to represent the US population and to account for the intricate survey design. Results A total of 8584 people were included in the final study population. We found that diabetes was significantly associated with reduced forced expiratory volume in one second (FEV1) and forced vital capacity. We further found L-shaped associations between hemoglobin A1c (HbA1c) and pulmonary function. There was a negative association between HbA1c and FEV1 in diabetes participants with good glucose control (HbA1c < 7.0%), but not in patients with poor glucose control. A non-linear association was also found with fasting plasma glucose, 2 h-plasma glucose after oral glucose tolerance test, insulin resistance, and CRP. Finally, we found that diabetes duration did not affect pulmonary function, and the deleterious effect of diabetes on pulmonary function was mediated by hyperglycemia, insulin resistance, low-grade chronic inflammation (CRP), and obesity. Conclusions Diabetes mellitus is non-linearly associated with pulmonary function. Our finding of a negative association between HbA1c and FEV1 in diabetes patients with good glucose control but not in patients with poor glucose control indicates that a stricter glycemic target should be applied to diabetic patients to improve pulmonary function. Given, the cross-sectional nature of this research, a longitudinal study is still needed to validate our findings.