Purpose To develop and validate a deep learning-based feature ensemble model using multiparametric magnetic resonance imaging (MRI) for predicting tumor budding (TB) grading in patients with rectal cancer (RC). Methods A retrospective cohort of 458 patients with pathologically confirmed rectal cancer (RC) from three institutions was included. Among them, 355 patients from Center 1 were divided into two groups at a 7:3 ratio: the training cohort (n = 248) and the internal validation cohort (n = 107). An additional 103 patients from two other centers served as the external validation cohort. Deep learning models were constructed for T2-weighted imaging (T2WI) and diffusion-weighted imaging (DWI) based on the CrossFormer architecture, and deep learning features were extracted. Subsequently, a feature ensemble module based on the attention mechanism of Transformer was used to capture spatial interactions between different imaging sequences, creating a multiparametric ensemble model. The predictive performance of each model was evaluated using the area under the curve (AUC), calibration curves, and decision curve analysis (DCA). Results The deep learning model based on T2WI achieved AUC values of 0.789 (95% CI: 0.680-0.900) and 0.720 (95% CI: 0.591-0.849) in the internal and external validation cohorts, respectively. The deep learning model based on DWI had AUC values of 0.806 (95% CI: 0.705-0.908) and 0.772 (95% CI: 0.657-0.887) in the internal and external validation cohorts, respectively. The multiparametric ensemble model demonstrated superior performance, with AUC values of 0.868 (95% CI: 0.775-0.960) in the internal validation cohort and 0.839 (95% CI: 0.743-0.935) in the external validation cohort. DeLong test showed that the differences in AUC values among the models were not statistically significant in both the internal and external test sets (P > 0.05). The DCA curve demonstrated that within the 10-80% threshold range, the fusion model provided significantly higher clinical net benefit compared to other models. Conclusion Compared to single-sequence deep learning models, the attention mechanism-based multiparametric MRI fusion model enables more effective individualized prediction of TB grading in RC patients. It offers valuable guidance for treatment selection and prognostic evaluation while providing imaging-based support for personalized postoperative follow-up adjustments.