Purpose Studies suggest that regular use of metformin may decrease cancer mortality. We investigated the association between diabetes medication use and cancer survival. Materials and Methods The current study includes 633 breast, 890 colorectal, 824 lung, and 543 gastric cancer cases identified from participants of two population-based cohort studies in Shanghai. Information on diabetes medication use was obtained by linking to electronic medical records. The associations between diabetes medication use (metformin, sulfonylureas, and insulin) and overall and cancer-specific survival were evaluated using time-dependent Cox proportional hazards models. Results After adjustment for clinical characteristics and treatment factors, use of metformin was associated with better overall survival among colorectal cancer patients (hazards ratio [HR], 0.55; 95% confidence interval [CI], 0.34 to 0.88) and for all four types of cancer combined (HR, 0.75; 95% CI, 0.57 to 0.98). Ever use of insulin was associated with worse survival for all cancer types combined (HR, 1.89; 95% CI, 1.57 to 2.29) and for the four cancer types individually. Similar associations were seen for diabetic patients. Sulfonylureas use was associated with worse overall survival for breast or gastric cancer (HR, 2.87; 95% CI, 1.22 to 6.80 and HR, 2.05; 95% CI, 1.09 to 3.84, respectively) among diabetic patients. Similar association patterns were observed between diabetes medication use and cancer-specific survival. Conclusion Metformin was associated with improved survival among colorectal cancer cases, while insulin use was associated with worse survival among patients of four major cancers. Further investigation on the topic is needed given the potential translational impact of these findings.