ObjectiveInvolvement of multiple lymph node (LN) metastasis in papillary thyroid microcarcinoma (PTMC) may indicate a progressive disease. To assist treatment decision, we conducted a clinical study to develop and validate a prediction model for the preoperative evaluation of LN metastasis involving more than five lymph nodes in patients with clinical N0 (cN0) PTMC. Material and MethodsUsing data from 6,337 patients with cN0 PTMCs at Tianjin Medical University Cancer Institute and Hospital from 2013 to 2017, we identified and integrated risk factors for the prediction of multiple LN metastasis to build a nomogram. The predictive accuracy and discriminative ability of the nomogram were evaluated by the concordance index (C-index) and calibration curve. The model was validated using bootstrap resampling of the training cohort and an independent temporal validation cohort at the same institution. ResultsIn the training cohort (n = 3,209 patients), six independent risk factors were identified and included the prediction model (PTMC Active Surveillance or Surgery (ASOS) Model), including age, gender, multifocality, tumor size, calcification, and aspect ratio. The PTMC ASOS model was validated both internally and through the temporal validation cohort (n = 3,128 patients) from the same institute. The C-indexes of the prediction model in the training cohort were 0.768 (95% CI, 0.698-0.838), 0.768 and 0.771 in the internal validation and external validation cohorts, respectively. The area under the receiver operating characteristic curve (AUC) was 0.7068 and 0.6799. The calibration curve for probability of large-LN metastasis showed good agreement between prediction by nomogram and actual observation. DCA curves were used for comparison with another model, and IDI and NRI were also calculated. The cutoff value of our model was obtained by the ROC curve. Based on this model and cut point, a web-based dynamic nomogram was developed (https://tjmuch-thyroid.shinyapps.io/PTMCASOSM/). ConclusionWe established a novel nomogram that can help to distinguish preoperatively cN0 PTMC patients with or without metastasis of multiple lymph nodes. This clinical prediction model may be used in decision making for both active surveillance and surgery.