Tumor immunotherapy has achieved remarkable efficacy, with immune-checkpoint inhibitors as especially promising candidates for cancer therapy. However, some issues caused by immunotherapy have raised attention, such as limited efficacy for some patients, narrow antineoplastic spectrum, and adverse reactions, suggesting that using regulators of tumor immune response may prove to be more complicated than anticipated. Current evidence indicates that different factors collectively constituting the unique tumor microenvironment promote immune tolerance, and these include the expression of co-inhibitory molecules, the secretion of lactate, and competition for nutrients between tumor cells and immune cells. Furthermore, cancer-associated fibroblasts, the main cellular components of solid tumors, promote immunosuppression through inhibition of T cell function and extracellular matrix remodeling. Here, we summarize the research advances in tumor immunotherapy and the latest insights into the influence of microenvironment on tumor immunotherapy.