Background Gastric cancer (GC) is a highly heterogeneous tumor with different responses to immunotherapy. Identifying immune subtypes and landscape of GC could improve immunotherapeutic strategies. Methods Based on the abundance of tumor-infiltrating immune cells in GC patients from The Cancer Genome Atlas, we used unsupervised consensus clustering algorithm to identify robust clusters of patients, and assessed their reproducibility in an independent cohort from Gene Expression Omnibus. We further confirmed the feasibility of our immune subtypes in five independent pan-cancer cohorts. Finally, functional enrichment analyses were provided, and a deep learning model studying the pathological images was constructed to identify the immune subtypes. Results We identified and validated three reproducible immune subtypes presented with diverse components of tumor-infiltrating immune cells, molecular features, and clinical characteristics. An immune-inflamed subtype 3, with better prognosis and the highest immune score, had the highest abundance of CD8+ T cells, CD4+ T-activated cells, follicular helper T cells, M1 macrophages, and NK cells among three subtypes. By contrast, an immune-excluded subtype 1, with the worst prognosis and the highest stromal score, demonstrated the highest infiltration of CD4+ T resting cells, regulatory T cells, B cells, and dendritic cells, while an immune-desert subtype 2, with an intermediate prognosis and the lowest immune score, demonstrated the highest infiltration of M2 macrophages and mast cells, and the lowest infiltration of M1 macrophages. Besides, higher proportion of EVB and MSI of TCGA molecular subtyping, over expression of CTLA4, PD1, PDL1, and TP53, and low expression of JAK1 were observed in immune subtype 3, which consisted with the results from Gene Set Enrichment Analysis. These subtypes may suggest different immunotherapy strategies. Finally, deep learning can predict the immune subtypes well. Conclusion This study offers a conceptual frame to better understand the tumor immune microenvironment of GC. Future work is required to estimate its reference value for the design of immune-related studies and immunotherapy selection.