Single-cell transcriptome analysis has provided detailed insights into the ecosystem of liver cancer. However, the changes of the cellular and molecular components of liver tumors in comparison with normal livers have not been described at single-cell level. Here, we performed an integrative single-cell analysis of both normal livers and liver cancers. Principal component analysis was firstly performed to delineate the cell lineages in liver tissues. Differential gene expression within major cell types were then analyzed between tumor and normal samples, thus resolved the cell type-specific molecular alterations in liver cancer development. Moreover, a comparison between liver cancer derived versus normal liver derived cell components revealed that two subpopulations of fibroblasts were exclusively expanded in liver cancer tissues. By further defining subpopulation-specific gene signatures, characterizing their spatial distribution in tumor tissues and investigating their clinical significance, we found that the SPARCL1 positive fibroblasts, representing a group of tumor vessel associated fibroblasts, were related to reduced vascular invasion and prolonged survival of liver cancer patients. Through establishing an in-vitro endothelial-to-mesenchymal transition model, we verified the conversion of the fetal liver sinusoidal endothelial cells into the fibroblast-like cells, demonstrating a possible endothelial cell origination of the SPARCL1 positive fibroblasts. Our study provides new insights into the cell atlas alteration, especially the expanded fibroblasts in liver cancers.