SC-MEB: spatial clustering with hidden Markov random field using empirical Bayes

Spatial transcriptomics (ST) has been emerging as a powerful technique for resolving gene expression profiles while retaining the tissue spatial information. These spatially resolved transcriptomics make it feasible to examine the complex mulitcellular systems under different microenvironments. To answer scientific questions with spatial transcriptomics, the first step is to identify cell clusters by integrating available spatial information which would expand the understanding of how cell types and states are regulated by tissues. Here, we introduce SC-MEB, an empirical Bayes approach for spatial clustering analysis using hidden Markov random field. We also derive an efficient expectation-maximization (EM) algorithm based on iterative conditional mode (ICM) for SC-MEB. Compared with BayesSpace, a recently developed method, SC-MEB is not only computationally efficient and scalable to the sample size increment, but also is capable of choosing the smoothness parameter and the number of clusters as well. We then performed comprehensive simulation studies to demonstrate the effectiveness of SC-MEB over some existing methods. We first applied SC-MEB to perform clustering analysis in the spatial transcriptomics dataset from human dorsolateral prefrontal cortex tissues that were manually annotated. Our analysis results show that SC-MEB can achieve similar clustering performance with BayesSpace that uses the true number of clusters and fixed smoothness parameter. We then applied SC-MEB to analyze the dataset from a patient with colorectal cancer (CRC) and COVID-19, and further performed differential expression analysis to identify signature genes related to the clustering results. The heatmap for identified signature genes shows that the identified clusters from SC-MEB is more separable than those from BayesSpace. By using pathway analysis, we identified three immune-related clusters and further compared mean expressions of COVID-19 signature genes in immune regions with those in non-immune ones. As such, SC-MEB provides a valuable computational tool for investigating structural organizations of tissues from spatial transcriptomic data.