A comparative study of statistical methods for identifying differentially expressed genes in spatial transcriptomics
Open accessAbstract
Spatial transcriptomics (ST) provides unprecedented insights into gene expression patterns while retaining spatial context, making it a valuable tool for understanding complex tissue architectures, such as those found in cancers. Seurat, by far the most popular tool for analyzing ST data, uses the Wilcoxon rank-sum test by default for differential expression analysis. However, as a nonparametric method that disregards spatial correlations, the Wilcoxon test can lead to inflated false positive rates and misleading findings. This limitation highlights the need for a more robust statistical approach that effectively incorporates spatial correlations. To this end, we propose a Generalized Estimating Equations (GEE) framework as a robust solution for differential gene expression analysis in ST. We conducted a comprehensive comparison of the GEE-based tests with existing methods, including the Wilcoxon rank-sum test and z -test. By appropriately accounting for spatial correlations, extensive simulations showed that the GEE test with robust standard error, referred to as the Independent GEE, demonstrated superior Type I error control and comparable power relative to other methods. Applications to ST datasets from breast and prostate cancer showed poor calibration of the p-values and potential false positive findings from the Wilcoxon rank-sum test. Our comparative study based on simulations and real data applications suggests that the Independent GEE test is well-suited for ST data, offering more accurate identification of biologically relevant gene expression changes and complementing the Wilcoxon rank-sum test. We have implemented the proposed method in R package “SpatialGEE”, available on GitHub.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ST spot-based sequencing captures hundreds to thousands of genes across localized areas, yielding deeply granular, multidimensional datasets while maintaining anatomical structure, which single-cell or bulk RNA sequencing alone cannot provide.
“Despite our low number of cases, ST spot‐based sequencing captures hundreds to thousands of genes across localized areas, thereby yielding deeply granular, multidimensional datasets, while maintaining anatomical structure, 28 that single‐cell or bulk RNA sequencing alone cannot provide.”
From Disruption of hippocampal upstream regulators of mTOR and insulin pathways in Down syndrome with Alzheimer's disease neuropathology: Preliminary observations (2026) · cites this paper as [28]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredIn ST experiments, methods designed to control false-positive rates by accounting for spatial correlations were applied.
“In ST experiments, for comparisons of DEGs as well as alterations in upstream regulators from specific ROIs within corresponding clusters between DS‐AD and HC groups, we applied methods designed to control false‐positive rates by accounting for spatial correlations. 28”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity
White, H. (1980) · Econometrica · doi:10.2307/1912934 · Cited by 25,861 · see citing works
(1967) · Cited by 23,719 · see citing works
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550 · see citing works
Comprehensive Integration of Single-Cell Data
Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W. M., Hao, Y., Stoeckius, M., Smibert, P., & Satija, R. (2019) · Cell · doi:10.1016/j.cell.2019.05.031 · Cited by 15,453 · see citing works
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354 · see citing works
Integrating single-cell transcriptomic data across different conditions, technologies, and species
Butler, A., Hoffman, P., Smibert, P., Papalexi, E., & Satija, R. (2018) · Nature Biotechnology · doi:10.1038/nbt.4096 · Cited by 13,174 · see citing works
Generalized linear mixed models: a practical guide for ecology and evolution
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J.-S. S. (2009) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2008.10.008 · Cited by 8,910 · see citing works
The phosphatidylinositol 3-Kinase–AKT pathway in human cancer
Vivanco, I., & Sawyers, C. L. (2002) · Nature reviews. Cancer · doi:10.1038/nrc839 · Cited by 6,041 · see citing works
Dictionary learning for integrative, multimodal and scalable single-cell analysis
Hao, Y., Stuart, T., Kowalski, M. H., Choudhary, S., Hoffman, P., Hartman, A., Srivastava, A., Molla, G., Madad, S., Fernandez-Granda, C., et al. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01767-y · Cited by 5,631 · see citing works
(1967) · Project Euclid (Cornell University) · Cited by 4,720 · see citing works
Approximate Inference in Generalized Linear Mixed Models
Breslow, N. E., & Clayton, D. G. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594284 · Cited by 4,595 · see citing works
Models for Longitudinal Data: A Generalized Estimating Equation Approach
Zeger, S. L., Liang, K., & Albert, P. S. (1988) · Biometrics · doi:10.2307/2531734 · Cited by 4,505 · see citing works
Visualization and analysis of gene expression in tissue sections by spatial transcriptomics
Ståhl, P. L., Salmén, F., Vickovic, S., Lundmark, A., Navarro, J. F., Magnusson, J., Giacomello, S., Asp, M., Westholm, J. O., Huss, M., et al. (2016) · Science · doi:10.1126/science.aaf2403 · Cited by 3,834 · see citing works
Extracellular Matrix Degradation and Remodeling in Development and Disease
Lu, P., Takai, K., Weaver, V. M., & Werb, Z. (2011) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a005058 · Cited by 2,265 · see citing works
Kolberg, L., Raudvere, U., Kuzmin, I., Adler, P., Vilo, J., & Peterson, H. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkad347 · Cited by 1,673 · see citing works
Mitogen-Activated Protein (MAP) Kinase Pathways: Regulation and Physiological Functions
Pearson, G. (2001) · Endocrine Reviews · doi:10.1210/er.22.2.153 · Cited by 1,060 · see citing works
Rao, C. R. (1948) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100023987 · Cited by 728 · see citing works
SpatialDE: identification of spatially variable genes
Svensson, V., Teichmann, S. A., & Stegle, O. (2018) · Nature Methods · doi:10.1038/nmeth.4636 · Cited by 695 · see citing works
Integrins: roles in cancer development and as treatment targets
Jin, H., & Varner, J. A. (2004) · British Journal of Cancer · doi:10.1038/sj.bjc.6601576 · Cited by 574 · see citing works
Proteoglycans in cancer biology, tumour microenvironment and angiogenesis
Iozzo, R. V., & Sanderson, R. D. (2010) · Journal of Cellular and Molecular Medicine · doi:10.1111/j.1582-4934.2010.01236.x · Cited by 569 · see citing works
The expanding vistas of spatial transcriptomics
Tian, L., Chen, F., & Macosko, E. Z. (2023) · Nature Biotechnology · doi:10.1038/s41587-022-01448-2 · Cited by 539 · see citing works
Statistical analysis of spatial expression patterns for spatially resolved transcriptomic studies
Sun, S., Zhu, J., & Zhou, X. (2020) · Nature Methods · doi:10.1038/s41592-019-0701-7 · Cited by 502 · see citing works
Spatially resolved clonal copy number alterations in benign and malignant tissue
Erickson, A. M., He, M., Berglund, E., Marklund, M., Mirzazadeh, R., Schultz, N., Kvastad, L., Andersson, A., Bergenstråhle, L., Bergenstråhle, J., et al. (2022) · Nature · doi:10.1038/s41586-022-05023-2 · Cited by 307 · see citing works
Rank-Sum Tests for Clustered Data
Datta, S., & Satten, G. A. (2005) · Journal of the American Statistical Association · doi:10.1198/016214504000001583 · Cited by 219 · see citing works
Essential Statistical Inference
Boos, D. D., & Stefanski, L. A. (2013) · Springer texts in statistics · doi:10.1007/978-1-4614-4818-1 · Cited by 199 · see citing works
A Powerful and Adaptive Association Test for Rare Variants
Pan, W., Kim, J., Zhang, Y., Shen, X., & Wei, P. (2014) · Genetics · doi:10.1534/genetics.114.165035 · Cited by 172 · see citing works
Self-citationCell type-specific inference of differential expression in spatial transcriptomics
Cable, D. M., Murray, E. M., Shanmugam, V., Zhang, S., Zou, L. S., Diao, M., Chen, H., Macosko, E. Z., Irizarry, R. A., & Chen, F. (2022) · Nature Methods · doi:10.1038/s41592-022-01575-3 · Cited by 164 · see citing works
Molecular profile of bladder cancer progression to clinically aggressive subtypes
Guo, C. C., Lee, S., Lee, J. G., Chen, H., Zaleski, M. P., Choi, W., McConkey, D. J., Wei, P., & Czerniak, B. A. (2024) · Nature Reviews Urology · doi:10.1038/s41585-023-00847-7 · Cited by 94 · see citing works
Self-citationWei, P., & Pan, W. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm612 · Cited by 84 · see citing works
Self-citationMa, Y., & Zhou, X. (2024) · Nature Methods · doi:10.1038/s41592-024-02284-9 · Cited by 77 · see citing works
Zhang, Y., Xu, Z., Shen, X., & Pan, W. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.03.061 · Cited by 64 · see citing works
Wang, X., Lee, S., Zhu, X., Redline, S. S., & Lin, X. (2013) · Genetic Epidemiology · doi:10.1002/gepi.21763 · Cited by 56 · see citing works
Spatial Linear Mixed Models with Covariate Measurement Errors.
Li, Y., Tang, H., & Lin, X. (2009) · PubMed · Cited by 38 · see citing works
Ospina, O. E., Wilson, C. M., Soupir, A. C., Berglund, A., Smalley, I., Tsai, K. Y., & Fridley, B. L. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac145 · Cited by 28 · see citing works
The origin of bladder cancer from mucosal field effects
Bondaruk, J. E., Jaksik, R., Wang, Z., Cogdell, D. E., Lee, S., Chen, Y., Dinh, K. N., Majewski, T., Zhang, L., Cao, S., et al. (2022) · iScience · doi:10.1016/j.isci.2022.104551 · Cited by 27 · see citing works
Self-citationiIMPACT: integrating image and molecular profiles for spatial transcriptomics analysis
Jiang, X., Wang, S., Guo, L., Zhu, B., Wen, Z., Jia, L., Xu, L., Xiao, G., & Li, Q. (2024) · Genome biology · doi:10.1186/s13059-024-03289-5 · Cited by 25 · see citing works
spatialGE Is a User-Friendly Web Application That Facilitates Spatial Transcriptomics Data Analysis
Ospina, O. E., Manjarres-Betancur, R., Gonzalez-Calderon, G., Soupir, A. C., Smalley, I., Tsai, K. Y., Markowitz, J., Khaled, M. L., Vallebuona, E. J., Berglund, A., et al. (2024) · Cancer Research · doi:10.1158/0008-5472.can-24-2346 · Cited by 10 · see citing works
Addressing the mean–variance relationship in spatially resolved transcriptomics data with spoon
Shah, K., Guo, B., & Hicks, S. C. (2024) · Biostatistics · doi:10.1093/biostatistics/kxaf012 · Cited by 4 · see citing works
Lee, S., Jung, S. Y., Kuś, P., Bondaruk, J. E., Lee, J. G., Jaksik, R., Putluri, N., Dinh, K. N., Cogdell, D. E., Chen, H., et al. (2025) · The Journal of Pathology · doi:10.1002/path.6474 · Cited by 4 · see citing works
Self-citationTUSCAN: Tumor segmentation and classification analysis in spatial transcriptomics
Zang, C., Guo, C. C., Wang, Y., Wei, P., & Li, Z. G. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1014058 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
1 of the 42 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- [16]Lehmann EL, Romano JP. Testing statistical hypotheses. 4th ed. Springer.
- [27]McCulloch CE, Searle SR, Neuhaus JM. Generalized, linear, and mixed models. 2nd ed. Wiley.
Cited by
Sorted by citations
Differential expression analysis for spatially correlated data using smiDE
Vasconcelos, A. G., McGuire, D., Simon, N., Danaher, P. J., & Shojaie, A. (2026) · Genome biology · doi:10.1186/s13059-025-03867-1 · Cited by 3 · see citing works
Nordbeck, A. J., Martá‐Ariza, M., Ek Olofsson, H., Kanshin, E., Ueberheide, B., Jones, K., Sanford, B., Hamlett, E. D., Head, E., Mufson, E. J., et al. (2026) · Alzheimer's & Dementia · doi:10.1002/alz.71776
“…the DG. 72 Despite our low number of cases, ST spot-based sequencing captures hundreds to thousands of genes across localized areas, thereby yielding deeply granular, multidimensional datasets, while maintaining anatomical structure, 28 that single-cell or bulk RNA sequencing alone cannot provide.”
- Authors
- Wang, Y., Zang, C., Li, Z., Guo, C. C., Lai, D., & Wei, P.
- Affiliations
- The University of Texas MD Anderson Cancer Center (US)
- The University of Texas Health Science Center at Houston (US)
- Corresponding author
- Wei, Peng
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2026-02-11
- Volume
- 22
- Issue
- 2
- Pages
- e1013956
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC12912703
- OpenAlex
- W7128589789
- Cited by
- 2
- References
- 42
- Citations by year
- 2026: 2
- FWCI
- 4.13
- Type
- journal article
- Language
- en
- Update notices
- new_version (2026-02-17)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- Digital Commons-TMC (Texas Medical Center) (submittedVersion)
- PubMed Central (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Cancer Prevention and Research Institute of Texas (RP230166)
- National Institutes of Health (P50CA217674, P01CA296429)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Keywords
- spatial transcriptomics, differential gene expression, generalized estimating equations, spatial correlation, Mann-Whitney test, false positive rate, type I error control, Seurat, breast cancer, prostate cancer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
