Cluster stability in the analysis of mass cytometry data
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Manual gating has been traditionally applied to cytometry data sets to identify cells based on protein expression. The advent of mass cytometry allows for a higher number of proteins to be simultaneously measured on cells, therefore providing a means to define cell clusters in a high dimensional expression space. This enhancement, whilst opening unprecedented opportunities for single cell‐level analyses, makes the incremental replacement of manual gating with automated clustering a compelling need. To this aim many methods have been implemented and their successful applications demonstrated in different settings. However, the reproducibility of automatically generated clusters is proving challenging and an analytical framework to distinguish spurious clusters from more stable entities, and presumably more biologically relevant ones, is still missing. One way to estimate cell clusters' stability is the evaluation of their consistent re‐occurrence within‐ and between‐algorithms, a metric that is commonly used to evaluate results from gene expression. Herein we report the usage and importance of cluster stability evaluations, when applied to results generated from three popular clustering algorithms ‐ SPADE, FLOCK and PhenoGraph ‐ run on four different data sets. These algorithms were shown to generate clusters with various degrees of statistical stability, many of them being unstable. By comparing the results of automated clustering with manually gated populations, we illustrate how information on cluster stability can assist towards a more rigorous and informed interpretation of clustering results. We also explore the relationships between statistical stability and other properties such as clusters' compactness and isolation, demonstrating that whilst cluster stability is linked to other properties it cannot be reliably predicted by any of them. Our study proposes the introduction of cluster stability as a necessary checkpoint for cluster interpretation and contributes to the construction of a more systematic and standardized analytical framework for the assessment of cytometry clustering results. © 2016 International Society for Advancement of Cytometry
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Data-Driven Phenotypic Dissection of AML Reveals Progenitor-like Cells that Correlate with Prognosis
Levine, J. H., Simonds, E. F., Bendall, S. C., Davis, K. L., Amir, E.-A. D., Tadmor, M. D., Litvin, O., Fienberg, H. G., Jager, A., Zunder, E. R., et al. (2015) · Cell · doi:10.1016/j.cell.2015.05.047 · Cited by 2,303 · cited 4×
“We used these metrics to assess the stability of clusters generated by three commonly used algorithms – FLOCK (7), SPADE (2) and PhenoGraph (1) - on four published data sets spanning different cell types (whole blood, bone marrow and lung tissue) and organisms (human and mouse) (4,10,11).”
BackgroundMethodResultInfluentialBendall, S. C., Simonds, E. F., Qiu, P., Amir, E.-A. D., Krutzik, P. O., Finck, R., Bruggner, R. V., Melamed, R., Trejo, A., Ornatsky, O. I., et al. (2011) · Science · doi:10.1126/science.1198704 · Cited by 2,279
Amir, E.-A. D., Davis, K. L., Tadmor, M. D., Simonds, E. F., Levine, J. H., Bendall, S. C., Shenfeld, D. K., Krishnaswamy, S., Nolan, G. P., & Pe'er, D. (2013) · Nature Biotechnology · doi:10.1038/nbt.2594 · Cited by 1,550
Extracting a cellular hierarchy from high-dimensional cytometry data with SPADE
Qiu, P., Simonds, E. F., Bendall, S. C., Gibbs, K. D., Bruggner, R. V., Linderman, M. D., Sachs, K., Nolan, G. P., & Plevritis, S. K. (2011) · Nature Biotechnology · doi:10.1038/nbt.1991 · Cited by 956 · cited 2×
“We used these metrics to assess the stability of clusters generated by three commonly used algorithms – FLOCK (7), SPADE (2) and PhenoGraph (1) - on four published data sets spanning different cell types (whole blood, bone marrow and lung tissue) and organisms (human and mouse) (4,10,11).”
MethodCluster-wise assessment of cluster stability
Hennig, C. (2007) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.11.025 · Cited by 930
Cited by
Distinct cellular mechanisms underlie anti-CTLA-4 and anti-PD-1 checkpoint blockade
Wei, S. C., Levine, J. H., Cogdill, A. P., Zhao, Y., Anang, N.-A. A. S., Andrews, M. C., Sharma, P., Wang, J., Wargo, J. A., Pe’er, D., et al. (2017) · Cell · doi:10.1016/j.cell.2017.07.024 · Cited by 1,302
Wei, S. C., Anang, N.-A. A. S., Sharma, R., Andrews, M. C., Reuben, A., Levine, J. H., Cogdill, A. P., Mancuso, J. J., Wargo, J. A., Pe’er, D., et al. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821218116 · Cited by 359
TYRO3 induces anti–PD-1/PD-L1 therapy resistance by limiting innate immunity and tumoral ferroptosis
Jiang, Z., Lim, S.-O., Yan, M., Hsu, J. L., Yao, J., Wei, Y., Chang, S.-S., Yamaguchi, H., Lee, H.-H., Ke, B., et al. (2021) · Journal of Clinical Investigation · doi:10.1172/jci139434 · Cited by 325
Bengsch, B., Ohtani, T., Khan, O., Setty, M., Manne, S., O’Brien, S., Gherardini, P. F., Herati, R. S., Huang, A. C., Chang, K.-M., et al. (2018) · Immunity · doi:10.1016/j.immuni.2018.04.026 · Cited by 318
Subrahmanyam, P. B., Dong, Z., Gusenleitner, D., Giobbie-Hurder, A., Severgnini, M., Zhou, J., Manos, M., Eastman, L. M., Maecker, H. T., & Hodi, F. S. (2018) · Journal for ImmunoTherapy of Cancer · doi:10.1186/s40425-018-0328-8 · Cited by 190
Metadata
- Authors
- Melchiotti, R., Gracio, F., Kordasti, S., Todd, A. K., & de Rinaldis, E.
- Venue
- Cytometry Part A
- Publisher
- Wiley
- Published
- 2016-10-18
- Volume
- 91
- Issue
- 1
- Pages
- 73-84
- ISSN
- 1552-4922 (ISSN-L)
- 1552-4930
- OpenAlex
- W2533463831
- Cited by
- 29
- References
- 5
- FWCI
- 1.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Cluster stability in the analysis of mass cytometry data. https://app.cochat.ai/papers/10.1002/cyto.a.23001
- The paper
- Melchiotti, R., Gracio, F., Kordasti, S., Todd, A. K., & de Rinaldis, E. (2017). Cluster stability in the analysis of mass cytometry data. Cytometry Part A. https://doi.org/10.1002/cyto.a.23001
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.