scPerturb: harmonized single-cell perturbation data

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The total number of cells per dataset is usually restricted by experimental limitations although it has increased over time.

“The total number of cells per dataset is usually restricted by experimental limitations although it has increased over time.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Three CITE-seq datasets have protein and RNA counts that are downloadable separately.

“Three CITE-seq datasets have protein and RNA counts that are downloadable separately6,9.”

• cites [6] Frangieh, C. J. et al. Multimodal pooled Perturb-CITE-seq screens in patient models define mechanisms of cancer immune evasion. Nat. Genet. 53, 332–341 (2021). sourceNot checked yet

  Not checked in the free run.

• cites [9] Papalexi, E. et al. Characterizing the molecular regulation of inhibitory immune checkpoints with multimodal single-cell screens. Nat. Genet. 53, 322–331 (2021). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

32 datasets report scRNA-seq exclusively; the resource also includes scATAC-seq from three publications, one with simultaneous protein measurements.

“While 32 datasets report scRNA-seq exclusively, we also include scATAC-seq from three publications, one with simultaneous protein measurements39.”

• cites [39] Mimitou, E. P. et al. Multiplexed detection of proteins, transcriptomes, clonotypes and CRISPR perturbations in single cells. Nat. Methods 16, 409–412 (2019). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Cells in 32 datasets underwent perturbation using CRISPR, and in 9 datasets the perturbation was done using drugs.

“Cells in 32 datasets underwent perturbation using CRISPR, and in 9 datasets the perturbation was done using drugs.”

May combine two claims.

The authors describe E-distance and E-test as tools for statistical comparisons of sets of cells and benchmark their robustness and applicability for distinguishing perturbations across datasets and modalities.

“We also describe E-distance and E-test as tools for statistical comparisons of sets of cells and benchmark their robustness and applicability for distinguishing perturbations across datasets and modalities.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Five drugs in the sci-Plex 3 dataset also appear in other drug perturbation datasets.

“Five drugs in this dataset also appear in other drug perturbation datasets (Supplementary Table 4).”

May combine two claims.

The most extensive drug dataset is sci-Plex 3, which includes 188 drugs tested across three cell lines; 107 of those perturbations had significant effects on cell states according to E-test analysis.

“The most extensive drug dataset is sci-Plex 3, which includes 188 drugs tested across three cell lines52; 107 of those perturbations had significant effects on cell states according to E-test analysis (Supplementary Table 3).”

• cites [52] Srivatsan, S. R. et al. Massively multiplex chemical transcriptomics at single-cell resolution. Science 367, 45–51 (2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Despite the increase in E-distance with falling cell numbers, the number of significant perturbations correctly decreases with fewer cells, and only some datasets have saturated significance at the full number of cells in that dataset.

“Despite the increase in E-distance with falling cell numbers, the number of significant perturbations correctly decreases with fewer cells, and only some datasets have saturated significance at the full number of cells in that dataset (Fig. 5b).”

May combine two claims.

The number of significant perturbations under the E-test saturates at around 500 UMIs per cell, with most perturbations that were significant at the full measured read depth maintaining that significance even with far fewer counts per cell.

“The number of significant perturbations under the E-test, however, saturates at around 500 UMIs per cell, with most perturbations that were significant at the full measured read depth maintaining that significance even with far fewer counts per cell (Fig. 5d).”

May combine two claims.

The perturbations in the group more dissimilar to the unperturbed condition have a low E-distance to each other, suggesting a similar phenotype induced by perturbation of these genes (IFNGR1, IFNGR2, JAK2, STAT1).

“The perturbations in the group more dissimilar to the unperturbed condition have a low E-distance to each other, suggesting a similar phenotype induced by perturbation of these genes (IFNGR1, IFNGR2, JAK2, STAT1).”

May combine two claims.

Cells in the top three perturbations with the largest E-distance to unperturbed cells are easily distinguishable from the gray unperturbed cells, while cells from the bottom three weakest perturbations are part of a single, uniform cloud visually indistinguishable from the unperturbed cells.

“Cells in the top three perturbations with the largest E-distance to unperturbed cells are easily distinguishable from the gray unperturbed cells, while cells from the bottom three weakest perturbations are part of a single, uniform cloud visually indistinguishable from the unperturbed cells.”

May combine two claims.

The three perturbations with highest E-distance are double perturbations while the three with the lowest E-distance are not.

“Indeed, the three perturbations with highest E-distance are double perturbations while the three with the lowest E-distance are not.”

May combine two claims.

Many datasets supplied only processed data, or raw data were available only after institutional clearance.

“For all data types, many datasets supplied only processed data, or raw data were available only after institutional clearance.”

May combine two claims.

This dataset contains >2.5 million cells, is the largest in the database, and has an average number of cells per gene detected significantly higher than in other datasets.

“Containing >2.5 million cells, this dataset is the largest in our database, with the average number of cells, which each gene is detected in, being significantly higher than in other datasets.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The most frequently perturbed gene, MYC, is perturbed in nine datasets from three publications.

“The most frequently perturbed gene, MYC, is perturbed in nine datasets from three publications.”

May combine two claims.

In the NormanWeissman2019 dataset, pairwise E-distances between perturbations reveal clusters of similar perturbations, including one of strongly acting CEBPA-targeting perturbations.

“E-distances between all pairs of perturbations in the dataset NormanWeissman2019. The color scale is clipped at 5% highest and lowest percentiles. Clusters of similar perturbations are visible, for example a cluster of strongly acting perturbations targeting CEBPA at the top.”

May combine two claims.

The authors processed the datasets with uniform quality-control pipelines and harmonized feature annotations.

“We apply uniform quality control pipelines and harmonize feature annotations.”

Classified as probably not a substantial claim. May combine two claims.

Targeted perturbations can be modeled as affecting individual nodes of a regulatory network, which allows investigation of mechanisms and inference of regulatory interactions and their direction.

“Experiments with targeted perturbations can be modeled as affecting individual nodes of a regulatory network model, enabling investigation of mechanistic processes and inference of regulatory interactions and their directionality11.”

• cites [11] Pratapa, A., Jalihal, A. P., Law, J. N., Bharadwaj, A. & Murali, T. M. Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nat. Methods 17, 147–154 (2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The scPerturb resource supports development and testing of computational methods and makes comparison and integration across datasets easier.

“The resulting information resource, scPerturb, enables development and testing of computational methods, and facilitates comparison and integration across datasets.”

May combine two claims.

E-statistics can be used to quantify how similar perturbations are and how effective they are.

“We illustrate the application of E-statistics for quantifying similarity and efficacy of perturbations.”

• AssumptionRests on an assumption: There is presently no convention for statistical comparison of response profiles in perturbation studies.

May combine two claims.

E-statistics can quantify perturbation effects and test their significance.

“We describe energy statistics (E-statistics) for quantification of perturbation effects and significance testing”

• AssumptionRests on an assumption: There is presently no convention for statistical comparison of response profiles in perturbation studies.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The work offers recommendations for experimental design, including optimal cell counts and read depth.

“This work provides an information resource for researchers working with single-cell perturbation data and recommendations for experimental design, including optimal cell counts and read depth.”

Probably not a substantial claim: may not pass S2 (checkable).

Statistical comparisons between perturbations and quantification of perturbation strength should ideally use a multivariate distance measure between sets of cells.

“Ideally, statistical comparisons between perturbations and quantification of perturbation strength should be based on a multivariate distance measure between sets of cells.”

May combine two claims.

Cell-based statistical measures can identify successfully perturbed cells but currently do not quantify similarity of perturbation effects.

“Cell-based statistical measures can be used to identify successfully perturbed cells but do not currently quantify similarity of perturbation effects2,9.”

• cites [2] Dixit, A., Parnas, O., Li, B. & Chen, J. Perturb-seq: dissecting molecular circuits with scalable single-cell RNA profiling of pooled genetic screens. Cell 167, 1853–1866 (2016). sourceNot checked yet

  Not checked in the free run.

• cites [9] Papalexi, E. et al. Characterizing the molecular regulation of inhibitory immune checkpoints with multimodal single-cell screens. Nat. Genet. 53, 322–331 (2021). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Several distance measures for scRNA-seq have been explored, including Wasserstein distance, maximum mean discrepancy, neighborhood-based measures, and energy distance.

“Multiple distance measures for scRNA-seq have been explored by the single-cell community in recent years, including Wasserstein distance computed in an optimal transport framework22,23, maximum mean discrepancy24, neighborhood-based measures17,18, and energy distance (E-distance)20.”

• cites [22] Chen, W. S. et al. Uncovering axes of variation among single-cell cancer specimens. Nat. Methods 17, 302–310 (2020). sourceNot checked yet

  Not checked in the free run.

• cites [23] Schiebinger, G. et al. Optimal-transport analysis of single-cell gene expression identifies developmental trajectories in reprogramming. Cell 176, 928–943 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [24] Lotfollahi, M., Naghipourfar, M., Theis, F. J. & Wolf, F. A. Conditional out-of-distribution generation for unpaired data using transfer VAE. Bioinformatics 36, i610–i617 (2020). sourceNot checked yet

  Not checked in the free run.

• cites [17] Burkhardt, D. B. et al. Quantifying the effect of experimental perturbations at single-cell resolution. Nat. Biotechnol. 39, 619–629 (2021). sourceNot checked yet

  Not checked in the free run.

• cites [18] Dann, E., Henderson, N. C., Teichmann, S. A., Morgan, M. D. & Marioni, J. C. Differential abundance testing on single-cell data using k-nearest neighbor graphs. Nat. Biotechnol. 40, 245–253 (2022). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Perturbation datasets have generally been too small to capture the complexity of cellular systems, so predictive models of regulatory interactions remain hard to infer.

“Typically, however, perturbation datasets have been too small to elucidate the complexity of cellular systems; thus, accurately predictive models of regulatory interactions remain difficult to infer12.”

• cites [12] Gross, T. & Blüthgen, N. Identifiability and experimental design in perturbation studies. Bioinformatics 36, i482–i489 (2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

A distance measure between sets of cells describes difference or similarity between expression profiles of cells treated with distinct perturbations, which can infer unique or shared mechanisms or identify perturbation targets.

“Such a distance measure describes the difference or similarity between the expression profiles of cells treated with distinct perturbations, thus inferring unique or shared mechanisms or identifying perturbation targets, which tend to produce similar shifts in molecular profiles20,21.”

• cites [20] Replogle, J. M. et al. Mapping information-rich genotype–phenotype landscapes with genome-scale Perturb-seq. Cell 185, 2559–2575 (2022). sourceNot checked yet

  Not checked in the free run.

• cites [21] Tian, R. et al. Genome-wide CRISPRi/a screens in human neurons link lysosomal failure to ferroptosis. Nat. Neurosci. 24, 1020–1034 (2021). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The energy test (E-test) is a statistically reliable tool for computational diagnostics of information content for a specific perturbation and can inform experimental design and data selection for training statistical models of perturbation effects.

“The associated energy test (E-test) is a statistically reliable tool for computational diagnostics of information content for a specific perturbation, and can inform design of experiments and data selection for training statistical models of perturbation effects.”

• AssumptionRests on an assumption: There is presently no convention for statistical comparison of response profiles in perturbation studies.

Probably not a substantial claim: may not pass S2 (checkable).

Analyzing increasingly large perturbation datasets reliably requires efficient statistical tools that can handle many cells and perturbations.

“Reliable analysis of increasingly large perturbation datasets requires efficient statistical tools to harness large numbers of cells and perturbations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Studies with mixtures of cell types have developed complex methods for quantifying similarity between heterogeneous cell populations.

“Studies with mixtures of cell types have developed complex methods for quantifying similarity between heterogeneous cell populations17-19.”

• cites [17] Burkhardt, D. B. et al. Quantifying the effect of experimental perturbations at single-cell resolution. Nat. Biotechnol. 39, 619–629 (2021). sourceNot checked yet

  Not checked in the free run.

• cites [18] Dann, E., Henderson, N. C., Teichmann, S. A., Morgan, M. D. & Marioni, J. C. Differential abundance testing on single-cell data using k-nearest neighbor graphs. Nat. Biotechnol. 40, 245–253 (2022). sourceNot checked yet

  Not checked in the free run.

• cites [19] Gehring, J., Park, J. H., Chen, S., Thomson, M. & Pachter, L. Highly multiplexed single-cell RNA-seq by DNA oligonucleotide tagging of cellular proteins. Nat. Biotechnol. 38, 35–38 (2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Some studies combine all cells in a perturbation setting into pseudo-bulk calculations, which loses information about cell-to-cell variation in response.

“Some studies carry out pseudo-bulk calculations by combining all cells in a given perturbation setting1,7. This means losing information about the cell-to-cell variation in response.”

• cites [1] Datlinger, P. et al. Pooled CRISPR screening with single-cell transcriptome readout. Nat. Methods 14, 297–301 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [7] Adamson, B. et al. A multiplexed single-cell CRISPR screening platform enables systematic dissection of the unfolded protein response. Cell 167, 1867–1882 (2016). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

E-distance is a fundamental statistical measure of distances between point clouds that can be used in a statistical test to identify strong or weak perturbations and to distinguish perturbations affecting distinct cellular sub-processes.

“Here, we exclusively use E-distance, a fundamental statistical measure of distances between point clouds that can be used in a statistical test to identify strong or weak perturbations as well as to distinguish between perturbations affecting distinct cellular sub-processes.”

• cites [20] Replogle, J. M. et al. Mapping information-rich genotype–phenotype landscapes with genome-scale Perturb-seq. Cell 185, 2559–2575 (2022). sourceNot checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: There is presently no convention for statistical comparison of response profiles in perturbation studies.

Probably not a substantial claim: may not pass S2 (checkable).

Moving from single-dataset to multi-dataset analysis requires developing quantitative approaches to perturbation biology.

“Moving from single-dataset to multi-dataset analysis requires development of quantitative approaches to perturbation biology.”

Classified as probably not a substantial claim.

The quantification of perturbation strength and comparison of experiment-specific variables may serve as a reference for optimal design of future single-cell experiments.

“Our quantification of perturbation strength and comparison of experiment-specific variables, such as the number of perturbations and the number of cells per perturbation, may serve as a reference for optimal design of future single-cell experiments.”

May combine two claims.

Increasing sequencing depth increases UMI counts measured even for low-expression genes, reducing uncertainty associated with zero counts.

“Increasing sequencing depth increases the UMI counts measured even for low-expression genes, reducing the uncertainty associated with zero counts34,47.”

• cites [34] Svensson, V., da Veiga Beltrame, E. & Pachter, L. A curated database reveals trends in single-cell transcriptomics. Database 2020, baaa073 (2020). sourceNot checked yet

  Not checked in the free run.

• cites [47] Haque, A., Engel, J., Teichmann, S. A. & Lönnberg, T. A practical guide to single-cell RNA-sequencing for biomedical research and clinical applications. Genome Med. 9, 75 (2017). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Unified frameworks for accessing single-cell data are in active development but do not currently support perturbation datasets or standardize perturbation annotations.

“Unified frameworks for accessing single-cell data are in active development, but do not currently support perturbation datasets or standardize perturbation annotations37,38.”

• cites [37] Fischer, D. S. et al. Sfaira accelerates data and model reuse in single cell genomics. Genome Biol. 22, 248 (2021). sourceNot checked yet

  Not checked in the free run.

• cites [38] Chan Zuckerberg CELLxGENE Discover. Cellxgene Data Portal. https://cellxgene.cziscience.com/Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

E-distance is a statistical measure of the distance between two distributions that provides intuition about the signal-to-noise ratio in a dataset.

“To compare and evaluate the effect and strength of perturbations in each dataset we used the E-distance, a statistical measure of the distance between two distributions, which provides intuition about the signal-to-noise ratio in a dataset48.”

• cites [48] Székely, G. J. & Rizzo, M. L. Energy statistics: a class of statistics based on distances. J. Stat. Plan. Inference 143, 1249–1272 (2013). sourceNot checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: There is presently no convention for statistical comparison of response profiles in perturbation studies.

May combine two claims.

Existing collections of datasets are mainly for filtering and do not provide a unified format for perturbations.

“Existing collections of datasets are primarily a means for filtering and do not supply a unified format for perturbations33-35.”

• cites [33] Lance, C. et al. Multimodal single cell data integration challenge: results and lessons learned. Preprint at https://doi.org/10.1101/2022.04.11.487796 (2022). sourceNot checked yet

  Not checked in the free run.

• cites [34] Svensson, V., da Veiga Beltrame, E. & Pachter, L. A curated database reveals trends in single-cell transcriptomics. Database 2020, baaa073 (2020). sourceNot checked yet

  Not checked in the free run.

• cites [35] Broad Institute. Single Cell Portal. https://singlecell.broadinstitute.org/single_cell (2022).Not checked yet

  Not checked in the free run.

May combine two claims.

Although several large databases of perturbations with bulk readouts exist, single-cell perturbation technologies are newer and their data are not unified.

“Although several large databases of perturbations with bulk readouts exist, single-cell perturbation technologies are newer and the data are not unified31,32.”

• cites [31] Stathias, V. et al. LINCS Data Portal 2.0: next generation access point for perturbation–response signatures. Nucleic Acids Res. 48, D431–D439 (2020). sourceNot checked yet

  Not checked in the free run.

• cites [32] Tsherniak, A. et al. Defining a cancer dependency map. Cell 170, 564–576 (2017). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The exact value of E-distance depends on dataset-specific parameters such as sequencing depth and how cells are distributed; the E-test accounts for these differences by creating a null distribution using permutations of the data.

“The exact value of the E-distance depends on dataset-specific parameters such as sequencing depth (Supplemental Note Section 4) and how exactly the cells are distributed; the E-test accounts for these differences by creating a null distribution using permutations of the data.”

Classified as probably not a substantial claim.

As a baseline for significant perturbations as defined by the E-test, the authors suggest at least 200-500 cells per perturbation and an average of 1,000 UMIs per cell as an experimental guideline for distinguishable perturbations.

“As a baseline for significant perturbations, as defined by the E-test, we suggest at least 200–500 cells per perturbation (Fig. 5b) and an average of 1,000 UMIs per cell (Fig. 5d) as an experimental guideline for distinguishable perturbations.”

May combine two claims.

A low E-distance indicates that a perturbation did not induce a large shift in expression profiles, reflecting technical problems, weak perturbation effect, or resistance to perturbation.

“A low E-distance indicates that a perturbation did not induce a large shift in expression profiles, reflecting technical problems in the experiment, weak effect of the perturbation, or resistance to perturbation.”

Classified as probably not a substantial claim.

For two groups of cells, E-distance relates the distribution of distances between groups (signal) to the distribution of distances within each group (noise).

“For two groups of cells, it relates the distribution of distances of cells between the groups (‘signal’), to the distribution of distances within each group ('noise').”

May combine two claims.

Any disruption of a gene in this cascade leads to a similar outcome and therefore a similar transcriptome profile.

“In agreement with this, we observe that any disruption of a gene in this cascade leads to a similar outcome and, therefore, a similar transcriptome profile.”

May combine two claims.

The saturation point depends on perturbation strength and dataset heterogeneity: if all cells are similar, a small set of cells will sufficiently describe every possible response to a perturbation.

“This saturation point depends on perturbation strength and on dataset heterogeneity: if all cells are similar, a small set of cells will sufficiently describe every possible response to a perturbation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The optimal lower bounds on UMI and cell counts for a given experiment depend on downstream specific modeling tasks.

“The optimal lower bounds on UMI and cell counts for a given experiment depend on downstream specific modeling tasks, as discussed in more detail elsewhere12.”

• cites [12] Gross, T. & Blüthgen, N. Identifiability and experimental design in perturbation studies. Bioinformatics 36, i482–i489 (2020). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Another drug dataset applies combinations of three drug perturbations at varying concentrations across samples.

“Another drug dataset applies combinations of three drug perturbations at varying concentrations across samples19.”

• cites [19] Gehring, J., Park, J. H., Chen, S., Thomson, M. & Pachter, L. Highly multiplexed single-cell RNA-seq by DNA oligonucleotide tagging of cellular proteins. Nat. Biotechnol. 38, 35–38 (2020). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Starting the analysis from sequencing reads may have improved the interoperability of datasets in this resource, but guide assignment procedures and demultiplexing algorithms are experimental set-up specific.

“Starting the analysis from sequencing reads may have improved the interoperability of datasets in this resource, but guide assignment procedures and demultiplexing algorithms are experimental set-up specific.”

Classified as probably not a substantial claim.

Different experimental designs result in various covariates that may need to be accounted for in statistical testing.

“However, different experimental designs result in various covariates that may need to be accounted for in statistical testing.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The sharing of spatial data is hampered by large file sizes and non-standardized analysis pipelines.

“The sharing of spatial data is hampered by large file sizes and non-standardized analysis pipelines58.”

• cites [58] Bredikhin, D., Kats, I. & Stegle, O. MUON: multimodal omics analysis framework. Genome Biol. 23, 42 (2022). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Adding more datasets to this resource, or creating similar resources in the future, would be easier if there were standard formats for sharing perturbation data and single-cell annotations.

“Adding more datasets to this resource, or the creation of similar resources in the future, would be easier if there were standard formats for sharing perturbation data, and, more generally, standard formats for sharing single-cell annotations.”

• AssumptionRests on an assumption: Lack of standardization in data sharing and processing across different research groups and studies hampered the creation of this resource.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Covariates arising from different experimental designs can affect the E-distance, and the authors recommend balancing cell numbers across experimental conditions if in doubt.

“In general, covariates arising from different experimental designs can affect the E-distance, and we recommend balancing cell numbers across experimental conditions if in doubt (Supplemental Note Section 8).”

May combine two claims.

Although many processed datasets were available on GEO, there is no standard format for sharing CRISPR barcode assignments and other metadata.

“Although many processed datasets were available on the National Center for Biotechnology Information (NCBI) Gene Expression Omnibus (GEO)55, there is no standard format for sharing CRISPR barcode assignments and other metadata.”

• cites [55] Barrett, T. et al. NCBI GEO: archive for functional genomics data sets – update. Nucleic Acids Res. 41, D991–D995 (2013). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

For scATAC feature assignments specific to CRISPR perturbations, known loci of action could be used to improve feature calls.

“In particular, for scATAC feature assignments specific to CRISPR perturbations, known loci of action could be used to improve feature calls40.”

• cites [40] Chen, H. et al. Assessment of computational methods for the analysis of single-cell ATAC-seq data. Genome Biol. 20, 241 (2019). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The uniform annotations in this resource enable data integration and benchmarking as well as exploration of the effects of shared perturbations across datasets.

“The uniform annotations in this resource enable data integration and benchmarking as well as exploration of the effects of shared perturbations across datasets.”

• AssumptionRests on an assumption: Unified datasets are key to developing generalizable machine learning methods and establishing multimodal data integration.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Such a standardization effort would enable simpler addition of datasets to the resource.

“Such a standardization effort would enable simpler addition of datasets to the resource.”

• AssumptionRests on an assumption: Lack of standardization in data sharing and processing across different research groups and studies hampered the creation of this resource.

Probably not a substantial claim: may not pass S3 (load-bearing).

There is increasing interest in spatial techniques, including single-cell resolution spatial transcriptomics, epigenomics and proteomics.

“Moreover, there is increasing interest in spatial techniques, including single-cell resolution spatial transcriptomics, epigenomics and proteomics57.”

• cites [57] Tian, L., Chen, F. & Macosko, E. Z. The expanding vistas of spatial transcriptomics. Nat. Biotechnol. 41, 773–782 (2023). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The unified annotations and perturbation significance testing should prove especially useful to the machine learning community for training computational models on this data.

“The unified annotations and perturbation significance testing should prove especially useful to the machine learning community for training computational models on this data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Within each dataset, p-values from the E-test were adjusted for multiple comparisons with the Holm-Sidak procedure.

“We corrected for multiple testing using the Holm–Sidak method per dataset.”

Probably not a substantial claim: may not pass S4 (based).

It is difficult to ascertain to what extent low E-distances between perturbed and unperturbed cells are caused by technical noise or simply weak perturbation effects compared with biological variation in groups of cells before and after perturbation.

“Unfortunately, it is difficult to ascertain to what extent low E-distances between perturbed and unperturbed cells are caused by technical noise or simply weak perturbation effects compared with biological variation in groups of cells before and after perturbation.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For the E-distance analysis, cells with at least 1,000 UMIs and genes detected in at least 50 cells were kept, 2,000 highly variable genes were chosen, and counts were normalized and log-transformed without z-scaling.

“Cells were retained if they had a minimum of 1,000 UMIs, and genes, with a minimum of 50 cells. A total of 2,000 highly variable genes were selected using scanpy.pp.find_variable_genes with flavor seurat_v3. We normalized the count matrix using scanpy.pp.normalize_total and log-transformed the data using scanpy.pp.log1p; we did not z-scale the data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

For the Spear-ATAC and CRISPR_sciATAC datasets, a barcode was assigned the most abundant sgRNA only when that guide's count was above 20 and it made up at least 80% of all guide counts.

“We assigned the sgRNA with the highest counts to a cell barcode if the sgRNA count exceeded 20 and if that sgRNA comprised at least 80% of all sgRNA counts.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Experiment design choices, such as the optimal number of cells per perturbation and the sequencing depth for each cell, depend on the questions that a particular perturbation study is intended to answer, and on the strength and uniqueness of the gene expression changes caused by the perturbations.

“Experiment design choices, such as the optimal number of cells per perturbation and the sequencing depth for each cell, depend on the questions that a particular perturbation study is intended to answer, and on the strength and uniqueness of the gene expression changes caused by the perturbations.”

Classified as probably not a substantial claim. May combine two claims.

For perturbation distinguishability as defined by the E-test, regardless of experiment parameters, the authors recommend measuring at least 200-500 cells per perturbation and an average of 1,000 UMIs per cell.

“For perturbation distinguishability as defined by the E-test, regardless of experiment parameters, we recommend measuring at least 200–500 cells per perturbation and an average of 1,000 UMIs per cell.”

Classified as probably not a substantial claim. May combine two claims.

Cells with under 1,000 or over 100,000 mapped fragments were removed, and a minimum transcription start site enrichment score of 4 was required, to keep quality consistent and signal-to-noise sufficient across datasets.

“To ensure a consistent and homogenous quality throughout the different datasets, we filtered out cells with fewer than 1,000 and more than 100,000 mapped fragments. Furthermore, we required a minimum transcription start site enrichment score of 4 to ensure a sufficient signal-to-noise ratio.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The scATAC-seq features used here were originally developed for non-perturbed cells, for purposes such as identifying cell types, discovering cell-type-specific regulatory elements, and reconstructing differentiation trajectories.

“Note that these features were originally developed for scATAC-seq data on non-perturbed cells, with goals such as the identification of cell type, discovery of cell type-specific regulatory elements, or reconstruction of cellular differentiation trajectories43,61.”

• cites [43] Buenrostro, J. D. et al. Single-cell chromatin accessibility reveals principles of regulatory variation. Nature 523, 486–490 (2015). sourceNot checked yet

  Not checked in the free run.

• cites [61] Satpathy, A. T. et al. Massively parallel single-cell chromatin landscapes of human immune cell development and intratumoral T cell exhaustion. Nat. Biotechnol. 37, 925–936 (2019). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

To avoid bias from unequal cell numbers per perturbation, each dataset was subsampled so that all perturbations had equal cell counts, after perturbations with fewer than 50 cells were removed.

“To avoid problems due to different numbers of cells per perturbation, we subsampled each dataset such that all perturbations had the same number of cells. We removed all perturbations with fewer than 50 cells and then subsampled to the number of cells in the smallest perturbation left after filtering.”