CoChat

Predicting cellular responses to complex perturbations in high‐throughput screens

Open access
Lotfollahi, Mohammad, Klimovskaia Susmelj, Anna, De Donno, Carlo, Hetzel, Leon, Ji, Yuge, Ibarra, Ignacio L, Srivatsan, Sanjay R, Naghipourfar, Mohsen, Daza, Riza M, Martin, Beth, Shendure, Jay, McFaline‐Figueroa, Jose L, Boyeau, Pierre, Wolf, F Alexander, Yakubova, Nafissa, Günnemann, Stephan, Trapnell, Cole, Lopez‐Paz, David, Theis, Fabian J · 2023 · Molecular Systems Biology

The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index; 4 of the 18 sources its claims cite were checked for support.

Abstract

Recent advances in multiplexed single‐cell transcriptomics experiments facilitate the high‐throughput study of drug and genetic perturbations. However, an exhaustive exploration of the combinatorial perturbation space is experimentally unfeasible. Therefore, computational methods are needed to predict, interpret, and prioritize perturbations. Here, we present the compositional perturbation autoencoder (CPA), which combines the interpretability of linear models with the flexibility of deep‐learning approaches for single‐cell response modeling. CPA learns to in silico predict transcriptional perturbation response at the single‐cell level for unseen dosages, cell types, time points, and species. Using newly generated single‐cell drug combination data, we validate that CPA can predict unseen drug combinations while outperforming baseline models. Additionally, the architecture's modularity enables incorporating the chemical representation of the drugs, allowing the prediction of cellular response to completely unseen drugs. Furthermore, CPA is also applicable to genetic combinatorial screens. We demonstrate this by imputing in silico 5,329 missing combinations (97.6% of all possibilities) in a single‐cell Perturb‐seq experiment with diverse genetic interactions. We envision CPA will facilitate efficient experimental design and hypothesis generation by enabling in silico response prediction at the single‐cell level and thus accelerate therapeutic applications using single‐cell technologies.

Overview

CoChat Summary

TL;DR

The paper introduces the compositional perturbation autoencoder (CPA), a deep-learning model that learns disentangled, additive latent embeddings of perturbations and covariates to predict single-cell transcriptional responses to unseen drug, dose, time, species, and genetic combinations. CPA is validated on multiple public datasets and a newly generated drug-combination dataset, and extended by chemCPA to predict responses to completely unseen drugs using chemical descriptors.

Key findings

  • CPA predicted unseen drug combinations in a new A549 combination dataset (combosciplex, n = 63,378 cells, 32 perturbation/combination conditions), including a held-out Panobinostat + Alvespimycin combination with an R2 of 0.81; it outperformed a random baseline and a linear model.
  • On sci-Plex3 (n = 290,889 cells; 188 compounds; A549, K562, MCF7; four doses), CPA outperformed scGen by 1.54% on mean prediction and 35.85% on variance prediction, and learned drug embeddings that clustered compounds by mechanism, including epigenetic, tyrosine kinase signaling, and protein formation/degradation pathways.
  • In a Perturb-seq genetic screen (108,497 cells; 284 conditions; 131 unique double-gene combinations), CPA imputed 5,329 missing combinations (97.6% of all possibilities); however, a simple baseline performed on par with CPA and linear models for mean whole-transcriptome prediction, suggesting limited nonlinear effects at that level.
  • chemCPA, which incorporates chemical descriptors, improved prediction for unseen drugs and combinations; when single-drug Panobinostat observations were held out, median R2 on DEGs dropped from 0.85 to 0.38, but the single-drug Panobinostat effect could still be predicted with high accuracy.

Methods

  • CPA is an autoencoder that encodes gene expression into a basal latent state, adversarially removes perturbation and covariate information from that state, then additively composes learned perturbation and covariate embeddings with nonlinear dose/time scalers before decoding to gene expression.
  • The model was trained and evaluated on six public datasets plus a newly generated A549 drug-combination dataset; datasets included Kang et al. PBMCs, sci-Plex2, sci-Plex3, Hagai et al. cross-species LPS phagocytes, Norman et al. Perturb-seq, and L1000 bulk data.
  • Evaluation used train/test/out-of-distribution splits, R2 on means and variances, Wasserstein distance, clustering metrics, and a distance-based uncertainty score; benchmarks included random baseline, linear baseline, and scGen.

Limitations

  • The paper states that CPA will fail if test data are very different from training data, that predictions cannot replace experimental validation, and that the uncertainty score is a condition-level heuristic rather than a cell-specific or Bayesian uncertainty estimate.
  • The paper notes that CPA is deterministic, uses gene expression alone, cannot directly capture post-transcriptional, signaling, or cell-communication effects, and may require systematic hyperparameter sweeping for best performance.
  • The text makes evident that CPA cannot predict completely unseen drugs without an extension such as chemCPA, and that some held-out combinations, such as Alvespimycin + Panobinostat, showed distribution shifts that neither CPA nor chemCPA fully identified (R2 about 0.3 on DEGs).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.15252/msb.202211517

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Interpretable single‐cell perturbation modeling using a compositional perturbation autoencoder (CPA) Given a matrix of gene expression per cell together with annotated potentially quantitative perturbations d and other covariates such as cell line, patient, or species, CPA learns the combined perturbation response for a single‐cell. It encodes gene expression using a neural network into a lower dimensional latent space that is eventually decoded back to an approximate gene expression matrix, as close as possible to the original one. To make the latent space interpretable in terms of perturbation and covariates, the encoded gene expression vector is first mapped to a “basal state” by feeding the signal to discriminators to remove any signal from perturbations and covariates. The basal state is then composed with perturbations and covariates, with potentially reweighted dosages, to reconstruct the gene expression. All encoder, decoder, and discriminator weights as well as the perturbation and covariate dictionaries are learned during training.Features of CPA are interpreted via plotting of the two learned dictionaries, interpolating covariate‐specific dose–response curves and predicting novel unseen drug combinations.
Figure 1 Interpretable single‐cell perturbation modeling using a compositional perturbation autoencoder (CPA) Given a matrix of gene expression per cell together with annotated potentially quantitative perturbations d and other covariates such as cell line, patient, or species, CPA learns the combined perturbation response for a single‐cell. It encodes gene expression using a neural network into a lower dimensional latent space that is eventually decoded back to an approximate gene expression matrix, as close as possible to the original one. To make the latent space interpretable in terms of perturbation and covariates, the encoded gene expression vector is first mapped to a “basal state” by feeding the signal to discriminators to remove any signal from perturbations and covariates. The basal state is then composed with perturbations and covariates, with potentially reweighted dosages, to reconstruct the gene expression. All encoder, decoder, and discriminator weights as well as the perturbation and covariate dictionaries are learned during training.Features of CPA are interpreted via plotting of the two learned dictionaries, interpolating covariate‐specific dose–response curves and predicting novel unseen drug combinations.CC BY · PubMed Central
Figure 2 Learning drug and cell line latent representations from massive single‐cell screens of 188 drugs across cancer cell lines UMAP representation of sci‐Plex samples (n = 290,889) of A549, K562, and MCF7 cell‐lines colored by pathway targeted by the compounds to which cells were exposed.Distribution of top 5 differentially expressed genes in A549 cells after treatment with Momelotinib, a JAK inhibitor, at the highest dose for real, control and CPA predicted cells.Mean gene expression of 5,000 genes and top 50 DEGs between CPA predicted and real cells together with the top five DEGs highlighted in red for four compounds for which the model did not see any examples of the highest dose.Box plots of R2 scores for predicted and real cells for 36 compounds and 108 unique held out conditions across different cell lines. Baseline indicates comparison of each condition with a mean derived from randomly sampled cells. R2 scores box plot for all and top 50 DEGs. Each column represents a scenario where cells exposed with specific dose for all compounds on a cell line or combinations of cell lines were held out from training and later predicted.Latent representation as learned by CPA of 82 cell lines from the L1000 dataset, with some cancer cell lines colored by tissue of origin.Two‐dimensional representation of latent drug embeddings as learned by the CPA. Compounds associated with epigenetic regulation, tyrosine kinase signaling, and protein folding/degradation pathways are colored by their respectively known pathways. The smaller upper right panel shows latent covariate embedding for three cell lines in the data, indicating no specific similarity preference.Latent drug embedding of CPA model trained on the bulk‐RNA cell line profiles of the top 1,000 most tested compounds from the L1000 dataset. Compounds overlapping with the sci‐Plex experiment in (A) are colored according to the same pathway labels as in (G). Data information: Box plots in (E and F) indicate the median (center lines) and interquartile range (hinges), and whiskers represent minimum and maximum values.
Figure 2 Learning drug and cell line latent representations from massive single‐cell screens of 188 drugs across cancer cell lines UMAP representation of sci‐Plex samples (n = 290,889) of A549, K562, and MCF7 cell‐lines colored by pathway targeted by the compounds to which cells were exposed.Distribution of top 5 differentially expressed genes in A549 cells after treatment with Momelotinib, a JAK inhibitor, at the highest dose for real, control and CPA predicted cells.Mean gene expression of 5,000 genes and top 50 DEGs between CPA predicted and real cells together with the top five DEGs highlighted in red for four compounds for which the model did not see any examples of the highest dose.Box plots of R2 scores for predicted and real cells for 36 compounds and 108 unique held out conditions across different cell lines. Baseline indicates comparison of each condition with a mean derived from randomly sampled cells. R2 scores box plot for all and top 50 DEGs. Each column represents a scenario where cells exposed with specific dose for all compounds on a cell line or combinations of cell lines were held out from training and later predicted.Latent representation as learned by CPA of 82 cell lines from the L1000 dataset, with some cancer cell lines colored by tissue of origin.Two‐dimensional representation of latent drug embeddings as learned by the CPA. Compounds associated with epigenetic regulation, tyrosine kinase signaling, and protein folding/degradation pathways are colored by their respectively known pathways. The smaller upper right panel shows latent covariate embedding for three cell lines in the data, indicating no specific similarity preference.Latent drug embedding of CPA model trained on the bulk‐RNA cell line profiles of the top 1,000 most tested compounds from the L1000 dataset. Compounds overlapping with the sci‐Plex experiment in (A) are colored according to the same pathway labels as in (G). Data information: Box plots in (E and F) indicate the median (center lines) and interquartile range (hinges), and whiskers represent minimum and maximum values.CC BY · PubMed Central
Figure 3 Validation of predictions in new large‐scale drug combination dataset UMAP representation of the combosciplex dataset comprised of n = 63,378 cells and 32 perturbation and perturbation combinations in A549 cells. The left UMAP highlights the split used for the following results. The right UMAP shows the five out‐of‐distribution conditions selected and their expression pattern amongst the clusters.The five out‐of‐distribution conditions shown in (A) and model performance per condition. The circles to the left of the condition names indicate a qualitative difficulty assessment of the prediction. Conditions with the green label are dominated by one of the two compounds in the condition and should be relatively easy for the model to predict, while combinations containing Alvespimycin are more transcriptionally dissimilar from conditions seen in training.Benchmark of CPA vs. a linear model vs. the aforementioned random baseline, as measured by R2 on both highly variable genes and the top 50 differentially expressed genes. Box plots indicate the median (center lines) and interquartile range (hinges), and whiskers represents minimum and maximum values.Predicted vs. true, post‐treatment expression values, with the top 5 DEGs colored in red.Violin plots of the top two DEGs per out‐of‐distribution condition and the pre‐stimulation, post‐stimulation, and CPA predicted expression values.UMAP representation of the combination latent vectors learned by CPA. Four individual conditions and the combinations they appear in are highlighted.
Figure 3 Validation of predictions in new large‐scale drug combination dataset UMAP representation of the combosciplex dataset comprised of n = 63,378 cells and 32 perturbation and perturbation combinations in A549 cells. The left UMAP highlights the split used for the following results. The right UMAP shows the five out‐of‐distribution conditions selected and their expression pattern amongst the clusters.The five out‐of‐distribution conditions shown in (A) and model performance per condition. The circles to the left of the condition names indicate a qualitative difficulty assessment of the prediction. Conditions with the green label are dominated by one of the two compounds in the condition and should be relatively easy for the model to predict, while combinations containing Alvespimycin are more transcriptionally dissimilar from conditions seen in training.Benchmark of CPA vs. a linear model vs. the aforementioned random baseline, as measured by R2 on both highly variable genes and the top 50 differentially expressed genes. Box plots indicate the median (center lines) and interquartile range (hinges), and whiskers represents minimum and maximum values.Predicted vs. true, post‐treatment expression values, with the top 5 DEGs colored in red.Violin plots of the top two DEGs per out‐of‐distribution condition and the pre‐stimulation, post‐stimulation, and CPA predicted expression values.UMAP representation of the combination latent vectors learned by CPA. Four individual conditions and the combinations they appear in are highlighted.CC BY · PubMed Central
Figure 4 CPA extensibility enables predicting the response to unseen drugs Proposed architecture change for CPA to include chemical prior knowledge. The molecule encoder G maps the chemical information of a compound to a latent drug embedding hdi. This module can be based on a pre‐trained graph encoder or molecular fingerprints like here. During training only the perturbation module M and the amortized dosage scaler S are optimized.Comparison between CPA and chemCPA, including a baseline that ignores all drug‐induced perturbation effects. Scores are computed on the sci‐Plex3 (n = 290,889) data on a test set that consists of nine compounds. Both the whole genes set (left) and the DEGs (right) are shown.Performance comparison between the models from (B) for two histone deactylation drugs, Givinostat and Dacinostat, across all doses and cell‐lines (shades).Median scores for the models from (B) for the highest dose value of 10 μM, including the result of chemCPA for the setting in which the nine test compounds are completely unseen and excluded from the training and validation.Comparison between CPA and chemCPA on the new combosciplex (n = 63,378) including the same type of baseline as in (B). The test set consists of all nine drug combinations that include Panobinostat.Detailed performance comparison between the models from (E) for the two conditions with the highest R2 difference on the DEGs.Median scores for the models from (E), including the result of chemCPA where also the single‐drug Panobinostat observations were held‐out. The median score on the DEGs reduces from 0.85 to 0.38.Showing how well chemCPA is able to predict the single‐drug effect of Panobinostat when it is held out. This is compared to the achieved score when Panobinostat is included in the training set (E). Data information: Box plots indicate the median (center lines) and interquartile range (hinges), and whiskers represent minimum and maximum values.
Figure 4 CPA extensibility enables predicting the response to unseen drugs Proposed architecture change for CPA to include chemical prior knowledge. The molecule encoder G maps the chemical information of a compound to a latent drug embedding hdi. This module can be based on a pre‐trained graph encoder or molecular fingerprints like here. During training only the perturbation module M and the amortized dosage scaler S are optimized.Comparison between CPA and chemCPA, including a baseline that ignores all drug‐induced perturbation effects. Scores are computed on the sci‐Plex3 (n = 290,889) data on a test set that consists of nine compounds. Both the whole genes set (left) and the DEGs (right) are shown.Performance comparison between the models from (B) for two histone deactylation drugs, Givinostat and Dacinostat, across all doses and cell‐lines (shades).Median scores for the models from (B) for the highest dose value of 10 μM, including the result of chemCPA for the setting in which the nine test compounds are completely unseen and excluded from the training and validation.Comparison between CPA and chemCPA on the new combosciplex (n = 63,378) including the same type of baseline as in (B). The test set consists of all nine drug combinations that include Panobinostat.Detailed performance comparison between the models from (E) for the two conditions with the highest R2 difference on the DEGs.Median scores for the models from (E), including the result of chemCPA where also the single‐drug Panobinostat observations were held‐out. The median score on the DEGs reduces from 0.85 to 0.38.Showing how well chemCPA is able to predict the single‐drug effect of Panobinostat when it is held out. This is compared to the achieved score when Panobinostat is included in the training set (E). Data information: Box plots indicate the median (center lines) and interquartile range (hinges), and whiskers represent minimum and maximum values.CC BY · PubMed Central
Figure 5 Predicting combinatorial genetic perturbations UMAP inferred latent space using CPA for 281 single and double‐gene perturbations obtained from Perturb‐seq (n = 108,497). Each dot represents a genetic perturbation. Coloring indicates known gene programs associated with perturbed genes from the original publications.Measured and CPA‐predicted gene expression for cells linked to a synergistic gene pair (CBL + CNN1). Gene names taken from the original publication.As (b) for an epistatic (DUSP9 + ETS) gene pair. Top 10 DEGs of DUSP9 + ETS co‐perturbed cells versus control cells are shown.R2 values of mean gene‐expression of measured and predicted cells for all genes (black) or top 100 DEGs for the prediction of all 131 combinations for CPA, linear model, and the baseline (13 trained models, with ≈10 tested combinations each time).R2 values of predicted and real mean gene‐expression versus number of cells in the real data.R2 values for predicted and real cells versus number of combinations seen during training.UMAP of mean gene expression in each measured (n = 284, red dots) and CPA‐predicted (n = 5,329, gray dots) perturbation combinations.As (g), showing measurement uncertainty (cosine similarity).As (g), showing dominant genes in Leiden clusters (25 or more observations). KLF1 cluster is highlighted with red dotted circle.UMAP of mean expression of cells with KLF1 as a common co‐perturbed gene.Hierarchical clustering of linear regression associated metrics between KLF1 with co‐perturbed genes, in measured and predicted cells. Each row indicates summary parameters obtained when fitting a linear regression for double predictions using single perturbations as predicting variables, and relationships between coefficients c1 and c2. For definitions see Materials and Methods.Scaled gene expression changes (versus control) of RF‐selected genes (x‐axis) in measured (purple) and predicted (yellow) perturbations (y‐axis). Headers indicate gene‐wise regression coefficients, and interaction mode suggested by (Norman et al, 2019). Data information: Box plots indicate the median (center lines) and interquartile range (hinges), and whiskers represent minimum and maximum values.
Figure 5 Predicting combinatorial genetic perturbations UMAP inferred latent space using CPA for 281 single and double‐gene perturbations obtained from Perturb‐seq (n = 108,497). Each dot represents a genetic perturbation. Coloring indicates known gene programs associated with perturbed genes from the original publications.Measured and CPA‐predicted gene expression for cells linked to a synergistic gene pair (CBL + CNN1). Gene names taken from the original publication.As (b) for an epistatic (DUSP9 + ETS) gene pair. Top 10 DEGs of DUSP9 + ETS co‐perturbed cells versus control cells are shown.R2 values of mean gene‐expression of measured and predicted cells for all genes (black) or top 100 DEGs for the prediction of all 131 combinations for CPA, linear model, and the baseline (13 trained models, with ≈10 tested combinations each time).R2 values of predicted and real mean gene‐expression versus number of cells in the real data.R2 values for predicted and real cells versus number of combinations seen during training.UMAP of mean gene expression in each measured (n = 284, red dots) and CPA‐predicted (n = 5,329, gray dots) perturbation combinations.As (g), showing measurement uncertainty (cosine similarity).As (g), showing dominant genes in Leiden clusters (25 or more observations). KLF1 cluster is highlighted with red dotted circle.UMAP of mean expression of cells with KLF1 as a common co‐perturbed gene.Hierarchical clustering of linear regression associated metrics between KLF1 with co‐perturbed genes, in measured and predicted cells. Each row indicates summary parameters obtained when fitting a linear regression for double predictions using single perturbations as predicting variables, and relationships between coefficients c1 and c2. For definitions see Materials and Methods.Scaled gene expression changes (versus control) of RF‐selected genes (x‐axis) in measured (purple) and predicted (yellow) perturbations (y‐axis). Headers indicate gene‐wise regression coefficients, and interaction mode suggested by (Norman et al, 2019). Data information: Box plots indicate the median (center lines) and interquartile range (hinges), and whiskers represent minimum and maximum values.CC BY · PubMed Central

Verification

Claims the paper makes

4 Valid · 0 Not valid · 11 Not checked · 91 Own results and interpretations

53 more claims

Create a free account to see all 106 claims and to verify the 12 references still open.

See all claims — free account
  • It may state two separate claims as one. 37
  • Probably not a substantial claim. 13

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Research works printed but not linked: 6

  1. Gal Y, Ghahramani Z (2016) Dropout as a bayesian approximation: representing model uncertainty in deep learning. In International Conference on Machine Learning, pp 1050–1059. PMLR
  2. Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning. Cambridge: MIT Press
  3. Hetzel L, Böhm S, Kilbertus N, Günnemann S, Lotfollahi M, Theis FJ (2022) Predicting cellular responses to novel drug perturbations at a single‐cell resolution. In NeurIPS 2022
  4. Landrum G (2006) RDKit: Open source cheminformatics https://www.rdkit.org
  5. Sohn K, Lee H, Yan X (2015) Learning structured output representation using deep conditional generative models. In Advances in Neural Information Processing Systems 28, Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds), pp 3483–3491. New York, NY: Curran Associates, Inc
  6. Zhao J, Kim Y, Zhang K, Rush A, LeCun Y (2018) Adversarially regularized autoencoders. In International Conference on Machine Learning, pp 5902–5911. PMLR

Cited by

See all 307

Metadata

Authors
Lotfollahi, M., Klimovskaia Susmelj, A., De Donno, C., Hetzel, L., Ji, Y., Ibarra, I. L., Srivatsan, S. R., Naghipourfar, M., Daza, R. M., Martin, B., Shendure, J., McFaline‐Figueroa, J. L., Boyeau, P., Wolf, F. A., Yakubova, N., Günnemann, S., Trapnell, C., Lopez‐Paz, D., & Theis, F. J.
Affiliations
  • Wellcome Sanger Institute (GB)
  • Helmholtz Munich (DE)
  • University of Washington (US)
  • University of California, Berkeley (US)
  • Howard Hughes Medical Institute (US)
  • Technical University of Munich (DE)
Corresponding author
Theis, Fabian J
Venue
Molecular Systems Biology
Published
2023-05-08
Volume
19
Issue
6
Pages
e11517-e11517
ISSN
1744-4292 (ISSN-L)
OpenAlex
W4375955255
Cited by
307
References
50
Citations by year
2026: 118 · 2025: 97 · 2024: 71 · 2023: 19
FWCI
26.94
Type
journal article
Language
en
Open access
Licence
Locations
  • Molecular Systems Biology (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • PuSH - Publication Server of Helmholtz Zentrum München (submittedVersion, cc-by)
Funders
  • Bundesministerium für Bildung und Forschung (L031L0214A, 01IS18036A, 01IS18053A)
  • Chan Zuckerberg Initiative (2018‐182835, 2019‐207271)
  • EC | Horizon 2020 Framework Programme (874656)
  • Helmholtz Association (ZT‐I‐0007, ZT‐I‐PF‐5‐01)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Drug Discovery › Computational Drug Discovery Methods
Keywords
single-cell transcriptomics, Perturb-seq, chemical representation
MeSH terms
Single-Cell Gene Expression Analysis, Computational Biology, Gene Expression Profiling, High-Throughput Screening Assays

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.