Massively parallel functional dissection of mammalian enhancers in vivo
Open accessAbstract
The functional consequences of genetic variation in mammalian regulatory elements are poorly understood. We report the in vivo dissection of three mammalian enhancers at single-nucleotide resolution through a massively parallel reporter assay. For each enhancer, we synthesized a library of >100,000 mutant haplotypes with 2-3% divergence from the wild-type sequence. Each haplotype was linked to a unique sequence tag embedded within a transcriptional cassette. We introduced each enhancer library into mouse liver and measured the relative activities of individual haplotypes en masse by sequencing the transcribed tags. Linear regression analysis yielded highly reproducible estimates of the effect of every possible single-nucleotide change on enhancer activity. The functional consequence of most mutations was modest, with ∼22% affecting activity by >1.2-fold and ∼3% by >2-fold. Several, but not all, positions with higher effects showed evidence for purifying selection, or co-localized with known liver-associated transcription factor binding sites, demonstrating the value of empirical high-resolution functional analysis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
DeepLiver predictions of the effect of enhancer mutations correlate with experimental saturation mutagenesis data on six enhancers from earlier studies (correlation ranging between 0.36 and 0.75).
“DeepLiver predictions of the effect of enhancer mutations correlate with experimental results (with a correlation ranging between 0.36 and 0.75; Extended Data Fig. 7f).”
From Single-cell spatial multi-omics and deep learning dissect enhancer-driven gene regulatory networks in liver zonation (2024) · cites this paper as [41]
The Abstract and Results section report experimental saturation-mutagenesis measurements of mutation effects for three mammalian liver enhancers, so this paper supports the identification of an earlier experimental dataset. It does not report six enhancers or a comparison with DeepLiver predictions; the correlations of 0.36–0.75 are not established by this source. Page numbers are not available in our copy. (LLM, conf=0.87)
Experimental — accuracy not yet measuredDeepLiver in silico mutagenesis was validated by calculating the correlation between the effect of mutations predicted by DeepLiver and experimental saturation mutagenesis data on six enhancers from earlier studies (three in vivo, three HepG2).
“To validate DeepLiver in silico mutagenesis, we calculated the correlation between the effect of the mutations predicted by DeepLiver and experimental saturation mutagenesis data on six enhancers from earlier studies (three enhancers from in vivo studies and three enhancers from HepG2 cells)41,42.”
From Single-cell spatial multi-omics and deep learning dissect enhancer-driven gene regulatory networks in liver zonation (2024) · cites this paper as [41]
Massively Parallel Reporter Assays have been established to directly test and identify active enhancers.
“new experimental approaches, such as Massively Parallel Reporter Assays (MPRAs), have been established to directly test and identify active enhancers14–18.”
From Enhancer grammar of liver cell types and hepatocyte zonation states (2022) · cites this paper as [14]
DeepLiver in silico mutagenesis was validated by calculating the correlation between predicted mutation effects and experimental saturation mutagenesis data on six enhancers from earlier studies (three in vivo and three HepG2).
“To validate DeepLiver in silico mutagenesis, we calculated the correlation between the effect of the mutations predicted by DeepLiver and experimental saturation mutagenesis data on six enhancers from earlier studies (three enhancers from in vivo studies and three enhancers from HepG2)14,60.”
From Enhancer grammar of liver cell types and hepatocyte zonation states (2022) · cites this paper as [14]
DeepLiver in silico mutagenesis correlates with experimental saturation mutagenesis in the AldoB enhancer.
“Correlation between DeepLiver in silico mutagenesis and experimental saturation mutagenesis in the AldoB enhancer.”
From Enhancer grammar of liver cell types and hepatocyte zonation states (2022) · cites this paper as Patwardhan et al., 2012
A major advantage of MPRAs is that sequences to be tested can simply be synthesized, enabling straightforward saturation mutagenesis and programming synthetic enhancers.
“A major advantage of MPRAs is that sequences to be tested can simply be synthesized, enabling straightforward saturation mutagenesis [G] of enhancers67,184,185 as well as programming synthetic enhancers to inform modelling of their properties186,187.”
From Towards a comprehensive catalogue of validated and target-linked human enhancers (2020) · cites this paper as [67]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A map of human genome variation from population-scale sequencing.
The 1000 Genomes Project Consortium (2010) · Nature · doi:10.1038/nature09534 · Cited by 7,877 · see citing works
Hindorff, L. A., Sethupathy, P., Junkins, H. A., Ramos, E. M., Mehta, J. P., Collins, F. S., & Manolio, T. A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0903103106 · Cited by 4,013 · see citing works
Flagged first, then by citations
Research works printed but not linked: 23
- [1]Consortium EP A user’s guide to the encyclopedia of DNA elements (ENCODE) PLoS Biol 9 e1001046 2011 21526222 PMC3079585
- [2]Ernst J Mapping and analysis of chromatin state dynamics in nine human cell types Nature 473 43 49 2011 21441907 PMC3088773
- [3]Visel A ChIP-seq accurately predicts tissue-specific activity of enhancers Nature 457 854 858 2009 19212405 PMC2745234
- [4]Cooper GM Shendure J Needles in stacks of needles: finding disease-causal variants in a wealth of genomic data Nat Rev Genet 12 628 640 2011 21850043
- [5]Kleinjan DA van Heyningen V Long-range control of gene expression: emerging mechanisms and disruption in disease Am J Hum Genet 76 8 32 2005 15549674 PMC1196435
- [6]Noonan JP McCallion AS Genomics of long-range regulatory elements Annu Rev Genomics Hum Genet 11 1 23 2010 20438361
- [7]VanderMeer JE Ahituv N cis-regulatory mutations are a genetic cause of human limb malformations Dev Dyn 240 920 930 2011 21509892 PMC3174732
- [9]Sabourin JC An intronic enhancer essential for tissue-specific expression of the aldolase B transgenes J Biol Chem 271 3469 3473 1996 8631949
- [10]Gregori C Expression of the rat aldolase B gene: a liver-specific proximal promoter and an intronic activator Biochem Biophys Res Commun 176 722 729 1991 2025285
- [11]Gregori C Porteu A Mitchell C Kahn A Pichard AL In vivo functional characterization of the aldolase B gene enhancer J Biol Chem 277 28618 28623 2002 12034748
- [12]Kim MJ Functional characterization of liver enhancers that regulate drug-associated transporters Clinical pharmacology and therapeutics 89 571 578 2011 21368754 PMC3227682
- [13]Dang Q Structure of the hepatic control region of the human apolipoprotein E/C-I gene locus J Biol Chem 270 22577 22585 1995 7673250
- [14]Patwardhan RP High-resolution analysis of DNA regulatory elements by synthetic saturation mutagenesis Nat Biotechnol 27 1173 1175 2009 19915551 PMC2849652
- [15]Hiatt JB Patwardhan RP Turner EH Lee C Shendure J Parallel, tag-directed assembly of locally derived short sequence reads Nature methods 7 119 122 2010 20081835 PMC2848820
- [16]Zhang G Budker V Wolff JA High levels of foreign gene expression in hepatocytes after tail vein injections of naked plasmid DNA Hum Gene Ther 10 1735 1737 1999 10428218
- [17]Kel AE MATCH: A tool for searching transcription factor binding sites in DNA sequences Nucleic acids research 31 3576 3579 2003 12824369 PMC169193
- [18]Loots GG Identification of a coordinate regulator of interleukins 4, 13, and 5 by cross-species sequence comparisons Science 288 136 140 2000 10753117
- [19]Margulies EH Analyses of deep mammalian sequence alignments and constraint predictions for 1% of the human genome Genome Res 17 760 774 2007 17567995 PMC1891336
- [20]Visel A Ultraconservation identifies a small subset of extremely constrained developmental enhancers Nat Genet 40 158 160 2008 18176564 PMC2647775
- [21]Blow MJ ChIP-Seq identification of weakly conserved heart enhancers Nat Genet 42 806 810 20729851 PMC3138496
- [22]Schmidt D Five-vertebrate ChIP-seq reveals the evolutionary dynamics of transcription factor binding Science 328 1036 1040 2010 20378774 PMC3008766
- [23]Cooper GM Distribution and intensity of constraint in mammalian genomic sequence Genome Res 15 901 913 2005 15965027 PMC1172034
- [25]Botstein D Shortle D Strategies and applications of in vitro mutagenesis Science 229 1193 1201 1985 2994214
Cited by
Sorted by citations
A general framework for estimating the relative pathogenicity of human genetic variants
Kircher, M., Witten, D. M., Jain, P., O'Roak, B. J., Cooper, G. M., & Shendure, J. (2014) · Nature Genetics · doi:10.1038/ng.2892 · Cited by 6,183 · see citing works
Self-citationMali, P., Aach, J., Stranges, P. B., Esvelt, K. M., Moosburner, M., Kosuri, S., Yang, L., & Church, G. M. (2013) · Nature Biotechnology · doi:10.1038/nbt.2675 · Cited by 1,752 · see citing works
Transcriptional enhancers: from properties to genome-wide predictions
Shlyueva, D., Stampfel, G., & Stark, A. (2014) · Nature Reviews Genetics · doi:10.1038/nrg3682 · Cited by 1,356 · see citing works
Guidelines for investigating causality of sequence variants in human disease
MacArthur, D. G., Manolio, T. A., Dimmock, D. P., Rehm, H. L., Shendure, J., Abecasis, G. R., Adams, D. R., Altman, R. B., Antonarakis, S. E., Ashley, E. A., et al. (2014) · Nature · doi:10.1038/nature13127 · Cited by 1,224 · see citing works
Self-citationGenome-Wide Quantitative Enhancer Activity Maps Identified by STARR-seq
Arnold, C. D., Gerlach, D., Stelzer, C., Boryń, Ł. M., Rath, M., & Stark, A. (2013) · Science · doi:10.1126/science.1232542 · Cited by 1,125 · see citing works
Detection of ultra-rare mutations by next-generation sequencing
Schmitt, M. W., Kennedy, S. R., Salk, J. J., Fox, E. J., Hiatt, J. B., & Loeb, L. A. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1208715109 · Cited by 1,047 · see citing works
Self-citationBCL11A enhancer dissection by Cas9-mediated in situ saturating mutagenesis
Canver, M. C., Smith, E. C., Sher, F., Pinello, L., Sanjana, N. E., Shalem, O., Chen, D. D., Schupp, P. G., Vinjamur, D. S., Garcia, S. P., et al. (2015) · Nature · doi:10.1038/nature15521 · Cited by 886 · see citing works
The Post-GWAS Era: From Association to Function
Gallagher, M. D., & Chen-Plotkin, A. S. (2018) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2018.04.002 · Cited by 822 · see citing works
Bauer, D. E., Kamran, S. C., Lessard, S., Xu, J., Fujiwara, Y., Lin, C., Shao, Z., Canver, M. C., Smith, E. C., Pinello, L., et al. (2013) · Science · doi:10.1126/science.1242088 · Cited by 653 · see citing works
Absence of a simple code: how transcription factors read the genome
Slattery, M., Zhou, T., Yang, L., Dantas Machado, A. C., Gordân, R., & Rohs, R. (2014) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2014.07.002 · Cited by 537 · see citing works
Tewhey, R., Kotliar, D., Park, D. S., Liu, B., Winnicki, S., Reilly, S. K., Andersen, K. G., Mikkelsen, T. S., Lander, E. S., Schaffner, S. F., et al. (2016) · Cell · doi:10.1016/j.cell.2016.04.027 · Cited by 480 · see citing works
Synthetic DNA Synthesis and Assembly: Putting the Synthetic in Synthetic Biology
Hughes, R. A., & Ellington, A. D. (2017) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a023812 · Cited by 411 · see citing works
Towards a comprehensive catalogue of validated and target-linked human enhancers
Gasperini, M., Tome, J. M., & Shendure, J. (2020) · Nature Reviews Genetics · doi:10.1038/s41576-019-0209-0 · Cited by 396 · see citing works
Self-citationInoue, F., Kircher, M., Martin, B., Cooper, G. M., Witten, D. M., McManus, M. T., Ahituv, N., & Shendure, J. (2017) · Genome Research · doi:10.1101/gr.212092.116 · Cited by 331 · see citing works
Self-citationSystematic Functional Dissection of Common Genetic Variation Affecting Red Blood Cell Traits
Ulirsch, J. C., Nandakumar, S. K., Wang, L., Giani, F. C., Zhang, X., Rogov, P., Melnikov, A., McDonel, P., Do, R., Mikkelsen, T. S., et al. (2016) · Cell · doi:10.1016/j.cell.2016.04.048 · Cited by 328 · see citing works
The Causes and Consequences of Genetic Interactions (Epistasis)
Domingo, J., Baeza-Centurion, P., & Lehner, B. (2019) · Annual Review of Genomics and Human Genetics · doi:10.1146/annurev-genom-083118-014857 · Cited by 242 · see citing works
Decoding enhancers using massively parallel reporter assays
Inoue, F., & Ahituv, N. (2015) · Genomics · doi:10.1016/j.ygeno.2015.06.005 · Cited by 272 · see citing works
Self-citationMany human accelerated regions are developmental enhancers
Capra, J. A., Erwin, G. D., McKinsey, G., Rubenstein, J. L. R., & Pollard, K. S. (2013) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0025 · Cited by 235 · see citing works
The road to precision psychiatry: translating genetics into disease mechanisms
Gandal, M. J., Leppa, V., Won, H., Parikshak, N. N., & Geschwind, D. H. (2016) · Nature Neuroscience · doi:10.1038/nn.4409 · Cited by 223 · see citing works
A systematic evaluation of the design and context dependencies of massively parallel reporter assays
Klein, J. C., Agarwal, V., Inoue, F., Keith, A., Martin, B., Kircher, M., Ahituv, N., & Shendure, J. (2020) · Nature Methods · doi:10.1038/s41592-020-0965-y · Cited by 226 · see citing works
Wang, X., He, L., Goggin, S. M., Saadat, A., Wang, L., Sinnott-Armstrong, N., Claussnitzer, M., & Kellis, M. (2018) · Nature Communications · doi:10.1038/s41467-018-07746-1 · Cited by 159 · see citing works
Activity-enhancing mutations in an E3 ubiquitin ligase identified by high-throughput mutagenesis
Starita, L. M., Pruneda, J. N., Lo, R. S., Fowler, D. M., Kim, H. J., Hiatt, J. B., Shendure, J., Brzovic, P. S., Fields, S., & Klevit, R. E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1303309110 · Cited by 183 · see citing works
Self-citationGenome-scale high-resolution mapping of activating and repressive nucleotides in regulatory regions
Ernst, J., Melnikov, A., Zhang, X., Wang, L., Rogov, P., Mikkelsen, T. S., & Kellis, M. (2016) · Nature Biotechnology · doi:10.1038/nbt.3678 · Cited by 162 · see citing works
Genomic Perspectives of Transcriptional Regulation in Forebrain Development
Nord, A. S., Pattabiraman, K., Visel, A., & Rubenstein, J. L. R. (2015) · Neuron · doi:10.1016/j.neuron.2014.11.011 · Cited by 152 · see citing works
The power of multiplexed functional analysis of genetic variants
Gasperini, M., Starita, L., & Shendure, J. (2016) · Nature Protocols · doi:10.1038/nprot.2016.135 · Cited by 160 · see citing works
Self-citationJacobs, J., Atkins, M., Davie, K., Imrichova, H., Romanelli, L., Christiaens, V., Hulselmans, G., Potier, D., Wouters, J., Taskiran, I. I., et al. (2018) · Nature Genetics · doi:10.1038/s41588-018-0140-x · Cited by 155 · see citing works
Genome-wide mapping of autonomous promoter activity in human cells
van Arensbergen, J., FitzPatrick, V. D., de Haas, M., Pagie, L., Sluimer, J., Bussemaker, H. J., & van Steensel, B. (2017) · Nature Biotechnology · doi:10.1038/nbt.3754 · Cited by 140 · see citing works
Functional assessment of human enhancer activities using whole-genome STARR-sequencing
Liu, Y., Yu, S., Dhiman, V. K., Brunetti, T., Eckart, H., & White, K. P. (2017) · Genome Biology · doi:10.1186/s13059-017-1345-5 · Cited by 136 · see citing works
A scalable platform for the development of cell-type-specific viral drivers
Hrvatin, S., Tzeng, C. P., Nagy, M. A., Stroud, H., Koutsioumpa, C., Wilcox, O. F., Assad, E. G., Green, J., Harvey, C. D., Griffith, E. C., et al. (2019) · eLife · doi:10.7554/elife.48089 · Cited by 118 · see citing works
A new class of temporarily phenotypic enhancers identified by CRISPR/Cas9-mediated genetic screening
Diao, Y., Li, B., Meng, Z., Jung, I., Lee, A. Y., Dixon, J., Maliskova, L., Guan, K.-L., Shen, Y., & Ren, B. (2016) · Genome Research · doi:10.1101/gr.197152.115 · Cited by 122 · see citing works
Systematic interrogation of human promoters
Weingarten-Gabbay, S., Nir, R., Lubliner, S., Sharon, E., Kalma, Y., Weinberger, A., & Segal, E. (2019) · Genome Research · doi:10.1101/gr.236075.118 · Cited by 110 · see citing works
“Toaddressfundamentalquestionsin gene expression,we and others have developed massively parallel reporter assays (MPRAs) probing the expression of various regulatory regions (Kwasnieski et al. 2012; Melnikov et al. 2012; Patwardhan et al. 2012; Sharon et al. 2012; Shalem et al. 2015).”
Supervised enhancer prediction with epigenetic pattern recognition and targeted validation
Sethi, A., Gu, M., Gumusgoz, E., Chan, L., Yan, K.-K., Rozowsky, J., Barozzi, I., Afzal, V., Akiyama, J. A., Plajzer-Frick, I., et al. (2020) · Nature Methods · doi:10.1038/s41592-020-0907-8 · Cited by 109 · see citing works
Self-citationVockley, C. M., Guo, C., Majoros, W. H., Nodzenski, M., Scholtens, D. M., Hayes, M. G., Lowe, W. L., & Reddy, T. E. (2015) · Genome Research · doi:10.1101/gr.190090.115 · Cited by 99 · see citing works
Multiplex enhancer-reporter assays uncover unsophisticated TP53 enhancer logic
Verfaillie, A., Svetlichnyy, D., Imrichova, H., Davie, K., Fiers, M., Kalender Atak, Z., Hulselmans, G., Christiaens, V., & Aerts, S. (2016) · Genome Research · doi:10.1101/gr.204149.116 · Cited by 82 · see citing works
Human evolution: the non-coding revolution
Franchini, L. F., & Pollard, K. S. (2017) · BMC Biology · doi:10.1186/s12915-017-0428-9 · Cited by 90 · see citing works
Bravo González-Blas, C., Matetovici, I., Hillen, H., Taskiran, I. I., Vandepoel, R., Christiaens, V., Sansores-García, L., Verboven, E., Hulselmans, G., Poovathingal, S., et al. (2024) · Nature Cell Biology · doi:10.1038/s41556-023-01316-4 · Cited by 85 · see citing works
Maricque, B. B., Chaudhari, H. G., & Cohen, B. A. (2019) · Nature Biotechnology · doi:10.1038/nbt.4285 · Cited by 83 · see citing works
Krebs, A. R., Dessus-Babus, S., Burger, L., & Schübeler, D. (2014) · eLife · doi:10.7554/elife.04094 · Cited by 81 · see citing works
Transcriptional enhancers and their communication with gene promoters
Ray-Jones, H., & Spivakov, M. (2021) · Cellular and Molecular Life Sciences · doi:10.1007/s00018-021-03903-w · Cited by 81 · see citing works
Functional characterization of enhancer evolution in the primate lineage
Klein, J. C., Keith, A., Agarwal, V., Durham, T., & Shendure, J. (2018) · Genome Biology · doi:10.1186/s13059-018-1473-6 · Cited by 63 · see citing works
Self-citationCis-regulatory landscapes of four cell types of the retina
Hartl, D., Krebs, A. R., Jüttner, J., Roska, B., & Schübeler, D. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkx923 · Cited by 37 · see citing works
Gauge fixing for sequence-function relationships
Posfai, A., Zhou, J., McCandlish, D. M., & Kinney, J. B. (2024) · doi:10.1101/2024.05.12.593772 · Cited by 11 · see citing works
Interpreting noncoding genetic variation in complex traits and human disease
Ward, L. D., & Kellis, M. (2012) · Nature Biotechnology · doi:10.1038/nbt.2422 · Cited by 6 · see citing works
Enhancer grammar of liver cell types and hepatocyte zonation states
González-Blas, C. B., Matetovici, I., Hillen, H., Taskiran, I. I., Vandepoel, R., Christiaens, V., Sansores-García, L., Verboven, E., Hulselmans, G., Poovathingal, S., et al. (2022) · doi:10.1101/2022.12.08.519575 · Cited by 5 · see citing works
- Authors
- Patwardhan, R. P., Hiatt, J. B., Witten, D. M., Kim, M. J., Smith, R. P., May, D., Lee, C., Andrie, J. M., Lee, S.-I., Cooper, G. M., Ahituv, N., Pennacchio, L. A., & Shendure, J.
- Affiliations
- University of Washington (US)
- University of California, San Francisco (US)
- Lawrence Berkeley National Laboratory (US)
- HudsonAlpha Institute for Biotechnology (US)
- Corresponding author
- Ahituv, Nadav
- Pennacchio, Len A
- Shendure, Jay
- ORCID
- Venue
- Nature Biotechnology
- Published
- 2012-02-26
- Volume
- 30
- Issue
- 3
- Pages
- 265-270
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC3402344
- OpenAlex
- W1989004527
- Cited by
- 536
- References
- 2
- Citations by year
- 2026: 26 · 2025: 41 · 2024: 19 · 2023: 30 · 2022: 25 · 2021: 29 · 2020: 34 · 2019: 41 · 2018: 40 · 2017: 43 · 2016: 51 · 2015: 57 · 2014: 43 · 2013: 34 · 2012: 22
- FWCI
- 20.07
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Institutes of Health (DP5OD009145)
- National Institute on Aging (F30AG039173)
- National Human Genome Research Institute (R01HG003988)
- National Institute of General Medical Sciences (U19GM061390, T32GM007175, U01GM061390)
- National Institute of Diabetes and Digestive and Kidney Diseases (P30DK026743)
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01HD059862)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Chromatin Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › CRISPR and Genetic Engineering
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Keywords
- enhancer function, single nucleotide variants, massively parallel reporter assay, mouse liver, purifying selection, transcription factor binding sites
- MeSH terms
- Animals, Binding Sites, Enhancer Elements, Genetic, Haplotypes, Humans, Liver, Mutation, Transcription Factors, Transcription, Genetic, Linear Models, Mutagenesis, Genes, Reporter, Evolution, Molecular, Mice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
