Rethinking expertise: A multifactorial gene–environment interaction model of expert performance.
The DOI matches its Crossref record; 254 of the 255 references it cites were found in a registry or an index.
Abstract
Scientific interest in expertise-superior performance within a specific domain-has a long history in psychology. Although there is a broad consensus that a long period of practice is essential for expertise, a long-standing controversy in the field concerns the importance of other variables such as cognitive abilities and genetic factors. According to the influential deliberate practice theory, expert performance is essentially limited by a single variable: the amount of deliberate practice an individual has accumulated. Here, we provide a review of the literature on deliberate practice, expert performance, and its neural correlates. A particular emphasis is on recent studies indicating that expertise is related to numerous traits other than practice as well as genetic factors. We argue that deliberate practice theory is unable to account for major recent findings relating to expertise and expert performance, and propose an alternative multifactorial gene-environment interaction model of expertise, which provides an adequate explanation for the available empirical data and may serve as a useful framework for future empirical and theoretical work on expert performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There is also substantial evidence that deliberate practice contributes to superior performance across a wide variety of domains, including chess, music performance, sports, board games, video games, and dance”
From Gifted Students (2019) · cites this paper as Ullén et al. 2016
In the citing paper
“demonstrating that a multitude of factors contribute to both the development of expertise and expert performance.”
From Gifted Students (2019) · cites this paper as Ullén et al. 2016
In the citing paper
“All frameworks also agree that unique genes generate individual differences that are important predictors of successful performance in some domains, such as height in many sports”
From Deliberate Practice and Proposed Limits on the Effects of Practice on the Acquisition of Expert Performance: Why the Original Definition Matters and Recommendations for Future Research (2019) · cites this paper as Ullén et al. (2015)
In the citing paper
“and that genes in the DNA are expressed in response to practice activities, and these genes play a central role mediating the biological changes of body and nervous system.”
From Deliberate Practice and Proposed Limits on the Effects of Practice on the Acquisition of Expert Performance: Why the Original Definition Matters and Recommendations for Future Research (2019) · cites this paper as Ullén et al. (2015)
In the citing paper
“All of them agree that extended practice is necessary to attain expert performance”
From Deliberate Practice and Proposed Limits on the Effects of Practice on the Acquisition of Expert Performance: Why the Original Definition Matters and Recommendations for Future Research (2019) · cites this paper as Ullén et al. (2015)
Citations
Cites
Items 201–254 of 254
Oikkonen, J., Huang, Y., Onkamo, P., Ukkola-Vuoti, L., Raijas, P., Karma, K., Vieland, V. J., & Järvelä, I. (2015) · Molecular Psychiatry · doi:10.1038/mp.2014.8 · Cited by 81 · cited 1×
“Musical aptitude is influenced by genetic factors (Drayna, Manichaikul, de Lange, Snieder, & Spector, 2001; Oikkonen et al., 2015; Ukkola-Vuoti et al., 2013), and Mosing, Madison, et al. (2014) found genetic pleiotropy to be the major factor behind associations between music practice and musical…”
BackgroundJabusch, H., Alpers, H., Kopiez, R., Vauth, H., & ALTENMÜLLER, E. O. (2008) · Human Movement Science · doi:10.1016/j.humov.2008.08.001 · Cited by 81
Specialization Effect and Its Influence on Memory and Problem Solving in Expert Chess Players
Bilalić, M., McLeod, P., & Gobet, F. (2009) · Cognitive Science · doi:10.1111/j.1551-6709.2009.01030.x · Cited by 80
Simple musical tests as measures of spearman's g
Lynn, R., Wilson, R., & Gault, A. (1989) · Personality and Individual Differences · doi:10.1016/0191-8869(89)90173-6 · Cited by 80
Elmer, S., Hänggi, J., Meyer, M. E., & Jäncke, L. (2010) · Human Brain Mapping · doi:10.1002/hbm.21169 · Cited by 79
A Model for the Transfer of Perceptual-Motor Skill Learning in Human Behaviors
Rosalie, S. M., & Müller, S. (2012) · Research Quarterly for Exercise and Sport · doi:10.1080/02701367.2012.10599876 · Cited by 77 · cited 1×
“Expert athletes may show superior performance in related sports (Rosalie & Müller, 2012, 2014).”
BackgroundWoods, E. A., Hernández, A. E., Wagner, V. E., & Beilock, S. L. (2014) · Brain and Cognition · doi:10.1016/j.bandc.2014.03.007 · Cited by 72
Measuring Working Memory Capacity With Greater Precision in the Lower Capacity Ranges
Nutley, S. B., Söderqvist, S., Bryde, S., Humphreys, K., & Klingberg, T. (2009) · Developmental Neuropsychology · doi:10.1080/87565640903325741 · Cited by 71
Vollmann, H., Ragert, P., Conde, V., Villringer, A., Claßen, J., Witte, O. W., & Steele, C. J. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00245 · Cited by 70 · cited 1×
“…such as the corpus callosum (Bengtsson et al., 2005; Schlaug, Jäncke, Huang, Staiger, & Steinmetz, 1995; Steele, Bailey, Zatorre, & Penhune, 2013; Vollmann et al., 2014), the superior longitudinal fasciculus (Oechslin, Van De Ville, Lazeyras, Hauert, & James, 2013), and the pyramidal tracts…”
On the effect of chess training on scholastic achievement
Bart, W. M. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00762 · Cited by 68 · cited 2×
“Studies of cognitive benefits of chess training have likewise yielded mixed results (Bart, 2014; Gliga & Flesner, 2014; Gobet & Campitelli, 2006).”
ResultNature, nurture, and expertise
Plomin, R., Shakeshaft, N. G., McMillan, A., & Trzaskowski, M. (2014) · Intelligence · doi:10.1016/j.intell.2013.06.008 · Cited by 67
Individual differences in expertise development over decades in a complex intellectual domain
Howard, R. W. (2009) · Memory & Cognition · doi:10.3758/mc.37.2.194 · Cited by 64 · cited 2×
“However, the distinction between deliberate practice and other forms of practice is in reality often very unclear (Howard, 2009).”
BackgroundThe architecture of the chess player׳s brain
Hänggi, J., Brütsch, K., Siegel, A. M., & Jäncke, L. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.07.019 · Cited by 63
Taxi drivers’ exceptional memory of street names
Kalakoski, V., & Saariluoma, P. O. (2001) · Memory & Cognition · doi:10.3758/bf03200464 · Cited by 63 · cited 2×
“Taxi drivers show exceptional memory for street names when these are organized according to driving routes, but not for randomly organized lists (Kalakoski & Saariluoma, 2001).”
BackgroundGenetic Pleiotropy Explains Associations between Musical Auditory Discrimination and Intelligence
Mosing, M. A., Pedersen, N. L., Madison, G., & Ullén, F. (2014) · PLoS ONE · doi:10.1371/journal.pone.0113874 · Cited by 63
Self-citationThe role of intelligence for performance in the prototypical expertise domain of chess
Grabner, R. H. (2013) · Intelligence · doi:10.1016/j.intell.2013.07.023 · Cited by 63
Robust expertise effects in right FFA
McGugin, R. W., Newton, A. T., Gore, J. C., & Gauthier, I. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.08.029 · Cited by 59
Bernardi, G., Cecchetti, L., Handjaras, G., Sani, L., Gaglianese, A., Ceccarelli, R., Franzoni, F., Galetta, F., Santoro, G., Goebel, R., et al. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00888 · Cited by 59 · cited 1×
“…& Jancke, 2011), painters (Lorains, Ball, & MacMahon, 2013), chess players (Hänggi, Brütsch, Siegel, & Jancke, 2014), professional car drivers (Bernardi et al., 2014), karate experts (Roberts, Bain, Day, & Husain, 2013), perfumers (Delon-Martin, Plailly, Fonlupt, Veyrac, & Royet, 2013), and…”
de Bruin, A. B. H., Kok, E. M., Leppink, J., & Camp, G. (2013) · Intelligence · doi:10.1016/j.intell.2013.07.004 · Cited by 58
Relationships among Impulsiveness, Locus of Control, Sex, and Music Practice
Miksza, P. (2006) · Journal of Research in Music Education · doi:10.1177/002242940605400404 · Cited by 57 · cited 2×
“For the domain of music, impulsivity has been shown to influence practice quality (Miksza, 2006, 2011).”
BackgroundJuggling revisited — A voxel–based morphometry study with expert jugglers
Gerber, P. A., Schlaffke, L., Heba, S., Greenlee, M. W., Schultz, T., & Schmidt‐Wilcke, T. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.04.023 · Cited by 55 · cited 2×
“…have also been found for other groups, including taxi drivers (Maguire et al., 2000; Woollett & Maguire, 2011), jugglers (Draganski et al., 2004; Gerber et al., 2014; Scholz et al., 2009), dancers (Hänggi, Koeneke, Bezzola, & Jäncke, 2010), professional simultaneous interpreters (Elmer, Hänggi,…”
BackgroundReading sheet music facilitates sensorimotor mu-desynchronization in musicians
Behmer, L. P., & Jantzen, K. J. (2011) · Clinical Neurophysiology · doi:10.1016/j.clinph.2010.12.035 · Cited by 54 · cited 1×
“Behmer and Jantzen (2011) used EEG to compare brain activity in musicians and nonmusicians during passive observation of sheet music.”
BackgroundHeritability of Myopia and Ocular Biometrics in Koreans: The Healthy Twin Study
Kim, M. H., Zhao, D., Kim, W., Lim, D.-H., Song, Y., Güallar, E., Cho, J., Sung, J., Chung, E., & Chung, T.-Y. (2013) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.12-11254 · Cited by 52
Abdul‐Kareem, I. A., Stancak, A., Parkes, L. M., & Sluming, V. (2011) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.22391 · Cited by 52 · cited 1×
“…(2002) found a correlation of r .39 between years of musical training and gray matter volume in Broca’s area in younger adult musicians, and Abdul-Kareem et al. (2011) reported a correlation of r .7 between the gray matter volume of this region and years of performance in orchestra…”
BackgroundRoberts, R. E., Bain, P. G., Day, B. L., & Husain, M. (2013) · Cerebral Cortex · doi:10.1093/cercor/bhs219 · Cited by 49
Anthropometry, Somatotypes, and Aerobic Power in Ballet, Contemporary Dance, and DanceSport
Liiv, H., Wyon, M. A., Jürimäe, T., Saar, M., Mäestu, J., & Jürimäe, J. (2013) · Medical Problems of Performing Artists · doi:10.21091/mppa.2013.4041 · Cited by 47 · cited 1×
“Dancers specializing in dances with different functional requirements show characteristic group differences in height and somatotype (Liiv et al., 2013).”
BackgroundLongitudinal effects of different types of practice on the development of chess expertise.
Howard, R. W. (2012) · Applied Cognitive Psychology · doi:10.1002/acp.1834 · Cited by 44 · cited 4×
“…based on single cases (Ericsson, Chase, & Faloon, 1980), or observational (Campitelli & Gobet, 2008; de Bruin, Kok, Leppink, & Camp, 2014; Howard, 2012; Hyde et al., 2009; Jabusch, Alpers, Kopiez, Vauth, & Altenmüller, 2009; McPherson, 2005) and randomized (Bilhartz, Bruhn, & Olson,…”
BackgroundInfluentialCognitive Benefits of Chess Training in Novice Children
Gliga, F., & Flesner, P. I. (2014) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2014.01.328 · Cited by 43 · cited 1×
“Studies of cognitive benefits of chess training have likewise yielded mixed results (Bart, 2014; Gliga & Flesner, 2014; Gobet & Campitelli, 2006).”
ResultPhenotypic and genetic relationships between vocational interests and personality
Harris, J. A., Vernon, P. A., Johnson, A. M., & Jang, K. L. (2006) · Personality and Individual Differences · doi:10.1016/j.paid.2005.11.024 · Cited by 43 · cited 1×
“…performance, such as personality, interests, and cognitive abilities, have been shown to be partly heritable, it is reasonable to also expect some genetic influences on variation in expert performance (Bouchard & McGue, 2003; Harris et al., 2006; Poldermann et al., 2015; Plomin & Spinath, 2004).”
BackgroundNeubauer, A. C., & Knorr, E. (1998) · Intelligence · doi:10.1016/s0160-2896(99)80058-0 · Cited by 42 · cited 1×
“Finally, intelligence shows weak to moderate correlations with RT (r .3 to .5; Deary, Der, & Ford, 2001; Jensen, 2006), processing speed (r .3 to .5; Neubauer & Knorr, 1998; Rindermann & Neubauer, 2004), and temporal and nontemporal sensory discrimination (r .2 to .5; Spearman, C. (1904); Rammsayer…”
BackgroundEducational benefits of chess instruction: A critical review
Gobet, F. R., & Campitelli, G. (2006) · Murdoch University Research Portal · Cited by 42
Detterman, D. K. (2014) · Intelligence · doi:10.1016/j.intell.2014.02.004 · Cited by 41 · cited 2×
“…innate talent (see Howe et al., 1998, with commentaries), the relative importance of deliberate practice and other variables for expert performance (Detterman, 2014; Hambrick, Altmann, Oswald, Meinz, & Gobet, 2014; Hambrick, Oswald, et al., 2014; Platz et al., 2014), and giftedness (see Ericsson…”
BackgroundBoth novelty and expertise increase action observation network activity
Liew, S., Sheng, T., Margetis, J. L., & Aziz‐Zadeh, L. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00541 · Cited by 41
Saariluoma, P. (1994) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640749408401130 · Cited by 39 · cited 2×
“Chess experts display superior memory for game positions from real games, but to a much lower degree for random positions, and the grouping of pieces into chunks is based on the strategic relations between pieces (Chase & Simon, 1973b; Gobet & Simon, 1996a, 1996b; Saariluoma, 1994; Simon & Chase, 1973).”
BackgroundEffect of skill level on recall of visually presented patterns of musical notes
Kalakoski, V. (2007) · Scandinavian Journal of Psychology · doi:10.1111/j.1467-9450.2007.00535.x · Cited by 38 · cited 2×
“Musicians display superior processing of wellformed, as opposed to random, musical structures (Halpern & Bower, 1982; Kalakoski, 2007; Meinz & Salthouse, 1998; Sloboda, 1976) and process musical materials using larger units than novices (Waters et al., 1997, 1998).”
BackgroundUnterrainer, J. M., Kaller, C. P., Leonhart, R., & Rahm, B. (2011) · The American Journal of Psychology · doi:10.5406/amerjpsyc.124.2.0213 · Cited by 38
Troche, S. J., & Rammsayer, T. H. (2009) · Personality and Individual Differences · doi:10.1016/j.paid.2009.02.001 · Cited by 37 · cited 1×
“…(r .3 to .5; Neubauer & Knorr, 1998; Rindermann & Neubauer, 2004), and temporal and nontemporal sensory discrimination (r .2 to .5; Spearman, C. (1904); Rammsayer & Brandler, 2007; Troche & Rammsayer, 2009)— basic capacities that appear likely to be important for expert performance in many domains.”
BackgroundExpertise Facilitates the Transfer of Anticipation Skill across Domains
Rosalie, S. M., & Müller, S. (2013) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2013.807856 · Cited by 37 · cited 1×
“Expert athletes may show superior performance in related sports (Rosalie & Müller, 2012, 2014).”
BackgroundEricsson, K. A., Roring, R. W., & Nandagopal, K. (2007) · High Ability Studies · doi:10.1080/13598130701351088 · Cited by 36 · cited 3×
“This has sometimes been brought forward as evidence that genetic factors are unimportant for expert performance, because it would appear unlikely that significant changes in relevant gene frequencies could have taken place during such a short time period (Ericsson et al., 2007a).”
Background“No level up!”: no effects of video game specialization and expertise on cognitive performance
Gobet, F. R., Johnston, S. J., Ferrufino, G., Johnston, M. S., Jones, M. B., Molyneux, A., Terzis, A., & Weeden, L. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01337 · Cited by 32 · cited 1×
“In a recent study Gobet et al. (2014) found no effects of video game expertise on an attention task (the Eriksen Flanker Compatibility Task), nor on a visual change detection task.”
BackgroundPerception of Contour in Music Reading
SLOBODA, J. A. (1978) · Perception · doi:10.1068/p070323 · Cited by 31 · cited 1×
“…(McKeithen, Reitman, Rueter, & Hirtle, 1981), taxi drivers (Kalakoski & Saariluoma, 2001), electronic technicians (Egan & Schwartz, 1979), and musicians (Halpern & Bower, 1982; Kalakoski, 2007; Sloboda, 1976, 1978; Waters, Townsend, & Underwood, 1998; Waters, Underwood, & Findlay, 1997).”
Facing facts about deliberate practice
Hambrick, D. Z., Altmann, E. M., Oswald, F. L., Meinz, E. J., & Gobet, F. R. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00751 · Cited by 28
Self-citationGains of item-specific training in visual working memory and their neural correlates
Zimmer, H. D., Popp, C., Reith, W., & Krick, C. M. (2012) · Brain Research · doi:10.1016/j.brainres.2012.05.019 · Cited by 28
Ullén, F., Mosing, M. A., & Madison, G. (2015) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.12630 · Cited by 23 · cited 3×
“Similarly, Ullén, Mosing, and Madison (2015) recently reported that the association between music practice and accuracy of motor timing disappears when controlling for genetics and shared environment.”
BackgroundInfluentialSelf-citationThe relation of musical ability to general intelligence and the major primaries
Lynn, R., & Gault, A. (1986) · Research in Education · doi:10.1177/003452378603600107 · Cited by 21 · cited 1×
“This ability is related to intelligence (Lynn & Gault, 1986; Lynn, Wilson, & Gault, 1989) and music practice (Ullén, Mosing, Holm, Eriksson, & Madison, 2014).”
Hutchinson, C. U., Sachs-Ericsson, N. J., & Ericsson, K. A. (2013) · High Ability Studies · doi:10.1080/13598139.2013.780966 · Cited by 20
Measurement of a number of indices of hand and movement angles in pianists with overuse disorders
Sakai, N., & SHIMAWAKI, S. (2010) · Journal of Hand Surgery (European Volume) · doi:10.1177/1753193409352405 · Cited by 18 · cited 1×
“Hand morphology influences dexterity in pianists, and a small hand makes it difficult to play wide chords and increases the risk for overuse disorders (Sakai & Shimawaki, 2010; Wagner, 1988).”
BackgroundPerceptual Skill in the Game of Othello
Wolff, A. S., Mitchell, D. H., & Frey, P. W. (1984) · The Journal of Psychology · doi:10.1080/00223980.1984.9712586 · Cited by 18
The interaction of drill and practice and error training with individual differences
Kluge, A., Ritzmann, S., Burkolter, D., & Sauer, J. (2010) · Cognition Technology & Work · doi:10.1007/s10111-010-0158-z · Cited by 15
Plasticity of the Human Auditory Cortex
Pantev, C., Weisz, N., Schulte, M., & Elbert, T. R. (2003) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195140132.003.0012 · Cited by 10 · cited 1×
“…auditory areas (Bermudez et al., 2009; Elmer, Hänggi, Meyer, & Jancke, 2013; Fauvel et al., 2014; Gaser & Schlaug, 2003; Hyde et al., 2009; Pantev & Herholz, 2011; Schneider et al., 2002; SeitherPreisler, Parncutt, & Schneider, 2014), the cerebellum (S. Hutchinson, Lee, Gaab, & Schlaug, 2003),…”
Genetic influences on musical giftedness, talent, and practice
Mosing, M. A., & Ullén, F. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199685851.003.0004 · Cited by 9
Self-citationKennedy, Q., Taylor, J. L., Noda, A. M., Adamson, M. M., Murphy, G. M., Zeitzer, J. M., & Yesavage, J. A. (2010) · Behavior Genetics · doi:10.1007/s10519-010-9436-z · Cited by 8 · cited 1×
“More generally, this also highlights the potential importance of more complex interplay between genes and environment—that is, gene–environment interactions (Kennedy et al., 2011; Schellenberg, 2015) and various forms of gene– environment covariation (Scarr, 1996), for expertise.”
BackgroundLehmann, A. C. (1998) · Behavioral and Brain Sciences · doi:10.1017/s0140525x9837123x · Cited by 7
Poveda, A., Jelenkovic, A., Susanne, C., & Rebato, E. (2010) · PubMed · Cited by 5
Cited by
Personality and Individual Differences
Hruska, E., & Bonneville-Roussy, A. (2022) · The Oxford Handbook of Music Performance, Volume 1 · doi:10.1093/oxfordhb/9780190056285.013.25 · Cited by 388
Ericsson, K. A., & Harwell, K. W. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02396 · Cited by 370
Domain-specific prior knowledge and learning: A meta-analysis
Simonsmeier, B. A., Flaig, M., Deiglmayr, A., Schalk, L., & Schneider, M. (2022) · Educational Psychologist · doi:10.1080/00461520.2021.1939700 · Cited by 255
Worrell, F. C., Subotnik, R. F., Olszewski-Kubilius, P., & Dixson, D. D. (2019) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-102846 · Cited by 245
“A relatively recent addition to the extant literature, the multifactorial gene–environment interaction model (MGIM) (Ullén et al. 2016) is a model of expertise rather than talent development.”
BackgroundInfluentialMoving from Novice to Expertise and Its Implications for Instruction
Persky, A. M., & Robinson, J. D. (2017) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe6065 · Cited by 212
“”(1)Ullén and colleagues recently found that expertise might have a genetic component which may explain why only a few people become experts in their domain.(7) Experts excel mainly in their domains, and there is little evidence that a person highly skilled in one area (eg, medicinal chemistry) can transfer or teach t…”
Background
Metadata
- Authors
- Ullén, F., Hambrick, D. Z., & Mosing, M. A.
- Affiliations
- Karolinska Institutet (SE)
- Michigan State University (US)
- Venue
- Psychological Bulletin
- Published
- 2016
- Volume
- 142
- Issue
- 4
- Pages
- 427-446
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2218640407
- Cited by
- 182
- References
- 255
- Citations by year
- 2026: 14 · 2025: 12 · 2024: 19 · 2023: 20 · 2022: 17 · 2021: 20 · 2020: 34 · 2019: 14 · 2018: 23 · 2017: 5 · 2016: 4
- FWCI
- 26.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Bank of Sweden Tercentenary Foundation
- Sven and Dagmar Salén Foundation
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Sport Psychology and Performance
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
- Keywords
- expertise, expert performance, gene-environment interaction, deliberate practice, cognitive ability, genetic factors
- MeSH terms
- Aptitude, Brain, Humans, Models, Theoretical, Practice, Psychological, Professional Competence, Gene-Environment Interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.